Opened 3 years ago

Closed 2 years ago

#55679 closed defect (fixed)

pypy3 @5.10.0: fails to fetch xz source during build and destroot; should not be downloading during build or destroot

Reported by: ryandesign (Ryan Schmidt) Owned by: danchr (Dan Villiom Podlaski Christiansen)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: jmroot (Joshua Root), ryandesign (Ryan Schmidt)
Port: pypy3

Description

pypy3 fails to destroot:

https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/16885/steps/install-port/logs/stdio

fetching archive https://tukaani.org/xz/xz-5.2.3.tar.gz
Traceback (most recent call last):
  File "package.py", line 351, in <module>
    retval, _ = package(*sys.argv[1:])
  File "package.py", line 330, in package
    return create_package(basedir, options, **kwds)
  File "package.py", line 89, in create_package
    embed_dependencies=options.embed_dependencies,
  File "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_pypy/pypy3/work/pypy3-v5.10.0-src/pypy/tool/build_cffi_imports.py", line 195, in create_cffi_import_libraries
    patches=patches)
  File "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_pypy/pypy3/work/pypy3-v5.10.0-src/pypy/tool/build_cffi_imports.py", line 89, in _build_dependency
    urlretrieve(url, archive)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 98, in urlretrieve
    return opener.retrieve(url, filename, reporthook, data)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 245, in retrieve
    fp = self.open(url, data)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 457, in open_https
    return self.http_error(url, fp, errcode, errmsg, headers)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 377, in http_error
    result = method(url, fp, errcode, errmsg, headers)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 641, in http_error_302
    data)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 667, in redirect_internal
    return self.open(newurl)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 457, in open_https
    return self.http_error(url, fp, errcode, errmsg, headers)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 377, in http_error
    result = method(url, fp, errcode, errmsg, headers)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 641, in http_error_302
    data)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 667, in redirect_internal
    return self.open(newurl)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 457, in open_https
    return self.http_error(url, fp, errcode, errmsg, headers)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 377, in http_error
    result = method(url, fp, errcode, errmsg, headers)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 641, in http_error_302
    data)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 667, in redirect_internal
    return self.open(newurl)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 213, in open
    return getattr(self, name)(url)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/urllib.py", line 443, in open_https
    h.endheaders(data)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/httplib.py", line 1049, in endheaders
    self._send_output(message_body)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/httplib.py", line 893, in _send_output
    self.send(msg)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/httplib.py", line 855, in send
    self.connect()
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/httplib.py", line 1274, in connect
    server_hostname=server_hostname)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/ssl.py", line 352, in wrap_socket
    _context=self)
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/ssl.py", line 577, in __init__
    self.do_handshake()
  File "/opt/local/lib/pypy-bootstrap/lib-python/2.7/ssl.py", line 806, in do_handshake
    self._sslobj.do_handshake()
IOError: [Errno socket error] [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure

Ports should download all needed files during the fetch phase, not later.

Though the xz distfile is the one that failed to download, and thus caused the port to fail to install, the port also downloads other files at build time that it should not:

fetching archive http://ftp.gnu.org/gnu/gdbm/gdbm-1.13.tar.gz
fetching archive http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.6.2.tar.gz
fetching archive https://tukaani.org/xz/xz-5.2.3.tar.gz

Change History (9)

comment:1 Changed 3 years ago by jmroot (Joshua Root)

Cc: jmroot added
Owner: changed from jmroot to danchr
Status: newassigned

This is a good demonstration that version updates are not the kind of minor changes that openmaintainer allows. Run them by the maintainer first please.

comment:2 in reply to:  1 Changed 3 years ago by danchr (Dan Villiom Podlaski Christiansen)

Replying to jmroot:

This is a good demonstration that version updates are not the kind of minor changes that openmaintainer allows. Run them by the maintainer first please.

Will do — I actually forgot that I wasn't the maintainer of PyPy, sorry! Ironically, I wrote the patch that caused the breakage, and did try to suppress it with the extra argument. Obviously, though, it didn't work… I'll look into it during the weekend!

Last edited 3 years ago by danchr (Dan Villiom Podlaski Christiansen) (previous) (diff)

comment:3 Changed 3 years ago by jmroot (Joshua Root)

If you want to take over as maintainer I'd be OK with that.

comment:4 in reply to:  3 Changed 3 years ago by danchr (Dan Villiom Podlaski Christiansen)

Replying to jmroot:

If you want to take over as maintainer I'd be OK with that.

Well, I finally got around to looking into this — so I suppose that's an argument against assuming maintainership…

I posted a pull request to fix this. The option to disable embedding was overwritten later in the file — and while at it, I bumped the port and addressed a warning. I did some brief testing of the binary, and basic imports work. In addition, the libraries appear to link to the correct locations:

% otool -L /opt/local/lib/pypy3/lib_pypy/**/*.so | sed 's,/opt/local,$prefix,g'
$prefix/lib/pypy3/lib_pypy/_audioop_cffi.pypy3-510-darwin.so:
        ./_audioop_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_curses_cffi.pypy3-510-darwin.so:
        ./_curses_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        $prefix/lib/libncurses.6.dylib (compatibility version 6.0.0, current version 6.0.0)
        $prefix/lib/libpanel.6.dylib (compatibility version 6.0.0, current version 6.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_curses_cffi_check.pypy3-510-darwin.so:
        ./_curses_cffi_check.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        $prefix/lib/libncurses.6.dylib (compatibility version 6.0.0, current version 6.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_decimal_cffi.pypy3-510-darwin.so:
        ./_decimal_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_gdbm_cffi.pypy3-510-darwin.so:
        ./_gdbm_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        $prefix/lib/libgdbm.5.dylib (compatibility version 6.0.0, current version 6.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_lzma_cffi.pypy3-510-darwin.so:
        ./_lzma_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        $prefix/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.3.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_pwdgrp_cffi.pypy3-510-darwin.so:
        ./_pwdgrp_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_pypy_openssl.pypy3-510-darwin.so:
        ./_pypy_openssl.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        $prefix/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
        $prefix/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_resource_cffi.pypy3-510-darwin.so:
        ./_resource_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_sqlite3_cffi.pypy3-510-darwin.so:
        ./_sqlite3_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        $prefix/lib/libsqlite3.0.dylib (compatibility version 9.0.0, current version 9.6.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
$prefix/lib/pypy3/lib_pypy/_syslog_cffi.pypy3-510-darwin.so:
        ./_syslog_cffi.pypy3-510-darwin.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
Last edited 3 years ago by danchr (Dan Villiom Podlaski Christiansen) (previous) (diff)

comment:5 Changed 2 years ago by danchr (Dan Villiom Podlaski Christiansen)

Resolution: fixed
Status: assignedclosed

In f45ad7cf932305679cac2fbfa332310ee8362adf/macports-ports (master):

pypy3: bump to 6.0.0, fix issues

  • address 'port lint' warning
  • properly suppress embedded dependencies

Closes: #55679

comment:6 Changed 2 years ago by danchr (Dan Villiom Podlaski Christiansen)

Resolution: fixed
Status: closedreopened

Oops, I didn't intend pushing that commit…

comment:7 Changed 2 years ago by danchr (Dan Villiom Podlaski Christiansen)

In c6234d57bcb5f66e56a36896a7ebe9d8fdf465a1/macports-ports (master):

pypy: oops, revert previous commit

I didn't intend to commit this.

#55679

comment:8 Changed 2 years ago by danchr (Dan Villiom Podlaski Christiansen)

Cc: ryandesign added
Status: reopenedassigned

Ryan commented the original PR; adding him as Cc for a forthcoming overhaul.

comment:9 Changed 2 years ago by danchr (Dan Villiom Podlaski Christiansen)

Resolution: fixed
Status: assignedclosed

In 0e70890ea90d14e0cb9274f7acf00bee05b3a82a/macports-ports (master):

pypy: bump to 6.0.0, fix issues

  • Bump to latest version, as mentioned.
  • Address issues identified by Ryan Schmidt in the PR referenced below, such as too much cleverness with replacements.
  • Properly suppress dependency embedding -- use ours instead.
  • Reorganise the port to avoid duplication wherever possible.
  • Add manual page.
  • Add some comments.
  • Use pypy2 for building pypy3.
  • Prevent tkinter from using the system-provided Tk.

Closes: #55679
Refs: https://github.com/macports/macports-ports/pull/1475

Note: See TracTickets for help on using tickets.