cmake 3.9.0: upgrade from 3.8.2_0 fails in make with urllib3 distribution not found (eventually)

[sambthompson (Sam Thompson)]

upgrading cmake port as part of upgrade outdated invocation, get error during build from source.

Tail of output from sudo port -v upgrade cmake is:

.
.
.
[ 11%] Built target cmsys
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.9.0'
make: *** [all] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.9.0'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/work/cmake-3.9.0" && /usr/bin/make -j8 -w all VERBOSE=ON 
Exit code: 2
Error: Failed to build cmake: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_cmake/cmake/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.

[Also attached main.log]

I can't see a message prior to the Exit code: 2 which explains why make -j8 -w all VERBOSE=ON fails.

Tried port clean, same result. Nothing stands out in the problem hotlist. No open tickets for cmake match these symptoms.

port uninstall and port install also fails. Now cmake is not installed!

Re-running the install command without cleaning allows it to proceed to 73% before it fails again, again with no clear message. The next time it gets to 94%, same results. Then 98%, then 100%. 6th and later invocations yield message:

pkg_resources.DistributionNotFound: The 'urllib3<1.22,>=1.21.1' distribution was not found and is required by requests

However,

% sudo port installed | grep urllib
   py27-urllib3 @1.22_0 (active)

Repeating the port install gets no further; message is the same.

This is on El-Capitan with MacPorts 2.4.1. Has been working OK, this was routine maintenance.

% uname -a
Darwin host.local 15.6.0 Darwin Kernel Version 15.6.0: Tue Apr 11 16:00:51 PDT 2017; root:xnu-3248.60.11.5.3~1/RELEASE_X86_64 x86_64

% xcodebuild -version
Xcode 8.2.1
Build version 8C1002

[michaelld (Michael Dickens)]

I see the same issue. Guessing this is because of a recent update to urllib3 ​https://github.com/macports/macports-ports/commit/ee9baca9dc101d99585dec12e65e51ffa91be897 . I'm looking into it.

[michaelld (Michael Dickens)]

Building "cmake -docs" works for me.

The issue is with Sphinx, when building documentation (try executing "sphinx-build-2.7 --help" to see the same error as generated when trying to build cmake). Sphinx uses py27-requests, which in turn uses urllib3. Thus, the actual issue is with py27-requests. It's init.py script does the check for urllib3, but I think there's an error in the code.

[michaelld (Michael Dickens)]

OK, so the actual issue is that py*-requests 2.18.1_0 has a "required" script that specifies "urllib3<1.22,>=1.21.1". There is a new version of requests out (2.18.2), which updates this to "urllib3<1.23,>=1.21.1". Thus, the issue is really that py*-requests just needs to be updated to the most recent version. Simple fix, really.

[michaelld (Michael Dickens)]

In 83b7e9c9f69e8cc56cda33bab5f856a64cccdb5a/macports-ports:

py-requests: update to 2.18.2.

Resolves requests runtime failure due to urllib3 update in ee9baca9.
Addresses #54531 .

[sambthompson (Sam Thompson)]

Thanks for the quick fix; I was able to selfupdate, upgrade py27-requests and then re-install cmake successfully, on the first attempt. However, isn't there still a worry about the lack of error messages in the first 5 install attempts? It took a fair bit of messing about just to get the error message about urllib3.

[michaelld (Michael Dickens)]

You're welcome & I'm glad the fix worked (for you, me, and others)!

I agree that the error message could be more obvious. The CMake build script should be able to figure out if Sphinx errored out & print something useful. Ditto for Py-Requests. Unfortunately, neither does -- and, that's not something we fix here in MacPorts-land; not generally at least.

I'd advise that if you want to see a useful error message, you contact the upstream projects (CMake, Sphinx, PyRequest) and request such changes. Point the developers to this ticket. I'll be happy to chime in. That said, don't hold your breath for quick or substantial changes -- developers generally have tons of "more important" fixes to implement or code to write. Can't hurt to try though!

[sambthompson (Sam Thompson)]

OK. I'd want to repro the issue outside of macports before going upstream, I'd probably start with reporting to the cmake project. Not sure when I can get time for the setup needed; thanks to macports, I normally never have to worry about the vagaries of separate tool installs and version requirements...