Distfile fetch for python39 hangs on 10.5

[fhgwright (Fred Wright)]

The distfile fetch for ​https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tar.xz hangs indefinitely on 10.5, both i386 and ppc. It does not hang under 10.4, so it's not a "too old" problem. It's successfully fetchable with command-line MacPorts curl, and with TenFourFox, so it's specific to the internal libcurl. Neither -v nor -d provides any useful additional information.

There's some evidence that this is a recent development. I haven't seen trouble with any other ports, though there haven't been any other updates that would fetch from python.org.

The workaround is to fetch the distfile manually, and place it in the proper directory.

[kencu (Ken)]

most people should pull the file from <​https://distfiles.macports.org/python39/>

[fhgwright (Fred Wright)]

But it tries the upstream source first, and if that hangs rather than failing, it never gets the chance to move on to try the mirror.

There are actually at least two bugs here - the hang, and the lack of a timeout.

[kencu (Ken)]

that is weird... does

sudo port -d fetch python39

show anything at all useful?

[ryandesign (Ryan Carsten Schmidt)]

Replying to fhgwright:

But it tries the upstream source first, and if that hangs rather than failing, it never gets the chance to move on to try the mirror.

MacPorts pings all available sites and tries them in an order that is supposed to result in the closest server being tried first. For you, the nearest server is apparently www.python.org.

You can add www.python.org to host_blacklist in macports.conf to tell MacPorts that you never want it to try to use that host.

There are actually at least two bugs here - the hang, and the lack of a timeout.

MacPorts has fetch timeouts. See ​https://github.com/macports/macports-base/blob/master/src/pextlib1.0/curl.c#L57-L60

[fhgwright (Fred Wright)]

Replying to kencu:

that is weird... does

sudo port -d fetch python39

show anything at all useful?

As I said above, "Neither -v nor -d provides any useful additional information."

Replying to ryandesign:

Replying to fhgwright:

But it tries the upstream source first, and if that hangs rather than failing, it never gets the chance to move on to try the mirror.

MacPorts pings all available sites and tries them in an order that is supposed to result in the closest server being tried first. For you, the nearest server is apparently www.python.org.

You can add www.python.org to host_blacklist in macports.conf to tell MacPorts that you never want it to try to use that host.

I didn't file the ticket to get a workaround; I filed the ticket to report a bug. :-)

There are actually at least two bugs here - the hang, and the lack of a timeout.

MacPorts has a fetch timeouts. See ​https://github.com/macports/macports-base/blob/master/src/pextlib1.0/curl.c#L57-L60

Empirically, the timeouts don't work in this case.

[kencu (Ken)]

I build macports-base against a newer libcurl, as I'm sure I explained to you before.

I have no problem at all.

macOS 10.5.8 9L31a
Xcode 3.1.4 9M2809

$ ls /opt/local/var/macports/distfiles/python39/Python-3.9.0.tar.xz
ls: /opt/local/var/macports/distfiles/python39/Python-3.9.0.tar.xz: No such file or directory

LeopardG5$ sudo port -v fetch python39
--->  Fetching distfiles for python39
--->  Python-3.9.0.tar.xz does not exist in /opt/local/var/macports/distfiles/python39
--->  Attempting to fetch Python-3.9.0.tar.xz from https://www.python.org/ftp/python/3.9.0/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 17.9M  100 17.9M    0     0  8336k      0  0:00:02  0:00:02 --:--:-- 8340k

[fhgwright (Fred Wright)]

Replying to kencu:

I build macports-base against a newer libcurl, as I'm sure I explained to you before.

I have no problem at all.

macOS 10.5.8 9L31a
Xcode 3.1.4 9M2809

$ ls /opt/local/var/macports/distfiles/python39/Python-3.9.0.tar.xz
ls: /opt/local/var/macports/distfiles/python39/Python-3.9.0.tar.xz: No such file or directory

LeopardG5$ sudo port -v fetch python39
--->  Fetching distfiles for python39
--->  Python-3.9.0.tar.xz does not exist in /opt/local/var/macports/distfiles/python39
--->  Attempting to fetch Python-3.9.0.tar.xz from https://www.python.org/ftp/python/3.9.0/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 17.9M  100 17.9M    0     0  8336k      0  0:00:02  0:00:02 --:--:-- 8340k

That could be a clue as to the problem, but it's not a solution for the masses.

There's also a good chance that the timeout is still broken, but happens not to be needed.

[kencu (Ken)]

At least we know it is in fact due to old curl/tls/https at least in part, as I see nothing wrong with base on 10.5 with current curl underpinnings. Why you don't see it on Tiger you say, I don't know.

I am uncertain why it is not first pulling from <​https://distfiles.macports.org>, even in my case?

MacPorts setup is designed to mirror the files there, where these older systems are supposed to go to get them. That's our MacPorts solution for the masses...

But there has always been something a bit weird about the python distfiles, I believe -- their https or TLS is somehow a bit different than others, I recall from previous tickets..

[fhgwright (Fred Wright)]

As Ryan explained above, it selects the source based on ping times, which may result in trying the primary source ahead of the mirrors. I'm not sure that behavior is desirable, but it's a separate issue.

I've just seen the same problem in fetches from cpan.org, so it's not just python.org.

[jmroot (Joshua Root)]

I currently get a fetch failure in an x86 Leopard VM, but not a hang:

$ sudo port -v fetch --no-mirror python39
--->  Fetching distfiles for python39
--->  Python-3.9.18.tar.xz does not exist in /opt/local/var/macports/distfiles/python39
--->  Attempting to fetch Python-3.9.18.tar.xz from https://www.python.org/ftp/python/3.9.18/

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
Error: Failed to fetch python39: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

[fhgwright (Fred Wright)]

Perhaps the bug causing the timeout to be ineffective no longer exists (due to some change in the past three years). Or perhaps there's another variable at play.