port install: fetch shouldn't abort the whole command on a 503 error

[greyhare]

I have no idea why I'm getting 503 errors from packages.macports.org, but IMHO port shouldn't immediately die on 4xx or 5xx errors. It should instead try the next download location.

$ sudo port upgrade xterm
--->  Computing dependencies for xterm
--->  Fetching archive for xterm
--->  Attempting to fetch xterm-369_0.darwin_20.x86_64.tbz2 from https://packages.macports.org/xterm
--->  Attempting to fetch xterm-369_0.darwin_20.x86_64.tbz2.rmd160 from https://packages.macports.org/xterm
Error: Failed to archivefetch xterm: Failed to fetch signature for archive: The requested URL returned error: 503 
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_x11_xterm/xterm/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.

The archivefetch lines from main.log:

:msg:archivefetch --->  Fetching archive for xterm
:debug:archivefetch Executing org.macports.archivefetch (xterm)
:debug:archivefetch euid/egid changed to: 0/0
:debug:archivefetch chowned /opt/local/var/macports/incoming to macports
:debug:archivefetch euid/egid changed to: 502/501
:info:archivefetch --->  xterm-369_0.darwin_20.x86_64.tbz2 doesn't seem to exist in /opt/local/var/macports/incoming/verified
:msg:archivefetch --->  Attempting to fetch xterm-369_0.darwin_20.x86_64.tbz2 from https://packages.macports.org/xterm
:msg:archivefetch --->  Attempting to fetch xterm-369_0.darwin_20.x86_64.tbz2.rmd160 from https://packages.macports.org/xterm
:debug:archivefetch The requested URL returned error: 503 
:debug:archivefetch     while executing
:debug:archivefetch "curl fetch --effective-url effectiveURL {*}$fetch_options ${file_url}.rmd160 $signature"
:error:archivefetch Failed to archivefetch xterm: Failed to fetch signature for archive: The requested URL returned error: 503 
:debug:archivefetch Error code: NONE
:debug:archivefetch Backtrace: Failed to fetch signature for archive: The requested URL returned error: 503 
:debug:archivefetch     while executing
:debug:archivefetch "portarchivefetch::fetchfiles"
:debug:archivefetch     (procedure "portarchivefetch::archivefetch_main" line 5)
:debug:archivefetch     invoked from within
:debug:archivefetch "$procedure $targetname"
:error:archivefetch See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_x11_xterm/xterm/main.log for details.

[ryandesign (Ryan Carsten Schmidt)]

Replying to greyhare:

I have no idea why I'm getting 503 errors from packages.macports.org, but IMHO port shouldn't immediately die on 4xx or 5xx errors. It should instead try the next download location.

It does.

Here, the problem seems to be that it successfully got the archive but then encountered a 503 error trying to download the archive's rmd160 signature. MacPorts base code did not anticipate that the fetch of one file from a server would succeed and an immediately following fetch of another file from the same server would fail.

I do not encounter this problem when I try it now. Can you try again? If it still fails, does this happen for all ports or just this one? Has it ever worked? Do you use a proxy server on your network? Do you have any software (antivirus, etc.) installed that inserts itself into your network stack that might be interfering?

[greyhare]

Whatever was going on server side is cleared now. (Maybe I caught it recomputing checksums? Or checksum files aren't really there, but are computed, cached, and served by a script, which broke? Or it was a temporary misconfiguration?) But it was occurring for a number of packages I was trying to upgrade with sudo port upgrade outdated.

In any case, maybe have *any* fetch error bail out to the next download site?

[ryandesign (Ryan Carsten Schmidt)]

Replying to greyhare:

Whatever was going on server side is cleared now. (Maybe I caught it recomputing checksums? Or checksum files aren't really there, but are computed, cached, and served by a script, which broke?

No. The .rmd160 files are generated right after the .tbz2 archives arrive on the buildmaster. The .rmd160 files exist on the filesystem right next to the .tbz2 files and all of the files are synchronized via rsync across our various mirror servers.

Or it was a temporary misconfiguration?)

The public packages/distfiles servers are all managed by other entities, so I cannot say what configuration changes might have been happening on those servers.

But it was occurring for a number of packages I was trying to upgrade with sudo port upgrade outdated.

In any case, maybe have *any* fetch error bail out to the next download site?

The code in question is here:

​https://github.com/macports/macports-base/blob/master/src/package1.0/portarchivefetch.tcl#L259-L289

Anyone may submit pull requests to improve the code.