Opened 4 months ago
Last modified 4 months ago
#73908 new defect
CI preferred hosts
| Reported by: | ryandesign (Ryan Carsten Schmidt) | Owned by: | admin@… |
|---|---|---|---|
| Priority: | Normal | Milestone: | |
| Component: | buildbot/mpbb | Version: | |
| Keywords: | Cc: | ||
| Port: |
Description
Last year these lines were added to the CI bootstrap.sh:
# Prefer hosts close to github echo "preferred_hosts mirror.fcix.net github.com *.github.com" | sudo tee -a /opt/local/etc/macports/macports.conf >/dev/null
I disagree with preferring the mirror.fcix.net mirror. It directly contradicts the intention of the preceding line, whose purpose was to ensure that we only hit distfiles.macports.org and packages.macports.org and not any of the mirrors:
# Only download from the CDN, not the mirrors echo "host_blacklist *.distfiles.macports.org *.packages.macports.org" | sudo tee -a /opt/local/etc/macports/macports.conf >/dev/null
The reasoning is that our primary URLs will have newly-mirrored distfiles and newly-built packages before the mirrors have had a change to sync them. If our primary URLs don't have the files, there's no point spending time checking any of the mirrors. Our primary URLs are on the Fastly CDN so they should be plenty fast.
I don't object to preferring github.com hosts. GitHub also uses the Fastly CDN so preferring github.com probably doesn't help very much, unless GitHub Actions uses custom hostname resolution that points github.com's file serving hostnames to a local server (which is conceivable, since it's what I do on the buildbot system).

{distfiles,packages}.macports.org are often unreasonably slow, as we've discussed before. If that were fixed, this could go away. Most of the time, the files that CI downloads from our mirrors are not newly added, so the hit rate is similar for mirror.fcix.net and the primary URLs. Anything truly new in a PR will be a miss on both of course, but trying two URLs isn't too bad and is still much better than trying most of the mirrors, as would happen with a default config.