Opened 3 years ago

Last modified 3 years ago

#63563 new defect

base and buildbots: certificate issues

Reported by: mascguy (Christopher Nielsen) Owned by: admin@…
Priority: Normal Milestone:
Component: buildbot/mpbb Version:
Keywords: certificate ssl Cc: chrstphrchvz (Christopher Chavez)
Port:

Description (last modified by mascguy (Christopher Nielsen))

It looks like we're experiencing certificate issues, both on the buildbots, as well as locally. It's not clear whether this is related to the certificate authorities, our own certs, or something else.

On multiple buildbots, I'm seeing different variations of certificate validation errors. And this is occurring across multiple mirrors; both our own, as well as those from 3rd-parties:

https://build.macports.org/builders/ports-10.10_x86_64-builder/builds/156470/steps/install-port/logs/stdio

--->  Attempting to fetch rrdtool-1.7.2.tar.gz from https://distfiles.macports.org/rrdtool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
DEBUG: Fetching distfile failed: SSL certificate problem: Invalid certificate chain

https://build.macports.org/builders/ports-10.6_x86_64-builder/builds/74262/steps/install-port/logs/stdio

--->  Attempting to fetch rrdtool-1.7.2.tar.gz from http://oss.oetiker.ch/rrdtool/pub/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
DEBUG: Fetching distfile failed: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/124365/steps/install-port/logs/stdio

--->  Attempting to fetch rrdtool-1.7.2.tar.gz from http://oss.oetiker.ch/rrdtool/pub/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
DEBUG: Fetching distfile failed: SSL certificate problem: certificate has expired

These appear to be sporadic, as they aren't occurring across-the-board.

In addition, I'm also seeing certificate errors locally, though those are also sporadic.

Change History (8)

comment:1 Changed 3 years ago by mascguy (Christopher Nielsen)

Description: modified (diff)

comment:2 Changed 3 years ago by mascguy (Christopher Nielsen)

Component: basebuildbot/mpbb
Owner: set to admin@…

comment:3 Changed 3 years ago by chrstphrchvz (Christopher Chavez)

Possibly related to Let’s Encrypt expiration on 10.14 and earlier: https://lists.macports.org/pipermail/macports-users/2021-October/050298.html

comment:4 Changed 3 years ago by chrstphrchvz (Christopher Chavez)

Cc: chrstphrchvz added

comment:5 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)

Yes.

I've added the new Let's Encrypt root certificate to the buildbot machines (workers for 10.6-10.11 and the 10.11 machine doing distfile mirroring) to fix this. I forced new builds of rrdtool and the few other ports that had failed due to this.

I don't have a solution for 10.13 and 10.14 yet.

comment:6 Changed 3 years ago by mascguy (Christopher Nielsen)

It looks like this commit fixed the issue for MacPorts Base:

Commit d8986b2 - Limit OS versions that use Let's Encrypt sites

Thanks Ryan!

Version 0, edited 3 years ago by mascguy (Christopher Nielsen) (next)

comment:7 Changed 3 years ago by mascguy (Christopher Nielsen)

Hmmmm, I spoke too soon: Even with the aforementioned commit, SSL errors are still occurring locally. Ugh...

comment:8 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)

Priority: HighNormal

On 10.13 and 10.14, SSL errors will continue when attempting to access sites with Let's Encrypt certificates (possibly only those that are still configured to send the old expired root certificate, which is Let's Encrypt's default) using /usr/bin/curl or /usr/lib/libcurl.dylib. If anyone knows a solution, let us know. 10.12 and 10.15 and later should not be affected. 10.11 and earlier can be fixed by installing the new root certificate locally; see ProblemHotlist#letsencrypt.

Note: See TracTickets for help on using tickets.