Opened 8 years ago

Last modified 6 years ago

#35474 new enhancement

RFE: Have curl-ca-bundle install individual .pem files in ${prefix}/etc/openssl/certs

Reported by: landonf (Landon Fuller) Owned by: ryandesign (Ryan Schmidt)
Priority: Normal Milestone: MacPorts Future
Component: ports Version:
Keywords: Cc: nonstop.server@…, larryv (Lawrence Velázquez), ryandesign (Ryan Schmidt), cooljeanius (Eric Gallager), neverpanic (Clemens Lang)
Port: curl-ca-bundle

Description

curl-ca-bundle currently installs a single cacerts.pem file. This makes it difficult to add new CA certificates to OpenSSL's default set, as cacerts.pem takes precedence over the ${prefix}/etc/openssl/certs directory.

Rather than install a single file, curl-ca-bundle could instead install individual certificates in ${prefix}/etc/openssl/certs. This would make it easy for users to add new certificates without modifying a port-installed cacerts file.

Debian/Ubuntu implement this mechanism in their ca-certificates package, along with some helpful scripts (update-ca-certificates) to generate the certs/ directory and keep it up-to-date: http://archive.ubuntu.com/ubuntu/pool/main/c/ca-certificates/ca-certificates_20111211.tar.gz

Note that Debian/Ubuntu use Mozilla's trust store instead of curl-ca-bundle; I'm not sure which is really better.

Attachments (2)

certsync.m (9.0 KB) - added by landonf (Landon Fuller) 7 years ago.
patch-add-certsync (7.2 KB) - added by landonf (Landon Fuller) 7 years ago.

Download all attachments as: .zip

Change History (29)

comment:1 Changed 8 years ago by landonf (Landon Fuller)

Cc: landonf@… added

comment:2 Changed 8 years ago by nonstop.server@…

Cc: nonstop.server@… added

Cc Me!

comment:3 Changed 8 years ago by jmroot (Joshua Root)

Cc: landonf@… removed
Port: curl-ca-bundle added
Version: 2.1.2

comment:4 Changed 8 years ago by ryandesign (Ryan Schmidt)

I haven't looked into this, but if you'd like to make this change, go ahead.

comment:5 Changed 7 years ago by landonf (Landon Fuller)

Owner: changed from ryandesign@… to landonf@…

Just a note, that we probably ought to also solve this problem for java, eg: http://archive.ubuntu.com/ubuntu/pool/main/c/ca-certificates-java/ca-certificates-java_20120721.tar.gz

A proper Mac OS X solution would actually be to synchronize the OpenSSL and Java keystore(s) from the system keychain (which would require writing some original software, and possibly slightly outside the purview of MacPorts). That said, this is somewhat necessary to make the keychain/openssl/java work nicely together, and would make sense for some level of integration into MacPorts.

comment:6 Changed 7 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… added

Cc Me!

comment:7 Changed 7 years ago by ryandesign (Ryan Schmidt)

Cc: ryandesign@… added

Cc Me!

comment:8 Changed 7 years ago by cooljeanius (Eric Gallager)

Cc: egall@… added

Cc Me!

comment:9 Changed 7 years ago by landonf (Landon Fuller)

I finally had time to look at this; it turned out to be pretty easy (well, the non-Java part, anyway). See the attached certsync.m; it exports anchor certificates from the system keychain. It should run on Mac OS X all the way back to 10.6 (and possibly earlier), but I only tested on 10.8.

To compile:

clang -mmacosx-version-min=10.6 certsync.m -o certsync -framework Foundation -framework Security -fobjc-arc

Example usage:

./certsync -o /opt/local/etc/openssl/cert.pem (this overwrites the curl-ca-bundle symlink)

This will write out all OS-included CA certs, as well as any CA certs that have been installed system-wide by the user -- this ought to make life a *lot* easier for anyone that works for a company that manages internal private CAs.

I'm open to suggestions on how best to integrate this into MacPorts; ideally it would be run automatically at activate-time, along with a user-executable script to update the certificates (or via cron, or via monitoring the keychain, or ...).

Since curl is configured with a non-standard certificate path, the above certsync command won't affect it. You can test with curl by overwriting the actual path curl is configured with:

./certsync -o /opt/local/etc/openssl/cert.pem (this overwrites the curl-ca-bundle symlink)

Short of patching curl to use SSL_CTX_set_default_verify_paths(), we should probably change the curl port to use:

--with-ca-bundle=${prefix}/etc/openssl/cert.pem
Last edited 7 years ago by landonf (Landon Fuller) (previous) (diff)

Changed 7 years ago by landonf (Landon Fuller)

Attachment: certsync.m added

comment:10 Changed 7 years ago by landonf (Landon Fuller)

Owner: changed from landonf@… to ryandesign@…

I just added an experimental certsync port; it should be a drop-in replacement for curl-ca-bundle:

sudo port -f deactivate curl-ca-bundle
sudo port install certsync

Bumping over to Ryan to get his input.

comment:11 Changed 7 years ago by landonf (Landon Fuller)

The port now also automatically updates the cert.pem when any of the keychain CAs are updated/changed/modified -- if you run port load certsync. As a side note, this is a rare case where I think a startup item should auto-enable.

comment:12 in reply to:  10 Changed 7 years ago by ryandesign (Ryan Schmidt)

Replying to landonf@…:

Bumping over to Ryan to get his input.

I don't have a particular opinion on this. I don't use custom certificate authorities so I haven't run into the problems this is designed to solve. If the new certsync port meets all the needs that curl-ca-bundle meets, and then some, I have no objection to retiring curl-ca-bundle. curl-ca-bundle is currently used by glib-networking, mercurial, osc and vantages in addition to curl, so we would want to make sure it works with all of those before retiring it.

comment:13 Changed 7 years ago by landonf (Landon Fuller)

The only thing that concerns me regarding throwing the switch is that the auto-updating only kicks in if the user runs 'port load'. I'm not sure how much of a problem that is?

Changed 7 years ago by landonf (Landon Fuller)

Attachment: patch-add-certsync added

comment:14 Changed 7 years ago by landonf (Landon Fuller)

Attached a patch for review that updates all the ports in question.

comment:15 Changed 7 years ago by ryandesign (Ryan Schmidt)

The portion of the patch that applies to certsync itself needn't increase the revision, since it does not affect any files the port installs; also, the deactivate hack has a copypasto; it still says kerberos5 on one line where it needs to say curl-ca-bundle.

In the name of a better user experience at the expense of some magic, we could have the certsync port automatically load itself in post-activate and unload in pre-deactivate. We could create the launchd plists manually, instead of using the startupitem commands, so that the user is not presented with any irrelevant instructions.

comment:16 in reply to:  15 ; Changed 7 years ago by neverpanic (Clemens Lang)

Replying to ryandesign@…:

In the name of a better user experience at the expense of some magic, we could have the certsync port automatically load itself in post-activate and unload in pre-deactivate.

I agree.

We could create the launchd plists manually, instead of using the startupitem commands, so that the user is not presented with any irrelevant instructions.

We could also extend the startupitem mechanism to support this out of the box.

comment:17 in reply to:  16 ; Changed 7 years ago by ryandesign (Ryan Schmidt)

Replying to cal@…:

We could also extend the startupitem mechanism to support this out of the box.

Yes; I was just trying to suggest a solution that would work immediately, without having to wait for a new release of MacPorts base.

comment:18 in reply to:  17 ; Changed 7 years ago by neverpanic (Clemens Lang)

Replying to ryandesign@…:

Yes; I was just trying to suggest a solution that would work immediately, without having to wait for a new release of MacPorts base.

We're in control of new base releases and we're not in a hurry with this change. I think we should rather do this right than quick.

comment:19 in reply to:  18 Changed 7 years ago by landonf (Landon Fuller)

Replying to cal@…:

We're in control of new base releases and we're not in a hurry with this change. I think we should rather do this right than quick.

Agreed on the lack of rush. I wouldn't mind seeing more testing, too.

How about 'startupitem.enable' as an option that defaults to no?

comment:20 Changed 7 years ago by neverpanic (Clemens Lang)

Cc: cal@… added

startupitem.autostart does this and has landed in r106810. Do we want to ship this in 2.2?

comment:21 in reply to:  20 Changed 7 years ago by landonf (Landon Fuller)

Replying to cal@…:

startupitem.autostart does this and has landed in r106810. Do we want to ship this in 2.2?

Awesome! An (obvious) +1 vote from me.

comment:22 Changed 7 years ago by neverpanic (Clemens Lang)

Milestone: MacPorts Future

Merged into release_2_2 in r109245.

comment:23 Changed 7 years ago by neverpanic (Clemens Lang)

Enabled startupitem.autostart, if available, for certsync in r109247.

comment:24 Changed 7 years ago by neverpanic (Clemens Lang)

Since the startupitem.autostart change shipped in 2.2.1, we could remove the forward compatibility hack committed in r109247. I think we might also consider this bug fixed. Opinions?

comment:25 Changed 7 years ago by ryandesign (Ryan Schmidt)

We would still want to replace curl-ca-bundle with certsync, but I'd like it to build on Tiger first (#41716).

comment:26 Changed 7 years ago by neverpanic (Clemens Lang)

Of course, but that doesn't affect this ticket, does it?

comment:27 in reply to:  10 Changed 6 years ago by dClauzel (Damien Clauzel)

Replying to landonf@…:

I just added an experimental certsync port; it should be a drop-in replacement for curl-ca-bundle:

sudo port -f deactivate curl-ca-bundle
sudo port install certsync

Bumping over to Ryan to get his input.

Tested with curl and wget. Works fine; I don't notice any bad side effects.

Note: See TracTickets for help on using tickets.