python27 @2.7.14: urlopen() with HTTPS fails with CERTIFICATE_VERIFY_FAILED

[artmatsak (Art Matsak)]

macOS High Sierra 10.13.3. Running the simple Python script below fails with the CERTIFICATE_VERIFY_FAILED error. It doesn't matter what URL is used as long as it's an HTTPS one.

from urllib2 import urlopen
urlopen('https://www.google.com/').read()

[ryandesign (Ryan Carsten Schmidt)]

Works for me.

[artmatsak (Art Matsak)]

Hm, are there any checks I can run to get to the bottom of this? I suspect that the CA certificates are not installed or accessible to python27. Apple's Python is OK.

[jmroot (Joshua Root)]

Do you have /opt/local/etc/openssl/cert.pem? If not you can install certsync (to use the system certs) or curl-ca-bundle.

[artmatsak (Art Matsak)]

/opt/local/etc/openssl/cert.pem is there. Could it be that Python is looking for the CA certificates in wrong locations?

[jmroot (Joshua Root)]

There may be some environment variables or other configuration that affect where the ssl module will load CA certificates from. You can check if it's finding any certs with something like this:

import ssl
s = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
s.load_default_certs()
s.cert_store_stats()

For me, the cert_store_stats() call returns:

{'x509': 133, 'x509_ca': 133, 'crl': 0}

[neverpanic (Clemens Lang)]

See also ​https://stackoverflow.com/questions/49029035/urllib2-in-macports-python-2-7-14-fails-with-certificate-verify-failed, so it's not an OpenSSL problem.

[artmatsak (Art Matsak)]

I desperately needed my Ansible scripts to work so I went ahead and installed Python 2.7.14 from ​https://www.python.org/downloads/mac-osx/. This somehow fixed the certificate problem with the MacPorts Python, too, even though I have now removed the python.org stuff from my machine. So everything is now working, although I still don't understand what the original issue was.

[jmroot (Joshua Root)]

I don't know what the problem could have been either, unless you were somehow not actually running the python from MacPorts to begin with.