py-cryptography@ 2.9.2_2 dlopen() symbol not found: _EVP_PKEY_size

[khepler]

I've rebuilt it and dependencies several times. Error happens with both py38-cryptography and py39-cryptography.

This manifests for me as a linking error when attempting to run certbot.

  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 16, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so, 2): Symbol not found: _EVP_PKEY_size
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so
  Expected in: dynamic lookup

I don't know if this is informative, but the _EVP_PKEY_size symbol appears in the library's indirect symbol table:

% otool -vI /opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so | grep _EVP_PKEY_size
0x00069794 12880 _EVP_PKEY_size
0x0007b500 12880 _EVP_PKEY_size

However, it does not appear in the any of its other tables, which are empty:

% otool -vS /opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so
% otool -vT /opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so
/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so:
Table of contents (0 entries)
module name      symbol name
% otool -vM /opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so
/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so:
Module table (0 entries)
% otool -vR /opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so
/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so:
Reference table (0 entries)
% otool -vH /opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so
/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so:
Two-level namespace hints table (0 hints)
index  isub  itoc
%

[khepler]

Full traceback from executing certbot

[neirbowj (John W. O'Brien)]

On bigsur arm64 and with 2.7.1, I observed a substantially similar traceback in py-cryptography (35.0.0_3) after diagnosing a problem affecting the ansible-vault utility from py-ansible.

$ python -c "from cryptography.hazmat.backends import openssl"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/__init__.py", line 6, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 82, in <module>
    from cryptography.hazmat.bindings.openssl import binding
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so, 2): Symbol not found: _ERR_put_error
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so
  Expected in: flat namespace
 in /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so

What I have managed to determine is that the proximate cause appears to be confusion between the version of OpenSSL py-cryptography was built against and which version it is being dynamically linked against. I came up with a short shell script to demonstrate the problem, which, on my system, affects 3.8, but not 3.7 nor 3.9.

# script
TESTCODE="from cryptography.hazmat.backends import openssl"
for i in 7 8 9; do
    echo "==> 3.${i}"
    port echo depof:py3${i}-cryptography | grep openssl
    SITEPKG=$(python3.${i} -c "import sys;print(sys.path[-1])")
    otool -L ${SITEPKG}/cryptography/hazmat/bindings/_openssl.abi3.so \
        | grep -oE "openssl[0-9]+"
    if python3.${i} -c "${TESTCODE}" 2> /dev/null; then
        echo succeeds
    else
        echo fails
    fi
done

# output
==> 3.7
openssl11
openssl11
openssl11
succeeds
==> 3.8
openssl11
openssl3
openssl3
fails
==> 3.9
openssl3
openssl3
openssl3
succeeds