Opened 5 weeks ago

Last modified 4 weeks ago

#63782 new defect

python38 @3.8.12 has a problem when using venv: ImportError: dynamic module does not define module export function (PyInit__posixsubprocess)

Reported by: casr (Chris Rawnsley) Owned by:
Priority: Normal Milestone:
Component: ports Version:
Keywords: monterey arm64 Cc: jmroot (Joshua Root)
Port: python38

Description (last modified by ryandesign (Ryan Schmidt))

I have built python38 with the +universal variant, however, when I use the arm64 version it fails to create a virtualenv directory and complains with:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 64, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/venv/__init__.py", line 10, in <module>
    import subprocess
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 69, in <module>
    import _posixsubprocess
ImportError: dynamic module does not define module export function (PyInit__posixsubprocess)

I am running macOS 12.0.1 21A559 arm64, Xcode 13.1 13A1030d.

To reproduce:

# works
( cd $(mktemp -d) && arch -x86_64 python3.8 -m venv venv )

# fails
( cd $(mktemp -d) && arch -arm64 python3.8 -m venv venv )

# fails
( cd $(mktemp -d) && python3.8 -m venv venv )

Seems similar to the problem described in #61282

Change History (6)

comment:1 Changed 5 weeks ago by ryandesign (Ryan Schmidt)

Cc: jmroot added
Description: modified (diff)
Keywords: monterey arm64 added
Port: python38 added

comment:2 Changed 5 weeks ago by jmroot (Joshua Root)

Summary: python38 @3.8.12 has a problem when using venv: ModuleNotFoundError: No module named 'msvcrt'python38 @3.8.12 has a problem when using venv: ImportError: dynamic module does not define module export function (PyInit__posixsubprocess)

comment:3 Changed 5 weeks ago by casr (Chris Rawnsley)

I noticed something peculiar with my MacPorts installation in that port was using a tclsh that was running an Intel binary. I've since reinstalled MacPorts making sure my terminal wasn't running in Rosetta-mode, retried the command and the virtual environment is created as expected.

I think this can be moved to fixed if you're happy to do so. Happy to provide any follow up in case it is useful too.

comment:4 Changed 4 weeks ago by casr (Chris Rawnsley)

Actually, I just tried the opposite way around and the same error occurred. In other words, this time around MacPorts is installed under arm64 and the x86_64 version is busted.

$ arch -x86_64 python3.8 -m venv venv
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 64, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/venv/__init__.py", line 10, in <module>
    import subprocess
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 69, in <module>
    import _posixsubprocess
ImportError: dynamic module does not define module export function (PyInit__posixsubprocess)

comment:5 Changed 4 weeks ago by jmroot (Joshua Root)

What kind of a binary is _posixsubprocess.cpython-38-darwin.so?

comment:6 Changed 4 weeks ago by casr (Chris Rawnsley)

It seems to be in order...

$ port contents python38 | grep -i posixsub
  /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/_posixsubprocess.cpython-38-darwin.so

$ file /opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/_posixsubprocess.cpython-38-darwin.so
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/_posixsubprocess.cpython-38-darwin.so: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64] [arm64:Mach-O 64-bit bundle arm64]
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/_posixsubprocess.cpython-38-darwin.so (for architecture x86_64):	Mach-O 64-bit bundle x86_64
/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/lib-dynload/_posixsubprocess.cpython-38-darwin.so (for architecture arm64):	Mach-O 64-bit bundle arm64
Note: See TracTickets for help on using tickets.