Pythons 3.x for ppc+ppc64 on Leopard: No module named 'msvcrt', '_posixsubprocess' errors, how to fix?

[barracuda156]

I am back to efforts to fix Pythons 3.x for Leopard as ppc+ppc64. I have built now python38 and python39 as +universal:

  python38 @3.8.13_0+universal (active) requested_variants='+universal' platform='darwin 9' archs='ppc ppc64' date='2022-04-01T01:06:28+0800'
  python39 @3.9.12_0+universal (active) requested_variants='+universal' platform='darwin 9' archs='ppc ppc64' date='2022-04-01T00:21:32+0800'

However installing modules fail. As an example:

--->  Building py38-setuptools
Executing:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py38-setuptools/work/setuptools-60.9.3" && /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -m build --wheel --no-isolation --outdir /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py38-setuptools/work --skip-dependency-check 
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 64, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/opt/local/share/py-bootstrap-modules/build/__init__.py", line 13, in <module>
    import subprocess
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 69, in <module>
    import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py38-setuptools/work/setuptools-60.9.3" && /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -m build --wheel --no-isolation --outdir /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-setuptools/py38-setuptools/work --skip-dependency-check

It is unclear why this happens. Any advice will be greatly appreciated.

16. S. With python27 universal is working with no issues:

svacchanda$ port -v installed py27-curl
The following ports are currently installed:
  py27-curl @7.43.0.3_2+universal (active) requested_variants='+universal' platform='darwin 9' archs='ppc ppc64' date='2022-04-01T01:24:12+0800'

[barracuda156]

Patch I made to fix ppc+ppc64 build:

[barracuda156]

In addition to the patch, this was changed in portfile:

        foreach arch ${configure.universal_archs} {
            lappend universal_arch_flags -arch ${arch}
            if {${arch} in {i386}} {
                lappend arch_run_32bit -${arch}
                lappend lipo_32bit_flags -extract ${arch}
            } elseif {${arch} in {ppc}} {
                lappend arch_run_32bit -${arch}
                lappend lipo_32bit_flags -extract ppc7400
            } else {
                set any64 yes
            }
        }

Because -extract ppc is a wrong flag and it doesn't work.

[barracuda156]

UPD. I have also tried now to use approach from python27, changing relevant section in python39 portfile to:

variant universal {
	patchfiles-append  patch-universal-powerpc.diff
    post-patch {
        reinplace \
            "s|__UNIVERSAL_ARCHFLAGS__|${configure.universal_cflags}|" \
            ${worksrcpath}/configure
    }
    if {${configure.sdkroot} ne ""} {
        configure.args-append   --enable-universalsdk=${configure.sdkroot}
    } else {
        configure.args-append   --enable-universalsdk=/
    }
    post-configure {
        set oldmtime [file mtime ${worksrcpath}/pyconfig.h]
        system -W ${worksrcpath} "ed - pyconfig.h < ${filespath}/pyconfig.h-universal.ed"
        file mtime ${worksrcpath}/pyconfig.h $oldmtime
    }

    post-destroot {
        foreach unversioned {python3-32 python3-intel64} {
            delete ${destroot}${prefix}/bin/${unversioned}
        }
    }
}

And then modifying patches; build itself again was successful, but trying to install any modules again fails:

--->  Building py39-curl
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 73, in <module>
    import msvcrt
ModuleNotFoundError: No module named 'msvcrt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-curl/py39-curl/work/pycurl-7.44.1/setup.py", line 11, in <module>
    import glob, os, re, sys, subprocess
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 78, in <module>
    import _posixsubprocess
ModuleNotFoundError: No module named '_posixsubprocess'
Command failed: /opt/local/Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 setup.py docstrings
Exit code: 1
Error: Failed to build py39-curl: command execution failed

[barracuda156]

Patch based on python27 one. (I also removed the last part of patch-configure.diff to match python27 patch.)

[barracuda156]

Opened an issue on Python Bugtracker: ​https://bugs.python.org/issue47240

[barracuda156]

Replying to mascguy:

So interestingly, all modules apparently get built correctly, if I just run sudo port -v build, and get placed in: /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.4/build/lib.macosx-10.5-powerpc-3.10. They have both archs:

/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.4/build/lib.macosx-10.5-powerpc-3.10/_posixsubprocess.cpython-310-darwin.so: Mach-O universal binary with 2 architectures
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.4/build/lib.macosx-10.5-powerpc-3.10/_posixsubprocess.cpython-310-darwin.so (for architecture ppc7400):	Mach-O bundle ppc
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.4/build/lib.macosx-10.5-powerpc-3.10/_posixsubprocess.cpython-310-darwin.so (for architecture ppc64):	Mach-O 64-bit bundle ppc64

However they get screwed at destroot, turning into "failed", i.e.: _posixsubprocess.cpython-310-darwin_failed.so. These failed ones get installed into framework, and surely fail to work. Failed ones only have ppc.

What's happening here? I had tried also using a patch identical to the one used for python27, which works for it, but here I get the same result. Also strangely, build does not fail, that's why these error went unnoticed for a while. And Python's binaries build normally, only these modules failing.

[barracuda156]

And with existing universal flags it builds in two separate folders, lib.macosx-10.5-powerpc-3.10 and lib.macosx-10.5-ppc64-3.10, second one contains broken files. And those broken get installed:

36-163:~ svacchanda$ file /opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload/_posixsubprocess.cpython-310-darwin_failed.so /opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload/_posixsubprocess.cpython-310-darwin_failed.so: Mach-O bundle ppc