python24, python25, python26, python27, python31, python32, python33, python34, python35: Fix building for strange architecture combinations

[land.apfel@…]

When trying to build a version of Python for a set of architectures not listed in ​https://hg.python.org/cpython/file/2.7/Lib/_osx_support.py#l471, the build fails with

  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.11/Lib/_osx_support.py", line 485, in get_platform_osx
    "Don't know machine value for archs=%r" % (archs,))
ValueError: Don't know machine value for archs=('ppc', 'ppc64')
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1

Upstream they appear to have good reasons to not extend the list of "machine values" (​http://bugs.python.org/issue26487).

I think that MacPorts should either restrict builds to architecture sets supported upstream or add a patch that generates machine values for all combinations (similar to ​http://bugs.python.org/file42073/patch-plat-ppc-fat.diff).

PS: This problem probably affects all Python versions (not only python27).

[ryandesign (Ryan Carsten Schmidt)]

Or use the muniversal portgroup.

[larryv (Lawrence Velázquez)]

Cc Me!

[MaddTheSane (C.W. Betts)]

The patch will probably work with a large portion of those with "strange architecture combinations:" PowerPC users who don't want to build Intel code.

[barracuda156]

And no fix till now?

I keep getting this on 10.5.8:

ValueError: Don't know machine value for archs=('ppc', 'ppc64')
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18" && /usr/bin/make -j4 -w all 
Exit code: 2
Error: Failed to build python27: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/main.log for details.

[ryandesign (Ryan Carsten Schmidt)]

Things only get fixed in MacPorts when someone who cares about an issue fixes it. Few people care about PowerPC systems these days. Since you do, you could try either of the two possible fixes suggested in this ticket (patchfile or muniversal portgroup). If you find one that works, you can report it here, or better yet, submit a pull request.

[barracuda156]

Replying to ryandesign:

Things only get fixed in MacPorts when someone who cares about an issue fixes it. Few people care about PowerPC systems these days. Since you do, you could try either of the two possible fixes suggested in this ticket (patchfile or muniversal portgroup). If you find one that works, you can report it here, or better yet, submit a pull request.

Thank you! I [wrongly] assumed that since the issue remained open, proposed solution didn't work.

I made the patch for python38 and Python built, however I got another error here:

cd Mac && /usr/bin/make pythonw
make[1]: Entering directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12/Mac'
/usr/bin/gcc-4.2 -arch ppc -arch ppc64 -L/opt/local/lib -Wl,-headerpad_max_install_names -arch ppc -arch ppc64 -DPYTHONFRAMEWORK='"Python"' -o pythonw \
		./Tools/pythonw.c -I.. -I./../Include \
		../Python.framework/Versions/3.8/Python
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12/Mac'
Creating directory /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin
if test "Python.framework" = "no-framework" ; then \
		/usr/bin/install -c python.exe /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
	else \
		/usr/bin/install -c -s Mac/pythonw /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
	fi
if test "3.8" != "3.8"; then \
		if test -f /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -o -h /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
		then rm -f /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
		fi; \
		(cd /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin; ln python3.8 python3.8); \
	fi
if test -f Python.framework/Versions/3.8/Python && test "Python.framework" = "no-framework" ; then \
		if test -n "" ; then \
			/usr/bin/install -c -m 755  /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin; \
		else \
			/usr/bin/install -c -m 755 Python.framework/Versions/3.8/Python /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/Python.framework/Versions/3.8/Python; \
			if test Python.framework/Versions/3.8/Python != Python.framework/Versions/3.8/Python; then \
				(cd /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib; ln -sf Python.framework/Versions/3.8/Python Python.framework/Versions/3.8/Python) \
			fi \
		fi; \
		if test -n ""; then \
			/usr/bin/install -c -m 755  /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/; \
		fi; \
	else	true; \
	fi
if test "x-extract ppc" != "x" ; then \
		rm -f /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/binpython3.8-32; \
		lipo -extract ppc \
			-output /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8-32 \
			/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
	fi
fatal error: lipo: -extract ppc specified but fat file: /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 does not contain that architecture
make: *** [altbininstall] Error 1
make: Leaving directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12'
Command failed:  cd "/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12" && /usr/bin/make -w frameworkinstall maninstall DESTDIR=/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot 
Exit code: 2
Error: Failed to destroot python38: command execution failed
Error: See /opt/local/var/macports/logs/_opt_PPCLeopardPorts_lang_python38/python38/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
Error: Processing of port python38 failed

I have checked the binary, and both architectures are present in fact:

36-225:~ svacchanda$ file /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: Mach-O universal binary with 2 architectures
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 (for architecture ppc7400):	Mach-O executable ppc
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 (for architecture ppc64):	Mach-O 64-bit executable ppc64

Any idea what should I fix now?

[kencu (Ken)]

the toolchain sometimes does not see ppc and ppc7400 as the same arch.

I needed to fix that in legacysupport a few years ago.

#60147

[barracuda156]

Replying to ryandesign:

Things only get fixed in MacPorts when someone who cares about an issue fixes it. Few people care about PowerPC systems these days. Since you do, you could try either of the two possible fixes suggested in this ticket (patchfile or muniversal portgroup). If you find one that works, you can report it here, or better yet, submit a pull request.

Okay, I have made it:

36-225:Python-3.8.12 svacchanda$ port -v installed python38
The following ports are currently installed:
  python38 @3.8.12_4+universal (active) requested_variants='+universal' platform='darwin 9' archs='ppc ppc64' date='2021-12-24T02:01:57+0800'
36-225:Python-3.8.12 svacchanda$

Could anyone check my patches? How should I submit those for review?

I will try now to fix python27.

[barracuda156]

16. S. I noticed this though with Python 3.8:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

[barracuda156]

Replying to barracuda156:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

It appears that while python27 is fixed and works, none of python3* works correctly: they all build with patches applied, but aren't usable due to some module errors.

If anyone got at idea how to fix that, please suggest me.

[barracuda156]

Replying to kencu:

the toolchain sometimes does not see ppc and ppc7400 as the same arch.

I needed to fix that in legacysupport a few years ago.

#60147

I tried to use portgroup legacysupport 1.0 now for Python310 (and discarding my patches), however I got these errors:

/usr/bin/gcc-4.2 -c -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch ppc -arch ppc64 -pipe -Os -I/opt/local/include/LegacySupport -arch ppc -arch ppc64   -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include -I/opt/local/include -I/opt/local/include/LegacySupport   -DPy_BUILD_CORE -o Python/structmember.o Python/structmember.c
In file included from /opt/local/include/LegacySupport/sys/random.h:25,
                 from Python/bootstrap_hash.c:17:
/usr/include/sys/random.h:37: error: expected declaration specifiers or ‘...’ before ‘u_int’
/usr/bin/gcc-4.2 -c -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch ppc -arch ppc64 -pipe -Os -I/opt/local/include/LegacySupport -arch ppc -arch ppc64   -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include -I/opt/local/include -I/opt/local/include/LegacySupport   -DPy_BUILD_CORE -o Python/symtable.o Python/symtable.c
In file included from /opt/local/include/LegacySupport/sys/random.h:25,
                 from Python/bootstrap_hash.c:17:
/usr/include/sys/random.h:37: error: expected declaration specifiers or ‘...’ before ‘u_int’
/usr/bin/gcc-4.2 -c -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch ppc -arch ppc64 -pipe -Os -I/opt/local/include/LegacySupport -arch ppc -arch ppc64   -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include -I/opt/local/include -I/opt/local/include/LegacySupport   -DPy_BUILD_CORE \
		-DABIFLAGS='""' \
		-DMULTIARCH=\"darwin\" \
		-o Python/sysmodule.o ./Python/sysmodule.c
fatal error: lipo: can't open input file: /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/.tmp/ccimd9EN.out (No such file or directory)
make: *** [Python/bootstrap_hash.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.1'
Command failed:  cd "/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.1" && /usr/bin/make -j4 -w all 
Exit code: 2
Error: Failed to build python310: command execution failed
Error: See /opt/local/var/macports/logs/_opt_PPCLeopardPorts_lang_python310/python310/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
Error: Processing of port python310 failed

This is on 10.5.8 PPC trying to build functional python3* as universal.