Opened 8 years ago

Last modified 2 years ago

#50821 new defect

python24, python25, python26, python27, python31, python32, python33, python34, python35: Fix building for strange architecture combinations

Reported by: land.apfel@… Owned by: jyrkiwahlstedt
Priority: Normal Milestone:
Component: ports Version: 2.3.4
Keywords: Cc: larryv (Lawrence Velázquez), MaddTheSane (C.W. Betts), barracuda156
Port: python24, python25, python26, python27, python31, python32, python33, python34, python35

Description

When trying to build a version of Python for a set of architectures not listed in https://hg.python.org/cpython/file/2.7/Lib/_osx_support.py#l471, the build fails with

  File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_lang_python27/python27/work/Python-2.7.11/Lib/_osx_support.py", line 485, in get_platform_osx
    "Don't know machine value for archs=%r" % (archs,))
ValueError: Don't know machine value for archs=('ppc', 'ppc64')
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1

Upstream they appear to have good reasons to not extend the list of "machine values" (http://bugs.python.org/issue26487).

I think that MacPorts should either restrict builds to architecture sets supported upstream or add a patch that generates machine values for all combinations (similar to http://bugs.python.org/file42073/patch-plat-ppc-fat.diff).

PS: This problem probably affects all Python versions (not only python27).

Change History (12)

comment:1 Changed 8 years ago by ryandesign (Ryan Carsten Schmidt)

Owner: changed from macports-tickets@… to jwa@…
Port: python24 python25 python26 python31 python32 python33 python34 python35 added
Summary: Python: Fix building for strange architecture combinationspython24, python25, python26, python27, python31, python32, python33, python34, python35: Fix building for strange architecture combinations

Or use the muniversal portgroup.

comment:2 Changed 8 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… added

Cc Me!

comment:3 Changed 5 years ago by MaddTheSane (C.W. Betts)

Cc: MaddTheSane added

comment:4 Changed 5 years ago by MaddTheSane (C.W. Betts)

The patch will probably work with a large portion of those with "strange architecture combinations:" PowerPC users who don't want to build Intel code.

comment:5 Changed 2 years ago by barracuda156

And no fix till now?

I keep getting this on 10.5.8:

ValueError: Don't know machine value for archs=('ppc', 'ppc64')
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/work/Python-2.7.18" && /usr/bin/make -j4 -w all 
Exit code: 2
Error: Failed to build python27: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_python27/python27/main.log for details.

comment:6 Changed 2 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: barracuda156 added

Things only get fixed in MacPorts when someone who cares about an issue fixes it. Few people care about PowerPC systems these days. Since you do, you could try either of the two possible fixes suggested in this ticket (patchfile or muniversal portgroup). If you find one that works, you can report it here, or better yet, submit a pull request.

comment:7 in reply to:  6 Changed 2 years ago by barracuda156

Replying to ryandesign:

Things only get fixed in MacPorts when someone who cares about an issue fixes it. Few people care about PowerPC systems these days. Since you do, you could try either of the two possible fixes suggested in this ticket (patchfile or muniversal portgroup). If you find one that works, you can report it here, or better yet, submit a pull request.

Thank you! I [wrongly] assumed that since the issue remained open, proposed solution didn't work.

I made the patch for python38 and Python built, however I got another error here:

cd Mac && /usr/bin/make pythonw
make[1]: Entering directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12/Mac'
/usr/bin/gcc-4.2 -arch ppc -arch ppc64 -L/opt/local/lib -Wl,-headerpad_max_install_names -arch ppc -arch ppc64 -DPYTHONFRAMEWORK='"Python"' -o pythonw \
		./Tools/pythonw.c -I.. -I./../Include \
		../Python.framework/Versions/3.8/Python
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12/Mac'
Creating directory /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin
if test "Python.framework" = "no-framework" ; then \
		/usr/bin/install -c python.exe /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
	else \
		/usr/bin/install -c -s Mac/pythonw /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
	fi
if test "3.8" != "3.8"; then \
		if test -f /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -o -h /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
		then rm -f /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
		fi; \
		(cd /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin; ln python3.8 python3.8); \
	fi
if test -f Python.framework/Versions/3.8/Python && test "Python.framework" = "no-framework" ; then \
		if test -n "" ; then \
			/usr/bin/install -c -m 755  /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin; \
		else \
			/usr/bin/install -c -m 755 Python.framework/Versions/3.8/Python /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/Python.framework/Versions/3.8/Python; \
			if test Python.framework/Versions/3.8/Python != Python.framework/Versions/3.8/Python; then \
				(cd /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib; ln -sf Python.framework/Versions/3.8/Python Python.framework/Versions/3.8/Python) \
			fi \
		fi; \
		if test -n ""; then \
			/usr/bin/install -c -m 755  /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/; \
		fi; \
	else	true; \
	fi
if test "x-extract ppc" != "x" ; then \
		rm -f /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/binpython3.8-32; \
		lipo -extract ppc \
			-output /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8-32 \
			/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8; \
	fi
fatal error: lipo: -extract ppc specified but fat file: /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 does not contain that architecture
make: *** [altbininstall] Error 1
make: Leaving directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12'
Command failed:  cd "/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/Python-3.8.12" && /usr/bin/make -w frameworkinstall maninstall DESTDIR=/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot 
Exit code: 2
Error: Failed to destroot python38: command execution failed
Error: See /opt/local/var/macports/logs/_opt_PPCLeopardPorts_lang_python38/python38/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
Error: Processing of port python38 failed

I have checked the binary, and both architectures are present in fact:

36-225:~ svacchanda$ file /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8: Mach-O universal binary with 2 architectures
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 (for architecture ppc7400):	Mach-O executable ppc
/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python38/python38/work/destroot/opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 (for architecture ppc64):	Mach-O 64-bit executable ppc64

Any idea what should I fix now?

comment:8 Changed 2 years ago by kencu (Ken)

the toolchain sometimes does not see ppc and ppc7400 as the same arch.

I needed to fix that in legacysupport a few years ago.

#60147

comment:9 in reply to:  6 Changed 2 years ago by barracuda156

Replying to ryandesign:

Things only get fixed in MacPorts when someone who cares about an issue fixes it. Few people care about PowerPC systems these days. Since you do, you could try either of the two possible fixes suggested in this ticket (patchfile or muniversal portgroup). If you find one that works, you can report it here, or better yet, submit a pull request.

Okay, I have made it:

36-225:Python-3.8.12 svacchanda$ port -v installed python38
The following ports are currently installed:
  python38 @3.8.12_4+universal (active) requested_variants='+universal' platform='darwin 9' archs='ppc ppc64' date='2021-12-24T02:01:57+0800'
36-225:Python-3.8.12 svacchanda$

Could anyone check my patches? How should I submit those for review?

I will try now to fix python27.

comment:10 Changed 2 years ago by barracuda156

  1. S. I noticed this though with Python 3.8:
Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

comment:11 in reply to:  10 Changed 2 years ago by barracuda156

Replying to barracuda156:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

It appears that while python27 is fixed and works, none of python3* works correctly: they all build with patches applied, but aren't usable due to some module errors.

If anyone got at idea how to fix that, please suggest me.

comment:12 in reply to:  8 Changed 2 years ago by barracuda156

Replying to kencu:

the toolchain sometimes does not see ppc and ppc7400 as the same arch.

I needed to fix that in legacysupport a few years ago.

#60147

I tried to use portgroup legacysupport 1.0 now for Python310 (and discarding my patches), however I got these errors:

/usr/bin/gcc-4.2 -c -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch ppc -arch ppc64 -pipe -Os -I/opt/local/include/LegacySupport -arch ppc -arch ppc64   -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include -I/opt/local/include -I/opt/local/include/LegacySupport   -DPy_BUILD_CORE -o Python/structmember.o Python/structmember.c
In file included from /opt/local/include/LegacySupport/sys/random.h:25,
                 from Python/bootstrap_hash.c:17:
/usr/include/sys/random.h:37: error: expected declaration specifiers or ‘...’ before ‘u_int’
/usr/bin/gcc-4.2 -c -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch ppc -arch ppc64 -pipe -Os -I/opt/local/include/LegacySupport -arch ppc -arch ppc64   -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include -I/opt/local/include -I/opt/local/include/LegacySupport   -DPy_BUILD_CORE -o Python/symtable.o Python/symtable.c
In file included from /opt/local/include/LegacySupport/sys/random.h:25,
                 from Python/bootstrap_hash.c:17:
/usr/include/sys/random.h:37: error: expected declaration specifiers or ‘...’ before ‘u_int’
/usr/bin/gcc-4.2 -c -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch ppc -arch ppc64 -pipe -Os -I/opt/local/include/LegacySupport -arch ppc -arch ppc64   -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal  -I. -I./Include -I/opt/local/include -I/opt/local/include/LegacySupport   -DPy_BUILD_CORE \
		-DABIFLAGS='""' \
		-DMULTIARCH=\"darwin\" \
		-o Python/sysmodule.o ./Python/sysmodule.c
fatal error: lipo: can't open input file: /opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/.tmp/ccimd9EN.out (No such file or directory)
make: *** [Python/bootstrap_hash.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory `/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.1'
Command failed:  cd "/opt/local/var/macports/build/_opt_PPCLeopardPorts_lang_python310/python310/work/Python-3.10.1" && /usr/bin/make -j4 -w all 
Exit code: 2
Error: Failed to build python310: command execution failed
Error: See /opt/local/var/macports/logs/_opt_PPCLeopardPorts_lang_python310/python310/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there
is a bug.
Error: Processing of port python310 failed

This is on 10.5.8 PPC trying to build functional python3* as universal.

Note: See TracTickets for help on using tickets.