Context Navigation

← Previous Ticket
Next Ticket →

#68545 assigned defect

py311-numpy @1.26.1 build failure - cannot find cython

Reported by:	RobK88	Owned by:	michaelld (Michael Dickens)
Priority:	Normal	Milestone:
Component:	ports	Version:	2.8.1
Keywords:	lion	Cc:	markmentovai (Mark Mentovai), RobK88
Port:	py-numpy

Description

I am unable to upgrade py311-numpy from 1.25.2_0 to @1.26.1 on Lion. It looks like the port cannot find the py311-cython library (which is installed).

The Meson build system
Version: 1.2.99
Source dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1
Build dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-9dakvmho/build
Build type: cross build
Project name: NumPy
Project version: 1.26.1
C compiler for the host machine: cc -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C linker for the host machine: cc -arch x86_64 ld64 136
C++ compiler for the host machine: c++ -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C++ linker for the host machine: c++ -arch x86_64 ld64 136

../../meson.build:1:0: ERROR: Unknown compiler(s): [['cython'], ['cython3']]
The following exception(s) were encountered:
Running `cython -V` gave "[Errno 2] No such file or directory: 'cython'"
Running `cython3 -V` gave "[Errno 2] No such file or directory: 'cython3'"

A full log can be found at /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-9dakvmho/build/meson-logs/meson-log.txt

ERROR Backend subprocess exited when trying to invoke build_wheel
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1" && /opt/local/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -m build --no-isolation --wheel --outdir /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work 
Exit code: 1
Error: Failed to build py311-numpy: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
bash-3.2$

Attachments (2)

main.log (1.1 MB) - added by RobK88 6 months ago.
main.2.log (1.1 MB) - added by RobK88 6 months ago.: Cannot find scipy-openblas dependency

Change History (16)

Changed 6 months ago by RobK88

Attachment:	main.log added

comment:1 Changed 6 months ago by RobK88

FYI -- see below. I do not know why py311-numpy cannot find the py311-cython library.

bash-3.2$ port installed py311-cython
The following ports are currently installed:
  py311-cython @0.29.36_0 (active)

comment:2 Changed 6 months ago by jmroot (Joshua Root)

Cc:	markmentovai added; michaelld removed
Owner:	set to michaelld
Port:	py-numpy added; py311-numpy removed
Status:	new → assigned
Summary:	Unable to upgrade py311-numpy @1.26.1 - cannot find cython → py311-numpy @1.26.1 build failure - cannot find cython

comment:3 Changed 6 months ago by markmentovai (Mark Mentovai)

python/py-numpy/files/patch-build_cython_path.diff and the build.env-append CYTHON=… was supposed to address this, and worked for me locally and on CI. Did that patch apply? Do you see CYTHON being set in the environment for the build phase?

comment:4 Changed 6 months ago by markmentovai (Mark Mentovai)

From the attached main.log I do see patch-build_cython_path.diff being applied:

:notice:patch --->  Applying patches to py311-numpy
:info:patch --->  Applying patch-build_cython_path.diff
[…]
:info:patch patching file vendored-meson/meson/mesonbuild/envconfig.py

And I do see the CYTHON environment variable being set:

:notice:build --->  Building py311-numpy
[…]
:debug:build Executing org.macports.build (py311-numpy)
:debug:build Environment: 
[…]
:debug:build CYTHON='/opt/local/bin/cython-3.11'

The build is being invoked correctly:

:info:build Executing:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1" && /opt/local/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -m build --no-isolation --wheel --outdir/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work 
:debug:build system:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1" && /opt/local/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -m build --no-isolation --wheel --outdir/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work

It’s using the correct Python (python3.11) and the correct Meson in vendored-meson (which has been patched above, and which identifies itself as 1.2.99):

:info:build * Getting build dependencies for wheel...
:info:build * Building wheel...
:info:build + /opt/local/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/vendored-meson/meson/meson.py setup /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-9dakvmho/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --cross-file /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-9dakvmho/build/meson-python-cross-file.ini --native-file=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-9dakvmho/build/meson-python-native-file.ini
:info:build The Meson build system
:info:build Version: 1.2.99

But the CYTHON environment variable isn’t having any effect.

:info:build ../../meson.build:1:0: ERROR: Unknown compiler(s): [['cython'], ['cython3']]
:info:build The following exception(s) were encountered:
:info:build Running `cython -V` gave "[Errno 2] No such file or directory: 'cython'"
:info:build Running `cython3 -V` gave "[Errno 2] No such file or directory: 'cython3'"

This error indicates that CYTHON is either unset at this point, or your build is somehow using a different (unpatched?) envconfig.py. When CYTHON is respected, even if it names a broken cython, the output looks a bit different, in that [['cython'], ['cython3']] doesn’t appear, but is replaced by the contents of the CYTHON variable. For example:

mark@arm-and-hammer zsh% CYTHON=/var/empty/enoent PATH=/opt/local/bin:/usr/bin:/bin python3.11 -m build --no-isolation --wheel
* Getting build dependencies for wheel...
* Building wheel...
[…]
The Meson build system
Version: 1.2.99
[…]
../../meson.build:1:0: ERROR: Unknown compiler(s): [['/var/empty/enoent']]
The following exception(s) were encountered:
Running `/var/empty/enoent -V` gave "[Errno 2] No such file or directory: '/var/empty/enoent'"

but that’s not what’s happening in your build log.

comment:5 Changed 6 months ago by RobK88

Here is a workaround. Now py311-numpy will find the py311-cython libraries.

bash-3.2$ port select --summary
Name        Selected  Options
====        ========  =======
clang       none      mp-clang-11 mp-clang-14 mp-clang-15 mp-clang-16 mp-clang-17 mp-clang-3.7 mp-clang-7.0 mp-clang-9.0 none
cython      none      cython27 cython311 none
gcc         none      llvm-gcc42 mp-gcc12 mp-gcc13 none
llvm        none      mp-llvm-11 mp-llvm-14 mp-llvm-15 mp-llvm-16 mp-llvm-17 mp-llvm-3.4 mp-llvm-3.7 mp-llvm-7.0 mp-llvm-9.0 none
pip         none      pip311 none
pip2        none      none
pip3        none      pip311 none
postgresql  none      postgresql15 none
pygments    none      py310-pygments py311-pygments none
python      none      python25-apple python26-apple python27 python27-apple python310 python311 python39 none
python2     none      python25-apple python26-apple python27 python27-apple none
python3     none      python310 python311 python39 none
ruby        none      ruby30 none
bash-3.2$ 

bash-3.2$ sudo port select --set cython cython311
Password:
Selecting 'cython311' for 'cython' succeeded. 'cython311' is now active.
bash-3.2$ 

bash-3.2$ port outdated
The following installed ports are outdated:
py311-numpy                    1.25.2_0 < 1.26.1_0       

bash-3.2$ sudo port clean py311-numpy
--->  Cleaning py311-numpy
bash-3.2$ 
bash-3.2$ sudo port -v upgrade outdated
--->  Computing dependencies for py311-numpy.
etc
etc

But the build fails since it could not find CMake and the Run-time dependency scipy-openblas:

The Meson build system
Version: 1.2.99
Source dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1
Build dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-y4fzpkbo/build
Build type: cross build
Project name: NumPy
Project version: 1.26.1
C compiler for the host machine: cc -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C linker for the host machine: cc -arch x86_64 ld64 136
C++ compiler for the host machine: c++ -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C++ linker for the host machine: c++ -arch x86_64 ld64 136
Cython compiler for the host machine: cython (cython 0.29.36)
C compiler for the build machine: ccache cc (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C linker for the build machine: cc ld64 136
C++ compiler for the build machine: ccache c++ (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C++ linker for the build machine: c++ ld64 136
Cython compiler for the build machine: cython (cython 0.29.36)
Build machine cpu family: x86
Build machine cpu: i386
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Program python3 found: YES (/opt/local/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11)
Found Pkg-config: NO
Run-time dependency python found: YES 3.11
Has header "Python.h" with dependency python: YES 
Compiler for C supports arguments -fno-strict-aliasing: NO 
Compiler for C supports arguments -ftrapping-math: NO 
Test features "SSE SSE2 SSE3" : Parial support, missing(SSE SSE2 SSE3) 
Test features "SSE" : Unsupported due to Implied feature "SSE2" is not supported
Test features "SSE2" : Unsupported due to Arguments "-msse, -msse2" are not supported
Test features "SSE3" : Unsupported due to Implied feature "SSE" is not supported
Test features "SSSE3" : Unsupported due to Implied feature "SSE" is not supported
Test features "SSE41" : Unsupported due to Implied feature "SSE" is not supported
Test features "POPCNT" : Unsupported due to Implied feature "SSE" is not supported
Test features "SSE42" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX" : Unsupported due to Implied feature "SSE" is not supported
Test features "F16C" : Unsupported due to Implied feature "SSE" is not supported
Test features "FMA3" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX2" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512F" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512CD" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_KNL" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_KNM" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_SKX" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_CLX" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_CNL" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_ICL" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_SPR" : Unsupported due to Implied feature "SSE" is not supported
Configuring npy_cpu_dispatch_config.h using configuration
Message: 
CPU Optimization Options
  baseline:
    Requested : min
    Enabled   : 
  dispatch:
    Requested : max -xop -fma4
    Enabled   : 

Library m found: YES
Found CMake: NO
Run-time dependency scipy-openblas found: NO (tried pkgconfig, framework and cmake)
WARNING: MKLROOT env var set to /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-y4fzpkbo/build/None, but not pointing to an MKL install
Run-time dependency mkl found: NO (tried pkgconfig and system)
WARNING: MKLROOT env var set to /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-y4fzpkbo/build/None, but not pointing to an MKL install
Run-time dependency mkl found: NO (tried pkgconfig and system)

../../numpy/meson.build:119:13: ERROR: Unhandled python exception

    This is a Meson bug and should be reported!

ERROR Backend subprocess exited when trying to invoke build_wheel
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1" && /opt/local/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -m build --no-isolation --wheel --outdir /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work 
Exit code: 1
Error: Failed to build py311-numpy: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.

Last edited 6 months ago by RobK88 (previous) (diff)

Changed 6 months ago by RobK88

Attachment:	main.2.log added

Cannot find scipy-openblas dependency

comment:6 Changed 6 months ago by markmentovai (Mark Mentovai)

The Run-time dependency scipy-openblas found: NO (tried pkgconfig, framework and cmake) thing isn’t what’s stopping the build. numpy has a number of options for linear algebra support of which scipy-openblas is one, although the fact that it’s looking for scipy-openblas tells me that maybe the MacPorts variant scheme isn’t fully communicating the desired option to py-numpy. That would be a separate problem worth looking into.

If you port select --set cython none and port deactivate meson, does that help?

comment:7 Changed 6 months ago by RobK88

No. The port py311-numpy still cannot find the py311-cython libraries. See below:

bash-3.2$ sudo port select --set cython none
Selecting 'none' for 'cython' succeeded. 'none' is now active.
bash-3.2$ 
bash-3.2$ sudo port deactivate meson
Note: It is not recommended to uninstall/deactivate a port that has dependents as it breaks the dependents.
The following ports will break: gutils-suite-core @1.1_3
Continue? [y/N]: Y
Warning: Deactivate forced.  Proceeding despite dependencies.
--->  Deactivating meson @1.2.2_0
--->  Cleaning meson
bash-3.2$ 
bash-3.2$ sudo port upgrade py311-numpy
--->  Computing dependencies for py311-numpy
etc
etc

:info:build The Meson build system
:info:build Version: 1.2.99
:info:build Source dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1
:info:build Build dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-wq36w8ae/build
:info:build Build type: cross build
:info:build Project name: NumPy
:info:build Project version: 1.26.1
:info:build C compiler for the host machine: cc -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C linker for the host machine: cc -arch x86_64 ld64 136
:info:build C++ compiler for the host machine: c++ -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C++ linker for the host machine: c++ -arch x86_64 ld64 136
:info:build ../../meson.build:1:0: ERROR: Unknown compiler(s): [['cython'], ['cython3']]
:info:build The following exception(s) were encountered:
:info:build Running `cython -V` gave "[Errno 2] No such file or directory: 'cython'"
:info:build Running `cython3 -V` gave "[Errno 2] No such file or directory: 'cython3'"
:info:build A full log can be found at /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1/.mesonpy-wq36w8ae/build/meson-logs/meson-log.txt
:info:build ERROR Backend subprocess exited when trying to invoke build_wheel
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work/numpy-1.26.1" && /opt/local/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11 -m build --no-isolation --wheel --outdir /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/work 
:info:build Exit code: 1
:error:build Failed to build py311-numpy: command execution failed
:debug:build Error code: CHILDSTATUS 61809 1
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$callback {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec -callback portprogress::target_progress_callback build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_python_py-numpy/py311-numpy/main.log for details.

comment:8 Changed 6 months ago by markmentovai (Mark Mentovai)

I noticed that you’re building +gfortran+openblas. The Portfile has a comment

    # However, +gfortran+openblas is broken on i386, see above

Does it make a difference if you build with the current default variant for i386, which should be +gfortran but not +openblas? (Probably not.)

Are you equipped to debug whether CYTHON is still set during the wheel build? You could try:

git clone https://github.com/numpy/numpy.git
cd numpy
git checkout v1.26.1
git submodule init
git submodule update
patch -p0 < /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/python/py-numpy/files/patch-build_cython_path.diff

You could also patch this in to help debug:

diff --git vendored-meson/meson/meson.py vendored-meson/meson/meson.py
index 6f3ba2a9ee46..5ffe4f8a45b4 100755
--- vendored-meson/meson/meson.py
+++ vendored-meson/meson/meson.py
@@ -37,4 +37,7 @@ if (meson_exe.parent / 'mesonbuild').is_dir():
 from mesonbuild import mesonmain
 
 if __name__ == '__main__':
+    import os
+    print('@@@ mesonmain = %r' % mesonmain, file=sys.stderr)
+    print('@@@ CYTHON = %r' % os.environ['CYTHON'], file=sys.stderr)
     sys.exit(mesonmain.main())

And build with CYTHON set:

PATH=/opt/local/bin:/usr/bin:/bin CYTHON=/opt/local/bin/cython-3.11 OPENBLAS=/opt/local/lib MKLROOT=None ATLAS=None SETUPTOOLS_USE_DISTUTILS=stdlib python3.11 -m build --no-isolation --wheel

When I build with this, I see at the top:

* Getting build dependencies for wheel...
* Building wheel...
+ /opt/local/bin/python3.11 /private/tmp/numpy/vendored-meson/meson/meson.py setup /private/tmp/numpy /private/tmp/numpy/.mesonpy-_hbx4qlc/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/private/tmp/numpy/.mesonpy-_hbx4qlc/build/meson-python-native-file.ini
@@@ mesonmain = <module 'mesonbuild.mesonmain' from '/private/tmp/numpy/vendored-meson/meson/mesonbuild/mesonmain.py'>
@@@ CYTHON = '/opt/local/bin/cython-3.11'
The Meson build system
[…]

showing that CYTHON is set and mesonbuild comes from the patched vendored-meson we expect.

If things are different for you, it’ll tell us where to look next.

And if things are the same for you and the build completes when done this way, it’ll tell us that there’s something else wrong with the build within port build but not when done manually.

comment:9 Changed 6 months ago by RobK88

I am building with +gfortran+openblas since they are the default variants.

You are right, building with just +gfortran made no difference. The build failed since the py311-cython libraries could not be found. P.S. I also tried sudo port install py311-numpy +gfortran -openblas. Same error.

I will try debugging as you suggested and will get back to you.

bash-3.2$ sudo port clean py311-numpy
--->  Cleaning py311-numpy
bash-3.2$ 
 
bash-3.2$ sudo port install py311-numpy +gfortran
Password:
--->  Computing dependencies for py311-numpy
--->  Fetching archive for py311-numpy
--->  Attempting to fetch py311-numpy-1.26.1_0+gfortran+openblas.darwin_11.x86_64.tbz2 from http://packages.macports.org/py311-numpy
--->  Attempting to fetch py311-numpy-1.26.1_0+gfortran+openblas.darwin_11.x86_64.tbz2 from http://ywg.ca.packages.macports.org/mirror/macports/packages/py311-numpy
--->  Attempting to fetch py311-numpy-1.26.1_0+gfortran+openblas.darwin_11.x86_64.tbz2 from http://mirror.fcix.net/macports/packages/py311-numpy
--->  Fetching distfiles for py311-numpy

etc

:info:build Build type: cross build
:info:build Project name: NumPy
:info:build Project version: 1.26.1
:info:build C compiler for the host machine: cc -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C linker for the host machine: cc -arch x86_64 ld64 136
:info:build C++ compiler for the host machine: c++ -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C++ linker for the host machine: c++ -arch x86_64 ld64 136
:info:build ../../meson.build:1:0: ERROR: Unknown compiler(s): [['cython'], ['cython3']]
:info:build The following exception(s) were encountered:
:info:build Running `cython -V` gave "[Errno 2] No such file or directory: 'cython'"
:info:build Running `cython3 -V` gave "[Errno 2] No such file or directory: 'cython3'
etc

Last edited 6 months ago by RobK88 (previous) (diff)

comment:10 Changed 6 months ago by RobK88

I tried to build py311-numpy from the git source (with the recommended patches).

The cython3 libraries / compiler was found. (i.e. I see @@@ CYTHON = '/opt/local/bin/cython-3.11'. But the build failed for another reason:

rob$ git clone https://github.com/numpy/numpy.git
Cloning into 'numpy'...
remote: Enumerating objects: 254138, done.
remote: Counting objects: 100% (1143/1143), done.
remote: Compressing objects: 100% (499/499), done.
remote: Total 254138 (delta 689), reused 963 (delta 629), pack-reused 252995
Receiving objects: 100% (254138/254138), 128.99 MiB | 984.00 KiB/s, done.
Resolving deltas: 100% (200239/200239), done.
Updating files: 100% (2124/2124), done.
Grinchs-Mac-Pro-2:git Repos rob$ 
Grinchs-Mac-Pro-2:git Repos rob$ 
Grinchs-Mac-Pro-2:git Repos rob$ cd numpy/
Grinchs-Mac-Pro-2:numpy rob$ git checkout v1.26.1
Updating files: 100% (1563/1563), done.
Note: switching to 'v1.26.1'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 411a55b9e Merge pull request #24928 from charris/update-version

Grinchs-Mac-Pro-2:numpy rob$ git submodule init
Submodule 'doc/source/_static/scipy-mathjax' (https://github.com/scipy/scipy-mathjax.git) registered for path 'doc/source/_static/scipy-mathjax'
Submodule 'numpy/core/src/npysort/x86-simd-sort' (https://github.com/intel/x86-simd-sort) registered for path 'numpy/core/src/npysort/x86-simd-sort'
Submodule 'numpy/core/src/umath/svml' (https://github.com/numpy/SVML.git) registered for path 'numpy/core/src/umath/svml'
Submodule 'vendored-meson/meson' (https://github.com/numpy/meson.git) registered for path 'vendored-meson/meson'
Submodule 'vendored-meson/meson-python' (https://github.com/numpy/meson-python.git) registered for path 'vendored-meson/meson-python'

Grinchs-Mac-Pro-2:numpy rob$ git submodule update
Cloning into '/Users/rob/Development/git Repos/numpy/doc/source/_static/scipy-mathjax'...
Cloning into '/Users/rob/Development/git Repos/numpy/numpy/core/src/npysort/x86-simd-sort'...
Cloning into '/Users/rob/Development/git Repos/numpy/numpy/core/src/umath/svml'...
Cloning into '/Users/rob/Development/git Repos/numpy/vendored-meson/meson'...
Cloning into '/Users/rob/Development/git Repos/numpy/vendored-meson/meson-python'...
Submodule path 'doc/source/_static/scipy-mathjax': checked out '36f4c898f2255e0c98eb6949cd67381552d5ffea'
Submodule path 'numpy/core/src/npysort/x86-simd-sort': checked out '85fbe7d1abca3b9a224ba1c62d52afe9a180f8ef'
Submodule path 'numpy/core/src/umath/svml': checked out '1b21e453f6b1ba6a6aca392b1d810d9d41576123'
Submodule path 'vendored-meson/meson': checked out '66ba7dbbfe2838983f65ad8fe16da1535ebf5b9d'
Submodule path 'vendored-meson/meson-python': checked out '206a31a96458af6cf5e29272b4ea1f6ea500b91b'
 
Grinchs-Mac-Pro-2:numpy rob$ patch -p0 < /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/python/py-numpy/files/patch-build_cython_path.diff
patching file vendored-meson/meson/mesonbuild/envconfig.py
 
Grinchs-Mac-Pro-2:numpy rob$ patch -p0 < /Users/rob/Documents/mesonPatch.diff 
patching file vendored-meson/meson/meson.py
 
Grinchs-Mac-Pro-2:numpy rob$ PATH=/opt/local/bin:/usr/bin:/bin CYTHON=/opt/local/bin/cython-3.11 OPENBLAS=/opt/local/lib MKLROOT=None ATLAS=None SETUPTOOLS_USE_DISTUTILS=stdlib python3.11 -m build --no-isolation --wheel
* Getting build dependencies for wheel...
* Building wheel...
+ /opt/local/bin/python3.11 /Users/rob/Development/git Repos/numpy/vendored-meson/meson/meson.py setup /Users/rob/Development/git Repos/numpy /Users/rob/Development/git Repos/numpy/.mesonpy-ew44b68j/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/Users/rob/Development/git Repos/numpy/.mesonpy-ew44b68j/build/meson-python-native-file.ini
@@@ mesonmain = <module 'mesonbuild.mesonmain' from '/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/mesonmain.py'>
@@@ CYTHON = '/opt/local/bin/cython-3.11'
The Meson build system
Version: 1.2.99
Source dir: /Users/rob/Development/git Repos/numpy
Build dir: /Users/rob/Development/git Repos/numpy/.mesonpy-ew44b68j/build
Build type: native build
Project name: NumPy
Project version: 1.26.1
C compiler for the host machine: ccache cc (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C linker for the host machine: cc ld64 136
C++ compiler for the host machine: ccache c++ (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
C++ linker for the host machine: c++ ld64 136
Cython compiler for the host machine: /opt/local/bin/cython-3.11 (cython 0.29.36)
Host machine cpu family: x86
Host machine cpu: i386
Program python found: YES (/opt/local/bin/python3.11)
Found pkg-config: /opt/local/bin/pkg-config (0.29.2)
Run-time dependency python found: YES 3.11
Has header "Python.h" with dependency python-3.11: YES 
Compiler for C supports arguments -fno-strict-aliasing: NO 
Compiler for C supports arguments -ftrapping-math: NO 
Test features "SSE SSE2" : Parial support, missing(SSE SSE2) 
Test features "SSE" : Unsupported due to Arguments "-msse, -msse2" are not supported
Test features "SSE2" : Unsupported due to Implied feature "SSE" is not supported
Test features "SSE3" : Unsupported due to Implied feature "SSE" is not supported
Test features "SSSE3" : Unsupported due to Implied feature "SSE" is not supported
Test features "SSE41" : Unsupported due to Implied feature "SSE" is not supported
Test features "POPCNT" : Unsupported due to Implied feature "SSE" is not supported
Test features "SSE42" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX" : Unsupported due to Implied feature "SSE" is not supported
Test features "F16C" : Unsupported due to Implied feature "SSE" is not supported
Test features "FMA3" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX2" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512F" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512CD" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_KNL" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_KNM" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_SKX" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_CLX" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_CNL" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_ICL" : Unsupported due to Implied feature "SSE" is not supported
Test features "AVX512_SPR" : Unsupported due to Implied feature "SSE" is not supported
Configuring npy_cpu_dispatch_config.h using configuration
Message: 
CPU Optimization Options
  baseline:
    Requested : min
    Enabled   : 
  dispatch:
    Requested : max -xop -fma4
    Enabled   : 

Library m found: YES
Found CMake: /opt/local/bin/cmake (3.24.4)
WARNING: CMake Toolchain: Failed to determine CMake compilers state
Run-time dependency scipy-openblas found: NO (tried pkgconfig, framework and cmake)
WARNING: MKLROOT env var set to /Users/rob/Development/git Repos/numpy/.mesonpy-ew44b68j/build/None, but not pointing to an MKL install
Run-time dependency mkl found: NO (tried pkgconfig and system)
WARNING: MKLROOT env var set to /Users/rob/Development/git Repos/numpy/.mesonpy-ew44b68j/build/None, but not pointing to an MKL install
Run-time dependency mkl found: NO (tried pkgconfig and system)
Traceback (most recent call last):
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/mesonmain.py", line 194, in run
    return options.run_func(options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/msetup.py", line 358, in run
    app.generate()
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/msetup.py", line 183, in generate
    return self._generate(env, capture, vslite_ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/msetup.py", line 228, in _generate
    intr.run()
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreter/interpreter.py", line 2997, in run
    super().run()
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 164, in run
    self.evaluate_codeblock(self.ast, start=1)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
    raise e
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 196, in evaluate_statement
    return self.function_call(cur)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 521, in function_call
    res = func(node, func_args, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 277, in wrapper
    return f(*nargs, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 596, in wrapper
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreter/interpreter.py", line 2409, in func_subdir
    self.evaluate_codeblock(codeblock)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
    raise e
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 206, in evaluate_statement
    return self.evaluate_if(cur)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 301, in evaluate_if
    self.evaluate_codeblock(i.block)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
    raise e
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 228, in evaluate_statement
    self.evaluate_foreach(cur)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 472, in evaluate_foreach
    self.evaluate_codeblock(node.block)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
    raise e
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 206, in evaluate_statement
    return self.evaluate_if(cur)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 306, in evaluate_if
    self.evaluate_codeblock(node.elseblock)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
    raise e
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 198, in evaluate_statement
    self.assignment(cur)
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 635, in assignment
    value = self.evaluate_statement(node.value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 196, in evaluate_statement
    return self.function_call(cur)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 521, in function_call
    res = func(node, func_args, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 826, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 826, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 826, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 4 more times]
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 109, in wrapped
    ret = f(*wrapped_args, **wrapped_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 127, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 277, in wrapper
    return f(*nargs, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 596, in wrapper
    return f(*wrapped_args, **wrapped_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreter/interpreter.py", line 1774, in func_dependency
    d = df.lookup(kwargs)
        ^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreter/dependencyfallbacks.py", line 355, in lookup
    dep = func(kwargs, func_args, func_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/interpreter/dependencyfallbacks.py", line 87, in _do_dependency
    dep = dependencies.find_external_dependency(name, self.environment, kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/dependencies/detect.py", line 124, in find_external_dependency
    d = c()
        ^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/dependencies/blas_lapack.py", line 524, in __init__
    if environment.machines[for_machine].is_darwin() and self.check_macOS_recent_enough():
                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/Development/git Repos/numpy/vendored-meson/meson/mesonbuild/dependencies/blas_lapack.py", line 529, in check_macOS_recent_enough
    sdk_version = subprocess.run(cmd, capture_output=True, check=True, text=True).stdout.strip()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['xcrun', '-sdk', 'macosx', '--show-sdk-version']' returned non-zero exit status 64.

../../numpy/meson.build:119:13: ERROR: Unhandled python exception

    This is a Meson bug and should be reported!

ERROR Backend subprocess exited when trying to invoke build_wheel
Grinchs-Mac-Pro-2:numpy rob$

comment:11 Changed 6 months ago by RobK88

Cc:	RobK88 added

comment:12 Changed 6 months ago by RobK88

I also tried the following:

bash-3.2$ sudo port select --set cython cython311
Selecting 'cython311' for 'cython' succeeded. 'cython311' is now active.
bash-3.2$ 
bash-3.2$ sudo port upgrade py311-numpy configure.compiler=macports-clang-11

But the build also failed. At least it found the cython3 compiler due to the port select command above. But for some reason, the port still tried to build using clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28. It did not try to build using clang-11

:info:build Build type: cross build
:info:build Project name: NumPy
:info:build Project version: 1.26.1
:info:build C compiler for the host machine: cc -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C linker for the host machine: cc -arch x86_64 ld64 136
:info:build C++ compiler for the host machine: c++ -arch x86_64 (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C++ linker for the host machine: c++ -arch x86_64 ld64 136
:info:build Cython compiler for the host machine: cython (cython 0.29.36)
:info:build C compiler for the build machine: ccache cc (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C linker for the build machine: cc ld64 136
:info:build C++ compiler for the build machine: ccache c++ (clang 4.2 "Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)")
:info:build C++ linker for the build machine: c++ ld64 136
:info:build Cython compiler for the build machine: cython (cython 0.29.36)
:info:build Build machine cpu family: x86
:info:build Build machine cpu: i386
:info:build Host machine cpu family: x86_64
:info:build Host machine cpu: x86_64
:info:build Target machine cpu family: x86_64
:info:build Target machine cpu: x86_64

comment:13 Changed 6 months ago by ryandesign (Ryan Carsten Schmidt)

Per your output, it's using cc and c++ rather than the compiler you asked for.

comment:14 Changed 6 months ago by RobK88

Thank Ryan. Maybe I do not understand how MacPorts works. I always thought that of one specifies the compiler on the command line, MacPorts would override cc and c++ with the specified compiler.

Note: See TracTickets for help on using tickets.

Download in other formats: