py39-aubio: fatal error: 'numpy/arrayobject.h' file not found

[ryandesign (Ryan Carsten Schmidt)]

py39-aubio does not build on macOS Monterey:

​https://build.macports.org/builders/ports-12_arm64-builder/builds/4101/steps/install-port/logs/stdio

In file included from python/ext/aubiomodule.c:2:
python/ext/aubio-types.h:17:10: fatal error: 'numpy/arrayobject.h' file not found
#include <numpy/arrayobject.h>
         ^~~~~~~~~~~~~~~~~~~~~
1 error generated.

[catap (Kirill A. Korinsky)]

@ryandesign can it be related to the fact that py-numpy wasn't build on the same machine? ​https://build.macports.org/builders/ports-12_arm64-builder/builds/2024

[ryandesign (Ryan Carsten Schmidt)]

I don't understand. If py39-numpy does not build, then the buildbot would never have attempted to build py39-aubio which depends on py39-numpy. But buildbot did attempt to build py39-aubio, therefore at that moment py39-numpy must have been successfully installed.

[catap (Kirill A. Korinsky)]

@ryandesign I'll upgrade my personal mac mini with m1 in next few days, after that I can try to reproduce it. Meanwhile may I ask you to re-run CI for py*-aubio ports?

[catap (Kirill A. Korinsky)]

Upgraded the mac and can reproduce it. Thinking.

[ryandesign (Ryan Carsten Schmidt)]

On the Big Sur build we had:

/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_python_py-aubio/py39-aubio/work/compwrap/cc/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch arm64 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -DAUBIO_VERSION=0.4.9 -DUSE_LOCAL_AUBIO=1 -DHAVE_STDLIB_H=1 -DHAVE_STDIO_H=1 -DHAVE_MATH_H=1 -DHAVE_STRING_H=1 -DHAVE_ERRNO_H=1 -DHAVE_C99_VARARGS_MACROS=1 -DHAVE_LIMITS_H=1 -DHAVE_STDARG_H=1 -DHAVE_MEMCPY_HACKS=1 -DHAVE_ACCELERATE=1 -DHAVE_SOURCE_APPLE_AUDIO=1 -DHAVE_SINK_APPLE_AUDIO=1 -DHAVE_WAVWRITE=1 -DHAVE_WAVREAD=1 -Ipython/ext -I/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include -Isrc -Ipython/gen -I/opt/local/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c python/ext/aubiomodule.c -o build/temp.macosx-11.0-arm64-3.9/python/ext/aubiomodule.o

But on the Monterey build we have:

/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_python_py-aubio/py39-aubio/work/compwrap/cc/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch arm64 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -DAUBIO_VERSION=0.4.9 -DUSE_LOCAL_AUBIO=1 -DHAVE_STDLIB_H=1 -DHAVE_STDIO_H=1 -DHAVE_MATH_H=1 -DHAVE_STRING_H=1 -DHAVE_ERRNO_H=1 -DHAVE_C99_VARARGS_MACROS=1 -DHAVE_LIMITS_H=1 -DHAVE_STDARG_H=1 -DHAVE_MEMCPY_HACKS=1 -DHAVE_ACCELERATE=1 -DHAVE_SOURCE_APPLE_AUDIO=1 -DHAVE_SINK_APPLE_AUDIO=1 -DHAVE_WAVWRITE=1 -DHAVE_WAVREAD=1 -Ipython/ext -Isrc -Ipython/gen -I/opt/local/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c python/ext/aubiomodule.c -o build/temp.macosx-12.0-arm64-3.9/python/ext/aubiomodule.o

Certainly the necessary flag -I/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include is missing on the Monterey build but I'm not sure why.

[catap (Kirill A. Korinsky)]

Upstream issue: ​https://github.com/aubio/aubio/issues/320

[catap (Kirill A. Korinsky)]

@ryandesign the root cause I feel that this issue: #63814

Inside setup.py is just ignores any error:

try:
    import numpy
    include_dirs += [numpy.get_include()]
except ImportError:
    pass

from ​https://github.com/aubio/aubio/blob/d3325ba1d97167dc17307cd116000c46c7aba1ec/setup.py#L24-L28

Seems that this isn't a bug or at lest not a bug on this port.

Do you agree?

[ryandesign (Ryan Carsten Schmidt)]

Maybe the bug is that aubio's setup.py ignores numpy import errors?

[catap (Kirill A. Korinsky)]

Probably, let me add a PR to fix it in upstream because without numpy it useless.

[catap (Kirill A. Korinsky)]

FYI: ​https://github.com/aubio/aubio/pull/350

[catap (Kirill A. Korinsky)]

@ryandesign is it possible to re-run build server? I feel that an issue should be fixed.

[ryandesign (Ryan Carsten Schmidt)]

Sure, I've queued rebuilds of py-aubio and its subports on the two macOS 12 builders. We have an extreme backlog of builds due to yesterday's openssl 3 switchover so these py-aubio builds will not happen for several days.

Is it #63864 which you think fixes this problem?

[catap (Kirill A. Korinsky)]

@ryandesign yes it help. I've confirmed it on my M1 which I've upgraded to macOS 12.

[ryandesign (Ryan Carsten Schmidt)]

Successful build on arm64 macOS 12: ​https://build.macports.org/builders/ports-12_arm64-builder/builds/9245