openblas @0.3.18_0+gcc11+lapack [+native] : (Intel) does not build because MAX_CPU_NUMBER is undefined

[snarkhunter (Steve Langer)]

On an iMac running Big Sur 11.6 and Xcode 13.0, openBLAS fails to build from source, whether or not the +native variant is specified. Strangely, on a MacBook Pro with the same OS and Xcode, it works properly. The machines have different processors (i9 vs i7), but the only macports configuration difference is that on the iMac the default variants are -x11 +quartz, which ought not to matter for openBLAS.

There are a bunch of these in the log file:

:info:build In file included from copy.c:40:
:info:build In file included from ../common.h:433:
:info:build ../common_x86_64.h:231:21: error: invalid token at start of a preprocessor expression
:info:build #if (MAX_CPU_NUMBER > 64)  
:info:build                     ^

-- Steve

[NicosPavlov]

Thanks for the detailed report. It is not necessarily surprising that OpenBLAS has different behaviors with different CPUs, as it tries to optimize the code for each type of architecture and generation.

I do not know why this happens at this point, but from the log you sent, the code fails because MAX_CPU_NUMBER is undefined. My guess is that there is an issue in the cpu characterization code of OpenBLAS, but this would have to be confirmed.

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/compwrap/cc/usr/bin/clang -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 -O3 -DMAX_STACK_ALLOC=2048 -DEXPRECISION -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DSMP_SERVER -DMAX_CPU_NUMBER= -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"1\" -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_ -DASMFNAME=__ -DNAME=_ -DCNAME= -DCHAR_NAME=\"_\" -DCHAR_CNAME=\"\" -DNO_AFFINITY -I. -O3 -DSMALL_MATRIX_OPT -DMAX_STACK_ALLOC=2048 -DEXPRECISION -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DSMP_SERVER -DMAX_CPU_NUMBER= -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"1\" -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_samin -DASMFNAME=_samin_ -DNAME=samin_ -DCNAME=samin -DCHAR_NAME=\"samin_\" -DCHAR_CNAME=\"samin\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c -DUSE_ABS -DUSE_MIN max.c -o samin.o

[NicosPavlov]

I just updated OpenBLAS-devel, which contains some fixes for detecting features on Apple, but this seems to be more related to M1 chips.

In principle, after attempting to build OpenBLAS, there should be a getarch program which determines the CPU characteristics, and especially MAX_CPU_NUMBER (which is determined by the NUM_CORES variable in the config files). Just to confirm the issue, would it be possible to post the output of running the command:

./getarch 0

?

[snarkhunter (Steve Langer)]

Well, I don't know what happened, but now it's building properly. The only thing different is that we had a power outage and the computer was shut off, but I don't see why that would matter.

[snarkhunter (Steve Langer)]

The laptop, which had not been powered off, still has the problem.

% ./getarch 0
CORE=HASWELL
LIBCORE=haswell
NUM_CORES=8
HAVE_MMX=1
HAVE_SSE=1
HAVE_SSE2=1
HAVE_SSE3=1
HAVE_SSSE3=1
HAVE_SSE4_1=1
HAVE_SSE4_2=1
HAVE_AVX=1
HAVE_AVX2=1
HAVE_FMA3=1
MAKE += -j 8

[NicosPavlov]

I am not sure to understand. In your initial description, you mentioned that the iMac had the issue, but that on the MacBook Pro, it worked properly. If indeed the laptop also has an issue when building as it appears now, is the log displaying the same issue?

[snarkhunter (Steve Langer)]

Sorry to be confusing, but I'm confused too. It's not behaving consistently.

On the MacBook just now, I saved the log file from the previously failed build. I'll attach it here as openblas.log1.

Then I ran "sudo port install -s openblas +native", without cleaning the failed build. It completed successfully (or at least without error messages).

Then I uninstalled and cleaned openblas, and built it again, with "sudo port install -s openblas +native". That failed. The log file is openblas.log2.

Next I tried to rebuild as before, without cleaning. This time it failed. The log file is openblas.log3.

[NicosPavlov]

Based on the logs, it seems that occasionally the script used to detect the cpu architecture (getarch) sometimes fails, such as in openblas.log3. That what led MAX_CPU_NUMBER to be undefined too.

Then, the other errors occur during testing phase (error 137).

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/compwrap/fc/opt/local/bin/gfortran-mp-11 -Os -m64 -O3 -m128bit-long-double -Wall -frecursive -fno-optimize-sibling-calls -m64  -msse3 -mssse3 -msse4.1 -mavx -mavx2 -mavx2 -O3 -m128bit-long-double -Wall -frecursive -fno-optimize-sibling-calls -m64  -msse3 -mssse3 -msse4.1 -mavx -mavx2 -mavx2 -fno-tree-vectorize -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -arch x86_64 -o zblat3 zblat3.o ../libopenblas-r1.a -lpthread -lgfortran -lpthread -lgfortran 
rm -f ?BLAT3.SUMM
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 ./sblat3 < ./sblat3.dat
make[1]: *** [level1] Killed: 9
make[1]: *** Waiting for unfinished jobs....
/bin/sh: line 1: 12687 Killed: 9               OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 ./sblat2 < ./sblat2.dat
make[1]: *** [level2] Error 137
/bin/sh: line 1: 12702 Killed: 9               OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 ./sblat3 < ./sblat3.dat
make[1]: *** [level3] Error 137
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/xianyi-OpenBLAS-efe4248/test'
make: *** [tests] Error 2

For this type of errors, I suspect reporting it upstream ​https://github.com/xianyi/OpenBLAS/issues would probably be more useful to find a solution. I will not report it myself though, as I would not be able to then provide details about the machine and error types as I can't reproduce the error.