Octave 7.1 Upgrade failed on Monterey: Undefined symbols for architecture x86_64: "_cgejsv_" and "_zgejsv_"

[neilt (Neil Tiffin)]

version:1
:debug:configure Starting logging for octave @7.1.0_0+accelerate+app+docs+gfortran+graphicsmagick+qt5+sound+sundials
:debug:sysinfo macOS 12.4 (darwin/21.5.0) arch i386
:debug:sysinfo MacPorts 2.7.2
:debug:sysinfo Xcode 13.4
:debug:sysinfo SDK 12
:debug:sysinfo MACOSX_DEPLOYMENT_TARGET: 12.0

:info:build libtool: link: /usr/bin/clang++ -std=gnu++11 -dynamiclib  -o liboctave/.libs/liboctave.9.dylib  liboctave/.libs/liboctave_la-liboctave-build-info.o liboctave/.libs/liboctave_la-version.o   -Wl,-force_load,liboctave/array/.libs/libarray.a -Wl,-force_load,liboctave/external/blas-xtra/.libs/libxerbla.a -Wl,-force_load,liboctave/external/ranlib/.libs/libranlib.a -Wl,-force_load,liboctave/external/.libs/libexternal.a -Wl,-force_load,liboctave/numeric/.libs/libnumeric.a -Wl,-force_load,liboctave/operators/.libs/liboperators.a -Wl,-force_load,liboctave/system/.libs/libsystem.a -Wl,-force_load,liboctave/util/.libs/libutil.a -Wl,-force_load,liboctave/wrappers/.libs/libwrappers.a -Wl,-force_load,libgnu/.libs/libgnu.a  -L/opt/local/lib -L/opt/local/libexec/qt5/lib -lcurl -lcholmod -lumfpack -lamd -lcamd -lcolamd -lccolamd -lcxsparse -lsuitesparseconfig -larpack -lqrupdate -lfftw3_threads -lfftw3 -lfftw3f_threads -lfftw3f -lvecLibFort -lreadline -lncurses -lpcre -ldl -L/opt/local/lib/gcc11 -lgfortran -lquadmath -liconv -lpthread -lm  -pthread -Os -stdlib=libc++ -arch x86_64 -Wl,-headerpad_max_install_names -Wl,-syslibroot -Wl,/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch x86_64 -Wl,-framework -Wl,CoreFoundation   -pthread -install_name  /opt/local/lib/octave/7.1.0/liboctave.9.dylib -compatibility_version 10 -current_version 10.0 -Wl,-single_module
:info:build Undefined symbols for architecture x86_64:
:info:build   "_cgejsv_", referenced from:
:info:build       octave::math::svd<FloatComplexMatrix>::gejsv(char&, char&, char&, char&, char&, char&, int, int, std::__1::complex<float>*, int, float*, std::__1::complex<float>*, std::__1::complex<float>*, int, std::__1::vector<std::__1::complex<float>, std::__1::allocator<std::__1::complex<float> > >&, int&, std::__1::vector<int, std::__1::allocator<int> >&, int&) in libnumeric.a(libnumeric_la-svd.o)
:info:build   "_zgejsv_", referenced from:
:info:build       octave::math::svd<ComplexMatrix>::gejsv(char&, char&, char&, char&, char&, char&, int, int, std::__1::complex<double>*, int, double*, std::__1::complex<double>*, std::__1::complex<double>*, int, std::__1::vector<std::__1::complex<double>, std::__1::allocator<std::__1::complex<double> > >&, int&, std::__1::vector<int, std::__1::allocator<int> >&, int&) in libnumeric.a(libnumeric_la-svd.o)
:info:build ld: symbol(s) not found for architecture x86_64
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build make[2]: *** [liboctave/liboctave.la] Error 1
:info:build make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_octave/octave/work/octave-7.1.0'
:info:build make[1]: *** [all-recursive] Error 1
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_octave/octave/work/octave-7.1.0'
:info:build make: *** [all] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_octave/octave/work/octave-7.1.0'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_octave/octave/work/octave-7.1.0" && /usr/bin/make -j8 -w all 
:info:build Exit code: 2
:error:build Failed to build octave: command execution failed
:debug:build Error code: CHILDSTATUS 67687 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$callback {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec -callback portprogress::target_progress_callback build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_octave/octave/main.log for details.

[neilt (Neil Tiffin)]

Log file.

[kencu (Ken)]

same for me, and buildbot:

​https://build.macports.org/builders/ports-12_x86_64-builder/builds/30844/steps/install-port/logs/stdio

[kencu (Ken)]

Something related, at least in part:

​https://lists.nongnu.org/archive/html/help-octave/2021-03/msg00017.html

I'm trying a sudo port -v install octave -accelerate +openblas build now.

[kencu (Ken)]

building against openblas works:

% port -v installed octave
The following ports are currently installed:
  octave @7.1.0_0+app+docs+gfortran+graphicsmagick+openblas+qt5+sound+sundials (active) requested_variants='+openblas-accelerate' platform='darwin 21' archs='x86_64' date='2022-06-01T10:24:39-0700'

[Schamschula (Marius Schamschula)]

Indeed! I have been building with +openblas for a while, and was wondering why the build bots failed, after I had success on both my Intel and M1 Monterey machines.

Perhaps need to make +openblas default.

[neilt (Neil Tiffin)]

I rarely select any variants except the default. My experience has not been good when I do. I don't know the reason why, but non-default variants tend to cause me problems. So if a variant does not work It should be removed. All variants should also be tested. I know for a fact that in many cases they are not. I also have success building with +openblas. So yes, please change the default.

[kencu (Ken)]

the default +accelerate variant always worked before, and works on every single system except Monterey. So this is new info.

And (AFAIK) nobody has tried to fix the accelerate variant yet - maybe it's easy, maybe it's impossible, nobody knows yet.

[kencu (Ken)]

I can't yet figure out how the accelerate variant works on older systems, as they don't seem to have cgejsv either, as far as I can tell -- this is 10.14:

$ cd /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Accelerate.framework

$ ag gejsv
Versions/A/Accelerate.tbd
587:                       _DGEHRD, _DGEHRD_, _DGEJSV, _DGEJSV_, _DGELQ2, _DGELQ2_, _DGELQF, 
704:                       _SGEHD2, _SGEHD2_, _SGEHRD, _SGEHRD_, _SGEJSV, _SGEJSV_, _SGELQ2, 
1011:                       _dgehd2_, _dgehrd, _dgehrd_, _dgejsv, _dgejsv_, _dgelq2, _dgelq2_, 
1128:                       _sgegv_, _sgehd2, _sgehd2_, _sgehrd, _sgehrd_, _sgejsv, _sgejsv_, 

Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.tbd
115:                       _DGEHRD, _DGEHRD_, _DGEJSV, _DGEJSV_, _DGELQ2, _DGELQ2_, _DGELQF, 
232:                       _SGEHD2, _SGEHD2_, _SGEHRD, _SGEHRD_, _SGEJSV, _SGEJSV_, _SGELQ2, 
539:                       _dgehd2_, _dgehrd, _dgehrd_, _dgejsv, _dgejsv_, _dgelq2, _dgelq2_, 
656:                       _sgegv_, _sgehd2, _sgehd2_, _sgehrd, _sgehrd_, _sgejsv, _sgejsv_, 

Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib.tbd
300:                       _DGEHRD, _DGEHRD_, _DGEJSV, _DGEJSV_, _DGELQ2, _DGELQ2_, _DGELQF, 
417:                       _SGEHD2, _SGEHD2_, _SGEHRD, _SGEHRD_, _SGEJSV, _SGEJSV_, _SGELQ2, 
724:                       _dgehd2_, _dgehrd, _dgehrd_, _dgejsv, _dgejsv_, _dgelq2, _dgelq2_, 
841:                       _sgegv_, _sgehd2, _sgehd2_, _sgehrd, _sgehrd_, _sgejsv, _sgejsv_, 

Versions/A/Frameworks/vecLib.framework/Versions/A/Headers/clapack.h
2593:int dgejsv_(char *__joba, char *__jobu, char *__jobv, char *__jobr,
5493:int sgejsv_(char *__joba, char *__jobu, char *__jobv, char *__jobr,

[kencu (Ken)]

Well, Marius, I give up.

It looks like octave is built against openblas mostly, upstream. The accelerate framework is said to be faster on Apple, and seems to be by anyone who tests it.

I don't see any definition for cgejsv in the accelerate framework, so I'm not sure how octave links on older systems without errors just now. Perhaps it is blocked from using that symbol somehow. I don't see any test for it though.

If you are only using openblas, and upstream is only using openblas for testing, then perhaps openblas it should be?

[Schamschula (Marius Schamschula)]

Well, the default used to be +atlas, but the macOS kernel completely broke those optimizations, and made atlas worthless.

I'll switch Monterey to default to +openblas.

[Schamschula (Marius Schamschula)]

In b51a2061565b838dc9580d4dc6c6e4b0bd35142e/macports-ports (master):

octave: make +openblas default for Monterey

See: #65260
clean up darwin platform references

[matsbn (Mats Bentsen)]

I had problem with the +accelerate variant on Monterey, but was successful installing octave with sudo port -v install octave -accelerate +openblas as suggested by kencu. However, with the latest change, setting +openblas as default for Monterey, I get the following when I run port -d selfupdate:

...

receiving file list ... done
./

sent 68 bytes  received 99 bytes  334.00 bytes/sec
total size is 96054784  speedup is 575178.35
DEBUG: successful verification with key /opt/local/share/macports/macports-pubkey.pem
DEBUG: system: /usr/bin/tar -C /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/tmp -xf /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports.tar
DEBUG: system: chmod -R a+r /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs
DEBUG: system: /opt/local/bin/portindex /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports
Creating port index in /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports
Error: octave: Variant accelerate conflicts with openblas
Failed to parse file math/octave/Portfile: Error evaluating variants

Total number of ports parsed:	1
Ports successfully parsed:	0
Ports failed:			1
Up-to-date ports skipped:	29274

--->  MacPorts base is already the latest version

I was wondering if this had anything to do with the current installation of octave and uninstalled/cleaned the port. Still, I get the variant conflict with selfupdate.

[Schamschula (Marius Schamschula)]

I didn't do a revision bump, as the build bots had failed to build the Monterey versions of octave. If you already had manually installed the +openblas variant your are up to date.

[jmroot (Joshua Root)]

You need to add +openblas to default_variants only if the conflicting variants, +atlas and +accelerate, are not set. Unfortunately the linear_algebra portgroup always adds +accelerate to default_variants if nothing else is specified by the user, so there will always be a conflict unless something changes.

[Schamschula (Marius Schamschula)]

What do you suggest?

I'm guessing perhaps that we may need to change the linear_algebra PG, as this issue may happen for other ports.

[Schamschula (Marius Schamschula)]

In 9f97bf9f8072a3045af1cda8f5734902495b767e/macports-ports (master):

octave: Make +openblas default in linear_algebra PG for Monterey

See: #65260