libjxl @0.9.0+tests does not build on macOS Sonoma, 14.2.1, because of problems with dylibs?

[ballapete (Peter "Pete" Dyballa)]

Quite early, at around 70 %, when test utilities were built, the avalanche of failures (bug report windows from macOS) goes off:

cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib && /opt/local/bin/cmake -E cmake_link_script CMakeFiles/jpegli_test.dir/link.txt --verbose=ON
/usr/bin/clang++ -pipe -Os -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -fno-rtti -DNDEBUG -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -mmacosx-version-min=14.0 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -fPIE -Xlinker -pie CMakeFiles/jpegli_test.dir/extras/jpegli_test.cc.o -o tests/jpegli_test  -Wl,-rpath,/opt/local/lib -lgmock libjxl_extras-internal.a libjxl_testlib-internal.a /opt/local/lib/libgtest_main.1.14.0.dylib /opt/local/lib/libgtest.1.14.0.dylib /opt/local/lib/giflib5/lib/libgif.dylib /opt/local/lib/libjpeg.dylib /opt/local/lib/libpng.dylib /opt/local/lib/libz.dylib /opt/local/lib/libOpenEXR-3_1.dylib /opt/local/lib/libOpenEXRUtil-3_1.dylib /opt/local/lib/libOpenEXRCore-3_1.dylib /opt/local/lib/libIex-3_1.dylib /opt/local/lib/libIlmThread-3_1.dylib /opt/local/lib/libImath-3_1.dylib libjxl_extras_nocodec-internal.a libjxl_threads.0.9.0.dylib libjxl-internal.a /opt/local/lib/libhwy.dylib /opt/local/lib/libbrotlidec.dylib /opt/local/lib/libbrotlicommon.dylib /opt/local/lib/libbrotlienc.dylib libjxl_cms.0.9.0.dylib 
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib && /opt/local/bin/cmake -D TEST_TARGET=jpegli_test -D TEST_EXECUTABLE=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/tests/jpegli_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=jpegli_test_TESTS -D CTEST_FILE=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/jpegli_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=240 -D TEST_XML_OUTPUT_DIR= -P /opt/local/share/cmake-3.24/Modules/GoogleTestAddTests.cmake
dyld[57958]: Library not loaded: @rpath/libjxl_threads.0.9.dylib
  Referenced from: <2E7B5E53-E706-3EB6-A6FF-E0A918BAC6AF> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/tests/jpegli_test
  Reason: tried: '/opt/local/lib/libjxl_threads.0.9.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/local/lib/libjxl_threads.0.9.dylib' (no such file), '/opt/local/lib/libjxl_threads.0.9.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/local/lib/libjxl_threads.0.9.dylib' (no such file)
CMake Error at /opt/local/share/cmake-3.24/Modules/GoogleTestAddTests.cmake:112 (message):
  Error running test executable.

    Path: '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/tests/jpegli_test'
    Result: Subprocess aborted
    Output:
      

Call Stack (most recent call first):
  /opt/local/share/cmake-3.24/Modules/GoogleTestAddTests.cmake:225 (gtest_discover_tests_impl)


make[2]: *** [lib/tests/jpegli_test] Error 1
make[2]: *** Deleting file `lib/tests/jpegli_test'
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build'
make[1]: *** [lib/CMakeFiles/jpegli_test.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 70%] Linking CXX executable benchmark_xl

Which looks as if building in a single thread might be adviced…

[ballapete (Peter "Pete" Dyballa)]

Main.log from macOS Sonoma, 14.2.1

[ballapete (Peter "Pete" Dyballa)]

port -vds build libjxl -tests works, the port is built.

[ballapete (Peter "Pete" Dyballa)]

Performing port -vds build libjxl +tests build.jobs=1 the build process fails at 60 %:

[ 60%] Linking CXX executable tests/codec_test
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib && /opt/local/bin/cmake -E cmake_link_script CMakeFiles/codec_test.dir/link.txt --verbose=ON
/usr/bin/clang++ -pipe -Os -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -fno-rtti -DNDEBUG -arch x86_64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -mmacosx-version-min=14.0 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -fPIE -Xlinker -pie CMakeFiles/codec_test.dir/extras/codec_test.cc.o -o tests/codec_test  -Wl,-rpath,/opt/local/lib -lgmock libjxl_extras-internal.a libjxl_testlib-internal.a /opt/local/lib/libgtest_main.1.14.0.dylib /opt/local/lib/libgtest.1.14.0.dylib /opt/local/lib/giflib5/lib/libgif.dylib /opt/local/lib/libjpeg.dylib /opt/local/lib/libpng.dylib /opt/local/lib/libz.dylib /opt/local/lib/libOpenEXR-3_1.dylib /opt/local/lib/libOpenEXRUtil-3_1.dylib /opt/local/lib/libOpenEXRCore-3_1.dylib /opt/local/lib/libIex-3_1.dylib /opt/local/lib/libIlmThread-3_1.dylib /opt/local/lib/libImath-3_1.dylib libjxl_extras_nocodec-internal.a libjxl_threads.0.9.0.dylib libjxl-internal.a /opt/local/lib/libhwy.dylib /opt/local/lib/libbrotlidec.dylib /opt/local/lib/libbrotlicommon.dylib /opt/local/lib/libbrotlienc.dylib libjxl_cms.0.9.0.dylib 
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib && /opt/local/bin/cmake -D TEST_TARGET=codec_test -D TEST_EXECUTABLE=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/tests/codec_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=codec_test_TESTS -D CTEST_FILE=/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/codec_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=240 -D TEST_XML_OUTPUT_DIR= -P /opt/local/share/cmake-3.24/Modules/GoogleTestAddTests.cmake
dyld[28400]: Library not loaded: @rpath/libjxl_threads.0.9.dylib
  Referenced from: <1A3C8BB5-082D-3029-9312-3C1729FE19BE> /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/tests/codec_test
  Reason: tried: '/opt/local/lib/libjxl_threads.0.9.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/local/lib/libjxl_threads.0.9.dylib' (no such file), '/opt/local/lib/libjxl_threads.0.9.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/local/lib/libjxl_threads.0.9.dylib' (no such file)
CMake Error at /opt/local/share/cmake-3.24/Modules/GoogleTestAddTests.cmake:112 (message):
  Error running test executable.

    Path: '/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/tests/codec_test'
    Result: Subprocess aborted
    Output:
      

Call Stack (most recent call first):
  /opt/local/share/cmake-3.24/Modules/GoogleTestAddTests.cmake:225 (gtest_discover_tests_impl)


make[2]: *** [lib/tests/codec_test] Error 1
make[2]: *** Deleting file `lib/tests/codec_test'
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build'
make[1]: *** [lib/CMakeFiles/codec_test.dir/all] Error 2
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build'
make: *** [all] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build" && /usr/bin/make -j1 -w all VERBOSE=ON 

Actually the directory with codec_test is empty:

pete 197 /\ ls -l /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_graphics_libjxl/libjxl/work/build/lib/tests
total 0

There seems to be a path to the built before dylibs is missing…

[jmroot (Joshua Root)]

I can't reproduce this on Ventura.

[lukaso (Lukas Oberhuber)]

I'm getting a slightly different problem but still with dylibs. And this is now on libjxl @ 0.9.2. Attaching the recommended output.

[lukaso (Lukas Oberhuber)]

output.log

[kencu (Ken)]

Fails for me.

It's a very common problem with the cmake PortGroups -- the defaults are basically set up to break testing. Sometimes the tests do work if you install it first, which confuses people.

Add this to the tests variant and you should be good to go:

    configure.args-replace  -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
                            -DCMAKE_BUILD_WITH_INSTALL_RPATH=OFF

[kencu (Ken)]

Yeah -- if you install it first, and then run the tests, you don't need that extra bit.

-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON tells cmake to use the full, final library paths during the build.

If the software has been installed already to the final location, then it finds the libraries and the test can build.

If you have not installed the software yet, then all the libraries exist only in the build tree. And as the build is looking for them in their final locations, the linker can't find them.

   configure.args-replace  -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
                            -DCMAKE_BUILD_WITH_INSTALL_RPATH=OFF

tells cmake to build it the normal, default way -- with rpaths referencing the build tree -- so that the build can find the libraries.

Now my recollection is that somehow this still works even on Tiger (which doesn't use rpaths) -- I think by using the full path to the libraries in the build tree. But it's been a long while since I tested that specific point, so someone will have to re-verify it.

At any rate, that block is the fix we use in many, many portfiles that use cmake to fix testing.

We should change the cmake portgroups to stop forcing the wrong build strategy I think. If that change breaks some things, those things should be properly fixed instead of going back to the current default.

[jmroot (Joshua Root)]

In 676280e02f9ff458303ec7d1ebf9525d8d476a01/macports-ports (master):

libjxl: update to 0.10.0

Also disable build with install rpath in +tests.

Fixes: #69053