Opened 8 years ago

Closed 8 years ago

#51929 closed defect (fixed)

ld64-latest (and others?): default variant llvm38 breaks linking with pre-3.8-clang

Reported by: Ionic (Mihai Moldovan) Owned by: jeremyhu (Jeremy Huddleston Sequoia)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc:
Port: ld64-latest

Description (last modified by Ionic (Mihai Moldovan))

Recently, I changed my variants on ld64-latest and cctools from whatever I used before (probably llvm33) to the new default variant (llvm38) on OS X 10.9+ (with me being on 10.9.5.)

While installing clang-3.8 as a dependency of something else, I hit this error:

-- Check for working C compiler: /opt/local/bin/clang-mp-3.7 -- broken
CMake Error at /opt/local/share/cmake-3.6/Modules/CMakeTestCCompiler.cmake:61 (message):
  The C compiler "/opt/local/bin/clang-mp-3.7" is not able to compile a
  simple test program.

  It fails with the following output:

   Change Dir: /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_llvm-3.8/clang-3.8/work/build/CMakeFiles/CMakeTmp



  Run Build Command:"/usr/bin/make" "cmTC_b11b8/fast"

  /Applications/Xcode.app/Contents/Developer/usr/bin/make -f
  CMakeFiles/cmTC_b11b8.dir/build.make CMakeFiles/cmTC_b11b8.dir/build

  Building C object CMakeFiles/cmTC_b11b8.dir/testCCompiler.c.o

  /opt/local/bin/clang-mp-3.7 -pipe -Os -arch x86_64
  -mmacosx-version-min=10.9 -o CMakeFiles/cmTC_b11b8.dir/testCCompiler.c.o -c
  /opt/local/var/macports/build/_opt_local_var_macports_sources_macports.rsync.ionic.de_release_ports_lang_llvm-3.8/clang-3.8/work/build/CMakeFiles/CMakeTmp/testCCompiler.c


  Linking C executable cmTC_b11b8

  /opt/local/bin/cmake -E cmake_link_script
  CMakeFiles/cmTC_b11b8.dir/link.txt --verbose=1

  /opt/local/bin/clang-mp-3.7 -pipe -Os -arch x86_64
  -mmacosx-version-min=10.9 -Wl,-search_paths_first
  -Wl,-headerpad_max_install_names -L/opt/local/lib
  -Wl,-headerpad_max_install_names -Wl,-rpath,@loader_path
  -Wl,-rpath,@loader_path/ CMakeFiles/cmTC_b11b8.dir/testCCompiler.c.o -o
  cmTC_b11b8

  dyld: Library not loaded: @executable_path/../lib/libLTO.dylib

    Referenced from: /opt/local/libexec/llvm-3.7/bin/ld
    Reason: Incompatible library version: ld requires version 1.0.0 or later, but libLTO.dylib provides version 0.0.0

  clang: error: unable to execute command: Trace/BPT trap: 5

  clang: error: linker command failed due to signal (use -v to see
  invocation)

Further investigation revealed:

root@nopileos~# ls -ldh /opt/local/libexec/llvm-3.7/bin/ld
lrwxr-xr-x 1 root 26 Jul 27 23:42 /opt/local/libexec/llvm-3.7/bin/ld -> /opt/local/libexec/ld64/ld

So when using clang-3.7, the system-default linker as installed by ld64/the other ld64-${version} ports are used, good.

root@nopileos~# otool -L /opt/local/libexec/llvm-3.7/lib/libLTO.dylib
/opt/local/libexec/llvm-3.7/lib/libLTO.dylib:
	/opt/local/libexec/llvm-3.7/lib/libLTO.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/libexec/llvm-3.7/lib/libLLVM-3.7.dylib (compatibility version 0.0.0, current version 0.0.0)
	[...]

llvm-3.7's libLTO has compat version 0.0.0, current version 0.0.0.

root@nopileos~# otool -L /opt/local/libexec/llvm-3.8/lib/libLTO.dylib
/opt/local/libexec/llvm-3.8/lib/libLTO.dylib:
	@rpath/libLTO.dylib (compatibility version 1.0.0, current version 3.8.1)
	@rpath/libLLVM.dylib (compatibility version 1.0.0, current version 3.8.1)
	[...]

llvm-3.8's libLTO changed the compat version to 1.0.0 and the current version to 3.8.1.

root@nopileos~# otool -L /opt/local/libexec/ld64/ld
/opt/local/libexec/ld64/ld:
	@executable_path/../lib/libLTO.dylib (compatibility version 1.0.0, current version 3.8.1)
	[...]

And the binary installed by ld64-latest (and symlinked via ld64) needs compat version 1.0.0.

clang-3.7 invokes /opt/local/libexec/llvm-3.7/bin/ld, so the expanded path to the library through @executable_path is /opt/local/libexec/llvm-3.7/lib/libLTO.dylib, which has (see above) a lower compat version of 0.0.0.

Breakage.

A fix might be to not use @executable_path, but rewrite the libLTO library dependency to always use the libLTO library as installed by the selected llvm*-variant version.

I haven't looked at the source code and how difficult that would be, though.

Change History (12)

comment:1 Changed 8 years ago by Ionic (Mihai Moldovan)

Description: modified (diff)

comment:2 Changed 8 years ago by jeremyhu (Jeremy Huddleston Sequoia)

$ otool -L llvm-3.*/lib/libLTO.dylib 
llvm-3.7/lib/libLTO.dylib:
	/opt/local/libexec/llvm-3.7/lib/libLTO.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/libexec/llvm-3.7/lib/libLLVM-3.7.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1237.0.0)
	/opt/local/lib/libffi.6.dylib (compatibility version 7.0.0, current version 7.4.0)
	/opt/local/lib/libedit.0.dylib (compatibility version 1.0.0, current version 1.54.0)
	/opt/local/lib/libncurses.6.dylib (compatibility version 6.0.0, current version 6.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.4.0)
llvm-3.8/lib/libLTO.dylib:
	@rpath/libLTO.dylib (compatibility version 1.0.0, current version 3.8.1)
	@rpath/libLLVM.dylib (compatibility version 1.0.0, current version 3.8.1)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
llvm-3.9/lib/libLTO.dylib:
	@rpath/libLTO.dylib (compatibility version 1.0.0, current version 3.9.0)
	@rpath/libLLVM.dylib (compatibility version 1.0.0, current version 3.9.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)

It would've been nice to have known about this before I revbumped all the llvm ports... sigh...

Basically two options:

  1. Change the compat version to 0.0.0 in libLTO in llvm-3.8+
  2. Change the compat and current versions in libLTO in llvm-3.7-
Last edited 8 years ago by jeremyhu (Jeremy Huddleston Sequoia) (previous) (diff)

comment:3 Changed 8 years ago by Ionic (Mihai Moldovan)

I think number 2 is cleaner.

To be fair, the old 0.0.0/0.0.0 behavior looks more like a bug than a carefully crafted compatibility-ensurance thing. It bites back for us, though.

But should ld use an llvm-specific libLTO in the first place? Shouldn't the "MacPorts system version" link against the libLTO version as selected by the llvm* variant? We seem to replace the llvm-provided linker binaries with a symlink to the ld64-provided symlink in the first place to ensure that "our" system linker is used. Pulling in different libLTO binaries dependent upon which clang version is being used sounds... wrong?

comment:4 Changed 8 years ago by jeremyhu (Jeremy Huddleston Sequoia)

No, this is the correct approach.

${prefix}/bin/ld does use the one provided by the llvm variant.

${prefix}/libexec/ld64/ld uses the one provided by the toolchain it is being used with.

comment:5 Changed 8 years ago by jeremyhu (Jeremy Huddleston Sequoia)

And yes, the unset compat version (resulting in 0.0.0) was a bug in older llvm. The issue here wasn't so much that it was consciously fixed so much as it was fixed by nature of the change to cmake as the build system. I could see an argument being made for setting the compat version to 0 for compatibility with the older autoconf-built versions, but the flip side is that they were certainly wrong and should be fixed as well.

comment:6 Changed 8 years ago by jeremyhu (Jeremy Huddleston Sequoia)

So yeah, I think I lean towards patching all the older llvm versions to use '-compatibility_version 1.0 -current_version 3.y.z' when linking libLTO.dylib and revbumping ld64 to force a relink.

comment:7 Changed 8 years ago by Ionic (Mihai Moldovan)

If that's intended and correct, than okay, one issue less. :)

Hmm... number 1 would mean less rebuilds, but we don't want to patch that around forever neither.

comment:8 Changed 8 years ago by Ionic (Mihai Moldovan)

Yep, I totally agree with that.

comment:9 Changed 8 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Testing out a solution now...

comment:10 Changed 8 years ago by Ionic (Mihai Moldovan)

  • trunk/dports/lang/llvm-3.3/Portfile

    a b  
    379383}
    380384
    381 if {${os.platform} eq "darwin" && ${os.major} > 15} {
     385if {${os.platform} eq "darwin" && ${os.major} > 25} {
    382386    depends_lib
    383387    depends_run

You should probably revert this change again?

comment:11 Changed 8 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Lol, yeah. I did that to test on Sierra and forgot to revert back, thanks.

comment:12 Changed 8 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.