libtool @2.4.6_9: link mode ignores specified compiler

[chrstphrchvz (Christopher Chavez)]

A long-known issue (assuming it's still the same one) in libtool is that when run in link mode, it ignores the compiler specified on the command line, and instead uses whichever compiler is hardcoded in libtool, i.e. the one it was compiled with. The "workaround" is to configure your own libtool hardcoded with the compiler you intend to build other things with, as if you're not supposed to reuse a single libtool installation with other compilers.

One way this causes problems is that on macOS 10.6 (​32 bit and presumably 64 bit) libtool is built with clang 3.7, and when other ports are built with another compiler (e.g. clang 9.0), libtool tries using clang 3.7 anyway during link mode. Example for recent libvterm 10.6 builds (​32 bit and ​64 bit):

glibtool --mode=link --tag=CC /opt/local/bin/clang-mp-9.0 -rpath /opt/local/lib -version-info 0:4:0 -o libvterm.la src/encoding.lo src/keyboard.lo src/mouse.lo src/parser.lo src/pen.lo src/screen.lo src/state.lo src/unicode.lo src/vterm.lo -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64
glibtool: link: /opt/local/bin/clang-mp-3.7 -dynamiclib -Wl,-undefined -Wl,dynamic_lookup -o .libs/libvterm.0.dylib  src/.libs/encoding.o src/.libs/keyboard.o src/.libs/mouse.o src/.libs/parser.o src/.libs/pen.o src/.libs/screen.o src/.libs/state.o src/.libs/unicode.o src/.libs/vterm.o   -L/opt/local/lib  -Wl,-headerpad_max_install_names -arch x86_64   -install_name  /opt/local/lib/libvterm.0.dylib -compatibility_version 1 -current_version 1.4 -Wl,-single_module
/opt/local/bin/glibtool: line 10550: /opt/local/bin/clang-mp-3.7: No such file or directory

Do any ports work around this? If so, are they expected to?

[ryandesign (Ryan Carsten Schmidt)]

The libtermkey port is one that runs into this problem:

​https://build.macports.org/builders/ports-10.6_x86_64-builder/builds/159291/steps/install-port/logs/stdio

glibtool --quiet --mode=compile --tag=CC /opt/local/bin/clang-mp-11 -Os -arch x86_64 -Wall -std=c99  -DHAVE_UNIBILIUM -o termkey.lo -c termkey.c
glibtool --quiet --mode=compile --tag=CC /opt/local/bin/clang-mp-11 -Os -arch x86_64 -Wall -std=c99  -DHAVE_UNIBILIUM -o driver-csi.lo -c driver-csi.c
glibtool --quiet --mode=compile --tag=CC /opt/local/bin/clang-mp-11 -Os -arch x86_64 -Wall -std=c99  -DHAVE_UNIBILIUM -o driver-ti.lo -c driver-ti.c
glibtool --quiet --mode=compile --tag=CC /opt/local/bin/clang-mp-11 -Os -arch x86_64 -Wall -std=c99  -DHAVE_UNIBILIUM -o demo.lo -c demo.c
glibtool --quiet --mode=compile --tag=CC /opt/local/bin/clang-mp-11 -Os -arch x86_64 -Wall -std=c99  -DHAVE_UNIBILIUM -o demo-async.lo -c demo-async.c
glibtool --quiet --mode=link --tag=CC /opt/local/bin/clang-mp-11 -rpath /opt/local/lib -version-info 15:2:14 -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -L/opt/local/lib -lunibilium -o libtermkey.la termkey.lo driver-csi.lo driver-ti.lo
/opt/local/bin/glibtool: line 10840: /opt/local/bin/clang-mp-3.7: No such file or directory
make: *** [libtermkey.la] Error 127

If this is the correct syntax for telling libtool what compiler you want it to use, then indeed it seems to be honoring it when compiling but ignoring it when linking. I would want to fix that in libtool, not in each individual port that uses libtool.

Christopher, you mentioned this is a long-known issue... do you know the upstream bug report URL?

[chrstphrchvz (Christopher Chavez)]

It has been too long for me to remember for certain, but I believe I had likely come across ​https://openwrt-devel.openwrt.narkive.com/dakgoLUj/cross-libtool and the page mentioned in one of the replies: ​https://web.archive.org/web/20210210153259/metastatic.org/text/libtool.html

I am not aware of an upstream libtool report for this issue.

[chrstphrchvz (Christopher Chavez)]

It turns out I did find an upstream report: ​https://debbugs.gnu.org/cgi/bugreport.cgi?bug=11911

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #43005.

Christopher, thanks for those links. I've read through them.

Replying to chrstphrchvz:

​https://openwrt-devel.openwrt.narkive.com/dakgoLUj/cross-libtool

This thread from 2012 is indeed a user reporting this problem and it is suggested that libtool is being used incorrectly but it is never clarified in what way it is being used wrong. The few suggested solutions are disproven by the OP.

​https://web.archive.org/web/20210210153259/metastatic.org/text/libtool.html

The second problem on that page from 2007, "The wrong-gcc problem", does describe this problem. The first solution presented there, using -rpath, does not sound relevant to me but I have not tried it. The second solution, building a dedicated libtool for each compiler that you want to use and instructing each build to use it, sucks. There is no reference to any upstream bug report or discussion.

Replying to chrstphrchvz:

​https://debbugs.gnu.org/cgi/bugreport.cgi?bug=11911

This bug from 2012 talks about unknown arguments passed to libtool when linking being ignored but doesn't mention the wrong compiler being used to do the link. If this is indeed the same problem, the developers of libtool may not recognize its severity given this lack of information.

I think a new bug report should be filed with the developers of libtool.

[kencu (Ken)]

libtool often works properly, otherwise we’d see rampant build errors:

	:info:build /bin/sh ../../../libtool  --tag=CXX   --mode=link /opt/local/bin/g++-mp-7 -std=c++11  -pipe -Os -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -arch ppc -no-undefined -version-info 26:1:20 -L/opt/local/lib -Wl,-headerpad_max_install_names -arch ppc -o libgpgmepp.la -rpath /opt/local/lib exception.lo context.lo key.lo trustitem.lo data.lo callbacks.lo eventloopinteractor.lo editinteractor.lo keylistresult.lo keygenerationresult.lo importresult.lo decryptionresult.lo verificationresult.lo signingresult.lo encryptionresult.lo engineinfo.lo gpgsetexpirytimeeditinteractor.lo gpgsetownertrusteditinteractor.lo gpgsignkeyeditinteractor.lo gpgadduserideditinteractor.lo gpggencardkeyinteractor.lo gpgaddexistingsubkeyeditinteractor.lo gpgrevokekeyeditinteractor.lo defaultassuantransaction.lo scdgetinfoassuantransaction.lo gpgagentgetinfoassuantransaction.lo statusconsumerassuantransaction.lo vfsmountresult.lo configuration.lo tofuinfo.lo swdbresult.lo util.lo  context_vanilla.lo   ../../../src/libgpgme.la -L/opt/local/lib -lassuan
1414	:info:build libtool: link: /opt/local/bin/g++-mp-7 -dynamiclib -arch ppc  -o .libs/libgpgmepp.6.dylib  .libs/exception.o .libs/context.o .libs/key.o .libs/trustitem.o .libs/data.o .libs/callbacks.o .libs/eventloopinteractor.o .libs/editinteractor.o .libs/keylistresult.o .libs/keygenerationresult.o .libs/importresult.o .libs/decryptionresult.o .libs/verificationresult.o .libs/signingresult.o .libs/encryptionresult.o .libs/engineinfo.o .libs/gpgsetexpirytimeeditinteractor.o .libs/gpgsetownertrusteditinteractor.o .libs/gpgsignkeyeditinteractor.o .libs/gpgadduserideditinteractor.o .libs/gpggencardkeyinteractor.o .libs/gpgaddexistingsubkeyeditinteractor.o .libs/gpgrevokekeyeditinteractor.o .libs/defaultassuantransaction.o .libs/scdgetinfoassuantransaction.o .libs/gpgagentgetinfoassuantransaction.o .libs/statusconsumerassuantransaction.o .libs/vfsmountresult.o .libs/configuration.o .libs/tofuinfo.o .libs/swdbresult.o .libs/util.o .libs/context_vanilla.o   -L/opt/local/lib ../../../src/.libs/libgpgme.dylib /opt/local/lib/libgpg-error.dylib /opt/local/lib/libassuan.dylib  -Os -arch ppc -Wl,-headerpad_max_install_names -arch ppc   -install_name  /opt/local/lib/libgpgmepp.6.dylib -compatibility_version 27 -current_version 27.1 -Wl,-single_module

[ryandesign (Ryan Carsten Schmidt)]

Can we figure out what the difference is? You example uses CXX; failures in this ticket use CC. Is that it or are there other differences? It could indeed be, as claimed in the openwrt thread, that libtool is being used wrong by the ports where it fails.

[kencu (Ken)]

I am thinking that ...

if you are using the system libtool, it has been installed with burned-in compiler selections that are apparently not easy to override at build time.

if you are building a software package that comes with it's own copy of libtool, then that would be configured freshly for each build with the selected compiler used during the build, and would know nothing about what compiler the system libtool might have been configured with...

[kencu (Ken)]

I have had success overriding CC and CXX with the system installed glibtool, but it is ignoring attempts to override the linker using LD so far. The libtool script is not very readable to see exactly why that is a problem.

[kencu (Ken)]

has duplicate #71707

[jmroot (Joshua Root)]

Replying to kencu:

if you are building a software package that comes with it's own copy of libtool, then that would be configured freshly for each build with the selected compiler used during the build, and would know nothing about what compiler the system libtool might have been configured with...

Yes, the normal way to use libtool is for the package's configure script to generate its own libtool from ltmain.sh. ​https://www.gnu.org/software/libtool/manual/html_node/Configuring.html

[barracuda156]

Replying to kencu:

I have had success overriding CC and CXX with the system installed glibtool, but it is ignoring attempts to override the linker using LD so far. The libtool script is not very readable to see exactly why that is a problem.

Script can patched in post-destroot to fix compilers from being hardcoded back to variables to be picked from env. Unfortunately, direct picking from env does not seem to work at least for some dependents, but explicit passing of correct CC and CXX values works.

[kencu (Ken)]

Replying to barracuda156:

Script can patched in post-destroot to fix compilers from being hardcoded back to variables to be picked from env.

that is a brute-force method I guess -- copy glibtool from ${prefix}/bin/glibtool to a custom location, and reinplace the linker somehow.

As a one-off hand-rolled solution to a problem on your own system "right now", easy enough.

As a general solution, it would be more difficult of course -- the system installed ${prefix}/bin/glibtool would be different on every OS, and also would change with Macports default compilers changing. So the reinplace would have to be robust enough to handle different exact lines in the libtool script that need to be reinplaced. There is some flexibility in SED to allow matching lines with some variability in them, so it's not impossible. I haven't looked at that.

Best would be to see why overriding LD is not working, whereas overriding CC and CXX does work. That seems like the only upstreamable fix. That is what I was looking into, when I got bogged down in the vagarities of shell code and moved along.

Everything is always possible, with enough effort.

[jmroot (Joshua Root)]

LD not being overridable is probably a real bug, but it's also irrelevant if you use libtool in the recommended way (configure script using LT_INIT, glibtoolize, etc).

[kencu (Ken)]

It looks like there are only a few cases where people have used "libtool" in Portfile code that need changing.

How many ports are building with the system libtool instead of regenerating their own libtool is pretty hard to guess, though.