clang >= 3.4: Unable to build/bootstrap on Intel Tiger

[dgelessus (dgelessus)]

It is currently not possible to build clang-3.4 or newer on Intel Tiger. As a result, any ports that require C++11 cannot be built on Tiger, because MacPorts selects a recent clang (currently 9.0) when looking for a C++11-capable compiler.

This issue is somewhat complex because a lot of bootstrapping is involved, and multiple ports in the bootstrap process don't work properly on Tiger. The (simplified) bootstrap dependency chain looks like this:

• Xcode 2.5 gcc
• apple-gcc42
• clang-3.4
• icu
• libxml2
• libcxx
• clang-3.7
• newer clang (e. g. 9.0)

So far I've run into the following individual issues in the chain:

• clang-3.4 requires libxml2 to build, which requires icu. icu requries a C++11 compiler, and MacPorts selects clang-3.4 here, which leads to a circular dependency.
  • The clang-3.4 Portfile already has a special case to handle this issue, but it doesn't get triggered on Tiger, because the cxx_stdlib is set to libstdcxx and not libcxx.
  • I've submitted a fix for this as ​PR 5771.
• llvm/clang-3.7 always wants libcxx as its C++ stdlib, but the libcxx port doesn't support Tiger.
  • I've attempted to make libcxx build on Tiger in ​a branch of my fork. This isn't tested very well yet. (I've previously submitted this as ​PR 5783, but these changes likely won't be merged into the main ports tree, to avoid breaking newer systems.)
  • Perhaps an alternative fix would be to make clang use libstdcxx instead of libcxx on Tiger. I haven't looked into this further yet, but @kencu has ​a customized llvm/clang-3.8 port that uses libstdcxx, so it seems to be possible.
• llvm-3.7's configure script tries to call clang-3.4 with the option -stdlib=macports-libstdc++, which that version of the clang port doesn't support. This option is patched in by MacPorts for clang-5.0 and newer with the +libstdcxx variant (which is enabled by default). clang ports older than 5.0 (including 3.4) don't have a +libstdcxx variant.
  • This is caused by a bug in MacPorts base, which has been reported in #59584.
  • As a workaround, the +libstdcxx patch from clang-5.0 can be applied to clang-3.4 (with minor modifications). This won't result in proper macports-libstdc++ support (because clang before 3.8 doesn't support compiling C++11 code against libstdc++), but it is enough to make the incorrect call from the configure script not fail.
• llvm-3.7 references a couple of macros that don't exist on Tiger.
  • This only requires some very minor patches, which I can submit once I've tested them.
• The clang-3.7 build fails to build the compiler-rt libraries for the clang_darwin target (the clang_macho_embedded ones get built properly). make thinks that there is nothing to be done for the clang_darwin target, so no libraries are built, which causes later parts of the build (that expect the libraries to exist) to fail.
  • This appears to be a similar issue to #54836 and #59157.
• There may be more issues after this; I'm currently working on getting clang-3.7 fully built.

I've already discussed some of these issues separately on IRC and GitHub (thanks to @jmroot and @kencu for the help so far!). This ticket is meant to be an overview of how the issues are related, as the issues further down the list usually can't be observed without first fixing/working around the previous issues.

Also I'm not sure what the best solutions are for some of the issues, or if it even makes sense to use clang (rather than gcc) as the C++11 compiler on Tiger. Any opinions and suggestions are welcome.

[kencu (Ken)]

So I have clang-3.8 running on Tiger with our macports-libstdc++ modification enabled. That is a pretty good start. Will help you over next few days to get up to there.

[kencu (Ken)]

may I first say --- are you sure you want to do this? If it's for fun and sport, fine. Otherwise, forget it. It's a massive sinkhole of frustrations, many things won't work, and the SDK is years inferior to Leopard (which is way better but still very frustrating). Go with 10.6.8 or 10.7.5 and you'll be miles and miles and miles more functional.

Tiger PPC is fun for MacOS9. Tiger intel is good for not much, other than crosscompiling for TigerPPC, which it really can't do until/unless I fix up a current gcc crosscompiler.

rpaths don't (currently) work on Tiger, so lots of build systems fail. etc.

If I haven't lost you yet, keep reading...

[kencu (Ken)]

you can expect almost nothing in the macports bootstrapping code (recently hatched) to work. It doesn't even work on Leopard Intel yet, which is a good place to start (meet Riccardo..). So have no expectations about that.

Almost nobody on MacPorts uses Tiger Intel (me, and .. uh.. ).

None of clang or llvm is ever tested on <10.6 except for a few of us diehards. There are many holes to fill.

With all that in mind, and your expectations hopefully suitably moderated, see <​https://github.com/kencu/TigerPorts/tree/master/lang/llvm-3.8> and associated software there...

[kencu (Ken)]

you can't backport macports-libstdc++ into clang 3.4, I believe, as the guts that make it work didn't show up until clang-3.9, and were backported into 3.8. but 3.4 is too far back to go, and --- why? use 3.8 instead.

And then perhaps that can be used to leap to clang-5.0. clang-3.4 can't build a working compiler newer than clang-3.7 (anything later might build but crashes in use).

[kencu (Ken)]

your go-to compiler on Tiger, at present, is gcc7, although there are many ports it won't build due to headers, build system assumptions, rpath missing, and general non-posix compliance in Tiger...

[dgelessus (dgelessus)]

are you sure you want to do this? If it's for fun and sport, fine. Otherwise, forget it.

My fun and sport is unusual ;) So far I haven't run into anything that seems unreasonably complicated to fix, so I want to see how far I can get. I'm aware that Intel Tiger doesn't have much practical use (and I have this machine dual-booted to 10.7, which I use for most things), but I'm happy to submit patches for any part of this that I can work out. (As long as MacPorts core accepts it - if this would be better off in a separate repo to not clutter up the core, let me know.)

you can't backport macports-libstdc++ into clang 3.4, I believe, as the guts that make it work didn't show up until clang-3.9, and were backported into 3.8.

Interesting. I didn't have much trouble adjusting the +libstdcxx patch file from 5.0 to work with 3.4 and 3.7, though I haven't tested if it actually works as expected. All I know is that the flag is accepted and clang doesn't crash as a result. Right now I only use -stdlib=macports-libstdc++ in clang-3.4 to work around some quirk with how clang-3.7's configure script is called (when checking that the C++ compiler is working, it includes the flag, even though the rest of the llvm/clang-3.7 build uses -stdlib=libc++). I need to look into that again in more detail, it's probably better to fix the configure script than to patch clang-3.4 just for that.

but 3.4 is too far back to go, and --- why? use 3.8 instead.

Currently I'm following the bootstrap chain that MacPorts suggested by default. clang-3.4 and libcxx are apparently working (nothing crashed yet at least) and the clang-3.7 build is in progress right now. That would allow building newer clangs, assuming everything worked up to that point. Of course, the newer clangs will most likely require more work to get running on Tiger...

I'm also fairly certain that this bootstrap sequence worked fine in the past (at some point during MacPorts 2.5.4, which I know isn't very specific). That installation broke at some point (couldn't upgrade most ports anymore, because almost everything triggered the dependency cycle in clang-3.4), so I uninstalled almost everything and tried to get the clang versions installed again.

I haven't tried anything using gcc yet - it's quite possible that that would be a better route than the old clang versions.

[kencu (Ken)]

the c++11 libstdc++ support in llvm landed here <​https://reviews.llvm.org/D18035> and here <​https://reviews.llvm.org/D17567> and that made it into clang-3.9. Debian backported that to clang-3.8 with moderate effort, and that's what formed part of my working clang-3.8-on-Tiger port above.

The MacPorts-specific part adds acceptance for the macports-libstdc++ flag, and changes some header lookup dirs, but does not actually function without the llvm support, of course, even if it accepts the flag.

The fact that macports base sends the macports-libstdc++ flag to clang 3.4 and clang 3.7 in the first place is a reported base bug. Those two compilers support c++11 only building against libc++.

MacPorts has dropped clang-3.6 to 4.0, and threatens to drop 5.0 soon, so that will influence stepping stones and focus effort.

[dgelessus (dgelessus)]

The MacPorts-specific part adds acceptance for the macports-libstdc++ flag, and changes some header lookup dirs, but does not actually function without the llvm support, of course, even if it accepts the flag.

Ah, I already expected something like that. At least the patch works well enough on the older versions that the accidental clang-3.4 -stdlib=macports-libstdc++ call runs without errors.

For reference, #59584 is the MacPorts base ticket for this issue.

I let my clang-3.7 build run overnight (llvm-3.7 already built successfully). Most of the build ran without problems, but now I'm getting this error in the compiler-rt build:

llvm[3]: Copying runtime library darwin/eprintf to build dir
cp /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/tools/clang/runtime/compiler-rt/clang_darwin/eprintf/libcompiler_rt.a /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/Release+Debug/lib/clang/3.7.1/lib/darwin/libclang_rt.eprintf.a
llvm[3]: Copying runtime library darwin/10.4 to build dir
cp /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/tools/clang/runtime/compiler-rt/clang_darwin/10.4/libcompiler_rt.a /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/Release+Debug/lib/clang/3.7.1/lib/darwin/libclang_rt.10.4.a
cp: /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/tools/clang/runtime/compiler-rt/clang_darwin/eprintf/libcompiler_rt.a: No such file or directory
gmake[3]: *** [Makefile:241: /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/Release+Debug/lib/clang/3.7.1/lib/darwin/libclang_rt.eprintf.a] Error 1
gmake[3]: *** Waiting for unfinished jobs....
cp: /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/tools/clang/runtime/compiler-rt/clang_darwin/10.4/libcompiler_rt.a: No such file or directory
gmake[3]: *** [Makefile:241: /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/Release+Debug/lib/clang/3.7.1/lib/darwin/libclang_rt.10.4.a] Error 1
rm /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/Release+Debug/lib/clang/3.7.1/lib/darwin/.dir /opt/local/var/macports/build/_Users_Shared_macports-ports_lang_llvm-3.7/clang-3.7/work/build/Release+Debug/lib/clang/3.7.1/lib/macho_embedded/.dir

This issue (or a very similar one) has been reported before as #54836, but it doesn't look like it was ever fixed. #59157 is a similar issue, but with a different cause (command-line tools vs. full Xcode installation, which is not an issue on Tiger). That ticket may be useful as it has a detailed analysis of what causes the issue, which might help with finding the root cause on Tiger.