boost @1.66.0_0 +universal has no i386 symbols

[devernay (Frédéric Devernay)]

I installed boost @1.66.0_0+no_single+no_static+python27+universal on a libc++-based snow leopard (10.6) system, building with macports-clang-5.0, and it seems that installed libraries do not contain any i386 symbols:

root@mistral:~$ file /opt/local/lib/libboost_regex-mt.dylib
/opt/local/lib/libboost_regex-mt.dylib: Mach-O universal binary with 2 architectures
/opt/local/lib/libboost_regex-mt.dylib (for architecture i386): Mach-O dynamically linked shared library i386
/opt/local/lib/libboost_regex-mt.dylib (for architecture x86_64):       Mach-O 64-bit dynamically linked shared library x86_64
$ nm -arch i386 /opt/local/lib/libboost_regex-mt.dylib  
         U dyld_stub_binder
$ nm -arch x86_64 /opt/local/lib/libboost_regex-mt.dylib  
0000000000099c50 s GCC_except_table0
0000000000099e08 s GCC_except_table0
0000000000099f04 s GCC_except_table0
... (lots of symbols)

reinstalling didn't fix this.

boost@1.65.1_3+no_single+no_static+python27+universal had no issue.

Cc-ing kencu who may be able to test this on his system.

[kencu (Ken)]

see this bit in the Portfile:

# clang++ produces broken boost libraries (https://trac.macports.org/ticket/31525)
# If Apple's clang is used on 32-bit systems, it seems to silently ignore the '-march=i386' flag.
# (https://trac.macports.org/ticket/38157)
compiler.blacklist {clang < 421} *llvm-gcc-4.2 *gcc-4.0 gcc-3.3

probably more info in 31525

I'm not sure if I've ever built boost +universal on this system...

[ryandesign (Ryan Carsten Schmidt)]

Replying to devernay:

it seems that installed libraries do not contain any i386 symbols

Ok, so I'm not crazy. I noticed the same on Sierra yesterday.

[kencu (Ken)]

see also 38157 and 35172

[devernay (Frédéric Devernay)]

I examined the main.log from boost 1.65.1 and boost 1.66, and found out that in the boost 1.66 build the arch flags are only passed at link time, but not at compile time. boost 1.65.1_3 passes the arch flags both at compile time and at link time. So this is not a compiler issue, but rather a build issue.

I managed to build a universal boost 1.66.0 by manually adding "-arch i386 -arch x86_64" to the write_jam line in the Portfile.

[devernay (Frédéric Devernay)]

This is an upstream bug (I tried building boost from source and got the same result). Filed as ​https://svn.boost.org/trac10/ticket/13454

[michaelld (Michael Dickens)]

Nice debugging! Anyone have any idea how to fix this issue? Looks like reverting some Boost commit or some such, but which one(s)? A quick search on the Boost repo between 1.65.1 and 1.66.0 doesn't turn up anything definitive (at least in my reading).

[devernay (Frédéric Devernay)]

As mentionned above, a dirty hack to build universal is to add ourselves the arch flags to the cxxflags in the right write_jam line in the Portfile.

[ryandesign (Ryan Carsten Schmidt)]

Replying to michaelld:

Looks like reverting some Boost commit or some such, but which one(s)? A quick search on the Boost repo between 1.65.1 and 1.66.0 doesn't turn up anything definitive (at least in my reading).

The ​two ​commits to builtin.jam that occurred after the release of boost 1.65.1 are huge and refactor a lot of stuff. Perhaps a mistake was made. It might be worth testing whether reverting one or both of those commits fixes the problem.

[michaelld (Michael Dickens)]

Yeah true it's hard to tell with those commits. Maybe they'll fix the issue & release 1.66.1 sooner rather than later ;) [so that we don't have to go this route to fix the issue]

[kencu (Ken)]

I am new to boost, but it looks to me like the -arch flags for darwin are added in the subroutine setup-address-model in this file: ​src/tools/darwin.jam

I note there are rules that call this subroutine for compiling *.m and *.mm files in that darwin.jam file, and rules that call this subroutine for linking, but I don't see any rules for calling it when compiling *.c and *.cpp files.

[kencu (Ken)]

there is also this file

<​https://github.com/boostorg/build/blob/0d0c656c1164efbfe8568a09353f374839c26ac4/src/tools/clang-darwin.jam>

that seems to be related to calling clang on darwin for the build, but there is no setup-address-model in that file at all.

So, I wonder if adding some entries for compile.c and compile.c++ similar to the ones for compile.m to src/tools/darwin.jam to call setup-address-model might be fruitful...

Alternatively, if that didn't work, adding setup-address-model to to src/tools/clang-darwin.jam might be a fallback idea.

I assume these cascade on each other, though.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

Based on the ​comment in gcc.jam,

            # For darwin, the model can be 32_64. darwin.jam will handle that
            # on its own.

I believe Ken is correct that the setup-address-model in ​clang.jam was supposed to override the setup-address-model in ​gcc.jam.
setup-address-mode was changed to set-address-model-options, however, and gcc.jam now calls gcc.set-address-model-options in ​this changeset.

I am not sure trying to fix the current version of boost is a good use of time since the address model code has ​undergone significant changes since version 1.66 was released.

Following comment:7, I have attached a patch that adds the arch flags manually.

[michaelld (Michael Dickens)]

I like this workaround. I'll test locally & report back; other dev folks: please do the same.

[ryandesign (Ryan Carsten Schmidt)]

Thanks, it works, but -arch flags should always be used, not just when building universal. I'll commit a revised version in a moment.

[ryandesign (Ryan Carsten Schmidt)]

In e9f6257913f62fc441befc62811e1d0b7ea95d70/macports-ports:

boost: Use -arch flags; fixes universal variant

Closes: #55857

[kencu (Ken)]

boost is rebuilding, but I'm not seeing any -arch flags in the *.cpp files ...

    "/opt/local/bin/clang++-mp-3.7"   -Os -stdlib=libc++  -stdlib=libc++ -O3 -Wall -dynamic -gdwarf-2 -fexceptions -Wno-inline -fPIC -m64 -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -DNDEBUG -I"bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/../src/tr1" -I"." -I"libs/math/src/tr1" -c -o "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/nextafterl.o" "libs/math/build/../src/tr1/nextafterl.cpp"

although they are there for the link

    "/opt/local/bin/clang++-mp-3.7" -dynamiclib -Wl,-single_module -install_name "/opt/local/lib/libboost_math_c99l-mt.dylib"  -o "stage/lib/libboost_math_c99l-mt.dylib" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/acoshl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/asinhl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/atanhl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/cbrtl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/copysignl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/erfcl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/erfl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/expm1l.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/fmaxl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/fminl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/fpclassifyl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/hypotl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/lgammal.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/llroundl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/log1pl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/lroundl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/nextafterl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/nexttowardl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/roundl.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/tgammal.o" "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/truncl.o"      -headerpad_max_install_names -Wl,-dead_strip -no_dead_strip_inits_and_terms -L/opt/local/lib -Wl,-headerpad_max_install_names -stdlib=libc++ -fPIC -m64 -arch x86_64 -undefined dynamic_lookup

of course, I'm not building universal ....

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

boost is rebuilding, but I'm not seeing any -arch flags in the *.cpp files ...

    "/opt/local/bin/clang++-mp-3.7"   -Os -stdlib=libc++  -stdlib=libc++ -O3 -Wall -dynamic -gdwarf-2 -fexceptions -Wno-inline -fPIC -m64 -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -DNDEBUG -I"bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/../src/tr1" -I"." -I"libs/math/src/tr1" -c -o "bin.v2/libs/math/build/darwin-darwin-4.2.1/release/threadapi-pthread/threading-multi/nextafterl.o" "libs/math/build/../src/tr1/nextafterl.cpp"

But note that -m64 does appear. I assumed that this and the other few compiler invocations that use -m64 but not -arch flags are used at build time only and are not installed.

I confirmed before committing that the universal build worked correctly (i.e. other universal ports that use boost that had failed to build before the change due to undefined i386 symbols now build successfully).

[ccorn]

For Darwin9/PPC64 I had to ​add a corresponding `<cflags>` entry with archflags to the `write_jam` line for correct compilation of alloc_lib.c. The need for this may be PPC-specific or due to the fact that I use extra compiler wrappers (to enable compiling +universal with MacPorts GCC), but if you run into linker architecture errors related to allocation function symbols, you might want to try an explicit <cflags> setting as well.

[ryandesign (Ryan Carsten Schmidt)]

ccorn, could you be more specific about what change you needed to make? Maybe attach a patch?

boost built fine on Darwin 9 PowerPC on our build server (but we don't build universal).

[ccorn]

Replying to ryandesign:

ccorn, could you be more specific about what change you needed to make? Maybe attach a patch?

I mean ​this patch. Since I have made other patches which are not in ideal shape for contribution, I am not exactly sure whether you actually need them.

boost built fine on Darwin 9 PowerPC on our build server (but we don't build universal).

Interesting. Here is my explanation attempt:

Boost.Container uses allocation functions. Some of those, with functionality like posix_memalign, are not available in older OS such as Darwin 9, therefore Boost.Container comes with a compatibility layer built from C sources, particularly alloc_lib.c. This is compiled using the configured compiler with -x c -fexceptions and not the usual <cxxflags> as configured in user-config.jam but with <cflags> instead. So if you compile for a non-default architecture on Darwin 9 or earlier, such as PPC64, you should get architecture mismatches when linking. I got those until I patched the user-config.jam to include <cflags>\"${configure.cflags} [get_canonical_archflags cc]\".

[kencu (Ken)]

In b11faed22522e827899f5c54e9187031ddbd592b/macports-ports (master):

boost: add asm and cflags to build

this passes the arch flags needed to enable +universal
closes: #58846
see: #55857

[michaelld (Michael Dickens)]

Thanks @kencu!