tor: build failure with Xcode/CLT 14: ld: archive member '__.SYMDEF SORTED' is not mach-o or llvm bitcode file for architecture x86_64

[i0ntempest]

:info:build ld: in src/test/libtor-testing.a(__.SYMDEF SORTED), archive member '__.SYMDEF SORTED' with length 496 is not mach-o or llvm bitcode file 'src/test/libtor-testing.a' for architecture x86_64
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build ld: in src/test/libtor-testing.a(__.SYMDEF SORTED), archive member '__.SYMDEF SORTED' with length 496 is not mach-o or llvm bitcode file 'src/test/libtor-testing.a' for architecture x86_64
:info:build make[1]: *** [src/test/fuzz/fuzz-consensus] Error 1
:info:build ld: in src/test/libtor-testing.a(__.SYMDEF SORTED), archive member '__.SYMDEF SORTED' with length 496 is not mach-o or llvm bitcode file 'src/test/libtor-testing.a' for architecture x86_64
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build make[1]: clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build *** [src/test/fuzz/fuzz-diff] Error 1
:info:build make[1]: *** [src/test/fuzz/fuzz-addressPTR] Error 1
:info:build ld: in src/test/libtor-testing.a(__.SYMDEF SORTED), archive member '__.SYMDEF SORTED' with length 496 is not mach-o or llvm bitcode file 'src/test/libtor-testing.a' for architecture x86_64
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build ld: in src/test/libtor-testing.a(__.SYMDEF SORTED), archive member '__.SYMDEF SORTED' with length 496 is not mach-o or llvm bitcode file 'src/test/libtor-testing.a' for architecture x86_64
:info:build make[1]: *** [src/test/fuzz/fuzz-diff-apply] Error 1
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build make[1]: *** [src/test/fuzz/fuzz-extrainfo] Error 1
:info:build mv -f src/test/.deps/test-test_util.Tpo src/test/.deps/test-test_util.Po
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_security_tor/tor/work/tor-0.4.7.10'
:info:build make: *** [all] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_security_tor/tor/work/tor-0.4.7.10'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_security_tor/tor/work/tor-0.4.7.10" && /usr/bin/make -j16 -w all 
:info:build Exit code: 2

This is a prolem I've noticed with Xcode betas on macOS 13, but now on macOS 12 as well since Xcode 14 was released. Seems ld has changed in some way in this new version of Xcode, and is affecting not just this port.

[jmroot (Joshua Root)]

Many of the static archives are working fine, but libtor.a is not. The difference seems to be that it was created by scripts/build/combine_libs. I would take a look at what that script is doing.

[i0ntempest]

It runs ar and ranlib to combine a bunch of archives into one:

#!/bin/sh

set -e

TMPDIR="$(mktemp -d -t tor_lib_combining.XXXXXX)"
ORIGDIR="$(pwd)"

trap 'cd "$ORIGDIR" && rm -rf "$TMPDIR"' 0

abspath() {
    echo "$(cd "$(dirname "$1")" >/dev/null && pwd)/$(basename "$1")"
}

TARGET=$(abspath "$1")

shift

for input in "$@"; do
    cd "$ORIGDIR"
    abs=$(abspath "$input")
    dir="$TMPDIR"/$(basename "$input" .a)
    mkdir "$dir"
    cd "$dir" >/dev/null
    "${AR:-ar}" x "$abs"
done

cd "$TMPDIR" >/dev/null
"${AR:-ar}" "${ARFLAGS:-cru}" library.tmp.a ./*/**
"${RANLIB:-ranlib}" library.tmp.a
mv -f library.tmp.a "$TARGET"

I can't see what it's doing wrong. Maybe someone can?

[jmroot (Joshua Root)]

​https://github.com/MagicalBitcoin/libtor-sys/blob/master/libtor-src/patches/tor-0009-remove-symdef-sorted-libs.patch

[jmroot (Joshua Root)]

Looks like only adding *.o rather than all files in the directory would also work. I don't know if this behaviour would be considered a bug in ar, but I think it's not guaranteed that .o files will be the only thing in an archive in any case.

[i0ntempest]

Thanks for pointing me to the patch, I had to remove all the __.SYMDEF files too or it fails the same way with that file.
I don't know if archives created by ar are commonly used for other stuff, but if used as static libs I think restricting to just object files should be fine?

[i0ntempest]

In 4b4383b53db935a6dc2a6deb253cce1dfa735931/macports-ports (master):

tor: fix build with Xcode/CLT 14

Closes: #65838

[jmroot (Joshua Root)]

The __.SYMDEF or __.SYMDEF SORTED files are the archive table of contents as created by ar s/ranlib (see the docs on the -s and -a options in the libtool manual for the difference). Those are definitely supposed to exist as archive members, I just don't know if any special handling of them may have previously happened. I can see how there could be problems if you extract one archive with a __.SYMDEF and one with a __.SYMDEF SORTED and then add all their contents to a new archive. I guess only one is treated as a table of contents and then the other looks like an an invalid object file.