zstd @1.4.5_0+universal: ar: libzstd.a: Inappropriate file type or format

[thetrial (alabay)]

On two machines with El Capitan the build failed.

Excerpt from the log:

:info:build compiling dynamic library 1.4.5
:info:build creating pkgconfig
:info:build compiling static library
:info:build warning: /Library/Developer/CommandLineTools/usr/bin/ranlib: archive library: libzstd.a will be fat and ar(1) will not be able to operate on it
:info:build compiling static library
:info:build ar: libzstd.a is a fat file (use libtool(1) or lipo(1) and ar(1) on it)
:info:build ar: libzstd.a: Inappropriate file type or format
:info:build make[1]: *** [libzstd.a] Error 1
:info:build make[1]: *** Waiting for unfinished jobs....
:info:build creating versioned links
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_zstd/zstd/work/zstd-1.4.5/lib'
:info:build make: *** [lib] Error 2
:info:build make: *** Waiting for unfinished jobs....
:info:build creating versioned links
:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_zstd/zstd/work/zstd-1.4.5/lib'
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_zstd/zstd/work/zstd-1.4.5'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_zstd/zstd/work/zstd-1.4.5" && /usr/bin/make -j4 -w allmost 
:info:build Exit code: 2
:error:build Failed to build zstd: command execution failed
:debug:build Error code: CHILDSTATUS 18073 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"

[JDLH (Jim DeLaHunt)]

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_archivers_zstd/zstd/main.log from macOS 10.13.6 High Sierra

[JDLH (Jim DeLaHunt)]

Also observed on macOS 10.13.6 High Sierra. I've attached the …/zstd/main.log file. I too see the same diagnostics:

:info:build compiling dynamic library 1.4.5
:info:build creating pkgconfig
:info:build compiling static library
:info:build warning: /opt/local/bin/ranlib: archive library: libzstd.a will be fat and ar(1) will not be able to operate on it
:info:build compiling static library
:info:build ar: libzstd.a is a fat file (use libtool(1) or lipo(1) and ar(1) on it)
:info:build ar: libzstd.a: Inappropriate file type or format
:info:build make[1]: *** [libzstd.a] Error 1
:info:build make[1]: *** Waiting for unfinished jobs....

I see multiple instances of another warning message:

:info:build ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)

I'm not sure if it is related to the primary failure.

[ryandesign (Ryan Carsten Schmidt)]

Confirmed. This is a regression from zstd @1.4.4 which built fine with the +universal variant.

This should be ​reported to the developers of zstd.

[kencu (Ken)]

You might be able to use cctools ranlib/ar/libtool instead...they are often more forgiving of universal builds.

Also, FWIW, I have disabled that warning:

:info:build ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)

in MacPorts ld64, so it will not pass it on, which may be helpful.

You can make Xcode's ld64 ignore the warning with -Wl,-w or some similar way of passing -w as a linker flag.

[thetrial (alabay)]

I wrote that issue to ​https://github.com/facebook/zstd/issues/2169 and made a reference back here. So what to do now? I can’t update my installations anymore.

[kencu (Ken)]

One interesting tidbit is that the universal build proceeded without any trouble on 10.6.8:

$ port -v installed zstd
The following ports are currently installed:
  zstd @1.4.5_0+universal (active) platform='darwin 10' archs='i386 x86_64' date='2020-05-23T17:00:52-0700'

and the apparently troublesome file is indeed fat

$ file /opt/local/lib/libzstd.a
/opt/local/lib/libzstd.a: Mach-O universal binary with 2 architectures: [x86_64:current ar archive random library] [i386:current ar archive random library]

This suggests it might just be the linker warning that causes trouble, but to be tested...

[thetrial (alabay)]

This is, what the developer wrote:

»As we develop and test a lot on macosx systems, it's worth noting that no similar issue has ever been experienced with one of our systems.

So I guess the issue seems related to MacPorts itself, which we don't usually employ, or even some local setting.«

[thetrial (alabay)]

Installing zstd works. Installing zstd +universal deosn’t. In the first case 21 files are broken. And macports again starts to try to build zstd +universal … because of tiff. So it’s a catch-22.

[kencu (Ken)]

Replying to thetrial:

Installing zstd works. Installing zstd +universal deosn’t.

+universal does work on 10.6.8. as I showed two boxes up

So it’s a catch-22.

I'll try to fix this for you today. I think I know the problem.

[thetrial (alabay)]

Thank you. It seems you you already have fixed something? Now it ran through and built. Shouldn’t we mention a point at the zstd-Page at Github?

[kencu (Ken)]

It's still broken, at least for me, building on 10.13 +universal fails.

There have been quite a few changes in lib/Makefile in 1.4.5. The last version @1.4.4 gave the warnings building universal, but didn't error:

compiling dynamic library 1.4.4
compiling static library
warning: /opt/local/bin/ranlib: archive library: libzstd.a will be fat and ar(1) will not be able to operate on it
ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)
creating versioned links
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_macports-ports_archivers_zstd/zstd/work/zstd-1.4.4/lib'

[thetrial (alabay)]

Why did it work here this time?

Are you going to discuss that with the developer? I think you oversee the issue better than me.

[kencu (Ken)]

but sure enough, as he says, building it +universal outside MacPorts manually does indeed work fine. So it is something we're doing. Sometimes, it appears, and sometimes not.

wget https://github.com/facebook/zstd/releases/download/v1.4.5/zstd-1.4.5.tar.gz
tar xzvf zstd-1.4.5.tar.gz
cd zstd*
CC="clang -arch i386 -arch x86_64" CXX="clang++ -arch i386 -arch x86_64" make allmost  V=1

[kencu (Ken)]

but building it outside MacPorts with 16 parallel jobs fails. So that's likely the problem now.

[kencu (Ken)]

In 1436fb5c3f907ef60cdbf70741841b160a6c8f17/macports-ports (master):

zstd: disable parallel building

at present, parallel building causes
universal builds to fail on most
attempts

closes: #60542

[JDLH (Jim DeLaHunt)]

This fix was effective for me on macOS 10.13.6 High Sierra. Thank you!

[szhorvat (Szabolcs Horvát)]

In 10a47f4585bf7aa056fc8c0d4e68725ed398463a/macports-ports (master):

zstd: update to 1.5.0

• update to 1.5.0
• remove obsolete workaround for #60542