rust @1.47.0: fails to build on Big Sur

[5nefarious (Arvin Ignaci)]

I'm trying to create a package and it looks like rust is one of the sub-dependencies (chromaprint > ffmpeg > librsvg > rust). However, it fails to build on the darwin_20 platform, which is what I'm currently running. Looking at the log output, it seems like it's failing to link an FFI library:

:info:build    Compiling regex v1.3.9
:info:build    Compiling globset v0.4.5
:info:build    Compiling ignore v0.4.16
:info:build    Compiling toml v0.5.6
:info:build    Compiling bootstrap v0.0.0 (/opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src/src/bootstrap)
:info:build     Finished dev [unoptimized + debuginfo] target(s) in 1m 02s
:info:build running: /opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src/build/bootstrap/debug/bootstrap build --stage 2 -v -j8
:info:build dyld: Library not loaded: /opt/local/lib/libffi.6.dylib
:info:build   Referenced from: /opt/local/libexec/llvm-9.0/lib/libLLVM.dylib
:info:build   Reason: image not found
:info:build thread 'main' panicked at 'command did not execute successfully: "/opt/local/libexec/llvm-9.0/bin/llvm-config" "--bindir"
:info:build expected success, got: signal: 6', src/build_helper/lib.rs:139:9
:info:build note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
:info:build Traceback (most recent call last):
:info:build   File "/opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src/src/bootstrap/bootstrap.py", line 1026, in <module>
:info:build     main()
:info:build   File "/opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src/src/bootstrap/bootstrap.py", line 1009, in main
:info:build     bootstrap(help_triggered)
:info:build   File "/opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src/src/bootstrap/bootstrap.py", line 995, in bootstrap
:info:build     run(args, env=env, verbose=build.verbose)
:info:build   File "/opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src/src/bootstrap/bootstrap.py", line 143, in run
:info:build     raise RuntimeError(err)
:info:build RuntimeError: failed to run: /opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src/build/bootstrap/debug/bootstrap build --stage 2 -v -j8
:info:build make: *** [all] Error 1
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_mports_macports-ports_lang_rust/rust/work/rustc-1.47.0-src" && /usr/bin/make -j8 -w all VERBOSE=1 BOOTSTRAP_ARGS="-v -j8" 
:info:build Exit code: 2
:error:build Failed to build rust: command execution failed
:debug:build Error code: CHILDSTATUS 81677 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$callback {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec -callback portprogress::target_progress_callback build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_mports_macports-ports_lang_rust/rust/main.log for details.

I checked and found that there is no /opt/local/lib/libffi.6.dylib, only /opt/local/lib/libffi.7.dylib. Is this something that needs to be fixed in MacPorts or an upstream problem?

macOS 11.0.1 Beta (20B5012d)
Apple clang version 12.0.0 (clang-1200.0.32.4)
Target: x86_64-apple-darwin20.1.0

[ryandesign (Ryan Carsten Schmidt)]

The port that provides /opt/local/libexec/llvm-9.0/lib/libLLVM.dylib (llvm-9.0) needed to have had its revision increased after the port that provided /opt/local/lib/libffi.6.dylib (libffi) was updated to a version that instead provided libffi.7.dylib. And that did happen 2 months ago in [bd5d6800828a3dcda1b65f3999fa748a365b168e/macports-ports]. When you run sudo port sync and port outdated, you should see that llvm-9.0 is outdated; you should then upgrade it. Or if that does not happen, then when you run sudo port rev-upgrade (and rev-upgrade also automatically runs after every port installation) it should tell you llvm-9.0 is broken and should rebuild it, which will also fix this problem.

[5nefarious (Arvin Ignaci)]

Replying to ryandesign:

The port that provides /opt/local/libexec/llvm-9.0/lib/libLLVM.dylib (llvm-9.0) needed to have had its revision increased after the port that provided /opt/local/lib/libffi.6.dylib (libffi) was updated to a version that instead provided libffi.7.dylib. And that did happen 2 months ago in [bd5d6800828a3dcda1b65f3999fa748a365b168e/macports-ports]. When you run sudo port sync and port outdated, you should see that llvm-9.0 is outdated; you should then upgrade it. Or if that does not happen, then when you run sudo port rev-upgrade (and rev-upgrade also automatically runs after every port installation) it should tell you llvm-9.0 is broken and should rebuild it, which will also fix this problem.

Thanks, Ryan. I'm currently unable to run port upgrade outdated, as qt5-qtbase is failing to build. It seems like the macOS 11 SDK is not being detected, as discussed in #61103. I guess I'll look into fixing that first.

[kencu (Ken)]

You could deactivate qt5-qtbase, do your upgrade outdated to fix llvm, and then reactivate it again and you'll be back to where you are now, but with a working llvm and other ports updated.

You can also just upgrade certain ports like llvm

sudo port -v upgrade llvm-9.0

etc

You can force MacPorts to do your bidding in just about any way you need, rather than be a slave to it.

[5nefarious (Arvin Ignaci)]

Replying to kencu: For some reason, even running port upgrade llvm-9.0, it's still trying to build qt5-qtbase.

[ryandesign (Ryan Carsten Schmidt)]

llvm-9.0 doesn't depend on qt5-qtbase. The only thing that could have happened is that you told MacPorts to upgrade llvm-9.0, MacPorts did upgrade it or though it was already up to date, MacPorts then ran rev-upgrade as it does after every installation or upgrade, found qt5-qtbase to be broken and tried to rebuild it, which failed for some reason.

What output do you get when you run:

port installed llvm-9.0 libffi

Depending on the output, we can advise you how to fix your llvm-9.0.

[5nefarious (Arvin Ignaci)]

Replying to ryandesign:

% port installed llvm-9.0 libffi
The following ports are currently installed:
  libffi @3.3_1 (active)
  llvm-9.0 @9.0.1_0 (active)

[ryandesign (Ryan Carsten Schmidt)]

Ok, so your llvm-9.0 is indeed simply out of date. The current version as of 2 months ago is 9.0.1_1. It should be shown as such in port outdated llvm-9.0. What exactly is the output of sudo port upgrade llvm-9.0?

[5nefarious (Arvin Ignaci)]

Replying to ryandesign:

% sudo port upgrade llvm-9.0
Password:
Warning: The macOS 11.0 SDK does not appear to be installed. Ports may not build correctly.
Warning: You can install it as part of the Xcode Command Line Tools package by running `xcode-select --install'.
--->  Computing dependencies for qt5-qtbase
--->  Fetching archive for qt5-qtbase
--->  Attempting to fetch qt5-qtbase-5.14.2_1+openssl.darwin_20.x86_64.tbz2 from https://packages.macports.org/qt5-qtbase
--->  Attempting to fetch qt5-qtbase-5.14.2_1+openssl.darwin_20.x86_64.tbz2 from https://ywg.ca.packages.macports.org/mirror/macports/packages/qt5/qt5-qtbase
--->  Attempting to fetch qt5-qtbase-5.14.2_1+openssl.darwin_20.x86_64.tbz2 from https://mse.uk.packages.macports.org/qt5-qtbase
--->  Building qt5-qtbase
Error: Failed to build qt5-qtbase: command execution failed
Error: See /opt/local/var/macports/logs/_opt_mports_macports-ports_aqua_qt5/qt5-qtbase/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.

[ryandesign (Ryan Carsten Schmidt)]

I wasn't expecting to see that.

As far as I can determine, the only mention of "qt" anywhere in the portfiles that llvm-9.0 depends on is in the cmake portfile, which has optional qt support in a variant:

$ port variants cmake
cmake has the variants:
   docs: Build documentation: HTML and manpages
   gui: Build Qt-based cmake-gui
   python35: Build documentation using Sphinx from Python 3.5
     * conflicts with python36 python37 python38
     * requires docs
   python36: Build documentation using Sphinx from Python 3.6
     * conflicts with python35 python37 python38
     * requires docs
   python37: Build documentation using Sphinx from Python 3.7
     * conflicts with python35 python36 python38
     * requires docs
   python38: Build documentation using Sphinx from Python 3.8
     * conflicts with python35 python36 python37
     * requires docs
   qt4: Build Qt GUI using Qt4
     * conflicts with qt5
     * requires gui
   qt5: Build Qt GUI using Qt5
     * conflicts with qt4
     * requires gui
   universal: Build for multiple architectures

Have you installed cmake with the gui variant? If so, try reinstalling it without that variant.

[5nefarious (Arvin Ignaci)]

Log for failing clang-9.0 build

[5nefarious (Arvin Ignaci)]

Replying to ryandesign:

It looks like I did build CMake with +gui +qt5. I was able to update LLVM, but Clang seems to be failing now.

[5nefarious (Arvin Ignaci)]

Build error log for clang-9.0

[ryandesign (Ryan Carsten Schmidt)]

Ok, the main.log says the problem is:

projects/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_posix.cc:169:31: error: invalid application of 'sizeof' to an incomplete type 'struct stat64'
  unsigned struct_stat64_sz = sizeof(struct stat64);
                              ^     ~~~~~~~~~~~~~~~

I haven't seen this before, but I see that you are trying to build a 3-way universal binary for arm64, i386 and x86_64, which you may be the first person to attempt, so it's not surprising you may run into some issues. 32-bit is dead as of macOS 10.15, so I suggest removing i386 from your universal_archs in macports.conf.

[kencu (Ken)]

It looks like you're trying to build against one of the iphone SDKs...that is not likely going to work out. llvm-9.0 will try to build libclangrt.a against all the archs that are supported on a given system, so presumably that is where this mess is coming from.

this is going to take some work. I admire your pluck, but -- building rust on a new OS version is not for the timid...

You most likely want librsvg? If so, there is a way to build the c++ version of that rather than the rust version, and that would be much easier at this juncture. Just edit the librsvg Portfile and force it to the c++ version -- you'll see where easily enough.

If nothing I'm saying makes any sense, then you're a little too far out on a limb, and you'll have to wait for MacPorts to catch up with BigSur and ARM and all the coming hoo-hah.

[5nefarious (Arvin Ignaci)]

Replying to kencu:

Well, it looks like I may have bitten off a bit more than I can chew here. I don't have a lot of time to debug all of this right now, so I may have to pick up on this later in December. I would like to get the Rust build working, if possible. Otherwise, I may just give up and try the C++ version. If all else fails, I guess I'll wait and see if the situation slowly improves after Big Sur's release.

Thanks for all of your help! I definitely wouldn't have figured all of this out without your input.