Opened 22 months ago

Closed 8 months ago

#62655 closed defect (fixed)

rust @1.51.0_1 fails to build under macOS 11.2.3 :info:destroot = note: Undefined symbols for architecture arm64

Reported by: ehjmx Owned by: g5pw (Aljaž Srebrnič)
Priority: Normal Milestone:
Component: ports Version: 2.6.4
Keywords: bigsur arm64 Cc: herbygillot (Herby Gillot)
Port: rust

Description

My system: macOS 11.2.3 macports 2.6.4 Apple M1 (arm64) MacBookPro with 16GB RAM.

rust@1.51.0_1 builds under macOS 11.2.3 but fails during the "Staging rust into destroot" process with the following error:

--->  Staging rust into destroot
Error: Failed to destroot rust: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_rust/rust/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port rust failed

Accordin to the attached main.log the error occures during "compiling openssl-sys v0.9.58". Compiling openssl-sys v0.9.58 fails with:

:info:destroot   = note: Undefined symbols for architecture arm64: 

This failure leads to this error:

:info:destroot -------------
:info:destroot error: could not compile `rls`

and in the end to the stagging error:

info:destroot Exit code: 2
:error:destroot Failed to destroot rust: command execution failed
:debug:destroot Error code: CHILDSTATUS 84222 2
:debug:destroot Backtrace: command execution failed
:debug:destroot     while executing
:debug:destroot "system {*}$notty {*}$nice $fullcmdstring"
:debug:destroot     invoked from within
:debug:destroot "command_exec destroot"
:debug:destroot     (procedure "portdestroot::destroot_main" line 2)
:debug:destroot     invoked from within
:debug:destroot "$procedure $targetname"

Attachments (4)

main.log.zip (171.0 KB) - added by ehjmx 22 months ago.
compressed main.log of the most recent failure
main.log-only_arm64.zip (178.5 KB) - added by ehjmx 22 months ago.
main.log for the build with the non-universal openssl
main.log.2.zip (188.4 KB) - added by ehjmx 22 months ago.
main.log rust build with all dependencies beeing arm64
main.log-2021-04-11.zip (170.7 KB) - added by ehjmx 22 months ago.
main.log rust build with more dependencies beeing arm64

Download all attachments as: .zip

Change History (25)

Changed 22 months ago by ehjmx

Attachment: main.log.zip added

compressed main.log of the most recent failure

comment:1 Changed 22 months ago by mf2k (Frank Schima)

In the future, please add the port maintainer(s) to Cc (port info --maintainers rust), if any.

comment:2 Changed 22 months ago by mf2k (Frank Schima)

Cc: herbygillot added
Owner: set to g5pw
Status: newassigned

comment:3 Changed 22 months ago by herbygillot (Herby Gillot)

Can you share what the architecture is for your openssl installation?

The following command can show this:

port -v installed | grep openssl
Last edited 22 months ago by herbygillot (Herby Gillot) (previous) (diff)

comment:4 Changed 22 months ago by ehjmx

port -v installed | grep openssl

gives me

openssl @1.1.1k_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2021-03-26T08:20:05+0100'
openssl10 @1.0.2u_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2021-03-21T16:29:41+0100'

I need the openssl10 as universal for Powershell. I tried it without +universal but then powershell wouldn't let me login. Maybe I can rebuild openssl without the universal.

comment:5 Changed 22 months ago by herbygillot (Herby Gillot)

Well I would most certainly suggest building Rust against a regular non-universal version of OpenSSL.

At least just to check if it's the universal variant of it that is causing problems here.

comment:6 Changed 22 months ago by ehjmx

Building against a non-universal OpenSSL breaks rust earlier then building against the universal-binary.

I had to edit Row 1493 Col 22 of builder.rs

out of

 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap

Instead of

panic!(out);

it needs

panic!("{}", out)

Lets see for how long rust builds this time.

Changed 22 months ago by ehjmx

Attachment: main.log-only_arm64.zip added

main.log for the build with the non-universal openssl

comment:7 Changed 22 months ago by ehjmx

Here is the new output for

port -v installed | grep openssl
openssl @1.1.1k_0 (active) platform='darwin 20' archs='arm64' date='2021-04-07T22:39:07+0200'
openssl10 @1.0.2u_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2021-03-21T16:29:41+0100'

comment:8 Changed 22 months ago by herbygillot (Herby Gillot)

OK, according to the log, we now need to examine libssh/libssh2.

port -v installed | grep libssh

comment:9 Changed 22 months ago by ehjmx

port -v installed | grep libssh

gives me

  libssh2 @1.9.0_0 (active) platform='darwin 20' archs='arm64' date='2021-04-07T22:49:49+0200'

I did check the other dependencies and here are the results

port -v installed | grep pkgconfig

gives me

  pkgconfig @0.29.2_0+universal platform='darwin 20' archs='arm64 x86_64' date='2021-04-05T10:44:21+0200'
  pkgconfig @0.29.2_0 (active) platform='darwin 20' archs='arm64' date='2021-04-08T14:21:51+0200'
port -v installed | grep ninja

gives me

  ninja @1.10.2_0 (active) platform='darwin 20' archs='arm64' date='2021-04-04T13:07:29+0200'
port -v installed | grep cmake

gives me

cmake @3.19.7_0 (active) platform='darwin 20' archs='arm64' date='2021-04-07T22:43:39+0200'
port -v installed | grep git

gives me

git @2.31.1_0+credential_osxkeychain+diff_highlight+doc+pcre+perl5_28 (active) platform='darwin 20' archs='arm64' date='2021-04-07T22:47:27+0200
mesa @19.0.8_1+osmesa+python27 (active) platform='darwin 20' archs='arm64' date='2021-03-21T16:47:27+0100'
  python2_select @0.0_3 (active) platform='darwin 20' archs='noarch' date='2021-03-21T18:01:13+0100'
  python3_select @0.0_2 (active) platform='darwin 20' archs='noarch' date='2021-03-21T16:20:10+0100'
  python27 @2.7.18_2 (active) platform='darwin 20' archs='arm64' date='2021-04-07T22:44:20+0200'
  python38 @3.8.9_0 (active) platform='darwin 20' archs='arm64' date='2021-04-05T08:19:52+0200'
  python39 @3.9.4_0 (active) platform='darwin 20' archs='arm64' date='2021-04-07T22:40:50+0200'
  python_select @0.3_9 (active) platform='darwin 20' archs='noarch' date='2021-03-21T16:20:10+0100'
  xorg-libxcb @1.14_0+python39 (active) platform='darwin 20' archs='arm64' date='2021-04-08T16:00:29+0200'
  xorg-xcb-proto @1.14.1_0+python39 (active) platform='darwin 20' archs='noarch' date='2021-03-21T16:24:05+0100'
  youtube-dl @2021.03.14_0+ffmpeg+python39 (active) platform='darwin 20' archs='noarch' date='2021-03-21T17:13:54+0100'

cctools gives

cctools @949.0.1_0+xcode (active) platform='darwin 20' archs='noarch' date='2021-04-04T13:07:29+0200'

For me my whole build environment looks arm64 only. Could it be that rust calls the architecture aarch64-apple-darwin while the clang tries to link it with arm64?

Attached you find the latest main.log with all the dependencies beeing arm64

Changed 22 months ago by ehjmx

Attachment: main.log.2.zip added

main.log rust build with all dependencies beeing arm64

comment:10 Changed 22 months ago by ehjmx

Here is the Tracking issue for supporting macOS on Apple Silicon in the rust githup repo.

comment:11 Changed 22 months ago by kencu (Ken)

IF you don't really care about rust on arm64, and I will guess you don't at present care about it, I assume you are trying to install something that wants libheif and libheif now includes rav1e, and that requires rust.

rust is a bear, and builds on reasonably few macOS systems. So this has caused some wreckage.

If you edit the libheif Portfile:

bbedit `port file libheif`

and remove the need for rav1e, you should be good to go.

comment:12 Changed 22 months ago by ehjmx

Thanks. I removed the rav1e depency. Now all my outdated ports youtube-dl, libbluray, libheif and gmim3 got updateed. You guessed right I don't care about rust as a port. I just got it as a depency for a depency.

@kencu: May I ask why libheif has rav1e, dav1d and aom. Isn't aom an encoder and decoder for av1? Why add another av1 encoder and decoder?

For the moment it looks like the solution in this issue is suitable for arm64 as well and may be considered until rust builds on arm64. In an ideal world it would be fine to have rust as a pre build portfile for arm64 if it is indeed such a bear.

comment:13 Changed 22 months ago by kencu (Ken)

libheif was picking these up and using them if they were installed, so they were explicitly added for consistent builds.

but then as always,all the fallout from such changes comes to light, so it is in the re-review stage.

rust can't build universal, so rav1e will likely have to be removed from all systems < darwin 18 for that reason.

and rust can't build on arm64 so it will have to be removed from there too, unless some new rust comes along that does build on arm64. Even so, it won't build universal there most likely.

IMHO rust is such a bear that it should never be routinely used for any software that has it as an optional dep -- but MacPorts is more of a loose school of fish than an orchestra, so this evolves.

comment:14 Changed 22 months ago by herbygillot (Herby Gillot)

and rust can't build on arm64

Just curious, the current version of Rust in MacPorts (1.51.0) does indeed build for arm64... perhaps I'm missing something?

arm64 support was first added for Rust in this commit: https://github.com/macports/macports-ports/commit/4c5878867bdeb08e5d8c394a76598343cefd081a#diff-318d6530788a550962f58d2caf0dca8f3c23ed7107cf2efada2f6b486d2b813f

comment:15 Changed 22 months ago by kencu (Ken)

Ah -- perhaps I was too trusting, with report after report above indicating it would not build, I did not go and quadruple-check.

This did not look encouraging <https://ports.macports.org/port/rust/summary> but now I do see the last version here: <http://packages.macports.org/rust/> (but with no good idea if it actually works).

I'm just wondering about when to buy my own arm64 Mac to know all this first hand -- but the pickings are not yet attractive enough for the outlay for a machine worth having, for me at least, at present. I do have access to a remote Apple Silicone system, but I never tried building rust on it out of pity for the owner :>.

Last edited 22 months ago by kencu (Ken) (previous) (diff)

comment:16 in reply to:  14 Changed 22 months ago by ehjmx

Replying to herbygillot:

and rust can't build on arm64

Just curious, the current version of Rust in MacPorts (1.51.0) does indeed build for arm64... perhaps I'm missing something?

I just tried to build rust 1.51.0 on arm64 and it still fails. I will attach the main.log so that you can take a look.

I again cleaned up some universal binaries and now glib2 +x11 and libffi are arm64 only.

As already stated I need the openssl@1.0 as universal binary for Powershell. So if this breaks the build I'm fine with it.

However I can confirm that rust 1.50 did indeed install fine on my M1 MacBookPro.

Would it help if I update rustc to 1.51 in the portfile? Is a sudo port clean rust sufficient or should I clean some directories manually?

Changed 22 months ago by ehjmx

Attachment: main.log-2021-04-11.zip added

main.log rust build with more dependencies beeing arm64

comment:17 Changed 22 months ago by herbygillot (Herby Gillot)

OK, so I just gained access temporarily to an M1 machine and was able to install Rust 1.51.0 from scratch without issue:

--->  No broken files found.
--->  No broken ports found.
--->  Some of the ports you installed have notes:
  libpsl has the following notes:
    libpsl API documentation is provided by the port 'libpsl-docs'.
  python39 has the following notes:
    To make this the default Python or Python 3 (i.e., the version run by the 'python' or 'python3' commands), run one or both of:

        sudo port select --set python python39
        sudo port select --set python3 python39
% rustc --version
rustc 1.51.0
% file `which rustc`
/opt/local/bin/rustc: Mach-O 64-bit executable arm64
% otool -L `which rustc`
/opt/local/bin/rustc:
        @rpath/librustc_driver-94d9528ffe19d5b1.dylib (compatibility version 0.0.0, current version 0.0.0)
        @rpath/libstd-fb33074c74598e54.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 904.4.0)
        /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)

According to your log file, the problem you're having is in compiling rls; Rust itself (rustc) and its standard library have built just fine.

rls requires OpenSSL, but on your system for some reason, it's also looking for symbols in libssh2. I don't know why, but perhaps it may have to do with custom variants you've set while installing ports on your system.

I would first try to rebuild libssh2 from scratch.

sudo port uninstall libssh2
sudo port build libssh2
sudo port install libssh2

The ideal thing afterwards would be to rebuild openssl as well.

comment:18 Changed 22 months ago by herbygillot (Herby Gillot)

@kencu - the MacPorts website is still somewhat broken, and has been since the buildbot issues Ryan was having during the storms and power issues in Texas.

Build information is still inaccurately represented and out-of-date for a lot of ports, and on top of that, there is currently no arm64 buildbot, so there is no information on whether recent ports and versions are building properly on arm64.

comment:19 in reply to:  17 ; Changed 22 months ago by ehjmx

Replying to herbygillot:

According to your log file, the problem you're having is in compiling rls; Rust itself (rustc) and its standard library have built just fine.

rls requires OpenSSL, but on your system for some reason, it's also looking for symbols in libssh2. I don't know why, but perhaps it may have to do with custom variants you've set while installing ports on your system.

I would first try to rebuild libssh2 from scratch.

sudo port uninstall libssh2
sudo port build libssh2
sudo port install libssh2

The ideal thing afterwards would be to rebuild openssl as well.

I did as suggested and did the same three steps for openssl. Still building rls fails. As I don't see any difference in the main log before and after doing the rebuilds should I still attach it?

What baffles me is, that after I did a

sudo port clean rust

today during the

sudo port upgrade outdated

rust, rustc 1.51 and rust, rustc, cargo 1.50 where downloaded

--->  Computing dependencies for rust
--->  Fetching archive for rust
--->  Attempting to fetch rust-1.51.0_1.darwin_20.arm64.tbz2 from https://packages.macports.org/rust
--->  Attempting to fetch rust-1.51.0_1.darwin_20.arm64.tbz2 from https://lil.fr.packages.macports.org/rust
--->  Attempting to fetch rust-1.51.0_1.darwin_20.arm64.tbz2 from https://cph.dk.packages.macports.org/rust
--->  Fetching distfiles for rust
--->  Attempting to fetch rustc-1.51.0-src.tar.gz from https://static.rust-lang.org/dist
--->  Attempting to fetch rust-std-1.50.0-aarch64-apple-darwin.tar.gz from https://static.rust-lang.org/dist
--->  Attempting to fetch rustc-1.50.0-aarch64-apple-darwin.tar.gz from https://static.rust-lang.org/dist
--->  Attempting to fetch cargo-1.50.0-aarch64-apple-darwin.tar.gz from https://static.rust-lang.org/dist

There is definitly something weired with my macports installation which causes the build failure for rls.

Last edited 22 months ago by ehjmx (previous) (diff)

comment:20 in reply to:  19 Changed 22 months ago by herbygillot (Herby Gillot)

Replying to ehjmx:

Replying to herbygillot:

According to your log file, the problem you're having is in compiling rls; Rust itself (rustc) and its standard library have built just fine.

rls requires OpenSSL, but on your system for some reason, it's also looking for symbols in libssh2. I don't know why, but perhaps it may have to do with custom variants you've set while installing ports on your system.

I would first try to rebuild libssh2 from scratch.

sudo port uninstall libssh2
sudo port build libssh2
sudo port install libssh2

The ideal thing afterwards would be to rebuild openssl as well.

I did as suggested and did the same three steps for openssl. Still building rls fails. As I don't see any difference in the main log before and after doing the rebuilds should I still attach it?

What baffles me is, that after I did a

sudo port clean rust

today during the

sudo port upgrade outdated

rust, rustc 1.51 and rust, rustc, cargo 1.50 where downloaded

--->  Computing dependencies for rust
--->  Fetching archive for rust
--->  Attempting to fetch rust-1.51.0_1.darwin_20.arm64.tbz2 from https://packages.macports.org/rust
--->  Attempting to fetch rust-1.51.0_1.darwin_20.arm64.tbz2 from https://lil.fr.packages.macports.org/rust
--->  Attempting to fetch rust-1.51.0_1.darwin_20.arm64.tbz2 from https://cph.dk.packages.macports.org/rust
--->  Fetching distfiles for rust
--->  Attempting to fetch rustc-1.51.0-src.tar.gz from https://static.rust-lang.org/dist
--->  Attempting to fetch rust-std-1.50.0-aarch64-apple-darwin.tar.gz from https://static.rust-lang.org/dist
--->  Attempting to fetch rustc-1.50.0-aarch64-apple-darwin.tar.gz from https://static.rust-lang.org/dist
--->  Attempting to fetch cargo-1.50.0-aarch64-apple-darwin.tar.gz from https://static.rust-lang.org/dist

There is definitly something weired with my macports installation which causes the build failure for rls.

So this is working as expected.

The way that Rust works, the previous release version is used to build the most recent release.

So to build release 1.51.0, version 1.50.0 is used. The build is a multi-phase process where the previous version Rust compiler, stdlib and cargo are used to build the latest Rust compiler, then that resulting compiler is then used to build the most recent standard lib and companion utilities.

Last edited 22 months ago by herbygillot (Herby Gillot) (previous) (diff)

comment:21 Changed 8 months ago by jmroot (Joshua Root)

Keywords: bigsur arm64 added; big sur removed
Resolution: fixed
Status: assignedclosed

The arm64 builds of rust @1.61.0 seem to have gone fine. Please reopen if you are still seeing this issue.

Note: See TracTickets for help on using tickets.