Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#62656 closed defect (fixed)

rust @1.51.0 does not build on 10.9: Dyld: lazy symbol binding failed: Symbol not found: _linkat (OS 10.9)

Reported by: cave-canem Owned by: Chris Jones <jonesc@…>
Priority: Normal Milestone:
Component: ports Version: 2.6.99
Keywords: mavericks Cc: herbygillot (Herby Gillot), Wowfunhappy (Jonathan), cjones051073 (Chris Jones)
Port: rust

Description (last modified by ryandesign (Ryan Carsten Schmidt))

I changed on line #93 of Portfile-rust ${os.major} < 14} to ${os.major} < 13, and tried to build "rust" with mp-clang-11.

Result:

dyld: lazy symbol binding failed: Symbol not found: _linkat
  Referenced from: /opt/MacPorts/var/macports/build/_opt_macports-ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo
  Expected in: /usr/lib/libSystem.B.dylib

The problem is that now the "mpv @ 0.33.1" and "youtube-dl @ 2021.04.01" ports (in particular) depend on the rav1e port, which depends on the "rust" port.

In other words, not only the rust port breaks, but other ports as well!

As a possible solution, can you add a static linking variant to the rust port?

See attached log with debug information.

See also link: https://github.com/rust-lang/libc/issues/2036

Attachments (4)

main.log.tar.gz (8.7 KB) - added by cave-canem 3 years ago.
main.log
new_main.log.tar.gz (8.5 KB) - added by cave-canem 3 years ago.
new_main.log
Screenshot 2021-04-15 в 0.03.08.png (80.4 KB) - added by cave-canem 3 years ago.
Screenshot 2021-04-15 в 0.03.08.png
rust-compiler-wrap_main.log (51.6 KB) - added by cave-canem 3 years ago.
rust-compiler-wrap_main.log

Download all attachments as: .zip

Change History (67)

Changed 3 years ago by cave-canem

Attachment: main.log.tar.gz added

main.log

comment:1 Changed 3 years ago by cave-canem

Description: modified (diff)

comment:2 Changed 3 years ago by cave-canem

Description: modified (diff)

comment:3 Changed 3 years ago by kencu (Ken)

rav1e was just removed from libheif for many systems. [aebe51e3cd62e27205f1a41df8ab0668395cf73b/macports-ports] and this should improve your immediate problem installing mpv once that rolls out to you. (We'll probably wind up removing it from more or all systems soon enough).

You can't static link rust to fix this.

(BTW, isn't it just wonderful that MacPorts supports Mavericks so well?! Please tell all your friends how great it is.)

Last edited 3 years ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:4 Changed 3 years ago by kencu (Ken)

Description: modified (diff)

comment:5 Changed 3 years ago by kencu (Ken)

Cc: cave-canem g5pw@… removed
Summary: rust @1.51.0 Dyld: lazy symbol binding failed: Symbol not found: _linkat (OS 10.9)rust @1.51.0 does not build on 10.9: Dyld: lazy symbol binding failed: Symbol not found: _linkat (OS 10.9)

comment:6 Changed 3 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: herbygillot added; herby.gillot@… removed
Description: modified (diff)
dyld: lazy symbol binding failed: Symbol not found: _linkat

I guess that's why the port doesn't allow you to install rust on 10.9. :)

comment:7 Changed 3 years ago by aeiouaeiouaeiouaeiouaeiouaeiou

Similar as #62639. It is worth trying to replace this string of code to:

if #[cfg(any(target_os = "vxworks", target_os = "redox", target_os = "android", target_os = "macos"))] {

comment:8 Changed 3 years ago by cave-canem

I tried to do what aeiouaeiouaeiouaeiouaeiouaeiou recommended: changed line #1085 in

/distfiles/rust/rustc-1.51.0-src.tar.gz/library/std/src/sys/unix/fs.rs

to

if # [cfg ( any (target_os = "vxworks", target_os = "redox", target_os = "android", target_os = "macos"))] {

Result:

dyld: lazy symbol binding failed: Symbol not found: _linkat
  Referenced from: /opt/MacPorts/var/macports/build/_opt_macports-ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _linkat
  Referenced from: /opt/MacPorts/var/macports/build/_opt_macports-ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo
  Expected in: /usr/lib/libSystem.B.dylib

See attached debug log file "new_main.log.tar.gz".

Changed 3 years ago by cave-canem

Attachment: new_main.log.tar.gz added

new_main.log

comment:9 Changed 3 years ago by kencu (Ken)

I believe rust downloads a prebuilt boot compiler, then builds itself.

The prebuilt boot compiler will still have the linkat error.

If you know how to insert libraries or generate a new temporary libSystem that links in legacysupport's linkat you might get it working. Look into ld64's export library feature, generate a new temporary libSystem somewhere safe, and change the downloaded rust bootstrap compiler to use it with dyld_library_path or install name tool.

Upstream would ideally fix this, though, otherwise this will be awkward at our level to workaround.

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:10 Changed 3 years ago by kencu (Ken)

here's a bit about insert libraries; the idea is you would try inserting legacysupport when running the bootstrap rust.

https://theevilbit.github.io/posts/dyld_insert_libraries_dylib_injection_in_macos_osx_deep_dive/

comment:11 Changed 3 years ago by kencu (Ken)

Well, I thought I would play around with this, on 10.9, and I did this:

From here <https://forge.rust-lang.org/infra/other-installation-methods.html> I downloaded this file to ~/

https://static.rust-lang.org/rustup/dist/x86_64-apple-darwin/rustup-init

Then had to chmod it

chmod 755 rustup-init

Then ran it:

./rustup-init

and everything installed just as it is supposed to, into $HOME/.cargo/bin.

So, then I added that to the PATH, and ran it rustc and got this:

$ ./rustc
Usage: rustc [OPTIONS] INPUT

Options:
    -h, --help          Display this message
        --cfg SPEC      Configure the compilation environment
    -L [KIND=]PATH      Add a directory to the library search path. The
                        optional KIND can be one of dependency, crate, native,
                        framework, or all (the default).
    -l [KIND=]NAME      Link the generated crate(s) to the specified native
                        library NAME. The optional KIND can be one of
                        static, framework, or dylib (the default).
... etc

SO -- that Just Seems To Work.

I then tried it with i386, and (although I didn't build anything) that seemed to work too.

So it appears we might have some work to do, I guess, on our ports. not 100% sure what the next step is.

comment:12 Changed 3 years ago by kencu (Ken)

but there is an SSL error on 10.9 trying to use cargo:

$ cargo search librsvg
    Updating crates.io index
error: failed to update registry `https://github.com/rust-lang/crates.io-index`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

so all is not necessarily roses yet.

Version 0, edited 3 years ago by kencu (Ken) (next)

comment:13 Changed 3 years ago by kencu (Ken)

10.7 doesn't work because rustup-init-i386 uses SSLCreateContext in the Security Framework, and that is 10.8 only.

comment:14 Changed 3 years ago by kencu (Ken)

To make cargo work on 10.9, you first make sure git is installed, then this magical incantation works:

export CARGO_NET_GIT_FETCH_WITH_CLI=true
RUSTFLAGS="-C link-args=/opt/local/lib/libMacportsLegacySupport.a" DYLD_FORCE_FLAT_NAMESPACE=1 DYLD_INSERT_LIBRARIES=/opt/local/lib/libMacportsLegacySupport.dylib cargo install ripgrep --verbose

and you get a very nice build of ripgrep with rust and it has libMacportsLegacySupport.a linked in statically, and (seems to) work:

$ uname -a
Darwin Kernel Version 13.4.0: Mon Jan 11 18:17:34 PST 2016; root:xnu-2422.115.15~1/RELEASE_X86_64 x86_64
$ rg
error: The following required arguments were not provided:
    <PATTERN>

USAGE:
    
    rg [OPTIONS] PATTERN [PATH ...]
    rg [OPTIONS] [-e PATTERN ...] [-f PATTERNFILE ...] [PATH ...]
    rg [OPTIONS] --files [PATH ...]
    rg [OPTIONS] --type-list
    command | rg [OPTIONS] PATTERN

For more information try --help

comment:15 Changed 3 years ago by kencu (Ken)

Everything I try to build (manually) on 10.9 with cargo/rust seems to build fine and works OK, AFAICT, including rav1e, using the above incantation.

$ rav1e
error: The following required arguments were not provided:
    <INPUT>
    --output <OUTPUT>

USAGE:
    rav1e <INPUT> --primaries <COLOR_PRIMARIES> --content-light <CONTENT_LIGHT> --keyint <KEYFRAME_INTERVAL> --limit <LIMIT> --mastering-display <MASTERING_DISPLAY> --matrix <MATRIX_COEFFICIENTS> --min-k

comment:16 Changed 3 years ago by kencu (Ken)

using the existing cargo it is possible to build a new cargo from source like this:

mkdir test
cd test
git clone https://github.com/rust-lang/cargo.git
cd cargo
export CARGO_NET_GIT_FETCH_WITH_CLI=true
RUSTFLAGS="-C link-args=/opt/local/lib/libMacportsLegacySupport.a" DYLD_FORCE_FLAT_NAMESPACE=1 DYLD_INSERT_LIBRARIES=/opt/local/lib/libMacportsLegacySupport.dylib OPENSSL_DIR=/opt/local cargo build --features curl-sys/force-system-lib-on-osx --verbose --release
cd target
cd release
./cargo

that seems unencumbered.

(I did have to deactivate libiconv for the link, as it found MacPorts version, but I'm sure that is surmountable). --> there are tickets all over about this issue when building the libgit part of cargo. An elegant solution looks like it would be appreciated.

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:17 Changed 3 years ago by cave-canem

Thanks Ken, I knew about this, so I wrote:

"As a possible solution, can you add a static linking variant to the rust port?"

When I did "cargo search librsvg" I got

cargo search librsvg
     Updating crates.io index
mdcat = "0.22.3" # cat for markdown: Show markdown documents in terminals

So, now in 10.9 there is rust, but it is not needed by itself, but as a tool for building other ports. It looks like I can't do this with rust in

~/.cargo/bin/rustc

. P.S.

Thanks especially for "ripgrep", a great tool.

comment:18 Changed 3 years ago by kencu (Ken)

"librsvg" is not something installable with cargo (when i tried) but "rsvg" and "rsvg-sys" are used by the "rav1e" binary which is installable by "cargo".

When you install "rav1e" it installs "rsvg" in the crates.

At least we know that with this library inserting method it can be forced to work on 10.9 (and probably 10.8, as it has the right SecurityFramework, but not (presently) on 10.7).

I'm not sure that I am feeling like spending the time to make the rust/cargo ports in macports do this dance -- if upstream are not game, then c'est la vie.

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:19 in reply to:  17 ; Changed 3 years ago by kencu (Ken)

Replying to cave-canem:

Thanks Ken, I knew about this, so I wrote:

"As a possible solution, can you add a static linking variant to the rust port?"

So there is no way to add a static linking variant to a downloaded rust port.

The symbols are in the executable (which is already 99% statically linked), and mortals can't really force them in there. (I actually think you can do it, with a decompiler/dissassembler, but ... not us).

So what you have to do is make the executable think it finds the symbols it wants. That is what the DYLD_INSERT_LIBRARIES feature does -- just jams another dylib into the mixture.

But this doesn't work either as it is, because rust/cargo expect the linkat symbol in libSystem.dylib. So that is how the flat namespace force helps us.

In this case.

The other method, generating a new libSystem.dylib using ld64's "export libraries" feature, is probably actually the better one, but more work, so I didn't bother.

comment:20 Changed 3 years ago by kencu (Ken)

what you can probably do is use the incantation above, as I did with cargo, to statically link in legacysupport to a new rustc. Perhaps that is what you were getting at.

Changed 3 years ago by cave-canem

Screenshot 2021-04-15 в 0.03.08.png

comment:21 Changed 3 years ago by cave-canem

Thanks for the answer, Ken.

You wrote:

"librsvg" is not something installable with cargo (when i tried) but "rsvg" and "rsvg-sys" are used by the "rav1e" binary which is installable by "cargo".

When you install "rav1e" it installs "rsvg" in the crates.

Immediately after installing rustup-init, I ran

cargo search librsvg

(without installing anything before this)

and got this result:

cargo search librsvg
     Updating crates.io index
mdcat = "0.22.3" # cat for markdown: Show markdown documents in terminals

See:

Screenshot 2021-04-15 в 0.03.08.png

You know better how to explain this fact.

comment:22 Changed 3 years ago by kencu (Ken)

"cargo search librsvg" gives me nothing as well. However "cargo search rsvg" is fruitful:

$ cargo search rsvg
rsvg = "0.4.0"                       # Rust bindings for the Rsvg library
rsvg-sys = "0.6.0"                   # Rust bindings for the Rsvg library (ffi)
mdcat = "0.22.3"                     # cat for markdown: Show markdown documents in terminals
rendersvg-meetmangukiya = "0.9.1"    # Original source and credits to :https://github.com/RazrFalcon/resvg/. Just published for e…

different names than what MacPorts uses, apparently.

and we have:

$ ls -la ~/.cargo/registry/cache/github.com-1ecc6299db9ec823/rsvg-0.4.0.crate
-rw-r--r--  1 me  staff  5110 13 Apr 21:11 ~/.cargo/registry/cache/github.com-1ecc6299db9ec823/rsvg-0.4.0.crate

$file ~/.cargo/registry/cache/github.com-1ecc6299db9ec823/rsvg-0.4.0.crate
~/.cargo/registry/cache/github.com-1ecc6299db9ec823/rsvg-0.4.0.crate: gzip compressed data, was "rsvg-0.4.0.crate", max compression

and here:

$ ls -la ~/.cargo/registry/src/github.com-1ecc6299db9ec823/rsvg-0.4.0
total 24
drwxr-xr-x    6 me  staff   204 13 Apr 21:11 .
drwxr-xr-x  195 me  staff  6630 13 Apr 22:37 ..
-rw-r--r--    1 me  staff     2 13 Apr 21:11 .cargo-ok
-rw-r--r--    1 me  staff  1288 13 Apr 21:11 Cargo.toml
-rw-r--r--    1 me  staff   595 19 Mar  2018 Cargo.toml.orig
drwxr-xr-x    7 me  staff   238 13 Apr 21:11 src
Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:23 Changed 3 years ago by kencu (Ken)

So this is not installing rust and cargo and associated ports with MacPorts, clearly.

What this is is demonstrating that it is indeed possible to run rust and cargo and associated software on 10.9 (and probably 10.8) with modest effort.

Someone could modify the macports' rust/cargo setup to make this work on 10.9 and 10.8 -- but it would be a pain to maintain this without upstream incorporating the support.

And -- the i686-apple-darwin target still exists as well, so for those interested in +universal with i386, that might be important.

comment:24 Changed 3 years ago by kencu (Ken)

$ rustc --print target-list | grep apple
aarch64-apple-darwin
aarch64-apple-ios
aarch64-apple-ios-macabi
aarch64-apple-tvos
armv7-apple-ios
armv7s-apple-ios
i386-apple-ios
i686-apple-darwin
x86_64-apple-darwin
x86_64-apple-ios
x86_64-apple-ios-macabi
x86_64-apple-tvos

comment:25 Changed 3 years ago by cave-canem

cargo search rsvg
rsvg = "0.4.0"                       # Rust bindings for the Rsvg library
rsvg-sys = "0.6.0"                   # Rust bindings for the Rsvg library (ffi)
mdcat = "0.22.3"                     # cat for markdown: Show markdown documents in terminals
rendersvg-meetmangukiya = "0.9.1"    # Original source and credits to :https://github.com/RazrFalcon/resvg/. Just published for e…
ls -la ~/.cargo/registry/cache/github.com-1ecc6299db9ec823/rsvg-0.4.0.crate
ls: /Users/ES/.cargo/registry/cache/github.com-1ecc6299db9ec823/rsvg-0.4.0.crate: No such file or directory
ls -la ~/.cargo/registry/cache/github.com-1ecc6299db9ec823 | grep "rsvg" && echo "Found" || echo "Not found"
Not found
ls -la ~/.cargo/registry/cache/github.com-1ecc6299db9ec823 | wc -l
      52
ls -la ~/.cargo/registry/src/github.com-1ecc6299db9ec823/rsvg-0.4.0
ls: /Users/ES/.cargo/registry/src/github.com-1ecc6299db9ec823/rsvg-0.4.0: No such file or directory
rustc --print target-list | grep apple
aarch64-apple-darwin
aarch64-apple-ios
aarch64-apple-ios-macabi
aarch64-apple-tvos
armv7-apple-ios
armv7s-apple-ios
i386-apple-ios
i686-apple-darwin
x86_64-apple-darwin
x86_64-apple-ios
x86_64-apple-ios-macabi
x86_64-apple-tvos

comment:26 Changed 3 years ago by cave-canem

date "+%s"; cargo search rsvg; date "+%s"; ls -la ~/.cargo/registry/cache/github.com-1ecc6299db9ec823 | grep "rsvg" && echo "Found" || echo "Not found"
1618498610
rsvg = "0.4.0"                       # Rust bindings for the Rsvg library
rsvg-sys = "0.6.0"                   # Rust bindings for the Rsvg library (ffi)
mdcat = "0.22.3"                     # cat for markdown: Show markdown documents in terminals
rendersvg-meetmangukiya = "0.9.1"    # Original source and credits to :https://github.com/RazrFalcon/resvg/. Just published for e…
1618498611
Not found

comment:27 Changed 3 years ago by kencu (Ken)

@cave-canem: I'm not exactly sure if you are trying to show me something here? I seem to have lost the thread.

If you would like to show rsvg installed on your system, you need to install something that calls for it, eg rav1e (with cargo, not MacPorts).

If you wonder when/if rust & cargo might be fixed to work on 10.8 and 10.9 with MacPorts, I'm not sure if it will be -- but the above shows it could be. Upstream rust needs to buy in better. Perhaps what they really want is someone like you to work with them to make sure that systems < 10.10 are properly supported! I'm sure if you volunteered to help them, test builds, etc, they would be responsive.

comment:28 Changed 3 years ago by Wowfunhappy (Jonathan)

Unless I'm misreading, upstream does plan to fix this and it's currently slated for the 1.49.0 milestone. https://github.com/rust-lang/rust/issues/80804.

comment:29 Changed 3 years ago by herbygillot (Herby Gillot)

Then that would mean the fix didn't work.

1.49.0 is an old version. Rust is at 1.51.0 today.

comment:30 Changed 3 years ago by kencu (Ken)

They would probably have the best success if they built rust against the MacOSX10.7.sdk, using -isysroot= and whatever in rust itself does that.

Otherwise it's hard to make sure it is working correctly.

Or just build it on a 10.7 VM.

comment:31 in reply to:  19 Changed 3 years ago by Wowfunhappy (Jonathan)

Replying to kencu:

Replying to cave-canem:

Thanks Ken, I knew about this, so I wrote:

"As a possible solution, can you add a static linking variant to the rust port?"

The symbols are in the executable (which is already 99% statically linked), and mortals can't really force them in there. (I actually think you can do it, with a decompiler/dissassembler, but ... not us).

I read this and my first thought was "can't I use install_name_tool"?

I wanted to play around, so I booted a clean 10.9 VM and built libMacportsLegacySupport.dylib. Then, I used optool to add libSystem.B.dylib as a sub-library of libMacportsLegacySupport.dylib:

optool install -c reexport -p /usr/lib/libSystem.B.dylib -t ~/Desktop/libMacportsLegacySupport.dylib

(I'm sure there's a way to do this at compile-time without optool, but I've been using optool.)

Next, I installed rust using rustup and went to create a new project.

curl -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env`
cargo new hello-rust
cd hello-rust/
cargo new
cargo run

Of course, cargo run gave me the expected Symbol not found: _linkat error. So I tried to use install_name_tool to swap out libSystem for LegacySupport:

install_name_tool -change /usr/lib/libSystem.B.dylib ~/Desktop/libMacportsLegacySupport.dylib /Users/jonathan/.cargo/bin/cargo

Unfortunately, this yielded:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: object: /Users/jonathan/.cargo/bin/cargo malformed object (unknown load command 10)

I'm not sure if this is the issue you ran into Ken, but it's not caused by some special Rust magic—Mavericks's install_name_tool was just too old. I copied the cargo binary to a 10.13 VM, re-ran the command, and copied the binary back to 10.9. In hindsight, I'm pretty sure I could have just installed an updated copy of the cctools port.

I had to also do the swap out on a couple of other binaries (but these all worked from within Mavericks):

install_name_tool -change /usr/lib/libSystem.B.dylib ~/Desktop/libMacportsLegacySupport.dylib /Users/jonathan/.rustup/toolchains/stable-x86_64-apple-darwin/bin/../lib/libstd-349f286494d73b18.dylib
install_name_tool -change /usr/lib/libSystem.B.dylib ~/Desktop/libMacportsLegacySupport.dylib /Users/jonathan/.rustup/toolchains/stable-x86_64-apple-darwin/bin/cargo

After that, cargo run started my hello world program without any additional incantations.

Now, what I wasn't able to do was install rav1e with cargo, but I suspect I just need to swap out libSystem for legacySupport in more binaries... I think? And if so, I don't think integrating this automatically would be so nuts.

---

P.S. To make cargo work, I installed https://jonathanalland.com/legacy-mac-proxy.html (disclaimer: my own package) and then set the environment variable https_proxy to localhost:3128. I really need to add this to the readme in my package—a lot of command line programs, including curl, won't use the proxy unless this is set.

Last edited 3 years ago by Wowfunhappy (Jonathan) (previous) (diff)

comment:32 Changed 3 years ago by Wowfunhappy (Jonathan)

Cc: Wowfunhappy added

comment:33 Changed 3 years ago by kencu (Ken)

I was going to use your proxy, but I thought this was simpler:

sudo port install git
export CARGO_NET_GIT_FETCH_WITH_CLI=true

re: optool

I haven't tried it, but I was thinking just make a stub dylib libSystem.dylib with a one-liner build line that just reexports legacysupport and /usr/lib/libSystem.dylib. In the end, using DYLD_INSERT_LIBRARIES was just easier (10 seconds).

but there are always 9 ways to skin a cat.

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:34 Changed 3 years ago by kencu (Ken)

All of this kinda remains a PITA unless upstream supports < 10.10, though. We could use this approach to build ourselves a rust and cargo that statically link in legacysupport.a, possibly even in MacPorts, but it's an ugly set of workarounds...

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:35 Changed 3 years ago by kencu (Ken)

MacPorts' cctools port is in fact pretty much the most current set of cctools available, by the way. If you install the cctools port, then all the tools you call (install_name_tool, etc) will be up-to-date.

I have (so far) managed to get the current cctools building all the way back to Tiger PPC (no doubt because it's written in pre-C11 c code).

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:36 Changed 3 years ago by Wowfunhappy (Jonathan)

I also hope upstream just fixes this. I'm only concerned because it seems like Rust in particular is about to take over the world.

comment:37 Changed 3 years ago by kencu (Ken)

i read recently the main sponsor pulled all the funding (250 engineers) -- but there is a new foundation to drive it <https://foundation.rust-lang.org>.

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:38 Changed 3 years ago by cave-canem

Dear Ken, since

it seems like Rust in particular is about to take over the world

please endure this

PITA

and do it at least

an ugly set of workarounds

for MacPorts

Thanks!

P.S.

I'm sure if you volunteered to help them, test builds, etc, they would be responsive.

I'm ready to do it, only I don’t know how to offer this help to upstream (+ my disgusting knowledge of English ...)

comment:39 Changed 3 years ago by cjones051073 (Chris Jones)

Cc: cjones051073 added

comment:40 Changed 3 years ago by cjones051073 (Chris Jones)

So, as it happens before finding this ticket I had been looking into getting rust going on older systems, just for kicks..

Anyway, the approach I took was quite similar to what was discussed here. Thanks though for the tips w.r.t. optool above as that simplified what I was trying to do somewhat...

I've pushed what I have to

https://github.com/macports/macports-ports/pull/10740

Works just fine on 10.9. Older systems have issues ... building rust there fails when the build starts with

Executing:  cd "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src" && /usr/bin/make -j6 -w all VERBOSE=1 BOOTSTRAP_ARGS="-vv -j6" 
DEBUG: system:  cd "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src" && /usr/bin/make -j6 -w all VERBOSE=1 BOOTSTRAP_ARGS="-vv -j6" 
make: Entering directory `/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src'
/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/bin/python3 /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py build --stage 2 -vv -j6
running: /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo build --manifest-path /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/Cargo.toml --verbose --frozen
error: Unknown option
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 1140, in <module>
    main()
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 1123, in main
    bootstrap(help_triggered)
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 1096, in bootstrap
    build.build_bootstrap()
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 855, in build_bootstrap
    run(args, env=env, verbose=self.verbose)
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 153, in run
    raise RuntimeError(err)

doing a web search for 'cargo error unknown option' finds a number of discussions (mostly how hard debugging cargo/rust is) but nothing yet that helps me figure out what is wrong..

comment:41 Changed 3 years ago by Wowfunhappy (Jonathan)

Older systems have issues ... building rust there fails when the build starts with

I may have led you astray with optool—it builds but does not function on 10.7 (see also #62276), and it wouldn't surprise me if it was broken on 10.8 as well. If so, that could plausibly lead to weird errors.

Last edited 3 years ago by Wowfunhappy (Jonathan) (previous) (diff)

comment:42 Changed 3 years ago by Wowfunhappy (Jonathan)

Strike that, I really should actually test things before speaking about them. Optool works fine in 10.8, so that's not the issue.

comment:43 Changed 3 years ago by kencu (Ken)

On 10.7 there is a juicey warning about an untypedefed enum after the patch, but optool builds through and appears to work.

Last edited 3 years ago by kencu (Ken) (previous) (diff)

comment:44 Changed 3 years ago by cjones051073 (Chris Jones)

Yeah, I fixed the build on 10.7. This bit is working fine on these platforms.

comment:45 Changed 3 years ago by cjones051073 (Chris Jones)

The closest I can get to debugging is, if I manually run the commands on 10.{7,9} I get

10.9

MacVM109 /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src > sudo /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/bin/python3 /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py build --stage 2 -vv -j6
Password:
running: /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo build --manifest-path /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/Cargo.toml --verbose --frozen
environ({'TERM': 'xterm-256color', 'SSH_AUTH_SOCK': '/tmp/launch-FIBjVb/Listeners', '__CF_USER_TEXT_ENCODING': '0x0:0:0', 'PATH': '/opt/local/bin:/opt/local/sbin:/Users/chris/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin', 'EDITOR': 'nano', 'LANG': 'en_GB.UTF-8', 'PS1': '\\[\\e[32;1m\\]\\h\\[\\e[0m\\] \\[\\e[33;1m\\]\\w\\[\\e[0m\\] > ', 'HOME': '/Users/chris', 'DISPLAY': '/tmp/launch-Zvsph3/org.macports:0', 'SHELL': '/bin/bash', 'MAIL': '/var/mail/root', 'LOGNAME': 'root', 'USER': 'root', 'USERNAME': 'root', 'SUDO_COMMAND': '/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/bin/python3 /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py build --stage 2 -vv -j6', 'SUDO_USER': 'chris', 'SUDO_UID': '501', 'SUDO_GID': '20'})
   Compiling proc-macro2 v1.0.19
   Compiling unicode-xid v0.2.1
<snip>

10.7

MacVM10 /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src > sudo /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/bin/python3 /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py build --stage 2 -vv -j6
Password:
running: /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo build --manifest-path /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/Cargo.toml --verbose --frozen
error: Unknown option
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 1140, in <module>
    main()
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 1123, in main
    bootstrap(help_triggered)
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 1096, in bootstrap
    build.build_bootstrap()
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 855, in build_bootstrap
    run(args, env=env, verbose=self.verbose)
  File "/opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/bootstrap.py", line 153, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo build --manifest-path /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/Cargo.toml --verbose --frozen

Interestingly, if I try and directly run the cargo command, I get the same on both

MacVM109 /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src > sudo  /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/build/stage0-x86_64/bin/cargo build --manifest-path /opt/local/var/macports/build/_Users_chris_Projects_MacPorts_ports_lang_rust/rust/work/rustc-1.51.0-src/src/bootstrap/Cargo.toml --verbose --frozen
Password:
error: could not execute process `rustc -vV` (never executed)

Caused by:
  No such file or directory (os error 2)

clearly the bootstrap.py script is doing something to the env to fix the above

Last edited 3 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:46 Changed 3 years ago by cave-canem

for Chris Jones:

I replaced files "_resources/port1.0/group/cargo_fetch-1.0.tcl", "lang/rust/Portfile", "devel/cargo/Portfile" to files from "93c3837bf5d62b44e018e6060f250c4248e01b9c", but couldn't find "rust-compiler-wrap"

sudo port -v install rust
Password:
--->  Computing dependencies for rust.
Error: Dependency 'rust-compiler-wrap' not found.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port rust failed

comment:47 Changed 3 years ago by cjones051073 (Chris Jones)

Looks like you didn't correctly update the portindex. rust-compiler-wrap is a sub-port of rust.

comment:48 Changed 3 years ago by cjones051073 (Chris Jones)

I don't know exactly how you are managing your local clone of the ports tree, but updating you should see something like

Oberon ~/Projects/MacPorts/ports > sudo portindex
Password:
Creating port index in /Users/chris/Projects/MacPorts/ports
Adding port devel/cargo
Adding subport cargo-bootstrap
Adding port lang/rust
Adding subport rust-compiler-wrap
Adding subport rust-src

Total number of ports parsed:	5 
Ports successfully parsed:	5 
Ports failed:			0 
Up-to-date ports skipped:	26201

comment:49 Changed 3 years ago by cjones051073 (Chris Jones)

So just to document a bit where my limited tests have got to on the issues on < 10.9, the following is a small reproducer.

wget -c www.hep.phy.cam.ac.uk/~jonesc/macports/hello.tar.bz2 -O - | tar -jx
cd hello
PATH=./stage0-x86_64/bin:$PATH cargo run

working output should be

   Compiling hello v0.1.0 (/Users/chris/Downloads/hello)
    Finished dev [unoptimized + debuginfo] target(s) in 2.11s
     Running `target/debug/hello`

on < 10.9 I get the frustratingly cryptic

error: Unknown option

Been poking at this a bit, with various Dtrace utilities, but not really getting anywhere, so if anyone else has ideas ?

Last edited 3 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:51 Changed 3 years ago by Chris Jones <jonesc@…>

Owner: set to Chris Jones <jonesc@…>
Resolution: fixed
Status: newclosed

In f918bf6a32b7c6bb3d7fcfccf05741601de022d6/macports-ports (master):

cargo_fetch PG, rust, cargo: Add support for using legacysupport PG on older systems

  • Patch bootstrapping binaries to use legacysupport dylib for missing libSystem calls
  • Wrap underlying compiler commands used by rust/cargo to enforce use of legacy support flags

Closes: #62656

comment:52 Changed 3 years ago by cave-canem

for Chris Jones:

Dear Chris, thank you so much for the work done "add support for using legacysupport PG on older systems".

One very small speck: Sub-port "rust-compiler-wrap" is not built by any compiler other than the default compiler:

Error: Unsupported compiler: macports-clang
Error: Failed to destroot rust-compiler-wrap: can't read "llvm_ver": no such variable
Error: Unsupported compiler: clang
Error: Failed to destroot rust-compiler-wrap: can't read "llvm_ver": no such variable

Thanks again!

comment:53 Changed 3 years ago by cjones051073 (Chris Jones)

Sorry, but can you provide details on exactly how you generated those errors, as I cannot reproduce them here.

comment:54 Changed 3 years ago by cjones051073 (Chris Jones)

b.t.w. please first ensure you are fully up to date with your ports tree.

comment:55 Changed 3 years ago by cjones051073 (Chris Jones)

I believe the issue you are seeing is related to something I fixed in

https://github.com/macports/macports-ports/commit/c4892534a78e24acc2b60c95f2d4dbf39b28e6a9#diff-318d6530788a550962f58d2caf0dca8f3c23ed7107cf2efada2f6b486d2b813f

so if you are still seeing it your ports cannot be up to date.

comment:56 Changed 3 years ago by cave-canem

so if you are still seeing it your ports cannot be up to date.

date -u; port outdated
Wed Apr 28 04:50:21 UTC 2021
The following installed ports are outdated:
mesa                           19.0.8_0 < 19.0.8_1
mpv                            0.33.0_2 < 0.33.1_1
date -u; sudo port clean --work rust-compiler-wrap; sudo port -vd install rust-compiler-wrap
Wed Apr 28 04:56:46 UTC 2021
--->  Cleaning rust-compiler-wrap

See further in the attached file for a debug log of attempting to build "rust-compiler-wrap" with "mp-clang-devel" (clang-13).

Changed 3 years ago by cave-canem

Attachment: rust-compiler-wrap_main.log added

rust-compiler-wrap_main.log

comment:57 Changed 3 years ago by cave-canem

P.S.

The line from "macports.conf" causing this error:

default_compilers macports-clang macports-gcc clang

comment:58 Changed 3 years ago by cave-canem

P.P.S.

#!/usr/bin/env bash

pushd "`pwd`" > /dev/null

cd /Volumes/MacOS/opt/macports\-ports

sudo git config pull.rebase true

sudo git checkout -- ./ && \

sudo git pull && \

sudo port -d sync && \

port outdated || \
	echo "Ошибка обновления MacPorts"

popd > /dev/null
Last edited 3 years ago by cave-canem (previous) (diff)

comment:59 Changed 3 years ago by cjones051073 (Chris Jones)

Ah, I see. Your set up is a bit 'individual', and causing some problems. I'll take a look.

Last edited 3 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:60 Changed 3 years ago by cjones051073 (Chris Jones)

Fixed by https://github.com/macports/macports-ports/commit/7a04eb80e914077f852cfea9a805bd9f095a09e0

MacVM109 ~/Projects/MacPorts/ports >  sudo port install rust-compiler-wrap configure.compiler=macports-clang
--->  Computing dependencies for rust-compiler-wrap
--->  Fetching archive for rust-compiler-wrap
Warning: Your DNS servers incorrectly claim to know the address of nonexistent hosts. This may cause checksum mismatches for some ports. See this page for more information: <https://trac.macports.org/wiki/MisbehavingServers>
--->  Attempting to fetch rust-compiler-wrap-1.51.0_3.darwin_13.noarch.tbz2 from https://packages.macports.org/rust-compiler-wrap
--->  Attempting to fetch rust-compiler-wrap-1.51.0_3.darwin_13.noarch.tbz2 from https://mse.uk.packages.macports.org/rust-compiler-wrap
--->  Attempting to fetch rust-compiler-wrap-1.51.0_3.darwin_13.noarch.tbz2 from https://fra.de.packages.macports.org/rust-compiler-wrap
--->  Fetching distfiles for rust-compiler-wrap
--->  Verifying checksums for rust-compiler-wrap
--->  Extracting rust-compiler-wrap
--->  Configuring rust-compiler-wrap
--->  Building rust-compiler-wrap
--->  Staging rust-compiler-wrap into destroot
--->  Installing rust-compiler-wrap @1.51.0_3
--->  Activating rust-compiler-wrap @1.51.0_3
--->  Cleaning rust-compiler-wrap
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  No broken files found.                             
--->  No broken ports found.

comment:61 Changed 3 years ago by cjones051073 (Chris Jones)

b.t.w. I strongly recommend against running with the settings you are using. By forcing the compiler to be macports-clang across the board, which then maps to whatever you happen to haveport select'ed for it, means you are over-riding the sometimes carefully selected compiler requirements ports have. Do this makes it quite likely you will run into problems with ports using a compiler their maintainer did not expect. I very much suggest you remove this option and just let the ports pick themselves what compiler they wish to use.

comment:62 Changed 3 years ago by cave-canem

for Chris Jones:

Now everything builds without any errors.

Thanks again and special thanks your valuable recommendations!

comment:63 Changed 3 years ago by aeiouaeiouaeiouaeiouaeiouaeiou

This bug will finally be fixed in the upstream version (but for how long?).

Note: See TracTickets for help on using tickets.