rust ports: link crashes across macOS releases; 10.6 failure for cargo-c blocking downstream ports

[mascguy (Christopher Nielsen)]

Link crashes are occurring for multiple ports, across macOS releases. They typically look something like this:

  = note: 0  0x106522b51  __assert_rtn + 144
1070	          1  0x106553063  ld::Fixup::Fixup(unsigned int, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 107
1071	          2  0x10652b4e9  mach_o::relocatable::Parser<x86_64>::addFixup(mach_o::relocatable::Parser<x86_64>::SourceLocation const&, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 33
1072	          3  0x106526751  mach_o::relocatable::Section<x86_64>::addRelocFixup(mach_o::relocatable::Parser<x86_64>&, macho_relocation_info<Pointer64<LittleEndian> > const*) + 1471
1073	          4  0x10654ebb7  mach_o::relocatable::Section<x86_64>::makeFixups(mach_o::relocatable::Parser<x86_64>&, mach_o::relocatable::Parser<x86_64>::CFI_CU_InfoArrays const&) + 91
1074	          5  0x10654a7c6  mach_o::relocatable::Parser<x86_64>::parse(mach_o::relocatable::ParserOptions const&) + 2034
1075	          6  0x10652e0fd  mach_o::relocatable::Parser<x86_64>::parse(unsigned char const*, unsigned long long, char const*, long, ld::File::Ordinal, mach_o::relocatable::ParserOptions const&) + 375
1076	          7  0x10655e9cc  archive::File<x86_64>::makeObjectFileForMember(archive::File<x86_64>::Entry const*) const + 758
1077	          8  0x10655e518  archive::File<x86_64>::justInTimeforEachAtom(char const*, ld::File::AtomHandler&) const + 122
1078	          9  0x10657419b  ld::tool::InputFiles::searchLibraries(char const*, bool, bool, bool, ld::File::AtomHandler&) const + 215
1079	          10  0x10657cc90  ld::tool::Resolver::resolveUndefines() + 160
1080	          11  0x10657ef63  ld::tool::Resolver::resolve() + 79
1081	          12  0x1065236c7  main + 689
1082	          13  0x10688e5fd  start + 1
1083	          A linker snapshot was created at:
1084	          /tmp/build_script_build-100b7420c9437e43-2022-06-04-173157.ld-snapshot
1085	          ld: Assertion failed: (name != NULL), function Fixup, file /SourceCache/ld64/ld64-241.9/src/ld/ld.hpp, line 510.
1086	          clang: error: linker command failed with exit code 1 (use -v to see invocation)
1087	

The current list of affected ports includes:

• cargo-c: link crash on 10.6_x86_64
• libopenraw: link crash on multiple macOS releases

[cjones051073 (Chris Jones)]

Its not obvious to me clang-13 is the ssiue here. If you check the log more closely I believe the seg. is coming from rust.

[cjones051073 (Chris Jones)]

@mascguy - Can you check the log you uploaded. its difficult to read as its got a bunch of html mixed in. Can you please make it a simple plain text file ?

[mascguy (Christopher Nielsen)]

Replying to cjones051073:

@mascguy - Can you check the log you uploaded. its difficult to read as its got a bunch of html mixed in. Can you please make it a simple plain text file ?

Fixed. And ignore the second version, forgot to check the box to replace the existing file. (Though both are identical anyway.)

[mascguy (Christopher Nielsen)]

In any case, the log is from our 10.9 buildbot. Simply wanted to attach it to the ticket, to preserve the detail once it ages out.

​https://ports.macports.org/port/libopenraw/details/

[mascguy (Christopher Nielsen)]

Replying to cjones051073:

Its not obvious to me clang-13 is the ssiue here. If you check the log more closely I believe the seg. is coming from rust.

Well, the link failure is only occurring on macOS 10.10 and earlier. So does that mean rust only has issues on those macOS releases...?

[cjones051073 (Chris Jones)]

I guess so, yes.

[jmroot (Joshua Root)]

Replying to mascguy:

the log is from our 10.9 buildbot.

Append /text to the URL to get a plaintext version.

[mascguy (Christopher Nielsen)]

Folks, we really need to find a fix for this, as the failures for cargo-c are blocking a large number of downstream ports on 10.6.

Thoughts as to how we move forward...?

[Christopher Nielsen <mascguy@…>]

In c3a13b6ceb9a424062952c8688cb583c4e1554f3/macports-ports (master):

libavif: disable rav1e for 10.6 and earlier
See: #65434

[Christopher Nielsen <mascguy@…>]

In 5919c5f984db126387391c2164adba4c3057d880/macports-ports (master):

libheif: enable rav1e for 10.7 and later

• Now supported on older macOS releases; presently blocked on 10.6 though

See: #62614
See: #65434

[mascguy (Christopher Nielsen)]

Kirill, any thoughts/insight into this? (Definitely not an emergency though.)

[catap (Kirill A. Korinsky)]

Christopher, I had discovered the same issue with retdec the last week: ​https://github.com/avast/retdec/issues/1114

[cjones051073 (Chris Jones)]

So, I had a moment to look into this a little more.

The problems here are definitely not the fault of the newer macports clang-X compilers.

In the case of libopenraw, the issue is in fact how the build internally configure rust, which is used by the build.

For parts of the build a newer macports clang is selected on older systems, due to the c++14 requirement. The build then later on passes the object files generated by these to rust, to link in with other rust modules. The problem is rust uses 'cc' to try and link these

error: linking with `cc` failed: exit status: 1
1045	  |
1046	  = note: "cc" "-m64" "-arch" "x86_64" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.0.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.1.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.10.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.11.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.12.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.13.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.14.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.15.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.2.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.3.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.4.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.5.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.6.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.7.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.8.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.build_script_build.3935ca6f-cgu.9.rcgu.o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f.2onfrlf4o5yrkhbk.rcgu.o" "-L" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/deps" "-L" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libstd-5ae51653233c3123.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-824f33e42152e8ad.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libobject-69ae51926f10bb57.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libmemchr-245445c928190b80.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libaddr2line-a0343ede66331927.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libgimli-d18052b3f2c44799.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-214396c7455eebbc.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libstd_detect-04fe1d4ba8284309.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-bb275db5534b8eac.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libminiz_oxide-eccddc93df08ba1a.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libadler-899dcd5e277779e4.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-f3953770a4f7c260.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libunwind-04937ee0dc84d4eb.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-3a8a8629c72cf068.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/liblibc-2c7b579ceb73107a.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/liballoc-efb34ae0db6de1f6.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-adb0cf783958c0b5.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libcore-b338f6e3114cd3ca.rlib" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-f3b83820ab311320.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/opt/local/lib/rustlib/x86_64-apple-darwin/lib" "-o" "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_graphics_libopenraw/libopenraw/work/libopenraw-0.3.2/lib/mp4/target/release/build/libc-e0df0c46b0db603f/build_script_build-e0df0c46b0db603f" "-Wl,-dead_strip" "-nodefaultlibs"
1047	  = note: 0  0x1075ddb51  __assert_rtn + 144
1048	          1  0x10760e063  ld::Fixup::Fixup(unsigned int, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 107
1049	          2  0x1075e64e9  mach_o::relocatable::Parser<x86_64>::addFixup(mach_o::relocatable::Parser<x86_64>::SourceLocation const&, ld::Fixup::Cluster, ld::Fixup::Kind, bool, char const*) + 33
1050	          3  0x1075e1751  mach_o::relocatable::Section<x86_64>::addRelocFixup(mach_o::relocatable::Parser<x86_64>&, macho_relocation_info<Pointer64<LittleEndian> > const*) + 1471
1051	          4  0x107609bb7  mach_o::relocatable::Section<x86_64>::makeFixups(mach_o::relocatable::Parser<x86_64>&, mach_o::relocatable::Parser<x86_64>::CFI_CU_InfoArrays const&) + 91
1052	          5  0x1076057c6  mach_o::relocatable::Parser<x86_64>::parse(mach_o::relocatable::ParserOptions const&) + 2034
1053	          6  0x1075e90fd  mach_o::relocatable::Parser<x86_64>::parse(unsigned char const*, unsigned long long, char const*, long, ld::File::Ordinal, mach_o::relocatable::ParserOptions const&) + 375
1054	          7  0x1076199cc  archive::File<x86_64>::makeObjectFileForMember(archive::File<x86_64>::Entry const*) const + 758
1055	          8  0x107619518  archive::File<x86_64>::justInTimeforEachAtom(char const*, ld::File::AtomHandler&) const + 122
1056	          9  0x10762f19b  ld::tool::InputFiles::searchLibraries(char const*, bool, bool, bool, ld::File::AtomHandler&) const + 215
1057	          10  0x107637c90  ld::tool::Resolver::resolveUndefines() + 160
1058	          11  0x107639f63  ld::tool::Resolver::resolve() + 79
1059	          12  0x1075de6c7  main + 689
1060	          A linker snapshot was created at:
1061	          /tmp/build_script_build-e0df0c46b0db603f-2022-06-04-173157.ld-snapshot
1062	          ld: Assertion failed: (name != NULL), function Fixup, file /SourceCache/ld64/ld64-241.9/src/ld/ld.hpp, line 510.
1063	          clang: error: linker command failed with exit code 1 (use -v to see invocation)

the issue is just using 'cc' means the ancient system clang compiler is picked, and on these older systems this is just too old to handle the newer object files generated by the more up to date clang compilers. The traceback above is in fact coming from this cc process, which it invokes an ancient 'ld' from it.

Note, libopenraw does not use the rust PG, which has logic in it to try and handle these sorts of things, to ensure the newer compiler toolkits are used when needed.

​https://github.com/macports/macports-ports/blob/a66c19532113565c4a73700b3888fa14823c3fc7/_resources/port1.0/group/rust-1.0.tcl#L689

It might be something similar is needed with this port.

[lukaso (Lukas Oberhuber)]

Not sure if this is the same thing, but seems close. It is about building rav1e using MACOSX_DEPLOYMENT_TARGET. I've got these two bug reports: ​https://gitlab.gnome.org/GNOME/gimp/-/issues/8930 and ​https://github.com/xiph/rav1e/issues/3080