rust: Update to 1.74.1

[Schamschula (Marius Schamschula)]

It looks like we've skipped rust 1.72.x. I have a local port that needs at least rust 1.72.0, but 1.73.0 was just released as I was researching the issue.

[Schamschula (Marius Schamschula)]

Update: as of this time (December 23, 2023) upstream rust is @ 1.74.1.

Unfortunately, updating the rust Portfile is not trivial, and there are no notes of how to go about it.

It is unclear how to use cargo2port to update the cargo packages. I manually merged those from rustc-1.74.1-src and rustc-1.74.1-src/src/bootstrap/, and got it to start building. However, it errored out.

[halostatue (Austin Ziegler)]

Replying to Schamschula:

Update: as of this time (December 23, 2023) upstream rust is @ 1.74.1.

Unfortunately, updating the rust Portfile is not trivial, and there are no notes of how to go about it.

It is unclear how to use cargo2port to update the cargo packages. I manually merged those from rustc-1.74.1-src and rustc-1.74.1-src/src/bootstrap/, and got it to start building. However, it errored out.

I think that MacPorts is going to need to have more responsive Rust updates moving forward *or* is going to need to adjust the Rust build steps to permit Rust's x.py to download the previous version (not always possible with older macOS versions, I would guess). In ​Bootstrapping the compiler (overview), the example shows bootstrapping the 1.64 compiler with 1.63. The current prebuilt version is 1.70.0, which suggests that we would need to build 1.71, 1.72 and 1.73 to get to 1.74, but we could then use 1.74.1 to build 1.75 when that is released. With our lack of Sonoma arm64 builders, this means that even when Rust 1.74.1 is available, unless we have the arm versions at hand, people on Sonoma arm64 would be unable to upgrade to the latest Rust.

At the moment, I have been following more or less the same steps and as far as I can tell, the build is failing because my bootstrap version is too old. (It’s *very* difficult to find in the errors in the MacPorts log, but I think that the first error is starting at 16855 of the attached build log and the main errors are at 16909–16961; these were found by copying the commands run and running them manually as the macports user.)

:info:build error: cannot find a built-in macro with name `ConstParamTy`
:info:build     --> library/core/src/marker.rs:1003:1
:info:build      |
:info:build 1003 | / pub macro ConstParamTy($item:item) {
:info:build 1004 | |     /* compiler built-in */
:info:build 1005 | | }
:info:build      | |_^
…
:info:build warning: `extern crate` is not idiomatic in the new edition
:info:build    --> /opt/local/libexec/rust/src/.home/.cargo/macports/compiler_builtins-0.1.101/build.rs:157:5
:info:build     |
:info:build 157 |     extern crate cc;
:info:build     |     ^^^^^^^^^^^^^^^^
:info:build     |
:info:build     = note: `-D unused-extern-crates` implied by `-D warnings`
:info:build help: convert it to a `use`
:info:build     |
:info:build 157 |     use cc;
:info:build     |     ~~~
:info:build warning: `compiler_builtins` (build script) generated 1 warning
:info:build      Running `/opt/local/var/macports/build/_Users_austin_dev_oss_forks_macports-ports_lang_rust/rust/work/rustc-1.74.1-src/build/aarch64-apple-darwin/stage0-std/release/build/compiler_builtins-da2e238ee913b174/build-script-build`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build   --> library/core/src/mem/transmutability.rs:10:1
:info:build    |
:info:build 10 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build   --> library/core/src/ptr/metadata.rs:53:1
:info:build    |
:info:build 53 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build    --> library/core/src/marker.rs:143:1
:info:build     |
:info:build 143 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build    --> library/core/src/marker.rs:176:1
:info:build     |
:info:build 176 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build    --> library/core/src/marker.rs:857:1
:info:build     |
:info:build 857 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build    --> library/core/src/marker.rs:962:1
:info:build     |
:info:build 962 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build    --> library/core/src/marker.rs:973:1
:info:build     |
:info:build 973 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`
:info:build error: malformed `rustc_deny_explicit_impl` attribute input
:info:build     --> library/core/src/marker.rs:1032:1
:info:build      |
:info:build 1032 | #[rustc_deny_explicit_impl(implement_via_object = false)]
:info:build      | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: must be of the form: `#[rustc_deny_explicit_impl]`

I’m attaching my build log *and* my current branch changes to see if this will help someone else unblock this.

[Schamschula (Marius Schamschula)]

That patch looks familiar!

I had to update the same patch files. I didn't diff the version for the various crates, but I think I likely ended up with something very similar.

[Schamschula (Marius Schamschula)]

I just checked on my FreeBSD server: Indeed the rust-bootstrap version is 1.74.0 while rust is at 1.74.1.

This may mean a multi step update path

However, it may also be possible to update rust-bootstrap before attempting to update rust.

[halostatue (Austin Ziegler)]

Note that even if we get this working, it’s not clear that it would be accepted for merge yet because of ongoing older macOS buildbot failures (see #68028).

[raimue (Rainer Müller)]

Replying to halostatue:

Note that even if we get this working, it’s not clear that it would be accepted for merge yet because of ongoing older macOS buildbot failures (see #68028).

It is actually #68015. In my opinion the legacy macOS versions 10.9 and 10.10 should not block an update of the rust port to a new version for the more recent and supported macOS versions. The new version might fix the problem or it does not build either. There is nothing to lose.

[halostatue (Austin Ziegler)]

Replying to raimue:

It is actually #68015. In my opinion the legacy macOS versions 10.9 and 10.10 should not block an update of the rust port to a new version for the more recent and supported macOS versions. The new version might fix the problem or it does not build either. There is nothing to lose.

According to the ​release announcement, the minimum ​Apple OS versions are:

Compatibility notes

As previously announced, Rust 1.74 has increased its requirements on Apple platforms. The minimum versions are now:
- macOS: 10.12 Sierra (First released 2016)
- iOS: 10 (First released 2016)
- tvOS: 10 (First released 2016)

I don't know exactly how this would be done, but I think that putting legacy versions of macOS on an older version (they could use up to 1.73.0, in theory) is the right choice here.

I will be looking at the patches required to build rust-bootstrap and rust from this perspective. As such, there’s a couple of emulated TLS patches for rust-bootstrap (one of which is used for rust) that I will be looking at dropping, and I will be reviewing other patches for pre macOS 10.12 support for the 1.74 release.

[halostatue (Austin Ziegler)]

Questions for interested parties:

1. What is the subport rust-bootstrap-transition and is it still needed?
2. Should we change the Python version from 3.11 to 3.12?

[Schamschula (Marius Schamschula)]

As is the custom for other ports, we can pin unsupported macOS versions to older versions of rust. It makes for a more complicated Portfile, but it is manageable.

I think it is still premature to move to Python 3.12. As it's a build dependency, that change should be made when Python 3.12 becomes default.

[halostatue (Austin Ziegler)]

Upstream is now 1.75.0, however I am failing to build either rust or rust-bootstrap. I’m mostly focusing on rust-bootstrap right now because it appears to be necessary to have the previous level working.

I’m not *entirely* sure what's going on in rust-bootstrap, but the most recent build failure (setting version_current to 1.72.1 and version_m1 to 1.71.1) is

:info:build Unexpected rustc version: 1.69.0, we should use 1.70.x/1.71.1 to build source with 1.71.1

It appears that this (1.69.0) is from rust.add_bootstrap_components (in _resources/port1.0/group/rust-1.0.tcl:146) and that points to https://github.com/MarcusCalhoun-Lopez/rust/releases/download/${version_current}:macports_vendor, which I think has been mentioned in other tickets or the mailing lists as a slightly odd dependency.

There's some dependencies that I don’t understand which, if I didn’t want to shift mostly to MacPorts for my package management, I would be seriously thinking about switching *back* to rustup and cargo install and/or cargo binstall for any rust-based dependencies.

It would be good to get some guidance from the port and port group owner on the right way to upgrade these ports, because I’m willing to spend some time helping out with this, but my Tcl-fu is not strong and there appear to be a number of undocumented procedural steps that are required (what's bootstrap-transition vs bootstrap, etc.).

This is made more difficult by the fact that I *know* that on macOS, the only supported architectures are x86_64 and arm64 and OS versions 10.12 or higher. I don't think that Rust has ever had powerpc-apple-darwin targets and i686-apple-darwin is a Tier 3 target (might work, might not, poorly supported), so some of the architecture choices seem questionable in the rust/rust-bootstrap portfiles.

I’m sure that there's good reasons behind the choices, but from my perspective, I can't reason them out.

[Schamschula (Marius Schamschula)]

I had a look at the Homebrew Formula. See: ​https://github.com/Homebrew/homebrew-core/blob/78fd5faf3c86b92bf7ec68e038eaac0acf3a9e59/Formula/r/rust.rb

A noticed couple of things:

1) there is no rust-bootstrap port, rather they fetch cargo bootstrap from ​https://static.rust-lang.org/

2) they patch src/bootstrap/compile.rs

FreeBSD has a Makefile for both rust-bootstrap (1.74.0 as I write this) and rust (1.74.1), however, the rust Makefile calls out version 1.73.0 for the bootstrap compiler.

[kencu (Ken)]

a lot of what is currently being done comes from here

​https://github.com/macports/macports-ports/pull/14277

Marcus has not been available for about a year now, however.

The current build system has been beyond the understanding of anyone else, unfortunately.

All the older system support, that macports prides itself on, may have to be abandoned with rust… and as several key parts of the gnome system only build with rust I understand, that spells the end of that….

Some pegged rust version might suffice for a short while….

[Schamschula (Marius Schamschula)]

I've just made some progress with rust-bootstrap with

    set version_current         1.74.1
    set version_m1              1.74.0
    set version_m2              1.73.0

I got the rust-llvm to build. Later, I got an error in

error: failed to select a version for the requirement `anyhow = "^1.0.62"` (locked to 1.0.71).

Apparently there are additional Cargo.lock files to consider:

src/tools/cargo/Cargo.lock

and

src/tools/rust-analyzer/Cargo.lock

After merging these I got a clean build.

marius@Mira ~ % port installed rust-bootstrap
The following ports are currently installed:
  rust-bootstrap @1.74.0_0+universal (active)

[Schamschula (Marius Schamschula)]

Note: rust-1.0.tcl and Portfile are rough:

In rust-1.0.tcl I only updated checksums for files I needed on my system (Ventura 13.6.3-arm).

The Portfile may have a number of unneeded cargo.crates.

Edit: I've re-uploaded rust-1.0.tcl as rust-1.0.2.tcl to fix a typo for version_m1.

[halostatue (Austin Ziegler)]

Replying to kencu:

a lot of what is currently being done comes from here

​https://github.com/macports/macports-ports/pull/14277

Thank you for the history; I’ll review that as much as I can.

The current build system has been beyond the understanding of anyone else, unfortunately.

All the older system support, that macports prides itself on, may have to be abandoned with rust… and as several key parts of the gnome system only build with rust I understand, that spells the end of that….

Some pegged rust version might suffice for a short while….

I think that’s going to be true of a lot of programming systems moving forward. I don’t think that Ruby has been built for PPC systems in several versions, and it seems there’s very little 32-bit support at ​https://rubyci.org.

[kencu (Ken)]

re ruby, maybe, but there are some very persistent folks around that may surprise you with their refusal to take no for an answer there…

[Schamschula (Marius Schamschula)]

And I have have a completed build for rust 1.74.1!

Now that I have an idea how the update process works, I'll go back and do this over for rust-bootstrap 1.74.0 and rust 1.75.0.

Once I get those done, I'll put them in a PR, and we can see where we want to go from there regarding older macOS versions.

[Schamschula (Marius Schamschula)]

For the record:

rust-bootstrap needed

cargo2port lang/rust-bootstrap/work/rustc-1.74.0-src/src/bootstrap/Cargo.lock
cargo2port lang/rust-bootstrap/work/rustc-1.74.0-src/Cargo.lock
cargo2port lang/rust-bootstrap/work/rustc-1.74.0-src/src/tools/cargo/Cargo.lock
cargo2port lang/rust-bootstrap/work/rustc-1.74.0-src/src/tools/rust-analyzer/Cargo.lock

while rust needed

cargo2port lang/rust/work/rustc-1.75.0-src/src/bootstrap/Cargo.lock
cargo2port lang/rust/work/rustc-1.75.0-src/Cargo.lock

[Schamschula (Marius Schamschula)]

PR: ​https://github.com/macports/macports-ports/pull/21985

[halostatue (Austin Ziegler)]

Those are probably worth adding to the portfiles.

That said, I think that we *may* want to push this as a couple of different releases because there is a hard drop from the Rust side of pre 10.12 in 1.74.0, and if we can get some level of support for older versions (as old as 10.8). I’ve been reading and adding comments to the rust PG file to try to make this process easier for others in the future. One point in particular that I’m adding is this:

# default macosx_deployment_target value of Rust function macos_default_deployment_target
# see [1] at 1.74.1 or earlier or [2] at 1.75.0 or later.
#
# [1] https://github.com/rust-lang/rust/blob/master/compiler/rustc_target/src/spec/apple_base.rs
# [2] https://github.com/rust-lang/rust/blob/master/compiler/rustc_target/src/spec/base/apple/mod.rs
#
# The Rust default for aarch64 (Apple Silicon) is 11.0 (`(11, 0)`) siunce 1.59.0,
# and the default for x86_64 depends on the Rust version. Before 1.71.0, the
# minimum version is 10.7 (`(10, 7)`). From 1.71.0 to 1.73.0, the support is 
# 10.8 (`(10, 8)`) for Haswell boards (x86_64h) and 10.7 for everything else.
# Starting with 1.74.0, the minimum supported version is 10.12 (`(10, 12)`).
#
# x86_64-apple-darwin is considered tier 1; aarch64-appled-darwin is considered
# tier 2 with host tools (and seems likely to end up as a tier 1 in the future
# when there is CI support).
#
# See https://doc.rust-lang.org/nightly/rustc/platform-support.html
default     rust.upstream_deployment_target {[expr {${os.arch} eq "arm" ? 11.0 : 10.7}]}

So if we can get things together for a 1.73.0 rust installation, then I think we can keep the support level we have for those, although we may need help from people with access to older hardware to build the bootstrap / transition compilers the way that Marcus Calhoun-Lopez did, and that will allow us to pin everything below 10.12 to 1.73.0 (not great, but unless there are people in the MacPorts community who are as committed to keeping Rust for those versions alive as there are for Ruby, I don't see much of an option here; I am *not* a compiler or rust language expert, so I am mostly useless on this part).

I think that a couple of things should happen, if possible, but will require assistance from the MacPorts organization maintainers on GitHub:

• We should fork ​https://github.com/MarcusCalhoun-Lopez/rust into the macports organization. We should *also* upload all of the releases from that into corresponding releases on the forked repo. Marcus did a release of 1.71.1 in August, so I think it would be *correct* to make sure he is still also a contributor on the forked version.
• We should add some documentation (to the wiki and/or the port files) on how and what order to submit these in (rust-bootstrap, then rust). Ideally this would include when to use rust-bootstrap-transition, but I haven't been able to figure that part out, and it may have literally been something required to *transition* into the new build system which could now be deleted since there are bootstraps available for a lot of previous versions. (To make this work correctly, we might need to do bootstrap builds for 1.72.)
• We should add a bit more documentation to the rust portfile.
• We should make version_m1 and version_m2 *computed* at all times:

proc rust.previous_version {version} {
    set list {}
    foreach element [split $version .] {
        if {[scan $element %d%s i trash] != 1} {
            return -code error "invalid version number: \"$version\""
        }

        if {[catch {incr i 0}]} {
            return -code error "invalid version number: \"$version\""
        }

        lappend list $i
    }

    if {[llength $list] > 3} {
        return -code error "invalid version number: \"$version\""
    }

    set list [lreplace $list 2 2 0]
    set list [lreplace $list 1 1 [expr [lindex $list 1] - 1]]

    return [join $list .]
}

    set version_current         1.71.1
    set version_m1              [rust.previous_version $version_current]
    set version_m2              [rust.previous_version $version_m1]

That might not be the best way to do this, but it would prevent some confusion.

[MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)]

In 342989ea6af61cc58d6b5a799367a447debad55a/macports-ports (master):

rust, rust-bootstrap, cargo: update & use build PG

Use new rust_build PortGroup.
Revamp comments.

Fixes #68364
Closes ​https://github.com/macports/macports-ports/pull/21985