gcc12 @12.2.0_2: gcc is baking in a fragile specific sdk because regex test is not mstching SDKs without a decimal point

[jsalort (Julien Salort)]

I get a linker error while trying to use gfortran-mp-12 on arm Ventura:

$ port version
Version: 2.8.1
$ port installed gcc12
The following ports are currently installed:
  gcc12 @12.2.0_2+stdlib_flag (active)
$ cat hello.f90 
program hello
    print*, 'Hello, World!'
end program hello
$ gfortran-mp-12 hello.f90 -o hello
ld: library not found for -lSystem
collect2: error: ld returned 1 exit status

[ryandesign (Ryan Carsten Schmidt)]

This says you have to set LIBRARY_PATH properly:

​https://stackoverflow.com/questions/56156520/gfortran-error-ld-library-not-found-for-lsystem-when-trying-to-compile

Maybe there is something we can do in the gcc ports so that setting LIBRARY_PATH manually is not necessary...

[ryandesign (Ryan Carsten Schmidt)]

Maybe #59727 is helpful as well.

[jsalort (Julien Salort)]

Replying to ryandesign:

Maybe #59727 is helpful as well.

Indeed.

First, I have checked that gcc is also broken (not just gfortran), i.e.:

$ cat hello.c
#include <stdio.h>

int main(void) {
    printf("Hello, World\n");
    return 0;
}
$ gcc-mp-12 -o hello hello.c
In file included from hello.c:1:
/opt/local/lib/gcc12/gcc/arm64-apple-darwin22/12.2.0/include-fixed/stdio.h:78:10: fatal error: _stdio.h: No such file or directory
   78 | #include <_stdio.h>
      |          ^~~~~~~~~~
compilation terminated.

It used to work. I realized that gfortran was broken because I needed it to recompile a project that compiled fine a few weeks ago. What changed is that the build bots have been brought online for Ventura ARM, so gcc12 was no longer built from source on my system.

So, I did as suggested in #59727,

sudo port uninstall gcc12
sudo port install -v -s gcc12

And now, both gcc-mp-12 and gfortran-mp-12 work as expected.

At the end of #59727, kencu says:

I believe this has been fixed now by changes in base and the noted ports to spec an existing SDK.

please reopen with new information if you find this is still and issue.

So I guess it is still relevant to have the ticket open.

[kencu (Ken)]

It would be good to see what SDK path is currently being baked into the gcc12 port, so we might get an idea what is going wrong with that. It is supposed to be a generic MacOSX.sdk at this point in time, pointing to the SDK installation in the Command Line Tools, which should exist on every system.

[kencu (Ken)]

so my buildbot-installed version is working fine, but it is not configured right.

% gcc-mp-12 -v 
Using built-in specs.
COLLECT_GCC=gcc-mp-12
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/arm64-apple-darwin22/12.2.0/lto-wrapper
Target: arm64-apple-darwin22
Configured with: /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_gcc12/gcc12/work/gcc-12.2.0/configure --prefix=/opt/local --build=arm64-apple-darwin22 --enable-languages=c,c++,objc,obj-c++,lto,fortran,jit --libdir=/opt/local/lib/gcc12 --includedir=/opt/local/include/gcc12 --infodir=/opt/local/share/info --mandir=/opt/local/share/man --datarootdir=/opt/local/share/gcc-12 --with-local-prefix=/opt/local --with-system-zlib --disable-nls --program-suffix=-mp-12 --with-gxx-include-dir=/opt/local/include/gcc12/c++/ --with-gmp=/opt/local --with-mpfr=/opt/local --with-mpc=/opt/local --with-isl=/opt/local --with-zstd=/opt/local --enable-checking=release --disable-multilib --enable-lto --enable-libstdcxx-time --with-build-config=bootstrap-debug --with-as=/opt/local/bin/as --with-ld=/opt/local/bin/ld --with-ar=/opt/local/bin/ar --with-bugurl=https://trac.macports.org/newticket --enable-host-shared --with-darwin-extra-rpath=/opt/local/lib/libgcc --with-libiconv-prefix=/opt/local --disable-tls --with-gxx-libcxx-include-dir=/opt/local/libexec/gcc12/libc++/include/c++/v1 --with-pkgversion='MacPorts gcc12 12.2.0_2+stdlib_flag' --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (MacPorts gcc12 12.2.0_2+stdlib_flag) 

as you can see, it's configured to look here for the SDK:

--with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk

but it is supposed to be configured to look here:

--with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

anyway, mine works because I happen to have something at --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk that looks like an SDK:

% ls -la /Library/Developer/CommandLineTools/SDKs
total 0
drwxr-xr-x  8 root  wheel  256  9 Jan 22:50 .
drwxr-xr-x  5 root  wheel  160  9 Jan 22:50 ..
lrwxr-xr-x  1 root  wheel   14  9 Jan 22:50 MacOSX.sdk -> MacOSX13.1.sdk
drwxr-xr-x  7 root  wheel  224 19 Nov  2021 MacOSX12.1.sdk
drwxr-xr-x  7 root  wheel  224  9 Jan 22:50 MacOSX12.3.sdk
lrwxr-xr-x  1 root  wheel   14  9 Jan 22:49 MacOSX12.sdk -> MacOSX12.3.sdk
drwxr-xr-x  7 root  wheel  224 12 Nov 13:18 MacOSX13.1.sdk
lrwxr-xr-x  1 root  wheel   14  9 Jan 22:49 MacOSX13.sdk -> MacOSX13.1.sdk

but that is probably just luck.

[kencu (Ken)]

Yeah, the procedure:

proc get_clean_sysroot {} {
    global configure.sdkroot
    # if the sdkroot is one of the current, rapidly changing SDKS, use the generic one
    return [regsub {MacOSX1[1-9]\.[0-9]+\.sdk} ${configure.sdkroot} {MacOSX.sdk}]
}

is broken, and no longer does what it is supposed to do.

It is returning:

/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk

instead of

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

[kencu (Ken)]

and that's because the SDK is named MacOSX13.sdk and so doesn't match the regsub test, which requires a decimal point in the SDK number.

[kencu (Ken)]

this can work

    return [regsub {MacOSX1[1-9]\S+} ${configure.sdkroot} {MacOSX.sdk}]

assuming there is no simple way to break that test too...

[kencu (Ken)]

wordy title, but describes the issue

[cjones051073 (Chris Jones)]

My regex skills have always been minimal at best, so if anyone knows how to change the above to address this be my guest….

[cjones051073 (Chris Jones)]

Maybe it is indeed enough to just drop the requirement that there is a dot in the SDK version, as I think this is something Apple themselves have dropped…

[cjones051073 (Chris Jones)]

Actually....

I am not sure the above proc is actually broken. The intention was to replace MacOSN.x.sdk with MacOSN.sdk on the assumption the later would always be available for a given major OS release. I was never the intention to replace it with the completely generic MacOS.sdk...

So I think I would like to understand a bit better why thesis not the case ?

[kencu (Ken)]

Or we just forget about all fancy the regex stuff, and force

--with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

on all newer systems.

Would break folks who don't install the CLTs, but we don't support that anyway.

[kencu (Ken)]

forcing MacOSX.sdk was always the intention I had in this.

[cjones051073 (Chris Jones)]

The regex when added was never intended to force MacOSX.sdk, it has never ever done that.

It was *specifically* only intended to deal with the fact that when macOS11 was released, in the early days of the SDKs they would change rapidly between minor versions. So 11.0 and 11.1 had different SDKs. MacPorts never expected this and it caused issues, so this regex was added. This happened because of how Apple changed the meaning of their OS versions. They went from 10.X being the major versions, with 10.X.n being the minor revision, to just X for the major and X.n for the minor.

We could think about just forcing the use of the completely version less SDk, but this is a change different to what we have done before. I would want to think through what side effects that could have. The current scheme is used as it ensures the build uses the SDK that matches their OS version. Switching to the versionless SDK could change this, e.g. on systems where Xcode provides multiple SDKs, and I am not sure we want that.

Also note, even this will *never* be a complete fix, it will still be possible for a user to have a different SDK path choice to whatever the binary installs are built with. As say the build bots built gcc using the Xcode SDK at a specific location, a user could still have only the CLT installed, or Xcode at a custom location. So the issue is still, at least in theory still there.

The correct solution here, which is what I always recommend, is for the user to *not* rely of the default SDK built into gcc, but to tell it what to use. The mechanisms to do this have been available for some time, and GCC (intentionally) uses the exact same methods as Clang (which incidentally has more or less the same need). The ways to do it are

1. Run through xcrun

> xcrun g++-mp-12 ...

xcrun creates the correct runtime environment, including info on the SDK, which gcc then uses.

2. Set SDKROOT environment var

> SDKROOT=/path/to/SDK g++-mp-12 ...

or if you prefer

> SDKROOT=/path/to/SDK
> g++-mp-12 ...

In both the above, gcc respects your SDKROOT setting and will use whatever it points to.

I personally use the second, and actually have

export SDKROOT=`xcrun --show-sdk-path`

in my shell profile, so whenever I start a SHELL this env var gets automatically set to whatever my current SDK choice happens to be.

Note the above is also exactly what you should also do with clang, to inform it what SDK to use. Macports clang ports in particular rely on this as they, like GCC, cannot assume any specific SDK location that the user will have installed.

Also note macports always sets SDKROOOT during port builds, so whenever it uses its own GCC or clang ports (or Xcode clang for that matter) it controls the SDK used via the exact same mechanisms as above.

For now I am inclined not to change anything here.

[kencu (Ken)]

Of course it was meant to force MacOSX.sdk… I added it to do exactly that:

​https://github.com/macports/macports-ports/commit/e8866c5019d60832527850b4e50fdc1de8878716

For the rest, leave it broken if you like.

[cjones051073 (Chris Jones)]

Sorry, my mistake, Yes when the regex matches it forces the completely versionless SDK.

However the intention was always that the regex would only match MacOSN.x.sdk and never MacOSN.sdk. That later SDK is stable for a given macOSN release and is fine to use. So the end result is the same, nothing needs changing here, and if the end user wants control over the SDK that they use then they should always, as before, do so by using xcrun or setting SDKROOT themselves.