base uses incorrect SDK for Big Sur 11.1 and Xcode 12.3

[mf2k (Frank Schima)]

base needs to be updated to correctly use the 11.1 SDK on Big Sur 11.1 and Xcode 12.3.

On many ports I see configure logs with this:

:debug:configure LDFLAGS='-L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -arch x86_64'

But that is not correct. It should be 11.1.

$ ls -l /Library/Developer/CommandLineTools/SDKs
total 0
lrwxr-xr-x  1 root  wheel   14 Dec 18 08:22 MacOSX.sdk -> MacOSX11.1.sdk
drwxr-xr-x  7 root  wheel  224 Nov 12  2019 MacOSX10.14.sdk
drwxr-xr-x  8 root  wheel  256 Jul  9 10:33 MacOSX10.15.sdk
drwxr-xr-x  4 root  wheel  128 Dec 14 13:41 MacOSX11.0.sdk
drwxr-xr-x  7 root  wheel  224 Nov 30 05:33 MacOSX11.1.sdk

I hacked my local copy of portconfigure.tcl so that SDKROOT is 11.1 and many ports build again. But I'm unsure of the proper fix to include the minor version of macOS for the SDK.

[kencu (Ken)]

what say we use MacOSX.sdk preferentially on BigSur and up?

That will save a lot of heartburn with all the gcc ports, python, perl, and all the others that bake in a path to the SDK.

And it's how Apple is going now anyway.

And helps ports build when they need a newer SDK than the versioned one that matches their system.

This was undesirable in the 10.4 to 10.13 era, but it may be time to do it now for 11.x and up.

[cjones051073 (Chris Jones)]

Base has already been updated to handle this

​https://github.com/macports/macports-base/commit/8aa347e8adaa0250fda0b45309d3a31266c7cedb

Of course thats just in the master branch, so not yet in a public release.

Personally, i think there is some merit to considering using by default the versionless sdk on big sur and newer.

[cjones051073 (Chris Jones)]

mf2k - What you see in your command line tools SDK area is not normal - You *shouldn't* have both the 11.0 and 11.1 SDKs there..

This is what I have on my system

Oberon ~/Projects/MacPorts/ports > ls -l /Library/Developer/CommandLineTools/SDKs
total 0
lrwxr-xr-x  1 root  wheel   14 15 Dec 12:02 MacOSX.sdk -> MacOSX11.1.sdk
drwxr-xr-x  8 root  wheel  256 15 Dec 12:02 MacOSX10.15.sdk
drwxr-xr-x  7 root  wheel  224 15 Dec 12:02 MacOSX11.1.sdk

which is what is expected I believe. Did you do something specific to add the other SDKs ?

Also note base, even the current released one, already will fallback to the versionless SDK if the specific version one it thinks it should use is not present. So normally what would happen is on mac OS11.1 it would (incorrectly) look for the 11.0 SDK, fail to find it, and then fallback to the versionless one. The fact for some reason you also have the 11.0 SDK on disk is actually the reason why you are using the wrong SDK.

[mf2k (Frank Schima)]

I am using the latest base from git master and it still does not correctly use the 11.1 SDK.

I did nothing special except keep upgrading Xcode to the latest version as new versions were released.

[mf2k (Frank Schima)]

I can manually delete the 11.0 SDK. But needing to do so is not documented anywhere. Not by Apple or Macports.

[jmroot (Joshua Root)]

Building against the 11.0 SDK if available is not wrong. We want to build binaries that will run on any macOS 11 release. If the 11.0 SDK is present as a directory but not usable for some reason, I guess we need some more checks to make sure an SDK is OK before using it.

[jmroot (Joshua Root)]

Replying to cjones051073:

Personally, i think there is some merit to considering using by default the versionless sdk on big sur and newer.

That will break things as soon as Xcode ships a macOS 12 SDK if not before. Almost everything checks for APIs at configure time, not at runtime.

[kencu (Ken)]

Replying to jmroot:

Replying to cjones051073:

Personally, i think there is some merit to considering using by default the versionless sdk on big sur and newer.

That will break things as soon as Xcode ships a macOS 12 SDK if not before. Almost everything checks for APIs at configure time, not at runtime.

It will break all over the world though, as every other build system is using the current xcode against the latest MacOS.sdk, and so is cmake defaulting to that now.

Headers are not a significant issue. It's the library symbols that might be found in the newer sdk, which should not be a significant issue for one system version (but could be...) and if we run into that, everyone using Xcode all over the place will also run into that, and it needs to be fixed for everyone anyway.

[cjones051073 (Chris Jones)]

Replying to mf2k:

I am using the latest base from git master and it still does not correctly use the 11.1 SDK.

I did nothing special except keep upgrading Xcode to the latest version as new versions were released.

I guess then the globbing that was added to find matching SDKs is likely matching both the 11.0 and 11.1 SDKs, but for some reason selecting the older one. If thats the case I think it should be changed to pick the latest match, not the oldest ?

[cjones051073 (Chris Jones)]

Replying to mf2k:

I can manually delete the 11.0 SDK. But needing to do so is not documented anywhere. Not by Apple or Macports.

When I updated to 11.1 and Xcode 12.3 + corresponding CLT, the 11.0 SDK was removed, and the 11.1 installed. I think its a bit odd this didnt happen for you as well.

[ryandesign (Ryan Carsten Schmidt)]

Replying to cjones051073:

I guess then the globbing that was added to find matching SDKs is likely matching both the 11.0 and 11.1 SDKs, but for some reason selecting the older one. If thats the case I think it should be changed to pick the latest match, not the oldest ?

The current code effectively picks a random macOS 11 SDK out of all available 11.x SDKs. (It returns the first item of the glob, but glob does not return results in any particular order. HFS+ seemed to return results in alphabetical order but APFS seems not to.) I agree that we should loop over the results, compare them with vercmp, and return the newest one.

[MaddTheSane (C.W. Betts)]

Attempting to build git +svn results in a build failure with subversion-perlbindings-5.28 with the following errors:

:info:build clang: warning: no such sysroot directory: '/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk' [-Wmissing-sysroot]
:info:build In file included from subversion/bindings/swig/perl/libsvn_swig_perl/swigutil_pl.c:24:
:info:build /opt/local/include/apr-1/apr.h:165:10: fatal error: 'sys/types.h' file not found
:info:build #include <sys/types.h>
:info:build          ^~~~~~~~~~~~~
:info:build 1 error generated.
:info:build make: *** [subversion/bindings/swig/perl/libsvn_swig_perl/swigutil_pl.lo] Error 1

Creating a symbolic link for 11.0 (sudo ln -s MacOSX11.1.sdk MacOSX11.0.sdk) makes it build farther, but then it runs into #61523.

[ryandesign (Ryan Carsten Schmidt)]

That's #61321.

Making a symlink to an SDK is not recommended.

[jmroot (Joshua Root)]

We should do a sanity check on the SDK regardless of the selection order. Can anyone provide an ls -R of the incomplete MacOSX11.0.sdk that is left after updating?

[jmroot (Joshua Root)]

In 3c0f70bb7d5b5e311538806022318a06d6e3c623/macports-base (master):

Add sanity check to find_close_sdk

Closes: #61866

[jmroot (Joshua Root)]

As no data has been forthcoming, I made my best guess as to what would distinguish a broken partial SDK from a complete one.