gmake @4.3_0+universal: build error on BigSur arm64

[kode54 (Christopher Snowhill)]

The x86 64 architecture of the universal variant is failing to build on arm64, due to some configuration issue that causes it to #define common variable types to other things, then importing the Darwin headers which typedef those variable types, only to attempt to create a typedef for something like "int" as "unsigned long long". Attaching build log.

[kode54 (Christopher Snowhill)]

Build log for gmake

[ryandesign (Ryan Carsten Schmidt)]

Could you attach the config.log as well? It will be in the directory identified by the command port work gmake.

[kode54 (Christopher Snowhill)]

Config log

[kencu (Ken)]

can you please make 100% sure you also have updated your command line tools to the latest version, to match xcode?

thanks

[kode54 (Christopher Snowhill)]

/Library/Developer/CommandLineTools/SDKs contains a MacOS11.3.sdk. Though I have xcode-select configured to use Xcode 12.5, not the Command Line Tools. I also have Xcode located on an external Thunderbolt drive.

[kencu (Ken)]

OK. You can be 100% sure by checking the versions of clang in the CLTs vs Xcode, but sounds like the CLTs are up to date.

Weird error, though.

Just by the way -- do you actually need a universal gmake for something? It's just a build tool, no libraries or anything, so there is no need for it to be universal, in general...

Sure, it would be nice if everything builds universal -- but I hope you aren't actually stuck with your project, whatever it is, due to this...

[kode54 (Christopher Snowhill)]

Attempting to install fluidsynth +universal builds all dependencies +universal, including gmake.

[kencu (Ken)]

try just installing gmake by itself first:

sudo port -v install gmake

and then install fluidsynth the way you want:

sudo port -v install fluidsynth +universal

That is likely to work; MacPorts idiosyncracy.

[kencu (Ken)]

On BigSur Intel this morning, I see a similar error to the one you saw.

Adding one of MacPorts' standard workarounds worked:

% port -v installed gmake
The following ports are currently installed:
  gmake @4.3_0+universal (active) platform='darwin 20' archs='arm64 x86_64' date='2021-05-08T11:28:45-0700'

Here's what I did:

% git diff
diff --git a/devel/gmake/Portfile b/devel/gmake/Portfile
index 7c3f814923f..3db0854196b 100644
--- a/devel/gmake/Portfile
+++ b/devel/gmake/Portfile
@@ -1,6 +1,7 @@
 # -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4
 
 PortSystem 1.0
+PortGroup muniversal 1.0
 
 name            gmake
 version         4.3

I won't try to take on whether this is the "one true fix", but if you need it fixed, this did it for me.

[ryandesign (Ryan Carsten Schmidt)]

I can reproduce this on macOS 11.0.1 Intel with Xcode 12.4 when building universal, but not when building non-universal.

Where things seem to start to go wrong in config.log is that several tests, such as whether the system has something basic like ssize_t, are failing because:

In file included from conftest.c:83:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/sys/types.h:81:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/machine/endian.h:35:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/i386/endian.h:99:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/sys/_endian.h:130:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/_OSByteOrder.h:80:
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/arm/OSByteOrder.h:15:1: error: redefinition of '_OSSwapInt16'
_OSSwapInt16(
^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/i386/_OSByteOrder.h:46:1: note: previous definition is here
_OSSwapInt16(
^
In file included from conftest.c:83:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/sys/types.h:81:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/machine/endian.h:35:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/i386/endian.h:99:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/sys/_endian.h:130:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/_OSByteOrder.h:80:
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/arm/OSByteOrder.h:25:1: error: redefinition of '_OSSwapInt32'
_OSSwapInt32(
^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/i386/_OSByteOrder.h:55:1: note: previous definition is here
_OSSwapInt32(
^
In file included from conftest.c:83:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/sys/types.h:81:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/machine/endian.h:35:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/i386/endian.h:99:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/sys/_endian.h:130:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/_OSByteOrder.h:80:
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/arm/OSByteOrder.h:41:1: error: redefinition of '_OSSwapInt64'
_OSSwapInt64(
^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/libkern/i386/_OSByteOrder.h:70:1: note: previous definition is here
_OSSwapInt64(
^
3 errors generated.

And because it thinks we don't have ssize_t it defines its own which causes everything that follows to be wrong.

Those errors look... terrible? I mean start with machine/endian.h which includes i386/endian.h if building for Intel or arm/endian.h if building for ARM with no consideration for what should happen if we want to build for both. Fortunately those files are are identical. But then we come to libkern/_OSByteOrder.h which includes libkern/i386/_OSByteOrder.h if building for Intel and libkern/arm/OSByteOrder.h if building for ARM, and they both try to define the same functions, so of course that can't work. I don't understand why the system headers appear to be designed to create conflicting definitions of OSSwapInt* when building universal. Given this problem that seems to occur just by including the ubiquitous <sys/types.h>, I don't understand how we're able to build anything universal on macOS 11 at all, without resorting to using muniversal for everything.

[ryandesign (Ryan Carsten Schmidt)]

Replying to kode54:

I have xcode-select configured to use Xcode 12.5, not the Command Line Tools.

MacPorts will use either the command line tools or Xcode to build a port, depending on what is installed. If both are installed, it will use the command line tools unless the port says it requires Xcode.

[kencu (Ken)]

Don't forget that when building with multiple arch flags, clang actually runs the build twice internally, once for each arch, and then lipos the results together.

Sort of like a little mini-mununiversal-portgroup.

So that's how this usually can work at all...

[ryandesign (Ryan Carsten Schmidt)]

Ok right, of course. I remember now. So only one of __i386__ or __x86_64__ or __arm__ or __arm64__ should be defined at a time. So we need to figure out what is causing __arm__ or __arm64__ to be defined when building the x86_64 part.

[ryandesign (Ryan Carsten Schmidt)]

Same issue reported in gtime in #62013 though there it looks more like interference from the bundled gnulib. That could be a problem here too, but I'm not sure.

[ryandesign (Ryan Carsten Schmidt)]

Same issue reported in libunistring in #61858.

[FaradayLight (Faraday Light)]

Replying to kencu:

OK. You can be 100% sure by checking the versions of clang in the CLTs vs Xcode, but sounds like the CLTs are up to date.

Weird error, though.

Just by the way -- do you actually need a universal gmake for something? It's just a build tool, no libraries or anything, so there is no need for it to be universal, in general...

Sure, it would be nice if everything builds universal -- but I hope you aren't actually stuck with your project, whatever it is, due to this...

I am experiencing the same problem under MacPorts 2.7.1 on arm64 big Sur.

The +universal option was set by default in a recent update to MacPorts in: /opt/local/etc/macports/variants.conf

[FaradayLight (Faraday Light)]

See attached main_gmake.LibreOffice for details, but some items:

:info:configure checking for eaccess... no                                                                                                                                                                        
:info:configure checking whether getdtablesize is declared... yes
:info:configure checking host CPU and C ABI... grep: conftest.s: No such file or directory
:info:configure arm

Installed using:

sudo port clean gmake
sudo port -v install gmake

I am seeing many type errors reported, and the the following at the end:

In file included from findprog-in.c:28:
In file included from ./unistd.h:40:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/unistd.h:76:
/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/sys/_types/_off_t.h:31:33: error: 'long type-name' is invalid
../src/config.h:1055:15: note: expanded from macro 'off_t'
3 error#define off_t long int
              ^
s generated.
3 errors generated.
2 errors generated.
make[3]: *** [dirname-lgpl.o] Error 1
make[3]: *** [concat-filename.o] Error 1
make[3]: *** [getprogname.o] Error 1
5 errors generated.
make[3]: *** [findprog-in.o] Error 1
make[3]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmake/gmake/work/make-4.3/lib'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmake/gmake/work/make-4.3/lib'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmake/gmake/work/make-4.3/lib'
make: *** [all-recursive] Error 1
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmake/gmake/work/make-4.3'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmake/gmake/work/make-4.3" && /usr/bin/make -j8 -w all 
Exit code: 2
Error: Failed to build gmake: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gmake/gmake/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug.
Error: Processing of port gmake failed

[kencu (Ken)]

Replying to FaradayLight:

The +universal option was set by default in a recent update to MacPorts in: /opt/local/etc/macports/variants.conf

Well if so, turn that off at warp 10.

There is nothing I know about in a MacPorts update that would lead to +universal being set in variants.conf, but if somehow it did get set for you, and you presumably have no need for it, get rid of it.

It will cause you nothing but trouble.

[ryandesign (Ryan Carsten Schmidt)]

Same issue reported in grep in #64135.

[potterpg]

I have this issue, and yes I have the default variant +universal in the conf file, but I want that, I think...

I intend to rebuild something that uses many libraries, that will all need to be universal, so this makes sense for to me to set this?

gmake - is trying to download a file that does not exist
gmake-43_0+universal.darwin_20.arm64-x86_64.tbz2

from 3 different locations:
​https://packages.macports.org/gmake
​https://mse.uk.packages.macports.org/gmake/
​https://ema.uk.packages.macports.org/gmake/

that seems to be the main issue for gmake build failure for me?

[kencu (Ken)]

this simple fix works

comment:8

[ryandesign (Ryan Carsten Schmidt)]

Replying to potterpg:

I have this issue, and yes I have the default variant +universal in the conf file, but I want that, I think...

I intend to rebuild something that uses many libraries, that will all need to be universal, so this makes sense for to me to set this?

Sure, if you like.

gmake - is trying to download a file that does not exist
gmake-43_0+universal.darwin_20.arm64-x86_64.tbz2

from 3 different locations:
​https://packages.macports.org/gmake
​https://mse.uk.packages.macports.org/gmake/
​https://ema.uk.packages.macports.org/gmake/

that seems to be the main issue for gmake build failure for me?

No, that is not the reason it fails.

MacPorts tries to download a binary first. If that fails, it tries to build from source. If that fails (as this bug report says it does), then that is a bug to be fixed.

We do not produce binaries with nonstandard variants such as +universal except in the special case that some other port requires that. There are many other possible reason why a binary might not exist for a particular port.

[kencu (Ken)]

Perhaps I will just add the muniversal Portgroup and close this.

[laszlo-pusztai]

Overriding the Portfile locally with the muniversal portgroup added did the trick for me as well. (macOS 12.2.1 and Xcode 13.2.1 on arm64).

[ryandesign (Ryan Carsten Schmidt)]

In 75f236983efc1435a044fad34b0f22bb18d73021/macports-ports (master):

gmake: Use muniversal portgroup

Closes: #62822