lima: Build failure on old OS that uses legacysupport

[ryandesign (Ryan Carsten Schmidt)]

lima fails to build on older OS versions that use legacy support, because it is trying to link Linux executables with the macOS legacy support library:

​https://build.macports.org/builders/ports-10.10_x86_64-builder/builds/166362/steps/install-port/logs/stdio

GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -ldflags="-s -w -X github.com/lima-vm/lima/pkg/version.Version=" -o _output/share/lima/lima-guestagent.Linux-x86_64 ./cmd/lima-guestagent
GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -ldflags="-s -w -X github.com/lima-vm/lima/pkg/version.Version=" -o _output/share/lima/lima-guestagent.Linux-aarch64 ./cmd/lima-guestagent
cp -a ./cmd/nerdctl.lima _output/bin/nerdctl.lima
cp -a ./cmd/lima _output/bin/lima
# calls the native resolver library and not the simplistic version in the Go library.
CGO_ENABLED=1 go build -ldflags="-s -w -X github.com/lima-vm/lima/pkg/version.Version=" -o _output/bin/limactl ./cmd/limactl
# github.com/lima-vm/lima/cmd/lima-guestagent
loadinternal: cannot find runtime/cgo
/opt/local/lib/go/pkg/tool/darwin_amd64/link: running /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_sysutils_lima/lima/work/compwrap/cc/usr/bin/clang failed: exit status 1
ld: warning: option -s is obsolete and being ignored
ld: warning: ignoring file /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_sysutils_lima/lima/work/.tmp/go-link-2121192670/go.o, file was built for unsupported file format ( 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 ) which is not the architecture being linked (x86_64): /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_sysutils_lima/lima/work/.tmp/go-link-2121192670/go.o
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

# github.com/lima-vm/lima/cmd/lima-guestagent
loadinternal: cannot find runtime/cgo
/opt/local/lib/go/pkg/tool/darwin_amd64/link: running /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_sysutils_lima/lima/work/compwrap/cc/usr/bin/clang failed: exit status 1
ld: warning: option -s is obsolete and being ignored
ld: warning: ignoring file /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_sysutils_lima/lima/work/.tmp/go-link-4108824845/go.o, file was built for unsupported file format ( 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 ) which is not the architecture being linked (x86_64): /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_sysutils_lima/lima/work/.tmp/go-link-4108824845/go.o
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

See #64174 for inspiration on what to do about this. (For example: do you really need to build the Linux executables?)

[ryandesign (Ryan Carsten Schmidt)]

This is still happening.

​https://build.macports.org/builders/ports-10.12_x86_64-builder/builds/176944/steps/install-port/logs/stdio

[ryandesign (Ryan Carsten Schmidt)]

Herby, any comment on this issue? Do you agree that the solution from #64174 could work here?

P.S.: 0.11.1 is out.

[herbygillot (Herby Gillot)]

Hey Ryan, I'm not sure that that would be a solution as those binaries are needed by lima to operate on guest instances that are internally Linux x86_64. Perhaps the right thing to do here would be to restrict lima from building on macOS versions that are too old.

[ryandesign (Ryan Carsten Schmidt)]

those binaries are needed by lima

Ok, good to know.

Perhaps [...] restrict lima from building on macOS versions that are too old

You could do this if you must, but if building Linux executables on macOS is something that other go ports might also need to do for similar reasons, then maybe we should take a moment to figure out why it doesn't work here and how to fix it.

MacPorts and the legacysupport portgroup are providing a number of ldflags to the port which are only appropriate when building macOS software, yet it seems like the build is using them to try to build Linux software, which unsurprisingly does not work.

The way that this appears to be happening is that the compiler_wrapper 1.0 portgroup is being used (the lima portfile includes the golang 1.0 portgroup which includes the legacysupport 1.1 and compiler_wrapper 1.0 portgroups, so this issue potentially affects all go ports) and the compiler_wrapper portgroup creates compiler wrappers and tells the port to use them instead of using the compiler directly. The compiler wrappers contain some additional flags.

On High Sierra, where legacy support is not in effect, the cc compiler wrapper contains:

#!/bin/bash
export CCACHE_DIR=/opt/local/var/macports/build/.ccache
exec /opt/local/bin/ccache /usr/bin/clang   -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk -pipe ${MACPORTS_LEGACY_SUPPORT_CPPFLAGS} "${@}"

so we see some cflags have been inserted. But on Sierra, where legacy support is in effect, it contains:

#!/bin/bash
export CCACHE_DIR=/opt/local/var/macports/build/.ccache
exec /opt/local/bin/ccache /usr/bin/clang   -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.12.sdk -pipe ${MACPORTS_LEGACY_SUPPORT_CPPFLAGS} -Wl,-headerpad_max_install_names -L/opt/local/lib -lMacportsLegacySupport "${@//-static/}" -Wno-error

so we see that not only cflags but also ldflags have been inserted, which seems like it might be the source of the problem.

Compiler wrappers can be troublesome since there may well be situations like the one lima is in now where it wants to compile some things without all of the flags added in the wrapper. MacPorts resisted the use of compiler wrappers for nearly two decades before this portgroup was created and managed to get by without them by instead supplying the correct flags in environment variables and ensuring that each port's build system obeys those environment variables (or other similar methods). Is there a way to make the golang portgroup and all go ports build this way, or at least the lima port?

Alternately, is there a way that the compiler wrappers could be enhanced to avoid this problem? I don't know the complete set of arguments or environment variables that go is sending to the compiler wrapper, but for example if the GOOS environment variable gets passed through, then the compiler wrapper could choose to only add the legacysupport ldflags when GOOS is "darwin" (or is unset).