muniversal portgroup header merging doesn't work on Ventura: diff: unrecognized option `--old-group-format=#ifdef __arm64__

[Morpheus-HH]

%~ $ sudo port clean libffi
--->  Cleaning libffi
%~ $ sudo port upgrade outdated
--->  Fetching archive for libffi
--->  Attempting to fetch libffi-3.4.3_0+universal.darwin_22.arm64-x86_64.tbz2 from https://fra.de.packages.macports.org/libffi
--->  Attempting to fetch libffi-3.4.3_0+universal.darwin_22.arm64-x86_64.tbz2 from https://packages.macports.org/libffi
--->  Attempting to fetch libffi-3.4.3_0+universal.darwin_22.arm64-x86_64.tbz2 from https://nue.de.packages.macports.org/libffi
--->  Fetching distfiles for libffi
--->  Verifying checksums for libffi
--->  Extracting libffi
--->  Applying patches to libffi
--->  Configuring libffi
--->  Building libffi                                    
--->  Staging libffi into destroot                       
Error: Failed to destroot libffi: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_libffi/libffi/main.log for details.

[Morpheus-HH]

Logfile

[ryandesign (Ryan Carsten Schmidt)]

The log says the problem is:

diff: unrecognized option `--old-group-format=#ifdef __arm64__

This usage of diff comes from the MacPorts muniversal portgroup which is used by libffi and many other ports.

Perhaps the diff utility changed in Ventura such that we can no longer do this. Another ticket suggested that the patch utility, which is related to the diff utility, changed in Ventura.

[nrlmol2018]

Probably the origin of this error comes earlier:

:info:destroot Command failed: /usr/bin/cmp -s "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_libffi/libffi/work/destroot-arm64//opt/local/include/ffitarget.h" "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_libffi/libffi/work/destroot-ppc-intel//opt/local/include/ffitarget.h" && /bin/cp -v "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_libffi/libffi/work/destroot-arm64//opt/local/include/ffitarget.h" "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_libffi/libffi/work/destroot//opt/local/include"
:info:destroot Exit code: 1

Because the cmp -s command returns 1, the cp command is never executed

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #66135.

[radarhere (Andrew Murray)]

Minor note - the 'ventura' keyword has been misspelt.

[astroboylrx (Rixin Li)]

log for failed build

[astroboylrx (Rixin Li)]

Updated: my apologies. I gave it a try this morning with a fresh start. It built successfully. A bit confused what happened.

---

I failed to build libffi. main.2.log attached above. Help would be greatly appreciated!

Relevant info seems to be

:info:build Undefined symbols for architecture arm64:
  :info:build   "_ffi_call", referenced from:
  :info:build       _ffi_raw_call in raw_api.o
  :info:build       _ffi_java_raw_call in java_raw_api.o
  :info:build   "_ffi_prep_cif_machdep", referenced from:
  :info:build       _ffi_prep_cif_core in prep_cif.o
  :info:build   "_ffi_prep_closure_loc", referenced from:
  :info:build       _ffi_prep_closure in prep_cif.o
  :info:build       _ffi_prep_raw_closure_loc in raw_api.o
  :info:build       _ffi_prep_raw_closure in raw_api.o
  :info:build       _ffi_prep_java_raw_closure_loc in java_raw_api.o
  :info:build       _ffi_prep_java_raw_closure in java_raw_api.o
  :info:build ld: symbol(s) not found for architecture arm64
  :info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
  :info:build make[3]: *** [libffi.la] Error 1

Also, I feel that this should be higher priority since lots of fundamental ports (e.g., gcc12, python39) depend on libffi

[dkoes (David Koes)]

I modified /opt/local/var/macports/sources/rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/muniversal-1.0.tcl to use a GNU diff instead of /usr/bin/diff and successfully built libffi.

[easye]

I second @astroboylrx's concern about building libffi for Ventura being fairly fundamental.

Is there some consensus on how to resolve this?

[kencu (Ken)]

We don't have the source code that Apple has used for diff in Ventura yet; I'm not sure why they would have removed this functionality from diff -- I can't think of a reason.

This feature of diff is used in muniversal to branch in different files based on arch. It might be hard to get along without it... there are hundreds if not thousands of ports that use this now, and who knows what breakage there might be without this feature? So we might need gnudiff.

As diff is used in the muniversal PortGroup, if we're going to dep muniversal on diffutils, then nothing in the build tree for diffutils can use muniversal or there will be a bootstrap conflict.

And there are a few things in the rdeps to worry about:

 % port rdeps diffutils
The following ports are dependencies of diffutils @3.8_0:
  xz
    gettext
      libiconv
        gperf
      libtextstyle
        ncurses
      gettext-runtime
      gettext-tools-libs

perhaps that list could be trimmed up with some looking for an option that doesn't use "xz".

[kencu (Ken)]

The dep list on older systems is onerous as older clangs are all blacklisted, so if this switch to diffutils is done, perhaps it would be only for Ventura+. And we really don't like that idea much either, as different diffs on different platforms is just another recipe for troubles.

[radarhere (Andrew Murray)]

A patch for the temporary fix using diffutils has been posted in ​https://trac.macports.org/ticket/66163

[Gcenx]

Replying to radarhere:

A patch for the temporary fix using diffutils has been posted in ​https://trac.macports.org/ticket/66163

The fun part of using diffutils is this also can’t be built as universal without adding one of the muniversal portgroups this casing a circular dependency.

[kencu (Ken)]

yup. luckily never need it universal!

[Gcenx]

Replying to kencu:

yup. luckily never need it universal!

Well it shouldn’t be needed but I’ve seen Macports force build deps be +universal many times, luckily for diffutils at least it’s universal build didn’t actual need to diff stage.

[kencu (Ken)]

That happens when you try to install a port as "universal" and it starts pulling in deps.

If the dep has a universal variant, the variant request propagates to the the dep, even if the dep is not needed to be universal.

Ports can indicate they don't supply libraries by setting "installs_libs no" in the Portfile, but when macports is running the deps it doesn't know what you want exactly, so goes for the universal install of the dep.

If we make a sequestered "diffutils" port that installs "diff" into ${prefix}/libexec, we can just set it to indicate it has no universal variant, and this won't happen.

[Ken <21211439+kencu@…>]

In 5dccc228b10aff819500672c6f4203da53849dc3/macports-ports (master):

diffutils-for-muniversal: v3.8 (​https://github.com/macports/macports-ports/pull/16642)

this commit adds a sequestered, isolated diff utility for use on Ventura (and above).
It also changes the muniversal PortGroups to use the new diff on Ventura+/

the new diff utility that comes with Ventura does not accept options that the muniversal PortGroup requires to properly function.

closes: #66163
closes: #66103