compilers portgroup is not correctly testing for gcc_v

[ballapete (Peter "Pete" Dyballa)]

boost176 could not be built because a dozen or more times -Wl,-rpath,/opt/local/lib/libgcc is passed to ld although it's known that on Tiger this does not work, see #65867. After some time I realised that the cause for the repeated faults is in the build system, via one of the included PortGroups. So I started to change compilers-1.0.tcl files, see below:

-rw-r--r--   1 root  x11    31485 16 Nov 23:03 /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/compilers-1.0.tcl
-rw-r--r--   1 root  admin  30243 16 Nov 22:31 /opt/local/var/macports/registry/portgroups/d0cf92b7e597517dcb1fc9ec40ac4d930855baa25e0198ecb05bdd3471c04584-30309/compilers-1.0.tcl
-rw-r--r--   1 root  admin  29895 21 Apr  2022 /opt/local/var/macports/registry/portgroups/ebfc4fd7146e39c2055b915a303f8cc3ce9bc8b07993fcb3342bf2ad2903f6bb-29895/compilers-1.0.tcl
-rw-r--r--   1 root  admin  29542 21 Jun  2021 /opt/local/var/macports/registry/portgroups/fdaf8a4848353f3177ce73946b3e062f5420bc49a55624bd3c3f2d46a5b82b3a-29542/compilers-1.0.tcl

I finally succeed and boost176 could be built overnight.

How can I remove old and unnecessary portgroups files?

[jmroot (Joshua Root)]

The portgroup files in the registry are deleted when you uninstall the ports that use them. They don't have any effect on installing new ports.

[ballapete (Peter "Pete" Dyballa)]

How can I find the portgroup file that is responsible for a particular port's version to be able to correct or inspect it?

[jmroot (Joshua Root)]

You don't need to (and shouldn't) change any files in the registry. The portgroups under _resources/port1.0/group in the ports tree(s) configured in sources.conf will always be used when building ports.

[ballapete (Peter "Pete" Dyballa)]

A portgroups file on Tiger that sets LDFLAGS to contain -Wl,-rpath,/opt/local/lib/libgcc is faulty.

[ballapete (Peter "Pete" Dyballa)]

boost176 uses a user-config.jam file that contains the faulty -Wl,-rpath,/opt/local/lib/libgcc. Since I do not know how it is built, I checked possible template files in the archive like boost_1_76_0/libs/beast/tools/user-config.jam or (less probable) boost_1_76_0/tools/build/example/user-config.jam. Both contain no hint of an ld setting like -Wl,-rpath,/opt/local/lib/libgcc. On Tiger it is introduced while creating the build environment on Tiger from the faulty new version of /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/compilers-1.0.tcl.

[ballapete (Peter "Pete" Dyballa)]

Since I succeeded to build icu a few more ports needed to be rebuilt, and boost176 is one of them. Again it failed to build because of -Wl,-rpath,/opt/local/lib/libgcc introduced by port selfupdate into /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/compilers-1.0.tcl. Removing that faulty line boost176 has again a clean build environment. Noting more changed. Which additional proof is needed to show that the new /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/compilers-1.0.tcl is faulty in Tiger?

[ballapete (Peter "Pete" Dyballa)]

This excerpt from the build log

DEBUG: Merging existing requested variants '+no_single+no_static+python39' into variants
DEBUG: new fully merged portvariants: no_static + python39 + no_single +
DEBUG: Changing to port directory: /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/devel/boost176
DEBUG: OS darwin/8.11.0 (macOS 10.4.11) arch powerpc
DEBUG: Sourcing PortGroup compiler_blacklist_versions 1.0 from /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/compiler_blacklist_versions-1.0.tcl
DEBUG: Sourcing PortGroup active_variants 1.1 from /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/active_variants-1.1.tcl
DEBUG: GCC versions for Darwin 8 powerpc - 5 6 7 8 9
DEBUG: Clang versions for Darwin 8 powerpc - 3.3 3.4
DEBUG: Sourcing PortGroup compilers 1.0 from /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/compilers-1.0.tcl
DEBUG: Sourcing PortGroup mpi 1.0 from /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/mpi-1.0.tcl
DEBUG: PortGroup active_variants 1.1 included previously, not sourcing again
DEBUG: compiler clang blacklisted because it's not installed or it doesn't work
DEBUG: Unmatched blacklisted compiler: clang
DEBUG: Reading variant descriptions from /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/variant_descriptions.conf
DEBUG: universal variant already exists, so not adding the default one
DEBUG: Executing variant python39 provides python39
DEBUG: Executing variant no_static provides no_static
DEBUG: Executing variant no_single provides no_single
DEBUG: Running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Chosen compiler macports-gcc-7 is provided by a port, adding dependency
DEBUG: Adding depends_build port:gcc7
DEBUG: Adding depends_lib path:share/doc/libgcc/README:libgcc
DEBUG: Adding depends_skip_archcheck gcc7
DEBUG: Finished running callback portconfigure::add_automatic_compiler_dependencies
DEBUG: Running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Finished running callback portbuild::add_automatic_buildsystem_dependencies
DEBUG: Running callback portstartupitem::add_notes
DEBUG: Finished running callback portstartupitem::add_notes
DEBUG: Running callback compilers::add_fortran_legacy_support
DEBUG: Finished running callback compilers::add_fortran_legacy_support
DEBUG: Running callback compilers::add_gcc_rpath_support
DEBUG: Finished running callback compilers::add_gcc_rpath_support
DEBUG: rev-upgrade override ... upgrading (from source)!

shows that the function add_gcc_rpath_support is called twice without need.

[ballapete (Peter "Pete" Dyballa)]

It's more likely that the TCL files are OK.

foreach PORT ( `port echo active | gawk '{ print $1 }'` )
    ggrep -i PortGroup `port file $PORT` /dev/null | ggrep compiler | gsed -e 's,^.*/\([^/].*/Portfile:\),\1 ,' | gawk '{ printf "%-32s%-28s%-8s", $1, $3, $4 }' && gprintf "@%s%11s\n" `port -q distfiles $PORT | ggrep size`	# less empty lines
end

gave me a list of ports that use this or that compiler related PortGroup file, and the size of their source archive files:

llvm-3.4/Portfile:              compiler_blacklist_versions 1.0     @           
ld64/Portfile:                  compiler_blacklist_versions 1.0     @size:     907171       ≈1 min
popt/Portfile:                  compiler_blacklist_versions 1.0     @size:     580569       ≈2 min
isl/Portfile:                   compiler_blacklist_versions 1.0     @size:    2261594       ≈3 min
dav1d/Portfile:                 compiler_blacklist_versions 1.0     @size:     962475       ≈5 min
rsync/Portfile:                 compiler_blacklist_versions 1.0     @size:    1149787       ≈5 min
grep/Portfile:                  compiler_blacklist_versions 1.0     @size:    1709536       ≈5 min
libusb/Portfile:                compiler_blacklist_versions 1.0     @size:     383960       ≈6 min
blackbox/Portfile:              compiler_blacklist_versions 1.0     @size:     558554       ≈8 min
findutils/Portfile:             compiler_blacklist_versions 1.0     @size:    2046252       ≈8 min
libpixman/Portfile:             compiler_blacklist_versions 1.0     @size:     756898       ≈8 min
libgcrypt/Portfile:             compiler_blacklist_versions 1.0     @size:    3778457       ≈9 min
py-cython/Portfile:             compiler_blacklist_versions 1.0     @size:    2088773      ≈12 min
ld64/Portfile:                  compiler_blacklist_versions 1.0     @size:     421947      ≈13 min
mpfr/Portfile:                  compiler_blacklist_versions 1.0     @size:    1525476      ≈14 min
unbound/Portfile:               compiler_blacklist_versions 1.0     @size:    6235060      ≈14 min
groff/Portfile:                 compiler_blacklist_versions 1.0     @size:    4137480      ≈16 min
kerberos5/Portfile:             compiler_blacklist_versions 1.0     @size:    8661660      ≈18 min
nettle/Portfile:                compiler_blacklist_versions 1.0     @size:    2406251      ≈19 min
xorg-libX11/Portfile:           compiler_blacklist_versions 1.0     @size:    1823712      ≈23 min
cairo/Portfile:                 compiler_blacklist_versions 1.0     @size:   41834076      ≈24 min
mesa/Portfile:                  compiler_blacklist_versions 1.0     @size:   16109944      ≈29 min
fftw-3/Portfile:                compiler_blacklist_versions 1.0     @size:    4144100      ≈30 min
gettext/Portfile:               compiler_blacklist_versions 1.0     @size:   24181849      ≈37 min
harfbuzz/Portfile:              compiler_blacklist_versions 1.0     @size:   17874260      ≈39 min
glib2/Portfile:                 compiler_blacklist_versions 1.0     @size:    4822784      ≈40 min
coreutils/Portfile:             compiler_blacklist_versions 1.0     @size:    5712104      ≈43 min
graphviz-devel/Portfile:        compiler_blacklist_versions 1.0     @size:   27582600      ≈43 min
gnutls/Portfile:                compiler_blacklist_versions 1.0     @size:    5639992   ≈1h 12 min
doxygen/Portfile:               compiler_blacklist_versions 1.0     @size:    5152094   ≈1h 27 min
dvisvgm/Portfile:               compiler_blacklist_versions 1.0     @size:    2709542   ≈1h 42 min
texlive-bin/Portfile:           compiler_blacklist_versions 1.0     @size:   24722892   ≈2h 15 min
OpenBLAS/Portfile:              compilers                   1.0     @size:   23738989   ≈2h 37 min
openssl3/Portfile:              compiler_blacklist_versions 1.0     @size:   15107575   ≈2h 44 min
gtk3/Portfile:                  compiler_blacklist_versions 1.0     @size:   21587592   ≈3h  5 min
(boost176/Portfile:              compiler_blacklist_versions 1.0     @size:  110073117   ≈3h 20 min)
xorg-server-devel/Portfile:     compiler_blacklist_versions 1.0     @size:    5844257   ≈ min
gcc7/Portfile:                  compiler_blacklist_versions 1.0     @size:   62783088   ≈ min
gcc6/Portfile:                  compiler_blacklist_versions 1.0     @size:   74355588   ≈ min
nss/Portfile:                   compiler_blacklist_versions 1.0     @size:   84717969   ≈ min
 

Then I started to build the ports (time recorded in last column). Some cannot be built (xorg-server-devel, nss), some would take too many days (GCC6/GCC7, llvm-3.4). None of the ports I tried, except boost176, failed to build because of a setting like -Wl,-rpath. They do use -rpath à la:

opt/local/var/macports/logs/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_devel_gettext/gettext/main.log::info:build /bin/sh ../libtool  --tag=CC   --mode=link /opt/local/bin/gcc-apple-4.2 -std=gnu99 -fvisibility=hidden -pipe -Os -arch ppc  -liconv -Wl,-framework -Wl,CoreFoundation  -no-undefined -export-symbols-regex '^([^g]|g[^l]|gl[^w]|glw[^t]|glwt[^h]|glwth[^r]|glwthr[^e]|glwthre[^a]|glwthrea[^d]).*' -version-info 10:0:2 -rpath /opt/local/lib -L/opt/local/lib -Wl,-headerpad_max_install_names -arch ppc -o libintl.la  bindtextdom.lo dcgettext.lo dgettext.lo gettext.lo finddomain.lo hash-string.lo loadmsgcat.lo localealias.lo textdomain.lo l10nflist.lo explodename.lo dcigettext.lo dcngettext.lo dngettext.lo ngettext.lo plural.lo plural-exp.lo localcharset.lo threadlib.lo lock.lo relocatable.lo langprefs.lo localename.lo localename-table.lo log.lo printf.lo setlocale.lo setlocale-lock.lo setlocale_null.lo version.lo xsize.lo osdep.lo intl-compat.lo

So my plaint is faulty – and so is boost176's Portfile. boost176 only builds when I remove line #844 from /opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/_resources/port1.0/group/compilers-1.0.tcl.

[kencu (Ken)]

How is that getting added, I wonder. The PG tests for gcc10 or greater, or devel:

  if { ${gcc_v} >= 10 || ${gcc_v} == "devel" } {
        configure.ldflags-delete  -Wl,-rpath,${prefix}/lib/libgcc
        configure.ldflags-append  -Wl,-rpath,${prefix}/lib/libgcc
    }

and you're using gcc7, so it should not be added to your build.

Anyway, soon enough there will be a gcc10 on Tiger, so the test might as well be fixed to not use rpaths on Tiger, assuming MacPorts is going to keep supporting Tiger much longer.

[ballapete (Peter "Pete" Dyballa)]

The bug is clearly in boost176 Portfile, since the other ports build, so this ticket can be closed.

[kencu (Ken)]

I can't build boost171 on Tiger unless I tweak the compilers PG as well. The compilers PortGroup is adding the rpath flag on Tiger, and that needs to be fixed.

The code to return the gcc version seems to be broken -- on Tiger, it returns UNKNOWN.

I guess to TCL, "UNKNOWN" is >= 10, so the rpath is added.

[kencu (Ken)]

Chris,

The compilers PG seems to be doing something off at least with the boost ports. We noted it is always adding the RPATH on Tiger, which started this, but the issue is not with Tiger, but with the way it is finding the current gcc_version to see if RPATH should be added.

On Mojave, testing with boost171, I added this to the compilers PG:

proc compilers::add_gcc_rpath_support {} {
    global prefix  
    set gcc_v [compilers::get_current_gcc_version]
    if { ${gcc_v} >= 10 || ${gcc_v} == "devel" } {
puts "adding RPATH, because gcc_v is:"
puts ${gcc_v}
        configure.ldflags-delete  -Wl,-rpath,${prefix}/lib/libgcc
        configure.ldflags-append  -Wl,-rpath,${prefix}/lib/libgcc
    }
}

and I see this, doing a "port info" with boost171:

$ port info
adding RPATH, because gcc_v is:
UNKNOWN
adding RPATH, because gcc_v is:
UNKNOWN
boost171 @1.71.0_6 (devel)

I see the same on Tiger, where gcc is gcc7. The error shows up on Tiger because rpath is not supported there, but Tiger is not the cause of the issue.

[kencu (Ken)]

This commit [013f72b85e6a840d47321d76be6d995ae76800af/macports-ports] takes the step of not adding rpath if the system is Tiger. That covers off the immediate Tiger rpath issue, and heads off the time in the future as well when gcc10+ is available on Tiger.

But the issue with the gcc_v returning as UNKNOWN remains, and still should be fixed separately.