perl5.28, perl5.30: forces modules to build with clang-3.4, though they don't declare a dependency on it

[ryandesign (Ryan Carsten Schmidt)]

See summary.

​https://build.macports.org/builders/ports-10.6_x86_64-builder/builds/11663/steps/install-port/logs/stdio

--->  Configuring p5.30-perlmagick
DEBUG: Preferred compilers: macports-clang-9.0 macports-clang-8.0 macports-clang-7.0 macports-clang-6.0 macports-clang-5.0 macports-clang-3.7 macports-clang-3.4
DEBUG: Using compiler 'MacPorts Clang 9.0'

/opt/local/bin/clang-mp-3.4 -c -I../ -I..   -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -I"/usr/include/ImageMagick"  -I../../ -I../..   -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -I"/usr/include/ImageMagick" -fno-common -DPERL_DARWIN -mmacosx-version-min=10.6 -pipe -Os  -fno-strict-aliasing -fstack-protector -I/opt/local/include -DPERL_USE_SAFE_PUTENV -I/opt/local/include/libxml2 -I/opt/local/include -I/opt/local/include -I/opt/local/include/libraw -I/opt/local/include -I/opt/local/include/libpng16 -I/opt/local/include -D_THREAD_SAFE -I/opt/local/include/OpenEXR -I/opt/local/include -I/opt/local/include/openjpeg-2.3 -I/opt/local/include -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -I/opt/local/include -I/opt/local/include/ossp -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -I/opt/local/include -I/opt/local/include  -I/opt/local/include   -g -O2 -Wall -fexceptions -D_FORTIFY_SOURCE=0 -pthread -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -arch x86_64 -isysroot/ -O3   -DVERSION=\"6.9.10\" -DXS_VERSION=\"6.9.10\"  "-I/opt/local/lib/perl5/5.30/darwin-thread-multi-2level/CORE" -D_LARGE_FILES=1 -DHAVE_CONFIG_H  -D_LARGE_FILES=1 -DHAVE_CONFIG_H Q16.c 
/bin/sh: /opt/local/bin/clang-mp-3.4: No such file or directory
make[1]: *** [Q16.o] Error 127

[jmroot (Joshua Root)]

Are you sure this happens with perl modules in general? You'll notice the configure phase is correctly showing /opt/local/bin/clang-mp-9.0 in use.

[ryandesign (Ryan Carsten Schmidt)]

Are you sure this happens with perl modules in general?

No, I am not sure.

You'll notice the configure phase is correctly showing /opt/local/bin/clang-mp-9.0 in use.

That's my point: the configure phase shows that the correct compiler is selected, yet by the build phase, the compiler that had been used to compile perl5.30 got used instead. Certainly the p5-perlmagick port does not specify clang-3.4, neither does the perlmagick build system, so the clang-3.4 compiler choice has to be coming from perl5.30's config files, and it seems to me that the bug is either perl5.30 doing that, or the perl5 portgroup not sufficiently overriding that.

Note that the problem does not occur with perl5.28:

$ sudo port -nd build subport=p5.28-perlmagick configure.compiler=macports-clang-8.0
...
DEBUG: Using compiler 'MacPorts Clang 8.0'
...
checking for gcc... /opt/local/bin/clang-mp-8.0
...
/opt/local/bin/clang-mp-8.0 -c -I../ -I..   -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  -I/opt/local/include  -I../../ -I../..   -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  -I/opt/local/include -fno-common -DPERL_DARWIN -mmacosx-version-min=10.13 -pipe -Os   -fno-strict-aliasing -fstack-protector-strong -I/opt/local/include -DPERL_USE_SAFE_PUTENV -I/opt/local/include/libxml2 -I/opt/local/include -I/opt/local/include -I/opt/local/include/libraw -I/opt/local/include -I/opt/local/include/libpng16 -I/opt/local/include  -I/opt/local/include/pango-1.0 -I/opt/local/include/harfbuzz -I/opt/local/include -I/opt/local/include/pango-1.0 -I/opt/local/include/fribidi -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1 -I/opt/local/include -I/opt/local/include/ossp -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -D_THREAD_SAFE -I/opt/local/include/OpenEXR -I/opt/local/include -I/opt/local/include/openjpeg-2.3 -I/opt/local/include -I/opt/local/include/graphviz -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -I/opt/local/include -I/opt/local/include/ossp -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -I/opt/local/include -I/opt/local/include  -I/opt/local/include   -g -O2 -Wall -fexceptions -D_FORTIFY_SOURCE=0 -pthread -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -arch x86_64 -arch i386 -isysroot/ -O3   -DVERSION=\"6.9.10\" -DXS_VERSION=\"6.9.10\"  "-I/opt/local/lib/perl5/5.28/darwin-thread-multi-2level/CORE" -D_LARGE_FILES=1 -DHAVE_CONFIG_H  -D_LARGE_FILES=1 -DHAVE_CONFIG_H Q16.c

But it does happen with perl5.30:

$ sudo port -nd build subport=p5.30-perlmagick configure.compiler=macports-clang-8.0
...
DEBUG: Using compiler 'MacPorts Clang 8.0'
...
checking for gcc... /opt/local/bin/clang-mp-8.0
...
/usr/bin/clang -c -I../ -I..   -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  -I/opt/local/include  -I../../ -I../..   -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16  -I/opt/local/include -fno-common -DPERL_DARWIN -mmacosx-version-min=10.13 -pipe -Os   -fno-strict-aliasing -fstack-protector-strong -I/opt/local/include -DPERL_USE_SAFE_PUTENV -I/opt/local/include/libxml2 -I/opt/local/include -I/opt/local/include -I/opt/local/include/libraw -I/opt/local/include -I/opt/local/include/libpng16 -I/opt/local/include  -I/opt/local/include/pango-1.0 -I/opt/local/include/harfbuzz -I/opt/local/include -I/opt/local/include/pango-1.0 -I/opt/local/include/fribidi -I/opt/local/include/cairo -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include -I/opt/local/include/pixman-1 -I/opt/local/include -I/opt/local/include/ossp -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -D_THREAD_SAFE -I/opt/local/include/OpenEXR -I/opt/local/include -I/opt/local/include/openjpeg-2.3 -I/opt/local/include -I/opt/local/include/graphviz -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -I/opt/local/include -I/opt/local/include/ossp -I/opt/local/include/freetype2 -I/opt/local/include/libpng16 -I/opt/local/include -I/opt/local/include -I/opt/local/include  -I/opt/local/include   -g -O2 -Wall -fexceptions -D_FORTIFY_SOURCE=0 -pthread -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 -arch x86_64 -arch i386 -isysroot/ -O3   -DVERSION=\"6.9.10\" -DXS_VERSION=\"6.9.10\"  "-I/opt/local/lib/perl5/5.30/darwin-thread-multi-2level/CORE" -D_LARGE_FILES=1 -DHAVE_CONFIG_H  -D_LARGE_FILES=1 -DHAVE_CONFIG_H Q16.c

This suggests to me that the bug is in perl5.30, not the perl5 portgroup.

[ryandesign (Ryan Carsten Schmidt)]

​Here's the same problem affecting p5.30-term-gnuplot, so it's not just a p5.30-perlmagick problem.

[ryandesign (Ryan Carsten Schmidt)]

Looks like [7080039472288d673d7b7cbd261095768bdc458f/macports-ports] fixed this for p5-perlmagick and p5-term-gnuplot.

But some other ports are still affected like ​p5.30-moose. Do we need to handle those on a case by case basis?

[jmroot (Joshua Root)]

I wonder if it might be worth replacing the build-time ${configure.cc} in Config_heavy.pl with something that will always exist, like /usr/bin/cc. Ports that aren't using the right compiler would still need to be fixed individually, but it would get past sanity checks like p5-moose uses and would allow end users to compile XS out of the box even if they don't have the exact clang port that was used to build perl.

[ryandesign (Ryan Carsten Schmidt)]

Another remaining example: ​p5.28-io-tty:

Now let's see what we can find out about your system
(logfiles of failing tests are available in the conf/ dir)...

ERROR: cannot run the configured compiler '/opt/local/bin/clang-mp-3.4'
(see conf/compilerok.log). Suggestions:
1) The compiler '/opt/local/bin/clang-mp-3.4' is not in your PATH. Add it
   to the PATH and try again. OR
2) The compiler isn't installed on your system. Install it. OR
3) You only have a different compiler installed (e.g. 'gcc').
   Either fix the compiler config in the perl Config.pm
   or install a perl that was built with the right compiler
   (you could build perl yourself with the available compiler).

Note: this is a system-administration issue, please ask your local
admin for help. Thank you.

It is so dumb that perl uses the compiler that was used to build itself to build other modules. What an idiotic design decision. Yes, let's replace it with /usr/bin/cc I guess.

[ryandesign (Ryan Carsten Schmidt)]

#60516 is another instance of this, with p5-sereal-decoder.

[kencu (Ken)]

This one bit me yet again.

$ sudo port -v destroot p5.28-xml-parser
--->  Computing dependencies for p5.28-xml-parser.
--->  Fetching distfiles for p5.28-xml-parser
--->  Verifying checksums for p5.28-xml-parser
--->  Checksumming XML-Parser-2.46.tar.gz
--->  Extracting p5.28-xml-parser
--->  Extracting XML-Parser-2.46.tar.gz
Executing:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_perl_p5-xml-parser/p5.28-xml-parser/work" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/perl5/XML-Parser-2.46.tar.gz' | /usr/bin/gnutar --no-same-owner -xf - 
--->  Configuring p5.28-xml-parser
Executing:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_perl_p5-xml-parser/p5.28-xml-parser/work/XML-Parser-2.46" && /opt/local/bin/perl5.28 Makefile.PL INSTALLDIRS=vendor CC="/opt/local/bin/clang-mp-9.0" LD="/opt/local/bin/clang-mp-9.0" 
Couldn't find your C compiler
Compilation failed in require at Makefile.PL line 4.
BEGIN failed--compilation aborted at Makefile.PL line 4.
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_perl_p5-xml-parser/p5.28-xml-parser/work/XML-Parser-2.46" && /opt/local/bin/perl5.28 Makefile.PL INSTALLDIRS=vendor CC="/opt/local/bin/clang-mp-9.0" LD="/opt/local/bin/clang-mp-9.0" 
Exit code: 2
Error: Failed to configure p5.28-xml-parser: configure failure: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_perl_p5-xml-parser/p5.28-xml-parser/main.log for details.

perl5 is hard-coded to build everything with clang-3.4. All is working again after reactivating clang-3.4:

$ perl5.28 -V
Summary of my perl5 (revision 5 version 28 subversion 2) configuration:
   
  Platform:
    osname=darwin
    osvers=10.8.0
    archname=darwin-thread-multi-2level
    uname='darwin kensmacbookpro.local 10.8.0 darwin kernel version 10.8.0: tue jun 7 16:33:36 pdt 2011; root:xnu-1504.15.3~1release_i386 i386 '
    config_args='-des -Dprefix=/opt/local -Dscriptdir=/opt/local/bin -Dvendorprefix=/opt/local -Dusemultiplicity=y -Dusethreads -Duseshrplib -Dcc=/opt/local/bin/clang-mp-3.4 -Dman1ext=1pm -Dman3ext=3pm -Dinstallstyle=lib/perl5 -Dman1dir=/opt/local/share/man/man1p -Dman3dir=/opt/local/share/man/man3p -Dsitebin=/opt/local/libexec/perl5.28/sitebin -Dsiteman1dir=/opt/local/share/perl5.28/siteman/man1 -Dsiteman3dir=/opt/local/share/perl5.28/siteman/man3 -Dvendorbin=/opt/local/libexec/perl5.28 -Dvendorman1dir=/opt/local/share/perl5.28/man/man1 -Dvendorman3dir=/opt/local/share/perl5.28/man/man3 -Dpager=/usr/bin/less -sR -Dperlpath=/opt/local/bin/perl5.28 -Dstartperl=#!/opt/local/bin/perl5.28 -Acppflags=-I/opt/local/include -Accflags=-pipe -Os   -Alddlflags=-L/opt/local/lib -Wl,-headerpad_max_install_names   -Aldflags=-L/opt/local/lib -Wl,-headerpad_max_install_names  '
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='/opt/local/bin/clang-mp-3.4'

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #62096.

[ryandesign (Ryan Carsten Schmidt)]

This would be a good issue to fix before the bulk add of p5.34 ports, since not fixing it causes many build failures on Mac OS X 10.6.

[kencu (Ken)]

I took steps over the past six months to remove all need for perl from the clang builds on < 10.7. So this issue with clang-3.4 should no longer exist, and perl5.X should be built by the default compiler, which is clang-9.0 at present.

We might need a dep on clang-9.0, but it is the default compiler and is hopefully always present for builds.

[ryandesign (Ryan Carsten Schmidt)]

The issue still exists as of today, as evidenced by the large number of p5.32 build failures on 10.6, such as for example ​p5.32-mac-systemdirectory:

Can't exec "/opt/local/bin/clang-mp-3.4": No such file or directory at inc/ExtUtils/HasCompiler.pm line 133.

Regardless, I recommend we modify the perl config files to reference a compiler that is guaranteed to exist (comment:5).

[kencu (Ken)]

I can't speak to that, but (AFAIK) none of the clang compilers declare a dep on perl, so (AFAIK) perl could build with any compiler.

Unless there is some way that some other thing that ultimately clang needs requires a macports perl -- can't say about that. Bin deps on perl confuse the rdeps too much for me to sort that out.

(Oh -- and what is already built with some baked in /opt/local/bin/clang-mp-3.4 is really hard to say.)

I don't know what the best course is, but the easiest course is to just require clang-3.4 I guess. Luckily there I have reduced the clang-3.4 footprint from about 800MB to 16 MB recently :>

[ryandesign (Ryan Carsten Schmidt)]

The best course of action is comment:5.

[kencu (Ken)]

Replying to ryandesign:

The best course of action is comment:5.

That makes it very simple then.

[kencu (Ken)]

BTW, on my system at least, perl5.30 is built with clang-9.0 and has that baked in:

  Compiler:
    cc='/opt/local/bin/clang-mp-9.0'

I haven't the slightest idea how that happened, but there she be.

[jmroot (Joshua Root)]

In f35a2c53331234cc730b090b1b866d4b1702a747/macports-ports (master):

perl5: address problematic baked-in values

Use generic /usr/bin/cc for compiler since the specific one used to
build perl may not be available at runtime. Remove flags specifying SDK
and deployment target since different ones may be specified in the
environment at runtime, and, similar to the compiler, the SDK that was
used at build time may not be available at runtime.

Fixes: #59786
Fixes: #62440