perl: hardcodes compiler and compiler options (packages from CPAN may not compile under a different Xcode version)

[matthew-macports@…]

I have perl5.22 and p5.22-app-cpanminus installed. Trying to install a CPAN module that requires some compiling, e.g. String::CRC32, fails with clang: error: unknown argument: '-fstack-protector-strong' (full output below). This is because /opt/local/lib/perl5/5.22/darwin-thread-multi-2level/Config.pm sets cc to /usr/bin/clang and /opt/local/lib/perl5/5.22/darwin-thread-multi-2level/Config_heavy.pl sets lddlflags to include -fstack-protector-strong when clang does not support that argument (and since XCode 5.1 unknown arguments are errors).

I'm not sure of the solution, sorry, given the arguments are set when perl is compiled and packaged, but it is not possible to use the packages as they currently are on my system (OS 10.10, XCode 6.1).

$ cpanm-5.22 -L local String::CRC32 --verbose
cpanm (App::cpanminus) 1.7040 on perl 5.022001 built for darwin-thread-multi-2level
Work directory is /Users/matthew/.cpanm/work/1458041438.63955
You have make /usr/bin/make
You have /opt/local/bin/wget
You have /usr/bin/tar: bsdtar 2.8.3 - libarchive 2.8.3
You have /usr/bin/unzip
Searching String::CRC32 () on cpanmetadb ...
--2016-03-15 11:30:38--  http://cpanmetadb.plackperl.org/v1.0/package/String::CRC32
Resolving cpanmetadb.plackperl.org (cpanmetadb.plackperl.org)... 185.31.18.249, 185.31.19.249
Connecting to cpanmetadb.plackperl.org (cpanmetadb.plackperl.org)|185.31.18.249|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 63 [text/yaml]
Saving to: ‘STDOUT’

-                                     100%[======================================================================>]      63  --.-KB/s    in 0s      

2016-03-15 11:30:38 (2.40 MB/s) - written to stdout [63/63]

--> Working on String::CRC32
Fetching http://www.cpan.org/authors/id/S/SO/SOENKE/String-CRC32-1.5.tar.gz ... --2016-03-15 11:30:38--  http://www.cpan.org/authors/id/S/SO/SOENKE/String-CRC32-1.5.tar.gz
Resolving www.cpan.org (www.cpan.org)... 94.242.223.198, 2607:f238:3::91:1
Connecting to www.cpan.org (www.cpan.org)|94.242.223.198|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6990 (6.8K) [application/x-gzip]
Saving to: ‘String-CRC32-1.5.tar.gz’

String-CRC32-1.5.tar.gz               100%[======================================================================>]   6.83K  --.-KB/s    in 0.006s  

2016-03-15 11:30:39 (1.08 MB/s) - ‘String-CRC32-1.5.tar.gz’ saved [6990/6990]

OK
Unpacking String-CRC32-1.5.tar.gz
x String-CRC32-1.5/
x String-CRC32-1.5/CRC32.xs
x String-CRC32-1.5/MANIFEST
x String-CRC32-1.5/CRC32.pm
x String-CRC32-1.5/t/
x String-CRC32-1.5/t/crc.t
x String-CRC32-1.5/t/testfile
x String-CRC32-1.5/crcgen.c
x String-CRC32-1.5/README
x String-CRC32-1.5/typemap
x String-CRC32-1.5/Makefile.PL
x String-CRC32-1.5/CRC32.pod
Entering String-CRC32-1.5
META.yml/json not found. Creating skeleton for it.
Running Makefile.PL
Configuring String-CRC32-1.5 ... Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for String::CRC32
Writing MYMETA.yml and MYMETA.json
OK
Checking dependencies from MYMETA.json ...
Checking if you have ExtUtils::MakeMaker 0 ... Yes (7.04_01)
Building and testing String-CRC32-1.5 ... cp CRC32.pod blib/lib/String/CRC32.pod
cp CRC32.pm blib/lib/String/CRC32.pm
Running Mkbootstrap for String::CRC32 ()
chmod 644 "CRC32.bs"
"/opt/local/bin/perl5.22" "/opt/local/lib/perl5/5.22/ExtUtils/xsubpp"  -typemap "/opt/local/lib/perl5/5.22/ExtUtils/typemap" -typemap "typemap"  CRC32.xs > CRC32.xsc && mv CRC32.xsc CRC32.c
/usr/bin/clang -c   -pipe -Os -fno-common -DPERL_DARWIN -I/opt/local/include -fno-strict-aliasing -fstack-protector-strong -I/opt/local/include -O3   -DVERSION=\"1.5\" -DXS_VERSION=\"1.5\"  "-I/opt/local/lib/perl5/5.22/darwin-thread-multi-2level/CORE"   CRC32.c
clang: error: unknown argument: '-fstack-protector-strong'
make: *** [CRC32.o] Error 1
FAIL
! Installing String::CRC32 failed. See /Users/matthew/.cpanm/work/1458041438.63955/build.log for details. Retry with --force to force install it.

[matthew-macports@…]

For completeness (I assume this file was autogenerated), but other things in Config_heavy.pl that include fstack-protector-strong are ccflags, cppflags, ldflags, ccflags_nolargefiles, and ldflags_nolargefiles. Removing -fstack-protector-strong from all those lines enables String::CRC32 to install.

[mf2k (Frank Schima)]

In the future, please Cc the port maintainers (port info --maintainers perl5.22), if any.

[mojca (Mojca Miklavec)]

Weird. I don't have -fstack-protector-strong in /opt/local/lib/perl5/5.22/darwin-thread-multi-2level/Config_heavy.pl.

I see it in Configure though and clang defines "GCC" variable that resolve to true in the code below:

	# on x86_64 (at least) we require an extra library (libssp) in the
	# link command line. This library is not named, so I infer that it is
	# an implementation detail that may change. Hence the safest approach
	# is to add the flag to the flags passed to the compiler at link time,
	# as that way the compiler can do the right implementation dependant
	# thing. (NWC)
	case "$gccversion" in
	?*)	set stack-protector-strong -fstack-protector-strong
		eval $checkccflag
		case "$dflt" in
		*-fstack-protector-strong*) ;; # It got added.
		*) # Try the plain/older -fstack-protector.
		   set stack-protector -fstack-protector
		   eval $checkccflag
		   ;;
		esac
		;;
	esac
	;;

# See note above about -fstack-protector
case "$ccflags" in
*-fstack-protector-strong*)
	case "$dflt" in
	*-fstack-protector-strong*) ;; # Don't add it again
	*) dflt="$dflt -fstack-protector-strong" ;;
	esac
	;;
*-fstack-protector*)
	case "$dflt" in
	*-fstack-protector*) ;; # Don't add it again
	*) dflt="$dflt -fstack-protector" ;;
	esac
	;;
esac

	case "$ldflags" in
	    *-fstack-protector-strong*)
		case "$dflt" in
		    *-fstack-protector-strong*) ;; # Don't add it again
		    *) dflt="$dflt -fstack-protector-strong" ;;
		esac
		;;
	    *-fstack-protector*)
		case "$dflt" in
		    *-fstack-protector*) ;; # Don't add it again
		    *) dflt="$dflt -fstack-protector" ;;
		esac
		;;
	esac

It would probably be helpful to talk to upstream developers once we investigate what exactly is going on (to make it easier for them to reproduce the problem).

First of all I would like to understand why I didn't get the same flags defined as you did (with a slightly older OS).

[mojca (Mojca Miklavec)]

Citation from ​https://reviews.freebsd.org/D3924:

Only use -fstack-protector-strong with supported compilers. This includes clang 3.5.0+, gcc 4.2.1, gcc 4.8.0+

After looking at XcodeVersionInfo I might have an explanation for that.

You said you were using Xcode 6.1 which is the oldest Xcode version ever released for 10.10. It officially ships clang "3.5svn", but that might be a version that doesn't support that flag yet. The latest version of Xcode supported on 10.10. seems to be 7.2.1 which ships a much later version of clang that most likely supports that flag.

But most likely you got your Perl as a binary package from the buildbot where the default system compiler supported this flag.

I guess that the problem would be solved if you upgrade your Xcode. I would need to bring this up to the developer mailing list to determine whether we should officially address issues like this one. I could imagine that you might run into similar problems elsewhere and the code to handle this special case would be pretty weird.

[mojca (Mojca Miklavec)]

See also

• ​Compiler name baked into our packages (Mailing list, October 2012)
• ​apr binary package misconfigured? (Mailing list, October 2012)

• #36474

  arb is not UsingTheRightCompiler for PERL2ARB

Here's what Joshua replied on the ​mailing list:

So it's also an upstream bug. You could patch perl to remove use of -fstack-protector-strong or replace it with -fstack-protector; that won't prevent anything from running since it's a hardening measure. Whether it "breaks" anything is a harder question. (Used on a bug-free program, it would do nothing but make it slightly slower.) Whether perl should even be choosing its own CFLAGS like this is left as an exercise for the reader. ;)

[matthew-macports@…]

Apologies for not CCing the port maintainers, and thanks for the investigation :) Upgrading Xcode is certainly the easiest solution, I'll do that. <time passes> Actually, simply uninstalling XCode meant it instead used the command line tools which *were* up to date to 7.2. Dunno what's been updating them but not XCode! (I realise MacPorts needs full XCode install, but good I don't have to wait for that 4.4G download for this example… :) ).