Opened 3 years ago

Closed 3 years ago

Last modified 7 months ago

#44596 closed defect (fixed)

gmp @6.0.0_0: Autotools incorrectly selects "-undefined suppress" symbol lookup on Yosemite

Reported by: wichert@… Owned by: larryv (Lawrence Velázquez)
Priority: Normal Milestone:
Component: ports Version:
Keywords: yosemite Cc: ryandesign (Ryan Schmidt), borodiychuk (Andriy Borodiychuk), skymoo (Adam Mercer), davidchambers (David Chambers), mtwest729@…, AP1010, eirnym@…, corsaroquad (Giovanni Grieco), platipodium (Carsten Lemmen), sultanqasim@…, hiro.masa.yoshi.moto@…, mww@…, Schamschula (Marius Schamschula), MarcusCalhoun-Lopez (Marcus Calhoun-Lopez), cjones051073, cooljeanius (Eric Gallager), jeremyhu (Jeremy Huddleston Sequoia), mojca (Mojca Miklavec)
Port: gmp

Description

The libgcc build fails when trying to compile unwind-dw2-fde-darwin.c with an internal compile error. main.log is attached.

This happens on a current yosemite / xcode 6b4 box.

Attachments (3)

main.log (4.7 MB) - added by wichert@… 3 years ago.
main_OSXleopard.log (4.8 MB) - added by mtwest729@… 3 years ago.
log from libgcc install attempt on machine running OSX 10.5.8
0001-gmp-Use-correct-symbol-lookup-on-Yosemite-44596.patch (2.6 KB) - added by larryv (Lawrence Velázquez) 3 years ago.
Fix for GMP build on Yosemite

Change History (59)

Changed 3 years ago by wichert@…

comment:1 Changed 3 years ago by ryandesign (Ryan Schmidt)

  • Cc ryandesign@… added; mww@… removed
  • Keywords yosemite added
  • Owner changed from macports-tickets@… to mww@…

comment:2 Changed 3 years ago by borodiychuk (Andriy Borodiychuk)

  • Cc ab@… added

Cc Me!

comment:3 Changed 3 years ago by skymoo (Adam Mercer)

  • Cc ram@… added

Cc Me!

comment:4 Changed 3 years ago by davidchambers (David Chambers)

  • Cc dc@… added

Cc Me!

comment:5 Changed 3 years ago by mtwest729@…

  • Cc mtwest729@… added

Cc Me!

Changed 3 years ago by mtwest729@…

log from libgcc install attempt on machine running OSX 10.5.8

comment:6 in reply to: ↑ description Changed 3 years ago by mtwest729@…

Replying to wichert@…:

From what I am able to tell from the log file, I am getting the same build error as wichert, but on a significantly older system. The main.log from sudo port -d install libgcc is attached and labeled accordingly.

  • OSX 10.5.8 on 2007 Intel Mac
  • Xcode 3.1.4

comment:7 Changed 3 years ago by ryandesign (Ryan Schmidt)

The log from Yosemite ends after 33000 lines with "internal compiler error: Segmentation fault: 11"; the log from Leopard ends after just 9000 lines with "collect2: error: ld returned 1 exit status". So they are different problems. You should probably file a new ticket for the problem you're seeing on Leopard since they'll probably have different solutions. In either case, they're issues the developers of gcc will probably have to solve; we just package and deliver the software they developer.

comment:8 Changed 3 years ago by AP1010

  • Cc arthur@… added

Cc Me!

comment:9 Changed 3 years ago by eirnym@…

  • Cc eirnym@… added

Cc Me!

comment:10 Changed 3 years ago by corsaroquad (Giovanni Grieco)

  • Cc giovanni.grc96@… added

Cc Me!

comment:11 Changed 3 years ago by larryv (Lawrence Velázquez)

  • Cc larryv@… added

Cc Me!

comment:12 Changed 3 years ago by larryv (Lawrence Velázquez)

Still seeing this with beta 6 (6A280e).

comment:13 Changed 3 years ago by mf2k (Frank Schima)

And I still see this with Xcode beta 7 (6A280n).

comment:14 Changed 3 years ago by platipodium (Carsten Lemmen)

  • Cc carsten@… added

Cc Me!

comment:15 follow-up: Changed 3 years ago by sultanqasim@…

This issue breaks almost every port on Yosemite because libgcc is so critical.

Last edited 3 years ago by sultanqasim@… (previous) (diff)

comment:16 Changed 3 years ago by sultanqasim@…

  • Cc sultanqasim@… added

Cc Me!

comment:17 Changed 3 years ago by garethsimons@…

  • Cc garethsimons@… added

Cc Me!

comment:18 Changed 3 years ago by garethsimons@…

  • Cc garethsimons@… removed

Cc Me!

comment:19 in reply to: ↑ 15 ; follow-up: Changed 3 years ago by larryv (Lawrence Velázquez)

Replying to sultanqasim@…:

This issue breaks almost every port on Yosemite because libgcc is so critical.

What? This is demonstrably hyperbolic. The vast majority of ports don’t use libgcc in any way. This only affects the gcc* ports and those that depend on them; there are certainly many, but nowhere near “almost every” one.

comment:20 in reply to: ↑ 19 ; follow-up: Changed 3 years ago by sultanqasim@…

Replying to larryv@…:

Replying to sultanqasim@…:

This issue breaks almost every port on Yosemite because libgcc is so critical.

What? This is demonstrably hyperbolic. The vast majority of ports don’t use libgcc in any way. This only affects the gcc* ports and those that depend on them; there are certainly many, but nowhere near “almost every” one.

While I admit a bit of an exaggeration, a huge number of ports have dependency chains that lead to libgcc. This has broken all kinds of ports for me ranging from py##-lxml to texlive-latex-recommended.

EDIT: I'm not sure why so many ports are trying to build libgcc. I traced the dependencies of the the ports I gave as examples and couldn't find why they wanted libgcc. For years I've noticed gcc-related dependencies on ports that seem unrelated to gcc. Sorry about derailing this discussion.

Last edited 3 years ago by sultanqasim@… (previous) (diff)

comment:21 in reply to: ↑ 20 Changed 3 years ago by ryandesign (Ryan Schmidt)

Replying to sultanqasim@…:

While I admit a bit of an exaggeration, a huge number of ports have dependency chains that lead to libgcc. This has broken all kinds of ports for me ranging from py##-lxml to texlive-latex-recommended.

The dependency chains of py*-lxml and texlive-latex-recommended do not seem to include libgcc, from what I can tell.

comment:22 Changed 3 years ago by sultanqasim@…

Sorry for this distraction again, but as an update, removing the gcc ports seemed to solve my issue of libgcc breaking many other ports. I found that py*-scipy was the port that led me to install gcc in the first place. It appears that macports was unhappy about broken files from when it tried to upgrade libgcc, and it often tried to rebuild libgcc when installing other ports.

Last edited 3 years ago by sultanqasim@… (previous) (diff)

comment:23 Changed 3 years ago by larryv (Lawrence Velázquez)

  • Port libgcc-devel added
  • Summary changed from libgcc @4.9.1 Build fails with internal compiler error to libgcc @4.9.1_0, libgcc-devel @4.10-20140810: Build fails with internal compiler error

Also applies to 4.10-20140810.

comment:24 follow-ups: Changed 3 years ago by larryv (Lawrence Velázquez)

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

comment:25 in reply to: ↑ 24 Changed 3 years ago by sultanqasim@…

Replying to larryv@…:

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

I am seeing this on an actual mid-2012 MacBook Pro running Yosemite natively.

comment:26 Changed 3 years ago by mf2k (Frank Schima)

Me too. Although on a 2013 MacBook Pro.

Last edited 3 years ago by mf2k (Frank Schima) (previous) (diff)

comment:27 Changed 3 years ago by larryv (Lawrence Velázquez)

Okay, thanks. I wanted to rule out Fusion shenanigans.

comment:28 in reply to: ↑ 24 Changed 3 years ago by platipodium (Carsten Lemmen)

Replying to larryv@…:

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

Yep, a 2014 MacBook Pro

comment:29 Changed 3 years ago by larryv (Lawrence Velázquez)

Still a problem on 10.10 14A343f with Xcode 6.1 6A1027 and clang-600.0.53 but not on 10.9.5 13F31 with the same dev tools.

comment:30 Changed 3 years ago by larryv (Lawrence Velázquez)

Builds fine with Homebrew. Might be a problem with our packaging or with a dependency.

comment:31 follow-up: Changed 3 years ago by hiro.masa.yoshi.moto@…

  • Cc hiro.masa.yoshi.moto@… added

I found a workaround to avoid this internal compile error. The steps are as follows;

  • step-1: downgrade gmp to 4.3.2
  • step-2: downgrade libmpc to 0.8.1
  • step-3: port install libgcc

For step 1 and 2, I used "port edit" to modify the version numbers and checksums. For step 2, I also wrote small patch;

--- src/acos.c.old~     2014-09-13 01:26:33.000000000 +0900
+++ src/acos.c  2014-09-13 01:26:53.000000000 +0900
@@ -189,7 +189,7 @@ mpc_acos (mpc_ptr rop, mpc_srcptr op, mp
     rnd_im = rnd_im == GMP_RNDU ? GMP_RNDD
       : rnd_im == GMP_RNDD ? GMP_RNDU
 #if MPFR_VERSION_MAJOR >= 3
-      : rnd_im == GMP_RNDA ? GMP_RNDZ
+      : rnd_im == MPFR_RNDA ? GMP_RNDZ
 #endif
       : rnd_im;
   rnd1 = RNDC(GMP_RNDN, rnd_im);

See http://lists.gforge.inria.fr/pipermail/mpc-discuss/2010-April/000679.html for details.

I hope this will help you :-)

Last edited 3 years ago by hiro.masa.yoshi.moto@… (previous) (diff)

comment:32 in reply to: ↑ 31 Changed 3 years ago by larryv (Lawrence Velázquez)

  • Owner changed from mww@… to larryv@…
  • Status changed from new to assigned

Thank you, but I’ve discovered the problem.

comment:33 Changed 3 years ago by larryv (Lawrence Velázquez)

  • Cc mww@… added; larryv@… removed

comment:34 Changed 3 years ago by Schamschula (Marius Schamschula)

  • Cc mschamschula@… added

Cc Me!

comment:35 follow-up: Changed 3 years ago by larryv (Lawrence Velázquez)

  • Cc mcalhoun@… added
  • Port gmp added; libgcc libgcc-devel removed
  • Summary changed from libgcc @4.9.1_0, libgcc-devel @4.10-20140810: Build fails with internal compiler error to gmp @6.0.0_0: Autotools incorrectly selects "-undefined suppress" symbol lookup on Yosemite
  • Version 2.3.1 deleted

The GCC ICE was a red herring. You’ll be unsurprised to hear that this is Autotools’ fault.

  1. We set the MACOSX_DEPLOYMENT_TARGET environment variable to “10.10”. (Homebrew doesn’t set it at all and thus doesn’t start down this road to perdition.)
  2. Libtool (and, by extension, configure scripts built with it) interprets “10.10” as “10.1*”.
  3. OS X did not support dynamic lookup of undefined symbols until 10.3. Since Libtool thinks 10.10 Yosemite is 10.1 Puma, it tells the linker to ignore undefined symbols.
    case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
        10.0,*86*-darwin8*|10.0,*-darwin[[91]]*)
            _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
        10.[[012]]*)
            _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
        10.*)
            _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
    esac
    
  4. GMP binaries contain several unresolved symbols that they no doubt expect dyld(1) to look up later.
  5. Thanks to Libtool, dyld(1) has no such plans. It makes this very clear during GMP’s test suite.
    Crashed Thread:        0  Dispatch queue: com.apple.main-thread
    
    Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
    Exception Codes:       EXC_I386_GPFLT
    
    Error Formulating Crash Report:
    Failed to read dyld_all_image_infos: Coudln't get the data
    
    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0   libdyld.dylib                 	0x7fff88c64432 stack_not_16_byte_aligned_error + 0
    
  6. This filters up to anything trying to use GMP. Like, say, GCC trying to optimize some code.

Marcus, I’m attaching my fix; may I commit it? (I bumped the revision because this problem only manifests at runtime.)

Last edited 3 years ago by larryv (Lawrence Velázquez) (previous) (diff)

Changed 3 years ago by larryv (Lawrence Velázquez)

Fix for GMP build on Yosemite

comment:36 Changed 3 years ago by wichert@…

That's some amazing detective work from larryv!

comment:37 Changed 3 years ago by cjones051073

  • Cc jonesc@… added

Cc Me!

comment:38 in reply to: ↑ 35 ; follow-up: Changed 3 years ago by ryandesign (Ryan Schmidt)

Replying to larryv@…:

  1. We set the MACOSX_DEPLOYMENT_TARGET environment variable to “10.10”. (Homebrew doesn’t set it at all and thus doesn’t start down this road to perdition.)

Ever since Mac OS X 10.5, the default value for MACOSX_DEPLOYMENT_TARGET is the currently running OS X version, so I would expect everyone else to be affected too.

comment:39 in reply to: ↑ 38 Changed 3 years ago by larryv (Lawrence Velázquez)

Perhaps Xcode or Clang/GCC have that default behavior, but there’s no magic default value for the environment variable. Unless it’s present in the environment, a script that tries to substitute $MACOSX_DEPLOYMENT_TARGET will get a null value.

So Libtool does the right thing if MACOSX_DEPLOYMENT_TARGET is absent from its environment (the first case alternative in the code snippet above). It only chokes when that variable is explicitly set to “10.10” (or “10.11” or “10.12” or “10.27”).

The variable is not present in Homebrew’s build environment, so their build works fine. (I personally tested both GCC and GMP.)

Last edited 3 years ago by larryv (Lawrence Velázquez) (previous) (diff)

comment:40 Changed 3 years ago by ryandesign (Ryan Schmidt)

Oh, you're right. I was thinking of the default value when the environment variable is not specified.

In any case, thanks for the patch! After rebuilding gmp with it I was able to build libgcc on Yosemite beta 2. (gcc itself is still building.)

comment:41 follow-up: Changed 3 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Should be fixed in r125350.
Thanks for the patch.

I submitted an upstream bug report at:
https://gmplib.org/list-archives/gmp-bugs/2014-September/thread.html

comment:42 in reply to: ↑ 41 Changed 3 years ago by larryv (Lawrence Velázquez)

  • Resolution set to fixed
  • Status changed from assigned to closed

Great, thanks!

(As for aclocal.m4, I just fixed that for completeness. Upstream should ultimately patch their Autotools, or this will crop up again.)

comment:43 Changed 3 years ago by cooljeanius (Eric Gallager)

  • Cc egall@… added

Cc Me!

comment:44 follow-ups: Changed 3 years ago by ryandesign (Ryan Schmidt)

As the gmp developers pointed out, this is a libtool problem. Larry fixed the libtool port in r125325. Did the problem also get reported to the libtool developers?

comment:45 in reply to: ↑ 44 Changed 3 years ago by larryv (Lawrence Velázquez)

I posted to libtool-patches in September, but the latest commit to their repository is from May.

Last edited 3 years ago by larryv (Lawrence Velázquez) (previous) (diff)

comment:46 in reply to: ↑ 44 Changed 3 years ago by larryv (Lawrence Velázquez)

Providentially, Libtool upstream will be cutting a release this weekend to take care of this problem.

comment:47 Changed 3 years ago by jeremyhu (Jeremy Huddleston Sequoia)

  • Cc jeremyhu@… added

Cc Me!

comment:48 Changed 3 years ago by jeremyhu (Jeremy Huddleston Sequoia)

-undefined dynamic_lookup ? Yuck. That usually indicates a bad design decision if it is used in shipping code. Why do we need to lookup undefined symbols at runtime? Why are they not available at build time?

Last edited 3 years ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:49 Changed 8 months ago by mojca (Mojca Miklavec)

  • Cc mojca added

comment:50 Changed 8 months ago by mojca (Mojca Miklavec)

I see Ryan committing changes to various ports referencing this ticket. What's the proper solution? Ask upstream to regenerate the configure files with a newer version of libtool? (Judging from the logs I assume that at least 2.4.3 or 2.4.4 is needed.)

comment:51 Changed 8 months ago by jeremyhu (Jeremy Huddleston Sequoia)

The ultimate proper solution is to make sure that binaries link correctly. Allow the linker to emit errors when it detects them and fix those errors instead of suppressing those errors and letting the user hit them at runtime.

Politely asking upstream to pickup the latest glibtool, automake, and autoconf is generally a good idea. There are many other bugs in older versions that bite us consistently (eg: older glibtool do not pass -stdlib=... during the link phase, which breaks LibCXXOnOlderSystems, and older glibtool also does not pass -fsanitize= which breaks ASan builds, etc).

comment:52 Changed 7 months ago by RJVB (René Bertin)

Can't concerned ports simply regenerate the configure script using autoreconf?

FWIW, if that step becomes usual it might be useful to record it in the statefile instead of doing it systematically each time one repeats the configure step.

comment:53 Changed 7 months ago by corsaroquad (Giovanni Grieco)

  • Cc corsaroquad added

comment:54 Changed 7 months ago by corsaroquad (Giovanni Grieco)

  • Cc corsaroquad removed

comment:55 Changed 7 months ago by jeremyhu (Jeremy Huddleston Sequoia)

It's very annoying to go in and add 'use_autoreconf ...' to Portfiles after they've been updated to a release where the release manager used an older version of autoconf. Additionally, those options usually persist long after they're needed (once upstream starts using newer tools), and even if they get removed, we might need to re-add it if someone later does a release with older tools...

The best option is to get upstream to use newer autotools.

comment:56 Changed 7 months ago by mojca (Mojca Miklavec)

I wonder if MacPorts should be detecting problems like too old tools used to generate the configure script and warn about it.

Note: See TracTickets for help on using tickets.