Opened 10 years ago

Closed 10 years ago

Last modified 6 years ago

#44596 closed defect (fixed)

gmp @6.0.0_0: Autotools incorrectly selects "-undefined suppress" symbol lookup on Yosemite

Reported by: wichert@… Owned by: larryv (Lawrence Velázquez)
Priority: Normal Milestone:
Component: ports Version:
Keywords: yosemite Cc: ryandesign (Ryan Carsten Schmidt), borodiychuk (Andriy Borodiychuk), skymoo (Adam Mercer), davidchambers (David Chambers), mtwest729@…, AP1010, eirnym (Arseny Nasokin), corsaroquad (Giovanni Grieco), platipodium (Carsten Lemmen), sultanqasim@…, hiro.masa.yoshi.moto@…, mww@…, Schamschula (Marius Schamschula), MarcusCalhoun-Lopez (Marcus Calhoun-Lopez), cjones051073 (Chris Jones), cooljeanius (Eric Gallager), jeremyhu (Jeremy Huddleston Sequoia), mojca (Mojca Miklavec)
Port: gmp

Description

The libgcc build fails when trying to compile unwind-dw2-fde-darwin.c with an internal compile error. main.log is attached.

This happens on a current yosemite / xcode 6b4 box.

Attachments (3)

main.log (4.7 MB) - added by wichert@… 10 years ago.
main_OSXleopard.log (4.8 MB) - added by mtwest729@… 10 years ago.
log from libgcc install attempt on machine running OSX 10.5.8
0001-gmp-Use-correct-symbol-lookup-on-Yosemite-44596.patch (2.6 KB) - added by larryv (Lawrence Velázquez) 10 years ago.
Fix for GMP build on Yosemite

Change History (59)

Changed 10 years ago by wichert@…

Attachment: main.log added

comment:1 Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: ryandesign@… added; mww@… removed
Keywords: yosemite added
Owner: changed from macports-tickets@… to mww@…

comment:2 Changed 10 years ago by borodiychuk (Andriy Borodiychuk)

Cc: ab@… added

Cc Me!

comment:3 Changed 10 years ago by skymoo (Adam Mercer)

Cc: ram@… added

Cc Me!

comment:4 Changed 10 years ago by davidchambers (David Chambers)

Cc: dc@… added

Cc Me!

comment:5 Changed 10 years ago by mtwest729@…

Cc: mtwest729@… added

Cc Me!

Changed 10 years ago by mtwest729@…

Attachment: main_OSXleopard.log added

log from libgcc install attempt on machine running OSX 10.5.8

comment:6 in reply to:  description Changed 10 years ago by mtwest729@…

Replying to wichert@…:

From what I am able to tell from the log file, I am getting the same build error as wichert, but on a significantly older system. The main.log from sudo port -d install libgcc is attached and labeled accordingly.

  • OSX 10.5.8 on 2007 Intel Mac
  • Xcode 3.1.4

comment:7 Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

The log from Yosemite ends after 33000 lines with "internal compiler error: Segmentation fault: 11"; the log from Leopard ends after just 9000 lines with "collect2: error: ld returned 1 exit status". So they are different problems. You should probably file a new ticket for the problem you're seeing on Leopard since they'll probably have different solutions. In either case, they're issues the developers of gcc will probably have to solve; we just package and deliver the software they developer.

comment:8 Changed 10 years ago by AP1010

Cc: arthur@… added

Cc Me!

comment:9 Changed 10 years ago by eirnym (Arseny Nasokin)

Cc: eirnym@… added

Cc Me!

comment:10 Changed 10 years ago by corsaroquad (Giovanni Grieco)

Cc: giovanni.grc96@… added

Cc Me!

comment:11 Changed 10 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… added

Cc Me!

comment:12 Changed 10 years ago by larryv (Lawrence Velázquez)

Still seeing this with beta 6 (6A280e).

comment:13 Changed 10 years ago by mf2k (Frank Schima)

And I still see this with Xcode beta 7 (6A280n).

comment:14 Changed 10 years ago by platipodium (Carsten Lemmen)

Cc: carsten@… added

Cc Me!

comment:15 Changed 10 years ago by sultanqasim@…

Cc Me! This issue breaks almost every port on Yosemite because libgcc is so critical.

Version 0, edited 10 years ago by sultanqasim@… (next)

comment:16 Changed 10 years ago by sultanqasim@…

Cc: sultanqasim@… added

Cc Me!

comment:17 Changed 10 years ago by garethsimons@…

Cc: garethsimons@… added

Cc Me!

comment:18 Changed 10 years ago by garethsimons@…

Cc: garethsimons@… removed

Cc Me!

comment:19 in reply to:  15 ; Changed 10 years ago by larryv (Lawrence Velázquez)

Replying to sultanqasim@…:

This issue breaks almost every port on Yosemite because libgcc is so critical.

What? This is demonstrably hyperbolic. The vast majority of ports don’t use libgcc in any way. This only affects the gcc* ports and those that depend on them; there are certainly many, but nowhere near “almost every” one.

comment:20 in reply to:  19 ; Changed 10 years ago by sultanqasim@…

Replying to larryv@…:

Replying to sultanqasim@…:

This issue breaks almost every port on Yosemite because libgcc is so critical.

What? This is demonstrably hyperbolic. The vast majority of ports don’t use libgcc in any way. This only affects the gcc* ports and those that depend on them; there are certainly many, but nowhere near “almost every” one.

While I admit a bit of an exaggeration, a huge number of ports have dependency chains that lead to libgcc. This has broken all kinds of ports for me ranging from py##-lxml to texlive-latex-recommended.

EDIT: I'm not sure why so many ports are trying to build libgcc. I traced the dependencies of the the ports I gave as examples and couldn't find why they wanted libgcc. For years I've noticed gcc-related dependencies on ports that seem unrelated to gcc. Sorry about derailing this discussion.

Last edited 10 years ago by sultanqasim@… (previous) (diff)

comment:21 in reply to:  20 Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

Replying to sultanqasim@…:

While I admit a bit of an exaggeration, a huge number of ports have dependency chains that lead to libgcc. This has broken all kinds of ports for me ranging from py##-lxml to texlive-latex-recommended.

The dependency chains of py*-lxml and texlive-latex-recommended do not seem to include libgcc, from what I can tell.

comment:22 Changed 10 years ago by sultanqasim@…

Sorry for this distraction again, but as an update, removing the gcc ports seemed to solve my issue of libgcc breaking many other ports. I found that py*-scipy was the port that led me to install gcc in the first place. It appears that macports was unhappy about broken files from when it tried to upgrade libgcc, and it often tried to rebuild libgcc when installing other ports.

Last edited 10 years ago by sultanqasim@… (previous) (diff)

comment:23 Changed 10 years ago by larryv (Lawrence Velázquez)

Port: libgcc-devel added
Summary: libgcc @4.9.1 Build fails with internal compiler errorlibgcc @4.9.1_0, libgcc-devel @4.10-20140810: Build fails with internal compiler error

Also applies to 4.10-20140810.

comment:24 Changed 10 years ago by larryv (Lawrence Velázquez)

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

comment:25 in reply to:  24 Changed 10 years ago by sultanqasim@…

Replying to larryv@…:

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

I am seeing this on an actual mid-2012 MacBook Pro running Yosemite natively.

comment:26 Changed 10 years ago by mf2k (Frank Schima)

Me too. Although on a 2013 MacBook Pro.

Last edited 10 years ago by mf2k (Frank Schima) (previous) (diff)

comment:27 Changed 10 years ago by larryv (Lawrence Velázquez)

Okay, thanks. I wanted to rule out Fusion shenanigans.

comment:28 in reply to:  24 Changed 10 years ago by platipodium (Carsten Lemmen)

Replying to larryv@…:

Anyone seeing this with Yosemite + Xcode 6 on actual hardware (i.e., not in a VM)?

Yep, a 2014 MacBook Pro

comment:29 Changed 10 years ago by larryv (Lawrence Velázquez)

Still a problem on 10.10 14A343f with Xcode 6.1 6A1027 and clang-600.0.53 but not on 10.9.5 13F31 with the same dev tools.

comment:30 Changed 10 years ago by larryv (Lawrence Velázquez)

Builds fine with Homebrew. Might be a problem with our packaging or with a dependency.

comment:31 Changed 10 years ago by hiro.masa.yoshi.moto@…

Cc: hiro.masa.yoshi.moto@… added

I found a workaround to avoid this internal compile error. The steps are as follows;

  • step-1: downgrade gmp to 4.3.2
  • step-2: downgrade libmpc to 0.8.1
  • step-3: port install libgcc

For step 1 and 2, I used "port edit" to modify the version numbers and checksums. For step 2, I also wrote small patch;

--- src/acos.c.old~     2014-09-13 01:26:33.000000000 +0900
+++ src/acos.c  2014-09-13 01:26:53.000000000 +0900
@@ -189,7 +189,7 @@ mpc_acos (mpc_ptr rop, mpc_srcptr op, mp
     rnd_im = rnd_im == GMP_RNDU ? GMP_RNDD
       : rnd_im == GMP_RNDD ? GMP_RNDU
 #if MPFR_VERSION_MAJOR >= 3
-      : rnd_im == GMP_RNDA ? GMP_RNDZ
+      : rnd_im == MPFR_RNDA ? GMP_RNDZ
 #endif
       : rnd_im;
   rnd1 = RNDC(GMP_RNDN, rnd_im);

See http://lists.gforge.inria.fr/pipermail/mpc-discuss/2010-April/000679.html for details.

I hope this will help you :-)

Last edited 10 years ago by hiro.masa.yoshi.moto@… (previous) (diff)

comment:32 in reply to:  31 Changed 10 years ago by larryv (Lawrence Velázquez)

Owner: changed from mww@… to larryv@…
Status: newassigned

Thank you, but I’ve discovered the problem.

comment:33 Changed 10 years ago by larryv (Lawrence Velázquez)

Cc: mww@… added; larryv@… removed

comment:34 Changed 10 years ago by Schamschula (Marius Schamschula)

Cc: mschamschula@… added

Cc Me!

comment:35 Changed 10 years ago by larryv (Lawrence Velázquez)

Cc: mcalhoun@… added
Port: gmp added; libgcc libgcc-devel removed
Summary: libgcc @4.9.1_0, libgcc-devel @4.10-20140810: Build fails with internal compiler errorgmp @6.0.0_0: Autotools incorrectly selects "-undefined suppress" symbol lookup on Yosemite
Version: 2.3.1

The GCC ICE was a red herring. You’ll be unsurprised to hear that this is Autotools’ fault.

  1. We set the MACOSX_DEPLOYMENT_TARGET environment variable to “10.10”. (Homebrew doesn’t set it at all and thus doesn’t start down this road to perdition.)
  2. Libtool (and, by extension, configure scripts built with it) interprets “10.10” as “10.1*”.
  3. OS X did not support dynamic lookup of undefined symbols until 10.3. Since Libtool thinks 10.10 Yosemite is 10.1 Puma, it tells the linker to ignore undefined symbols.
    case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
        10.0,*86*-darwin8*|10.0,*-darwin[[91]]*)
            _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
        10.[[012]]*)
            _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
        10.*)
            _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
    esac
    
  4. GMP binaries contain several unresolved symbols that they no doubt expect dyld(1) to look up later.
  5. Thanks to Libtool, dyld(1) has no such plans. It makes this very clear during GMP’s test suite.
    Crashed Thread:        0  Dispatch queue: com.apple.main-thread
    
    Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
    Exception Codes:       EXC_I386_GPFLT
    
    Error Formulating Crash Report:
    Failed to read dyld_all_image_infos: Coudln't get the data
    
    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0   libdyld.dylib                 	0x7fff88c64432 stack_not_16_byte_aligned_error + 0
    
  6. This filters up to anything trying to use GMP. Like, say, GCC trying to optimize some code.

Marcus, I’m attaching my fix; may I commit it? (I bumped the revision because this problem only manifests at runtime.)

Last edited 10 years ago by larryv (Lawrence Velázquez) (previous) (diff)

Changed 10 years ago by larryv (Lawrence Velázquez)

Fix for GMP build on Yosemite

comment:36 Changed 10 years ago by wichert@…

That's some amazing detective work from larryv!

comment:37 Changed 10 years ago by cjones051073 (Chris Jones)

Cc: jonesc@… added

Cc Me!

comment:38 in reply to:  35 ; Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

Replying to larryv@…:

  1. We set the MACOSX_DEPLOYMENT_TARGET environment variable to “10.10”. (Homebrew doesn’t set it at all and thus doesn’t start down this road to perdition.)

Ever since Mac OS X 10.5, the default value for MACOSX_DEPLOYMENT_TARGET is the currently running OS X version, so I would expect everyone else to be affected too.

comment:39 in reply to:  38 Changed 10 years ago by larryv (Lawrence Velázquez)

Perhaps Xcode or Clang/GCC have that default behavior, but there’s no magic default value for the environment variable. Unless it’s present in the environment, a script that tries to substitute $MACOSX_DEPLOYMENT_TARGET will get a null value.

So Libtool does the right thing if MACOSX_DEPLOYMENT_TARGET is absent from its environment (the first case alternative in the code snippet above). It only chokes when that variable is explicitly set to “10.10” (or “10.11” or “10.12” or “10.27”).

The variable is not present in Homebrew’s build environment, so their build works fine. (I personally tested both GCC and GMP.)

Last edited 10 years ago by larryv (Lawrence Velázquez) (previous) (diff)

comment:40 Changed 10 years ago by ryandesign (Ryan Carsten Schmidt)

Oh, you're right. I was thinking of the default value when the environment variable is not specified.

In any case, thanks for the patch! After rebuilding gmp with it I was able to build libgcc on Yosemite beta 2. (gcc itself is still building.)

comment:41 Changed 10 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Should be fixed in r125350.
Thanks for the patch.

I submitted an upstream bug report at:
https://gmplib.org/list-archives/gmp-bugs/2014-September/thread.html

comment:42 in reply to:  41 Changed 10 years ago by larryv (Lawrence Velázquez)

Resolution: fixed
Status: assignedclosed

Great, thanks!

(As for aclocal.m4, I just fixed that for completeness. Upstream should ultimately patch their Autotools, or this will crop up again.)

comment:43 Changed 10 years ago by cooljeanius (Eric Gallager)

Cc: egall@… added

Cc Me!

comment:44 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)

As the gmp developers pointed out, this is a libtool problem. Larry fixed the libtool port in r125325. Did the problem also get reported to the libtool developers?

comment:45 in reply to:  44 Changed 9 years ago by larryv (Lawrence Velázquez)

I posted to libtool-patches in September, but the latest commit to their repository is from May.

Last edited 9 years ago by larryv (Lawrence Velázquez) (previous) (diff)

comment:46 in reply to:  44 Changed 9 years ago by larryv (Lawrence Velázquez)

Providentially, Libtool upstream will be cutting a release this weekend to take care of this problem.

comment:47 Changed 9 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Cc: jeremyhu@… added

Cc Me!

comment:48 Changed 9 years ago by jeremyhu (Jeremy Huddleston Sequoia)

-undefined dynamic_lookup ? Yuck. That usually indicates a bad design decision if it is used in shipping code. Why do we need to lookup undefined symbols at runtime? Why are they not available at build time?

Last edited 9 years ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:49 Changed 7 years ago by mojca (Mojca Miklavec)

Cc: mojca added

comment:50 Changed 7 years ago by mojca (Mojca Miklavec)

I see Ryan committing changes to various ports referencing this ticket. What's the proper solution? Ask upstream to regenerate the configure files with a newer version of libtool? (Judging from the logs I assume that at least 2.4.3 or 2.4.4 is needed.)

comment:51 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

The ultimate proper solution is to make sure that binaries link correctly. Allow the linker to emit errors when it detects them and fix those errors instead of suppressing those errors and letting the user hit them at runtime.

Politely asking upstream to pickup the latest glibtool, automake, and autoconf is generally a good idea. There are many other bugs in older versions that bite us consistently (eg: older glibtool do not pass -stdlib=... during the link phase, which breaks LibcxxOnOlderSystems, and older glibtool also does not pass -fsanitize= which breaks ASan builds, etc).

Last edited 6 years ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:52 Changed 7 years ago by RJVB (René Bertin)

Can't concerned ports simply regenerate the configure script using autoreconf?

FWIW, if that step becomes usual it might be useful to record it in the statefile instead of doing it systematically each time one repeats the configure step.

comment:53 Changed 7 years ago by corsaroquad (Giovanni Grieco)

Cc: corsaroquad added

comment:54 Changed 7 years ago by corsaroquad (Giovanni Grieco)

Cc: corsaroquad removed

comment:55 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

It's very annoying to go in and add 'use_autoreconf ...' to Portfiles after they've been updated to a release where the release manager used an older version of autoconf. Additionally, those options usually persist long after they're needed (once upstream starts using newer tools), and even if they get removed, we might need to re-add it if someone later does a release with older tools...

The best option is to get upstream to use newer autotools.

comment:56 Changed 7 years ago by mojca (Mojca Miklavec)

I wonder if MacPorts should be detecting problems like too old tools used to generate the configure script and warn about it.

Note: See TracTickets for help on using tickets.