libgcc9 @9.5.0_1: builds fail on 10.8, 10.9, 10.10

[chillin-]

Undefined symbols for architecture x86_64:
  "_cfun", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_epilogue_completed", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_cf_protection", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_finite_math_only", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_fp_int_builtin_inexact", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_peephole2", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_pic", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_rounding_math", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_trapping_math", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_unsafe_math_optimizations", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_insn", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_arch_features", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_cmodel", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_fpmath", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_isa_flags", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_isa_flags2", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_pmode", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_tls_dialect", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_tune_features", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_operands", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_optimize", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_reload_completed", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_reload_in_progress", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_rtx_class", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_target_flags", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_this_target_flag_state", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_x86_prefetch_sse", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
ld: symbol(s) not found for architecture x86_64

​https://ports.macports.org/port/libgcc9/builds/

[chillin-]

libgcc build main.log

[ryandesign (Ryan Carsten Schmidt)]

The relevant errors in the log are:

Undefined symbols for architecture x86_64:
  "_cfun", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_epilogue_completed", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_cf_protection", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_finite_math_only", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_fp_int_builtin_inexact", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_peephole2", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_pic", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_rounding_math", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_trapping_math", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_flag_unsafe_math_optimizations", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_insn", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_arch_features", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_cmodel", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_fpmath", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_isa_flags", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_isa_flags2", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_pmode", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_tls_dialect", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_ix86_tune_features", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_operands", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_optimize", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_reload_completed", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_reload_in_progress", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_rtx_class", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_target_flags", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_this_target_flag_state", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
  "_x86_prefetch_sse", referenced from:
      ___cxx_global_var_init.99 in gencondmd.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [build/gencondmd] Error 1

[macdeport]

libgcc9 build darwin_14.x86_64 (Yosemite) main.log

[macdeport]

Build fail too on Yosemite but maybe for another reason.

[kencu (Ken)]

it's BBAAACCCKKK !

The recurrent problem with touchy gencondmd.c build failures, eg: 64316

and other similar tickets. Something about this file is always the canary in the coal mine for gcc builds.

[cjones051073 (Chris Jones)]

My recent change was just to include some additional dylibs in the distribution

​https://github.com/macports/macports-ports/commit/2ecaf92ddc13cce78e11edb8834ea96f3d4d4983

nothing fundamental in the build itself changed. The issue in fact started with

​https://github.com/macports/macports-ports/commit/445cb63580211cbbb788570a7cc3a9ec125e7359

[cjones051073 (Chris Jones)]

So, it appears all the failing platforms are using MP clang 14 to build, whereas 10.11, the oldest to be ok is not. Perhaps its enough to blacklist this compiler (and perhaps 13,12…) so fallback to a older one. Unfortunately the logs for the last builds to work on these platforms are not available to check what worked so its going to be a bit of trial and error…

[cjones051073 (Chris Jones)]

Replying to macdeport:

Build fail too on Yosemite but maybe for another reason.

Please open a separate ticket for this.

[cjones051073 (Chris Jones)]

Replying to cjones051073:

Replying to macdeport:

Build fail too on Yosemite but maybe for another reason.

Please open a separate ticket for this.

Scrap that. Probably is the same thing…

[cjones051073 (Chris Jones)]

just blacklist clang 14 did not help, as clang 13 had the same issue.

​https://github.com/macports/macports-ports/commit/eacc1edf719f8db5163e02a69cd8acbd3b90d5fe

which blacklists 12 and newer, so forces the use of clang-11 (a known sweet spot) seems to have perhaps done the trick, in so far as the builds seem to have got past gencondmd ok..

[mascguy (Christopher Nielsen)]

Well, if nothing else, this illustrates why most of us shouldn't directly commit upgrades of core build tools. Instead, it should be done via a PR, and left open for ample time for review.

Regardless, we need to fix this ASAP, or simply revert to 9.4.1. Thoughts on next steps...?

[cjones051073 (Chris Jones)]

Reverting is not necessary. It looks like the issues are now checksum diffs, which is an old chestnut we have run into before.

Comparing stages 2 and 3
warning: gcc/cc1obj-checksum.o differs
warning: gcc/cc1objplus-checksum.o differs
Bootstrap comparison failure!
gcc/alias.o differs
gcc/alloc-pool.o differs

newer GCCs (11, and 12 which I am working on right now) have logic to turn this off in certain cases. Almost certainly they are 'false' positives, so whilst not ideal doing this where required is acceptable I think.

[cjones051073 (Chris Jones)]

​https://github.com/macports/macports-ports/commit/e00d0a244534796e29aae5ef2854bb20f7172bf5

lets see if that helps.

[mascguy (Christopher Nielsen)]

Replying to cjones051073:

​https://github.com/macports/macports-ports/commit/e00d0a244534796e29aae5ef2854bb20f7172bf5

lets see if that helps.

Looks good, thanks Chris!

[macdeport]

Replying to cjones051073:

​https://github.com/macports/macports-ports/commit/e00d0a244534796e29aae5ef2854bb20f7172bf5

lets see if that helps.

I can confirm on 10.10 here. Thanks Chris.

[chillin-]

Sorry, I missed all this, months ago, apparently. I have learned to expect the bug report title changing, but it appears my original description was edited, which would have (should have) included that my build fail was on 10.14, not 10.8, 10.9, & 10.10. I vaguely recall an explanation on the mailinglist, but of course I have forgotten. Am I doomed to have libgcc9 seizing my "port -vsN upgrade outdated," forever?

[cjones051073 (Chris Jones)]

You shouldn't be using libgcc9 for anything any more. Only (lib)gcc10 or newer should be use on 10.14.

You need to though manually, yourself, identify which ports are still using libgcc9 on your system and manually reinstall them using newer variants of gcc. Run

port list installed | grep gcc9

and then, for each of those ports manually uninstall them and reinstall using the new default variants, which will be using (lib)gcc12 instead.

[macdeport]

Thank you to each of you for this streamlining.

[chillin-]

fyi, no ports were still using libgcc9, attempting uninstall warns this will break libgcc8. No ports use libgcc8, but attempt to uninstall libgcc8 warns it will break libgcc7. Attempt to uninstall libgcc7 warns it will break libgcc6. Attempt to uninstall libgcc6 warns it will break "gcc5" and pdftk.

pdftk is one of the few ports I manually requested that I actually use from time to time; most other ports I installed for "some day."

Looking at the dependencies of pdftk, it includes libgcc6, libgcc7, libgcc8, libgcc9, libgcc10, libgcc11 and libgcc12.

But I find it hard to believe pdftk won't build on 10.14 and above (because the older libgcc6-9 no longer build).

I need some instruction, please. Should I uninstall pdftk with --follow-dependencies and then rebuild it? Or will the subsequent pdftk build fail because libgcc6-9 won't build anymore on 10.14?

I don't want to lose pdftk.

[cjones051073 (Chris Jones)]

pdftk is obsolete and no longer maintained, and replaced by pdftk-java on newer systems. see

​https://github.com/macports/macports-ports/blob/master/textproc/pdftk/Portfile

Looking at that portfile, the obsolete pdftk indeed does still need gcc5 (or older) and does not offer variants for anything newer.

Unless you can find someone interested in updating that port to support a newer gcc, which might work, but also might not, your best course of action is to migrate to use pdftk-java instead which is the currently supported implementation.

[chillin-]

Research appreciated, Thank you! Maybe things have changed since my horrible experiences with java sucking. Computers have gotten much faster in the last 2 decades, but I doubt java could ever catch up with the performance of C++. I'll hang on to the old version until pdf changes enough to break the old pdftk and upgrade with "and not libgcc9." Thanks again, and this ticket may be closed and remain so, if not already.

[mascguy (Christopher Nielsen)]

Replying to chillin-:

Research appreciated, Thank you! Maybe things have changed since my horrible experiences with java sucking.

I've been developing with Java full-time, as part of my career, for more than two decades. And while JVM performance wasn't all that impressive in the first few years - prior to Sun and the industry aggressively pursuing just-in-time compilation - that hasn't been the case since the late 1990s.

Indeed, Java performance not only matches that of native C/C++ code, but can even exceed it in some cases. (It takes a brief period for the JVM/JIT to warmup, after a cold start. But once that's done, hold onto your hat!)

And the truly elegant thing about this design, is that the code doesn't have to be recompiled: Since it's pcode - with native compilation, optimization, etc, all done at runtime - it improves with every new JVM version.

The model works so well, that it's used extensively throughout the industry, for many many other languages and stacks. And indeed, Microsoft also went all-in on this paradigm, with C# and .NET as a whole.

So while you may have been less-than-impressed 20 years ago, things have changed significantly since.

[chillin-]

Some broken thoughts... My understanding is C# was Microsoft's attempt to compete with and kill Java, which failed but C# still became established.

I had heard that in some edge cases Java can outperform C++ due to JIT compilation, and I understand Java can reduce development time. But the machine in question is from 2012 and was discontinued in 2014. Whatever advantage I can get from native memory management vs Java's garbage collection I am happy to take.

I am pretty envious of your marketability, and greatly appreciate the bug fixes, but my box gets 40 rods to the hogshead and that's the way I likes it, so kindly get off my lawn.

[mascguy (Christopher Nielsen)]

Replying to chillin-:

I had heard that in some edge cases Java can outperform C++ due to JIT compilation, and I understand Java can reduce development time. But the machine in question is from 2012 and was discontinued in 2014. Whatever advantage I can get from native memory management vs Java's garbage collection I am happy to take.

The point is this: Hardware from 2012 - or even from 2006 - is more than powerful enough to run Java at incredible speeds. And your preconceived notions that you'll suffer a performance penalty are misguided, and completely wrong.

Furthermore, even the Java 6 JVM is mature enough, to provide native speeds. And the garbage collection was more than mature enough to ensure that it's not a significant factor in execution performance. (I've done extensive JVM/GC profiling and tuning over the years, and those are statements of fact.)

So the takeaway is this: Given that the Java 8 JVM is supported as far back as 10.8 - and it'll provide native for whatever you throw at it - this whole discussion is moot.

It sounds like the need/desire for libgcc9 is all predicated on a completely wrong assumption, which is that you'll get better performance than Java. That's completely and utterly wrong.

If you want to disregard guidance and advice you're being given here - and prefer to beat your head against the wall - that's fine. But we're trying to help you... and you aren't being receptive.

So why are you asking for help, if you refuse to listen to what folks are saying?

[chillin-]

But I did use the information, and I've made the best decision based on the information given and in deference to my older hardware, for which I have previously compiled pdftk for specifically. Being that the hardware is discontinued, and the vendor switched platforms, it is exceedingly unlikely that further development will occur with the platform architecture, so I'll never realize whatever performance advantages Java may -possibly- provide. Further, the pdftk-java readme estimates that due to the differences in Java and C++, that it is likely there are bugs in the Java version and further testing is required. Though I don't use it everyday, I consider it a production machine, and being that I will not be able to go back if I find problems with the Java version, I have stalled out of an abundance of caution. Though it is widely accepted that native code is generally more performant than interpreted, regardless of Java getting a boost from faster hardware, native code will always be more efficient than interpreted, and there's nothing Java will ever be able to do about that until there are Java hardware engines on every motherboard, which will likely never happen because it defeats the purpose of Java's portability. I was being silly towards the end and did not intend to offend anyone, didn't intend to start a flame war, but if anyone is delicate enough to take offense, I will reluctantly accommodate, and if it is war you want then it is war you shall have, because it is all beyond my ability to care.

Further, while I understand it is necessary to correct or add more information to original descriptions, and I don't doubt that I am making mistakes, I think it is a bad idea to vacate that description entirely and replace it. I reported the bug having discovered it on 10.14, and was shocked to see that all mention of 10.14 was removed from the description, and the title change, which I know is necessary because try as I may I haven't yet learned the title convention, it appears, incorrectly, this bug only existed on 10.8, 10.9 and 10.10. Leaving that information in the description would avoid future confusion. Just my opinion, and I am definitely not in charge, so just a suggestion.

[mascguy (Christopher Nielsen)]

There's no flaming going on here, I'm simply trying to educate. And this is all good, positive discussion!

So here's the bottom line: The JVM compiles Java pcode into true native code - highly-optimized code, comparable to the best C/C++ compilers - and then runs that native code at full speed. In short, there isn't any interpretation, beyond the first few seconds of execution. After that, it's all native code, which is how it matches or beats C/C++ performance.

As for garbage collection: In the very early days of the industry - before the days of multi-core CPUs, when speeds were still measured in MHz - GC did have a tangible impact on performance. And it could be quite intrusive, depending on the language/system.

Since then, GC techniques have evolved so far, that they're actually more efficient than C code that explicitly calls malloc() and free(). (Or the new and delete operators in C++, if you prefer.) That's because memory deallocation and reclamation can occur in the background - and in a batched, highly-efficient manner - such that it's inconsequential.

That's why even the newest languages like Google's Go - which was designed from the ground up as a C/C++ replacement - use GC for memory management. It's really that good!

All that said, if there's a legitimate concern - like pdftk-java not being as mature and reliable - then that's certainly reasonable.

Anyhow, I'm not trying to sell you on Java per se. Rather, I'm simply suggesting that you try to keep an open mind, and don't reject a solution simply because it's not pure C/C++.

Does that make sense?

[Christopher Nielsen <mascguy@…>]

In d97800da7384c3214964c929de2ecc9f61117b5a/macports-ports (master):

pdftk-java: update to 3.3.3
See: #65415

[kencu (Ken)]

if you really want pdftk and no other, with a little bit of manual effort I can show you how to have it.

It would be a self-contained pdftk version, built from macports binaries but thereafter remaining self-contained on your system. It will never be updated (but it isn't being updated anyway).

It would involve downloading some prebuilt binaries from <​http://packages.macports.com/pdftk/> and similar supporting libraries, and a bit of manual effort with otool and install_name_tool. It is actually interesting to do, in it's own way. 10 to 15 minutes, maximum should do it.

Let me know if that interests you.

[chillin-]

The JVM compiles Java pcode into true native code - highly-optimized code, comparable to the best C/C++ compilers - and then runs that native code at full speed. In short, there isn't any interpretation, beyond the first few seconds of execution. After that, it's all native code, which is how it matches or beats C/C++ performance.

p-code, or bytecode, is an intermediary between the Java language and the processor's native machine code. Modern C++ compilers typically compile to native machine code, though sometimes assembly is used as an intermediate step, but assembly is effectively human readable machine code. p-code is not human readable, and the code is interpreted on the JVM which itself runs on the processor. Java may be matching or exceeding performance of C++ native binaries, but it can't do so running on the bare metal without a JVM.

As for garbage collection

Java requires it, C++ doesn't need it. pdftk, as an example, will necessarily require less memory than pdftk-java.

That's why even the newest languages like Google's Go - which was designed from the ground up as a C/C++ replacement

This is contradictory. If it was designed as a replacement for C++, then C++ actually was the ground. For whatever reasons (my understanding is it was due to frequent version incompatibilities), Ken Thompson hated C++ from its early development (probably couldn't tolerate writing something more than once). That is why he co-developed Go, a reaction to C++, which is not a replacement for C++ as it doesn'r support direct calling of functions written in C/C++, and these languages are too engrained. One man's (well, three men's) intolerance for C++ is not a compelling reason to replace C++, and there are no other compelling reasons that could whelm me, as I am not a developer. Nevertheless, discussion of Go here is somewhat of a straw man.

Anyhow, I'm not trying to sell you on Java per se. Rather, I'm simply suggesting that you try to keep an open mind, and don't reject a solution simply because it's not pure C/C++.

I'm not a developer, only at times a lowly sysadmin. When a solution is not immediately forthcoming, I'll use whatever works, no matter how ugly it may be. Fortunately, I already have something that works just fine with only a trivial adjustment to my upgrade script to get the best package manager in the world to cooperate.

Does that make sense?

I don't think you really want to know.

---

with a little bit of manual effort I can show you how to have it... Let me know if that interests you.

That is incredibly generous, and appreciated, but I assure you it is not necessary. I'd honestly prefer the pdftk binary I rolled myself (with all the heavy lifting by MacPorts), which fortunately happens to already be installed on the system and which I briefly used earlier today to combine some pdfs that I wanted bundled together. I don't mind that pdftk is no longer in active development, and I honestly believe that most software would benefit from that strategy rather than continue endlessly on the death march of feature creep, which ruins so much good software it is ridiculous -for me personally and most recently, my iPad's ssh client. I want back the way it worked 2 years ago, but the developers insist it had rendering artifacts, which I never personally experienced and think they're making it up.

[kencu (Ken)]

Oh -- you already have pdftk.

Well problem solved then.