clang-16 and clang-17: Undefined symbols "std::__1::__libcpp_verbose_abort

[snowflake (Dave Evans)]

Here's the link error for poppler which is also reported by the build bots for macOS 10.11:

Undefined symbols for architecture x86_64:
  "std::__1::__libcpp_verbose_abort(char const*, ...)", referenced from:
      __ZNSt3__120__throw_out_of_rangeB8un170004EPKc in GooString.cc.o
      __ZNSt3__120__throw_length_errorB8un170004EPKc in GooString.cc.o
      __ZNSt3__120__throw_length_errorB8un170004EPKc in gbase64.cc.o
      __ZNSt3__120__throw_length_errorB8un170004EPKc in gbasename.cc.o
      __ZNSt3__120__throw_length_errorB8un170004EPKc in gfile.cc.o
      __ZNSt3__120__throw_length_errorB8un170004EPKc in grandom.cc.o
      __ZNSt3__120__throw_length_errorB8un170004EPKc in FoFiTrueType.cc.o
      ...

[kencu (Ken)]

more of this error….

can you try clang-16 instead of clang-17?

[snowflake (Dave Evans)]

Replying to kencu:

more of this error….

can you try clang-16 instead of clang-17?

Building with clang-16 works! Thank you.

sudo port -s upgrade poppler configure.compiler=macports-clang-16

[RobK88]

FYI - I had the same problem building poppler on Lion and Mtn Lion. But I was able to build poppler on Lion and Mtn Lion using clang-16.

[kencu (Ken)]

there are a few changes in clang-17 that are causing troubles

[kencu (Ken)]

chris this error has been showing up building with clang-17 on systems with older libc++ installations (see the recent mozjs pr for another example) , and there was another different error as well with clang-17…I’m sure you saw the ticket.

[kencu (Ken)]

chris bump

[yaobao1993 (Yao Bao)]

Hello,

Seems macOS Sierra (10.12.6) encounters the same issue.

Inspired by above comments, I tried:

$ sudo port install poppler configure.compiler=macports-clang-16

But the build failed with the same error.

I am new to MacPorts, seems above command does not make compiler switched to clang-16.

Here is the installed clang related packages:

clang-16 @16.0.6_2+analyzer (active)
clang-17 @17.0.4_0+analyzer (active)

How to make sure clang-16 is used for this build?

Thanks

[snowflake (Dave Evans)]

Yao, You need to tell Macports to build from source. This is done by the -s flag

sudo port -s install poppler configure.compiler=macports-clang-16

Without the -s flag, Macports will try to fetch a built archive from the central server. It probably won't exist as this port is not built on the server either.

[snowflake (Dave Evans)]

The latest version of Poppler, poppler @23.12.0_0+boost, has just built on my old El Capitan Mac without any special invocations for selecting a compiler.

[cjones051073 (Chris Jones)]

Replying to snowflake:

The latest version of Poppler, poppler @23.12.0_0+boost, has just built on my old El Capitan Mac without any special invocations for selecting a compiler.

That is only because clang-17 is no longer used on older systems that have this issue by default. If you force the use of it, you will probably see the issue is still there.

[ryandesign (Ryan Carsten Schmidt)]

Did anyone bother to file a bug report with the llvm developers?

[kencu (Ken)]

Well -- it's not a bug. It's a feature.

They don't support older libc++ versions on MacOS, and don't entertain any requests to do so.

macports-libcxx of a correct vintage might work (have to be newer than the one we have now, though, I think).

[kencu (Ken)]

it is not impossible that someone could create these symbols and add them to legacysupport (and the landing pads for the std::optional and other exceptions that cause issues as well, perhaps) and -- here's the part I don't know how to do -- make the linker accept them as coming from the libcxx namespace.

I don't know how to do that -- but someone might know.

[kencu (Ken)]

here is a replacement function used in flang for this:

​https://github.com/llvm/llvm-project/commit/ce6b9b3b58b6c9e51d87084c916fa7aef81401f1

and here is some useful discussion:

​https://reviews.llvm.org/D158957

[kencu (Ken)]

has duplicate #69603

[christophecvr (christophecvr)]

Replying to Dave-Allured:

Some info about this error. on macos 10.13.6 this errors shows up as from use of clang 15. With clang 14 no problems. There was a ticket about it

#69603 But also another ticket building totem failed during install phase when running a totem-scan check function when building with docs. When building without docs totem did build and installed but crashed up on start with segfault 11.

Ticket #69594

But since I downgraded clang to 14 all problems are solved. They all show up since clang-15

[christophecvr (christophecvr)]

I pushed to my fork of macports-ports master a temporary solution for macos 10.13.6 . This problem does not occur as from clang 17 but as from clang 15 . Clang 14 is the last stable compiler for macos 10.13.6 (i do not known for other macos like 10.11.... to 10.22....) But the use of clang higher then 14 does not only cause this issue but many more issues. p.s. apple dropped support for older os on clang.

​https://github.com/christophecvr/macports-ports/commit/310a49be0f3d93a5f6919bcf78778fed95210f53

[kencu (Ken)]

what is unclear at present is why clang-16 did not show this issue a few months ago (as above), but apparently now does show this issue, at least sometimes.

clang-16 does not appear to have changed in the interim. There is something more to this yet.

[christophecvr (christophecvr)]

Replying to kencu:

what is unclear at present is why clang-16 did not show this issue a few months ago (as above), but apparently now does show this issue, at least sometimes.

clang-16 does not appear to have changed in the interim. There is something more to this yet.

I do not know . Self i just started with mac ports 2 months ago. I’ve been out for a while. But before I used to work with linux . Actually I started so wath 20 years ago with debian and then changed to ubuntu. But the problems are there since longer I think and not since clang 16 . Oneof the causes is that a lot of packages are build and just needed to be downloaded. They were already build most probably before the clang was upgraded to versions above 14 . But then yes a new version somewhere needed upgrade for some libs and so and we must build locally with the result we see now. And typical to this is that just a function is only sometimes used and only with some packages . On top of it with error messages who are misleading . I start the more thinking that the real problem is there since clang 15 due to a function wichbis not compatible anymore with older macos guess all os below 10.14

[kencu (Ken)]

no, it's not that prebuilt binaries were downloaded.

see above: comment:2

more to figure out stil.

[christophecvr (christophecvr)]

Replying to kencu:

no, it's not that prebuilt binaries were downloaded.

see above: comment:2

more to figure out stil.

Yes but then below comment:9 says it does not work with clang16 same issue . That seems then again what I encountered ok with other packages he did not try to downgrage the clang further . I started harder and at firs tried out with clang 12 . Then it bild fines ok . Then i went up to clang 13 still ok then up to 14 still ok . Then up to 15 and bingo failure with the error. Back to 14 and ok . Did the test with another package as totem that had the issue segfault 11 that issue is showing up in a lot of tickets and yes also have a main origin clang compiler upgrade . So at first persons should test this out on more os versions then 10.13.6 . To find out . I also will try out next week on this poppler how it is an which clang version is fine . But then proceed with my own stuff trying to upgrade gnome/nautilus to at least 3.38.2 since the current version is one of the most buggy versions produced due to there works on gio. But that will be all for next week

[christophecvr (christophecvr)]

For info the package poppler does build fine on macos 10.13.6 with clang 17,16,15,14 very fun but it's a fact. I will post the terminal output log of successful build of poppler on macos 10.13.6 . This mean that this ticket is not the same as mine for gjs and totem. By the way gjs and totem needs clang14 or lower. But poppler builds fine with clang17

[christophecvr (christophecvr)]

Poppler build installog on macos 10.13.6 does work fine.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #69598 (building gegl with clang++ 16).

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #69664 (building mesa with clang++ 16).

[ryandesign (Ryan Carsten Schmidt)]

Since nobody has responded to my question about whether a bug was filed with the developers of llvm about this and I could not find one, I filed at ​https://github.com/llvm/llvm-project/issues/87590.

Can we downgrade the affected systems to clang 15?

[kencu (Ken)]

I reiterate this will not likely be considered an llvm bug, but more likely a misuse of llvm/libcxx.

eg see:

​https://github.com/llvm/llvm-project/issues/87012

I think this is most likely happening because the macports-libcxx port disables libc++ availability testing to allow new library symbols to be seen on the older MacOS system.

​https://github.com/macports/macports-ports/blob/master/lang/macports-libcxx/files/patch-disable-availabilty.diff

But it looks like turning off availability testing now tells newer clang versions that libc++ provides a verbose_abort function when it does not:

​https://github.com/llvm/llvm-project/blob/6009708b4367171ccdbf4b5905cb6a803753fe18/libcxx/include/__availability#L170

I think if we want some kind of fix for this on older MacOS systems, we are going to have to come up with it ourselves.

[ryandesign (Ryan Carsten Schmidt)]

Replying to kencu:

I think this is most likely happening because the macports-libcxx port disables libc++ availability testing to allow new library symbols to be seen on the older MacOS system.

​https://github.com/macports/macports-ports/blob/master/lang/macports-libcxx/files/patch-disable-availabilty.diff

What would happen if we didn't do that?

[cjones051073 (Chris Jones)]

Replying to ryandesign:

Since nobody has responded to my question about whether a bug was filed with the developers of llvm about this and I could not find one, I filed at ​https://github.com/llvm/llvm-project/issues/87590.

Can we downgrade the affected systems to clang 15?

There is nothing to downgrade. I have already prevented the selection of clang-17 or newer as a compiler on the affected older systems.

​https://github.com/macports/macports-ports/blob/master/_resources/port1.0/compilers/clang_compilers.tcl#L16

If ports then *specifically* select clang-17/18 in some way, thats their choice.

[ryandesign (Ryan Carsten Schmidt)]

The problem also occurs with clang 16. (comment:19, comment:29, comment:30)

[kencu (Ken)]

possibly if we patch clang 16+ to always define this macro in the installed clang headers

#  define _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT 0

things will work again through the fallback pathway.

​https://github.com/llvm/llvm-project/blob/1b761205f2686516cebadbcbc37f798197d9c482/libcxx/include/__verbose_abort#L39

Someone with a machine that is seeing this issue would have to test this -- and it's also possible this has been baked into some built software libraries I think.

A clear understanding of when and where this occurs is still evolving.

[kencu (Ken)]

Replying to ryandesign:

Replying to kencu:

I think this is most likely happening because the macports-libcxx port disables libc++ availability testing to allow new library symbols to be seen on the older MacOS system.

​https://github.com/macports/macports-ports/blob/master/lang/macports-libcxx/files/patch-disable-availabilty.diff

What would happen if we didn't do that?

macports-libcxx headers would obey Apple OS availability rules for the system libc++, ignoring the functionality of the newer libc++.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #69671 (building mesa with clang++ 16).

[kencu (Ken)]

this seems to fix this issue for clang-16 when using macports-libcxx, by defining the noted define when the availability settings are disabled:

$ diff -u /opt/local/libexec/llvm-16/include/c++/v1/__availability.orig /opt/local/libexec/llvm-16/include/c++/v1/__availability
--- /opt/local/libexec/llvm-16/include/c++/v1/__availability.orig	2024-04-07 07:21:22.000000000 -0700
+++ /opt/local/libexec/llvm-16/include/c++/v1/__availability	2024-04-07 07:22:05.000000000 -0700
@@ -159,7 +159,7 @@
     // This controls whether the library claims to provide a default verbose
     // termination function, and consequently whether the headers will try
     // to use it when the mechanism isn't overriden at compile-time.
-// #   define _LIBCPP_HAS_NO_VERBOSE_ABORT_IN_LIBRARY
+#   define _LIBCPP_HAS_NO_VERBOSE_ABORT_IN_LIBRARY
 
 #elif defined(__APPLE__)
 

I would suspect something similar will fix clang-17.

It needs a bit of broader testing, from a few interested folks, and then if it is indeed robust, it's a very very trivial patch to the installed clang cxx headers.

[kencu (Ken)]

for clang-17, this works:

$ diff -u /opt/local/libexec/llvm-17/include/c++/v1/__availability.orig /opt/local/libexec/llvm-17/include/c++/v1/__availability
--- /opt/local/libexec/llvm-17/include/c++/v1/__availability.orig	2024-04-07 07:45:37.000000000 -0700
+++ /opt/local/libexec/llvm-17/include/c++/v1/__availability	2024-04-07 07:50:37.000000000 -0700
@@ -166,8 +166,8 @@
     // This controls whether the library claims to provide a default verbose
     // termination function, and consequently whether the headers will try
     // to use it when the mechanism isn't overriden at compile-time.
-// #   define _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT
-#   define _LIBCPP_AVAILABILITY_VERBOSE_ABORT
+#   define _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT
+#   define _LIBCPP_AVAILABILITY_VERBOSE_ABORT __attribute__((unavailable))
 
     // This controls the availability of the C++17 std::pmr library,
     // which is implemented in large part in the built library.

[kencu (Ken)]

and for clang-17, this simpler one also works, although doesn't specificy the "unavailable" nature of the function specifically like the above patch does.

$ diff -u /opt/local/libexec/llvm-17/include/c++/v1/__availability.orig /opt/local/libexec/llvm-17/include/c++/v1/__availability
--- /opt/local/libexec/llvm-17/include/c++/v1/__availability.orig	2024-04-07 07:45:37.000000000 -0700
+++ /opt/local/libexec/llvm-17/include/c++/v1/__availability	2024-04-07 07:56:40.000000000 -0700
@@ -166,7 +166,7 @@
     // This controls whether the library claims to provide a default verbose
     // termination function, and consequently whether the headers will try
     // to use it when the mechanism isn't overriden at compile-time.
-// #   define _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT
+#   define _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT
 #   define _LIBCPP_AVAILABILITY_VERBOSE_ABORT
 
     // This controls the availability of the C++17 std::pmr library,

[kencu (Ken)]

In 1a99bb3d5d1a256f810535cb9a0d6e414534ad89/macports-ports (master):

clang-16 @16.0.6_3: disable verbose_abort in availability

when availability tests are disabled (macports-libcxx, darwin 10)
a fallback set of defines are used in

include/c++/v1/availability

however, the fallback defaults to indicating that libcxx contains
a verbose_abort function. This is currently always false for MacPorts
uses, and so link errors occur.

disable this by default, so a fallback pathway is used

see: #68640

this has no effect on the normal functionality of this header,
as the APPLE pathway is used otherwise.

[kencu (Ken)]

I will do clang-17 later -- it's already not being used at present due to this issue. Once it is shown to be fixed, clang-17 can be re-enabled on older systems.

[christophecvr (christophecvr)]

On macos 10.13.6 gjs does now build with clang16, Totem does not fail anymore with segfault 11.

[mascguy (Christopher Nielsen)]

Replying to christophecvr:

On macos 10.13.6 gjs does now build with clang16, Totem does not fail anymore with segfault 11.

Just forced rebuilds of mesa, and it is now fixed as well. (The port status page is delayed by a few minutes, but will show all green once it catches up.)

​https://ports.macports.org/port/mesa/details/

[kencu (Ken)]

I popped this fix up in this branch of macports-ports

​https://github.com/macports/macports-ports/commits/clang-17-fixes/

I did it like this so Chris and others could easily take a look.

There is another clang fix that is needed -- moving the libc++* parts into a subdirectory of the current library folder where they are placed now, so they are not found opportunistically. In the interests of efficient buildbot usage, might be best to do these together.

I would have done it today, but the current destrooting situation with the clang ports has become trickier in ways, so I'll leave this fix to Chris. For details on that, see #69725 and #68281