Opened 2 years ago

Last modified 5 months ago

#58898 assigned defect

libcxx bootstrapping needs a method to upgrade to the +emulatedtls variant on 10.6 and less

Reported by: kencu (Ken) Owned by: kencu (Ken)
Priority: Normal Milestone:
Component: ports Version:
Keywords: snowleopard leopard tiger Cc: jmroot (Joshua Root), mojca (Mojca Miklavec), cjones051073 (Chris Jones), kencu (Ken), dliessi (Davide Liessi)
Port: libcxx

Description

during automatic bootstrapping of 10.6 and less, libcxx is installed as a first step, building with clang-3.4.

As clang-3.4 does not support building libcxx +emulatedtls, libcxx is first built without +emulatedtls. The process then leads to the installation of clang-5.0 or greater, and these compilers do support building libcxx +emulatedtls.

The libcxx Portfile will build libcxx +emulatedtls if a suitable compiler is installed that can build it, but it presently requires a manual rebuild to force the upgrade.

For a fully functional older system, this upgrade should be automatic.

One option might be to put a revision increase by 1 in the Portfile, in the block that tests if libcxx +emulatedtls can be built and enables it if so.

Change History (24)

comment:1 Changed 2 years ago by kencu (Ken)

Owner: kencu deleted

comment:2 Changed 2 years ago by mf2k (Frank Schima)

Cc: jeremyhu removed
Owner: set to jeremyhu

comment:3 Changed 2 years ago by kencu (Ken)

jeremy is not going to fix this. At this point in time I might as well consider owning this port, even though I deleted myself from owning this ticket a week ago :}

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:4 Changed 2 years ago by mojca (Mojca Miklavec)

Cc: mojca added

comment:5 Changed 2 years ago by cjones051073 (Chris Jones)

Cc: cjones051073 added

comment:6 Changed 2 years ago by kencu (Ken)

so the revision +1 idea if a clang >= 5.0 is found is one way to force the update.

could also exist a libcxx-bootstrap port.

Maybe that is more appropriate?

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:7 Changed 2 years ago by kencu (Ken)

the revision +1 approach will get pretty confused by the buildbot's portindex. That's probably not a good idea, in the end...

comment:8 Changed 2 years ago by cjones051073 (Chris Jones)

I also would be wary of an approach that plays games with the revision... seems like a recipe for all sorts of problems.

Last edited 2 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:9 Changed 2 years ago by cjones051073 (Chris Jones)

I think a dedicated 'bootstrap' version of libcxx that the first bootrap phase of the clang builds use, is the way to go. It would then later on get replaced by the main libcxx build.

comment:10 Changed 2 years ago by kencu (Ken)

shame we can't depend on a port variant..

So libcxx on < 10.7 would have to depend on libcxx-bootstrap. Tha's simple enough. But it would also have to depend on some clang >= 5.0, and that is circular.

unless clang can be satisfied by libcxx-bootstrap. which it can.

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:11 Changed 2 years ago by cjones051073 (Chris Jones)

I was more thinking of something like.... clang-3.4 declares a path like dependency, something like path:lib/something.dylib:libcxx-bootstrap so when it is first built it triggers the installation of libcxx-bootstrap. Then, later on, when clang5 etc. are built they keep the port:libcxx dep. Finally, libcxx would have to have some logic added to forcibly remove libcxx-bootstrap when it is built/installed, so effectively it replaces it.

Last edited 2 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:12 Changed 2 years ago by kencu (Ken)

It's complex. clang-3.4 can't depend on any version of libcxx.

comment:13 Changed 2 years ago by kencu (Ken)

libcxx just overwrites to previous, so doesn't have to forcibly remove it ... but then you cannot tell which libcxx is installed with a path dependency either...

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:14 Changed 2 years ago by cjones051073 (Chris Jones)

Hmmmm. How about adding a libcxx-bootstrap sub-port to libcxx, that does everything identically apart from it does not enable emulatedtls whereas libcxx does. Basically give libcxx and libcxx +emulatedtls different port names so you can then more easily tell which is currently installed....

comment:15 Changed 2 years ago by cjones051073 (Chris Jones)

b.t.w. you say clang-3.4 does not itself depend on libcxx. But, something in the dep tree must be the thing that first triggers its installation ? Could you not then make that depend on path:lib/something.dylib:libcxx-bootstrap ?

comment:16 in reply to:  14 ; Changed 2 years ago by kencu (Ken)

Replying to cjones051073:

Hmmmm. How about adding a libcxx-bootstrap sub-port to libcxx, that does everything identically apart from it does not enable emulatedtls whereas libcxx does.

Chris - that is exactly what I suggested :>

But -- you can't tell which one is installed with a path dep.

I would work on this, but history tells me Josh will parachute in with the fix any second now...

comment:17 Changed 2 years ago by kencu (Ken)

Now -- if I actually was a computer science major, perhaps I would have folded the emulated_tls objects in somewhere else, like perhaps in ld64 and adding in tapi, like Iains appears to have done here <https://github.com/iains/tapi/commit/aeab0de3e9ec61c97d8f3cd86928983eba59de49> .

But I did it the way I understood how to do it, using existing libcxx tools, so we have the setup we have.

Of interest, it is pretty simple to build libcxx +emulated_tls with gcc with some modest hackery <https://github.com/kencu/macports-ports/commits/libcxxfixupgcc> . That's what I do on PowerPC, but I knew/know that is not a method acceptable to MacPorts, so never bothered to propose it here.

comment:18 in reply to:  16 Changed 2 years ago by cjones051073 (Chris Jones)

Replying to kencu:

Replying to cjones051073:

Hmmmm. How about adding a libcxx-bootstrap sub-port to libcxx, that does everything identically apart from it does not enable emulatedtls whereas libcxx does.

Chris - that is exactly what I suggested :>

But -- you can't tell which one is installed with a path dep.

I guess not with path... But we could have each of the flavours of libcxx install some special info file which says which it is (via the file name for instance), and then use that somehow to determine if libcxx-bootstrap is in fact installed, and thus needs to be forcibly replaced with libcxx....

I would work on this, but history tells me Josh will parachute in with the fix any second now...

Last edited 2 years ago by cjones051073 (Chris Jones) (previous) (diff)

comment:19 Changed 2 years ago by kencu (Ken)

You're right -- there is a special include file in /usr/include that I only include with +emulated_tls variant. We could path dep on that.

Edit: nope, I'm wrong. I just expose a new function in the header, but it's not a totally new header. So we would still need a breadcrumb, as Chris said.

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:20 Changed 21 months ago by kencu (Ken)

Cc: kencu added

comment:21 Changed 16 months ago by kencu (Ken)

I have been wrapping my head around this, but I always wind up back where I started with circular dependencies if there are to be no manual steps.

The simplest thing would be to make the +emulated_tls variant a default variant, and have Ryan build it manually one time on 10.6. Then it's in the archives, and can be downloaded and installed. But it's a manual step.

It is trivial to make up a port that could either be called libcxx-bootstrap or libcxx +bootstrap that builds the current non-emulated-tls libcxx. And then the standard libcxx port could enable emulated_tls by default. libcxx could depend on libcxx-bootstrap if there is nothing at /usr/lib/libc++.dylib, for example.

But for that to work, everything that is in the line to build libcxx would have also be satisfied by libcxx-bootstrap instead of just by libcxx, and that is a lot of ports leading all the way up to clang-9.0. It's messy.

Alternately, we could build libcxx 7.0 using gcc7 or gcc8. We can bootstrap to gcc7 right now from the base OS, and to gcc8. And those compilers can build libcxx 7.0. Totally different than what we do now, and there is going to be something messy about the libcxxabi ABI linking into gcc I think to be managed somehow.

I could build libcxx mostly with clang-3.4, but use gcc7 to build the troublesome cxa_thread_atexit.cpp file that needs a __thread supporting compiler. That's a bit ugly. Hard to see how that would be the best way forward.

Or I could just download a fully intact, pre-build libcxx+universal+emulated_tls from some webserver or github repo where I keep it, and be done with it. That is dead simple.

Last edited 16 months ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:22 Changed 13 months ago by kencu (Ken)

Owner: changed from jeremyhu to kencu

comment:23 Changed 9 months ago by dliessi (Davide Liessi)

Cc: dliessi added

comment:24 Changed 5 months ago by kencu (Ken)

I actually can build libcxx/libcxxabi by bootstrapping with gcc8.

gcc8 automatically builds llvm & clang, and then uses that clang to build libcxx/libcxxabi with all the toys enabled.

but gcc8 still has deps on things that require libc++, in the end, so we're no further ahead.

I am very (very) tempted to just put up my libcxx built system_roots for 10.6, 10.5, and 10.4, all of which have thread_local enabled properly in them, and have the libcxx port install those binaries. And if I ever get to releasing the clang-7.0 for PPC I have been using, I can add the libc++ for PPC there too.

I can leave some instructions in the wiki on how people can build their own.

Note: See TracTickets for help on using tickets.