cctools, ld64: Removing +llvm33 variant introduces circular dependencies on Leopard due to llvm-3.4 failing to build with Xcode 3.1's gcc

[ryandesign (Ryan Carsten Schmidt)]

cctools and ld64 cannot be installed on Leopard because they eventually depend on themselves.

​https://build.macports.org/builders/ports-10.5_ppc_legacy-builder/builds/2

--->  Dependencies to be installed: cctools llvm-3.4 apple-gcc42 ld64 ld64-127 libgcc-devel
DEBUG: dlist_eval: all entries in dependency list have unsatisfied dependencies; can't process
Error: The following dependencies were not installed: cctools llvm-3.4 apple-gcc42 ld64 ld64-127 libgcc-devel
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port gcc7 failed

cctools depends on llvm-3.4, which depends on apple-gcc42, which depends on cctools.

ld64 depends on ld64-127, which depends on llvm-3.4, which depends on apple-gcc42, which depends on ld64.

[larryv (Lawrence Velázquez)]

You have to use apple-gcc42 +bootstrap here, although I don’t know how you’d automate it on a buildslave.

[jeremyhu (Jeremy Huddleston Sequoia)]

Yeah, there's not really a way to do that short of rolling up a bootstrap package which lays down a toolchain in /opt/macports-bootstrap-toolchain or something similar. Of course, base would need to be updated to know how to use that toolchain (and make it part of the default toolchains, perhaps as macports-bootstrap-gcc and macports-bootstrap-clang). I'm not opposed to doing that and have considered doing so myself in the past. If that seems like it would be worthwhile, I'll see what I can do to come up with such a package.

[jeremyhu (Jeremy Huddleston Sequoia)]

We could also make 'apple-gcc42 +bootstrap' into a subport instead of a variant if that is desired.

[ryandesign (Ryan Carsten Schmidt)]

I like the subport idea, if apple-gcc42 now needs the bootstrap stuff on Leopard as well. I thought it only applied to Tiger, based on stuff like:

$ port variants apple-gcc42
apple-gcc42 has the variants:
   bootstrap: Variant to break a dependency cycle on Tiger by first building an
              apple-gcc42 using host ld and cctools
   gpl3: Merge in changes from gcc-4.2.4
   universal: Build for multiple architectures

[jeremyhu (Jeremy Huddleston Sequoia)]

Ah, ok. That was true in the past but isn't now.

In r135187, I added Leopard's apple-gcc to the blacklist for llvm-3.4, so to build llvm-3.4, we need either apple-gcc42 or clang-3.3.

In r150763 r150764, I removed the +llvm33 variant from cctools and ld64 and made the default on Leopard be +llvm34.

I think the simplest solution for now might be to revert r150763 and r150764.

[mojca (Mojca Miklavec)]

Cc Me!

[mojca (Mojca Miklavec)]

Replying to jeremyhu@…:

Yeah, there's not really a way to do that short of rolling up a bootstrap package which lays down a toolchain in /opt/macports-bootstrap-toolchain or something similar. Of course, base would need to be updated to know how to use that toolchain (and make it part of the default toolchains, perhaps as macports-bootstrap-gcc and macports-bootstrap-clang). I'm not opposed to doing that and have considered doing so myself in the past. If that seems like it would be worthwhile, I'll see what I can do to come up with such a package.

We started discussing using an external compiler for libc++ builds in the past, but Ryan warned against that idea because some compilers might leave traces (we could end up with a library linking against /opt/macports-bootstrap-toolchain/lib/libsomething.dylib or any other weird situation).

The problem we'll have on the libc++ build slaves will be that clang-3.x and all of its dependencies will have to be activated and deactivated over and over again.

[jeremyhu (Jeremy Huddleston Sequoia)]

I reverted back to +llvm33 as the default in r152009. Leaving this open as I want to find a better solution.

[mojca (Mojca Miklavec)]

Jeremy, why not using

if {${os.major} < 11} {
    variant llvm34 conflicts llvm33 llvm38 llvm39 llvmdev description {Use llvm-3.4 for libLTO and llvm-mc} {
        set llvm_version        3.4
        depends_lib-append      port:llvm-${llvm_version}

        set has_llvm_nm         false
        set has_llvm_size       false
    }
}

instead of

variant llvm34 conflicts llvm33 llvm38 llvm39 llvmdev description {Use llvm-3.4 for libLTO and llvm-mc} {
    set llvm_version        3.4
    depends_lib-append      port:llvm-${llvm_version}

    set has_llvm_nm         false
    set has_llvm_size       false

    if {${os.major} >= 11} {
        ui_error "The +llvm34 variant is not supported on Lion and later."
        error "Invalid variant selected"
    }
}

? I guess that conflicting variants could be adjusted with a loop, but then again conflicts with non-existing variants probably don't hurt anyway.

Similar for ld64:

devel/ld64/Portfile

@@ -42 +42 @@
                     sha256  307f73678a3e5c9ed4d1bcf77da7399d84efac32916c5df6cd477c3b5c36f953
 
 
+if {${os.major} < 14} {
 subport ld64-97 {
     # XCode 3.2.6
     version             97.17
@@ -62 +63 @@
         ld64-97-no-Availability.h.patch \
         ld64-97-BaseAtomImplicitDecl.patch \
         ld64-97-no-ppc-thread_status.patch
-
-    if {${os.major} >= 14} {
-        pre-fetch {
-            ui_error "$subport is not supported on Yosemite or later."
-            error "unsupported platform"
-        }
-    }
 }
+}
 
 subport ld64-127 {
     # XCode 4.2
@@ -94 +89 @@
         ld64-ppc-9610466.patch
 }
 
+if {${os.major} > 9} {
 subport ld64-136 {
     # XCode 4.6
     version             136
@@ -113 +109 @@
     if {${configure.cxx_stdlib} eq "libstdc++"} {
         patchfiles-append   ld64-136-hash_set.patch
     }
-
-    if {${os.major} <= 9} {
-        pre-fetch {
-            ui_error "$subport is not supported on Leopard or earlier.  It requires the blocks runtime."
-            error "unsupported platform"
-        }
-    }
 }
+}
 
+if {${os.major} > 9} {
 subport ld64-236 {
     # XCode 5.1
     version             236.3
@@ -146 +137 @@
     if {${configure.cxx_stdlib} eq "libstdc++"} {
         patchfiles-append   ld64-236-hash_set.patch
     }
-
-    if {${os.major} <= 9} {
-        pre-fetch {
-            ui_error "$subport is not supported on Leopard or earlier.  It requires the blocks runtime."
-            error "unsupported platform"
-        }
-    }
 }
+}
 
+# requires a C++11 runtime
+if {[file exists /usr/lib/libc++.dylib]} {
 subport ld64-latest {
     # XCode 7.3.1
     version             264.3.102
@@ -180 +167 @@
     configure.cxxflags-append -std=c++11
 
     supported_archs i386 x86_64
-
-    pre-fetch {
-        if {![file exists /usr/lib/libc++.dylib]} {
-            ui_error "$name requires a C++11 runtime, which your configuration does not allow"
-            error "unsupported configuration"
-        }
-    }
 }
+}
 
 subport ld64-xcode {
     version             1

Anyway, I'm not sure if it's just because I made a partial update of your commits (I only checked out ld64 and ran portindex), but this fails for me already during the portindex phase:

Adding port devel/ld64
Error: The +llvm34 variant is not supported on Lion and later.
Error: ld64: Error executing llvm34: Invalid variant selected
Failed to parse file devel/ld64/Portfile with subport 'ld64-97': Error evaluating variants

[jeremyhu (Jeremy Huddleston Sequoia)]

Because that will cause portindex to be created differently based on which OS you create it on.

[jeremyhu (Jeremy Huddleston Sequoia)]

You should 'sudo port -v -s upgrade ld64-97 -llvm34.

[mojca (Mojca Miklavec)]

Replying to jeremyhu@…:

Because that will cause portindex to be created differently based on which OS you create it on.

I'm not familiar with the infrastructure, but judging from ​Ryan's email:

Currently, there is only one PortIndex file generated on the server, and published to the rsync server, for each OS name/version/endianness combination.

I guess that except perhaps for

if {[file exists /usr/lib/libc++.dylib]} {

(until we make a separate PortIndex, in particular on < 10.7) this should work.

[larryv (Lawrence Velázquez)]

It would work technically, but it introduces implicit behavior (different variants visible on different systems) that might be considered undesirable.

[ryandesign (Ryan Carsten Schmidt)]

I don't think we have a policy on that: is it better to only show variants on systems where they can be used, or show them everywhere and make them error when they're not applicable? I think we have examples of both currently in the port tree.

[jeremyhu (Jeremy Huddleston Sequoia)]

FWIW, the variant-exclusions are being addressed by hiding them on newer OS versions (r152081, with r152082 reverting the accidental change to wiggle).

This is being left open to still consider the apple-gcc42-bootstrap subport option as a way of avoiding the need for the +llvm33 variant.

[mojca (Mojca Miklavec)]

I opened a new ticket for removing the variants: #52128.

I didn't understand why

sudo port -v -s upgrade ld64-97 -llvm34

because I don't have ld64-97 installed at all. But I just realized that I have

ld64-latest @264.3.102_2+llvm34 (active)

which the next upgrade should fix.

More likely the reason had at least something to do with #52127 though. I no longer experience the error now.

[jeremyhu (Jeremy Huddleston Sequoia)]

cctools removed the +llvm33 variant in 373d97e8/macports-ports in a way that should not make this be a problem (essentially dropping support for LTO on Leopard by default).

So on Leopard and Tiger, we should be able to build cctools without depending on llvm, but we still have the issue with ld64.

[kencu (Ken)]

fixed