Add support to provide a clang-based as wrapper

[howarth.at.macports@…]

The current clang-3.7 subport is forcing a downgrade in the clang-based assembler used on Xcode 7..

    depends_build       port:cctools
    depends_skip_archcheck-append cctools ld64 subversion

This seems like very undesirable behavior.

[ryandesign (Ryan Carsten Schmidt)]

Please use WikiFormatting. Also note that the Cc field requires complete email addresses.

[jeremyhu (Jeremy Huddleston Sequoia)]

What makes you come to this conclusion?

Actually, I think that Xcode 7.0's tools branched from llvm before the final 3.7.

If you really want a newer assembler than 3.7, install 3.8.

[howarth.at.macports@…]

Because when I tried 'sudo port -d -s install clang-3.7', the build dependencies include cctools. Inspection of the Portfile for llvm-3.7 shows that...

depends_build port:cctools

is defined for } elseif {${subport} eq "clang-${llvm_version}"} {. IMHO, that depends_build should be protected by a conditional that doesn't set it for Xcode 7 or later. Also, I see the same behavior in the llvm-3.8 Portfile for its dependencies required in the clang-3.8 subport.

[howarth.at.macports@…]

Also, the clang-based assembler is not part of the upstream llvm.org builds. This feature seems to be an Apple specific replacement for the legacy GNU based assembler. It is really unclear where Apple would contribute that code into the llvm.org project anyway as it isn't really part of cfe logically needs to build against it.

[howarth.at.macports@…]

Also, the crake-based build of llvm38 in fink only produces the expected llvm-as which is entirely different from the new clang-based assembler in Xcode 7 which reports back as...

% as -v

Apple LLVM version 7.0.0 (clang-700.0.72)
Target: x86_64-apple-darwin15.0.0
Thread model: posix
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1as -triple x86_64-apple-macosx10.11.0 -filetype obj -main-file-name - -target-cpu core2 -fdebug-compilation-dir /Users/howarth -dwarf-debug-producer Apple LLVM version 7.0.0 (clang-700.0.72) -o a.out -

IMHO, we don't want the GNU-based assembler ever to be installed against Xcode 7 as it would regress the opcode support for AVX provided in the new clang-based assembler from Xcode 7.

[jeremyhu (Jeremy Huddleston Sequoia)]

cctools is installed to provide a fallback. The default (clang-based, integrated) assembler is provided by the llvm-3.7 port itself.

[howarth.at.macports@…]

Replying to jeremyhu@…:

cctools is installed to provide a fallback. The default (clang-based, integrated) assembler is provided by the llvm-3.7 port itself.

llvm-as isn't the same thing as /usr/bin/as under Xcode 7. The Xcode 7 clang based assembler is a drop in replacement for the legacy GNU assembler as it understands the command flags, etc.

[howarth.at.macports@…]

My understanding is that the clang-based /usr/bin/as assembler in Xcode 7 is using the integrated assembler in clang rather than the llvm-as assembler while providing flag and syntax compatibility with the older GNU-assembler (so it is effectively a drop-in replacement).

[jeremyhu (Jeremy Huddleston Sequoia)]

Yes, and maybe if I get some cycles, I'll setup alternatives to support that as well out of the clang port, but there is nothing *wrong* with the current configuration just because Xcode 7 was released. The software that is installed on your system didn't magically change over the past week. It's just that Xcode's versions are newer than our latest versions and added support for an as-wrapper around cfe instead of using GNU as.

[howarth.at.macports@…]

I would argue that it creates unexpected behavior because the 'as' provided by cctools overrides the system 'as' from Xcode 7 for users on the command line outside of 'port'.

[jeremyhu (Jeremy Huddleston Sequoia)]

That is the *expected* behavior. We don't want our toolchains relying on the "system" / Xcode toolchain as that results in behavior that is dependent on outside factors (specifically, dependent on what version of Xcode is installed). We want builds to be as reproducible as possible.

[jeremyhu (Jeremy Huddleston Sequoia)]

I'm all for *replacing* the cctools dependency with something else that will provide an alternative wrapper, like what is shipped in Xcode. But I don't have time for that now.

[howarth.at.macports@…]

It would be nice to find out what Apple intends to release as open source for the 10.11 cctools. The 'as' binary in Xcode 7.0.1 is actually quite small...

 ls -l /Library/Developer/CommandLineTools/usr/bin/as
-rwxr-xr-x  1 root  admin  31696 Sep  4 16:41 /Library/Developer/CommandLineTools/usr/bin/as

which suggests that it might just be routing the assembly to the system clang compiler rather than performing the assembly itself.

[jeremyhu (Jeremy Huddleston Sequoia)]

Multiple things wrong with that statement.

1) cctools in OS X is only used to produce libmacho, not as.

2) as (and most of the other toolchain bits) in /usr/bin are all xcrun stubs. They just redirect to the toolchain in the selected Xcode.

3) The cctools provided by Xcode 7 are of more interest to you.

You just need to be patient.

[jeremyhu (Jeremy Huddleston Sequoia)]

which suggests that it might just be routing the assembly to the system clang compiler rather than performing the assembly itself.

Yes, exactly as I said above. If you feel like writing that yourself rather than waiting for the opensource drop, I'll certainly take your changes. However, I'm content on waiting for the OSS drop.

[howarth.at.macports@…]

Replying to jeremyhu@…:

Multiple things wrong with that statement.

1) cctools in OS X is only used to produce libmacho, not as.

That is not the behavior that I see on my 10.11 installation. The MacPorts cctools package shows a file...

-r-xr-xr-x  0 root   admin   27948 Sep 29 14:37 ./opt/local/bin/as

which is producing the output...

% /opt/local/bin/as -v
Apple Inc version cctools-870, GNU assembler version 1.38

Since Xcode 7 doesn't contain the legacy assembler, it must be coming from the MacPorts cctools package.

2) as (and most of the other toolchain bits) in /usr/bin are all xcrun stubs. They just redirect to the toolchain in the selected Xcode.

3) The cctools provided by Xcode 7 are of more interest to you.

You just need to be patient.

[jeremyhu (Jeremy Huddleston Sequoia)]

Replying to howarth.at.macports@…:

Replying to jeremyhu@…:

Multiple things wrong with that statement.

1) cctools in OS X is only used to produce libmacho, not as.

That is not the behavior that I see on my 10.11 installation. The MacPorts cctools package shows a file...

-r-xr-xr-x  0 root   admin   27948 Sep 29 14:37 ./opt/local/bin/as

which is producing the output...

% /opt/local/bin/as -v
Apple Inc version cctools-870, GNU assembler version 1.38

Since Xcode 7 doesn't contain the legacy assembler, it must be coming from the MacPorts cctools package.

The cctools project has many targets.

In OS X, it is used to build libmacho (specifically /usr/lib/system/libmacho.dylib). OS X does not ship any version of as.

In Xcode, it is used to build a version of libmacho for the tools as well as the GNU assembler and the as wrapper. And before you say "no" ... yes it does contain the legacy assembler. See: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/libexec/as

The as wrapper driver (​http://opensource.apple.com/source/cctools/cctools-870/as/driver.c) has an option of using clang, and that option is now on in Xcode 7. Previously, it was only on for arm64. See:

$ /opt/local/bin/as -arch arm64 --version
clang version 3.8.0 (trunk 246024)
Target: aarch64-apple-darwin15.0.0
Thread model: posix
InstalledDir: /opt/local/libexec/llvm-3.8/bin

I'm not comfortable turning that on in cctools-870 for other architectures (and probably won't even when we update to newer cctools) because I don't have time to qualify that and mess around with the resulting dependency nightmare. We'd have to ensure that the 'port select'ed clang is new enough or modify driver.c to always use a specific version of clang.

[kencu (Ken)]

I believe this has been fixed by recent changes made by jmr to the cctools port, to make it use clang by default as the assembler.

[jeremyhu (Jeremy Huddleston Sequoia)]

Ugg. Hopefully not as that can cause a dependency cycle. I’ll have to check what was done there.

[kencu (Ken)]

I hope you'll agree that it was quite elegant, as you would expect from Josh. It seemed quite solid to Chris and I as well. If you're also happy, we can close this ticket.