cctools as does not support AVX, consider using llvm

[wd11@…]

The gcc groups of compiler ports (gcc47 etc) depend on cctools' for the linker 'as'. However, that linker is deficient: it doesn't know about the AVX instruction set supported by Intel's sandy-bridge and ivy-bridge processors, which are used in the recent mac books. As a consequence, gcc cannot compile valid code, such as the attached c++ program (note: compile with the -march=native option on a intel corei7 chip)

Note that this is a known issue (​http://stackoverflow.com/questions/9840207/how-to-use-avx-pclmulqdq-on-mac-os-x-lion), but the suggested solution is extremely inelegant and does not mesh well with macports updates.

Instead, this problem should solved at its heart, i.e. by shipping the gcc ports with a better linker (or by convincing cctools to ship a useful linker).

[wd11@…]

valid source code that cannot be compiled.

[jeremyhu (Jeremy Huddleston Sequoia)]

as is not a linker. It's an assembler. ld is the linker and is provided by ld64.

[ltalirz (Leopold Talirz)]

Cc Me!

[jeremyhu (Jeremy Huddleston Sequoia)]

We should consider using 'clang -c -x assembler' instead of as...

[cooljeanius (Eric Gallager)]

Cc Me!

[jeremyhu (Jeremy Huddleston Sequoia)]

IMO, this is a gcc problem to work out. If the assembler doesn't support an instruction set, it shouldn't be used. gcc should either not use these instruction sets or should use llvm's assembler.

[cooljeanius (Eric Gallager)]

Replying to jeremyhu@…:

IMO, this is a gcc problem to work out. If the assembler doesn't support an instruction set, it shouldn't be used. gcc should either not use these instruction sets or should use llvm's assembler.

You're referring to the gcc port(s) here, right? I see no reason why gcc upstream would want to use the assembler of a competing compiler; they would probably just recommend that we update the binutils port and us the as that it provides instead...

[mouse07410 (Mouse)]

Any update on this issue? It's been over 5 years, and the problem remains.

[raimue (Rainer Müller)]

These AVX instructions are not emitted by gcc; the example code uses intrinsics. gcc does not validate that, it just passes it through to the assembler like it always does for inline assembler code.

I do not know or check whether gcc on macOS would emit AVX code from C code (keyword: auto-vectorization).

If we want to support this, we need to modify the gcc ports in MacPorts to use the LLVM assembler. The simplest option seems to be to use --with-as=/usr/bin/as (maybe the same for the other target tools that are now provided by cctools?).

But this can only be done on macOS and Xcode version combinations that already use the LLVM assembler by default and not the old cctools. You need to find out which these are. Older versions should continue to use cctools, as that is still newer than what is provided in /usr/bin.

Then test this setup and report back.

[kencu (Ken)]

gcc on macOS uses the ancient apple gcc assembler from about 1852. It is version 1.38, and hasn't been updated in years. The build of gcc-as in binutils is disabled as it is not apple-happy. I would love it if somebody would update that to something fresh.

However, gcc accepts -Wa,-q which causes it to use clang as the assembler. So you can add that to the cflags and cxxflags on a gcc build, and if clang calls to some newish version of clang, you're good to go.

There is a flag to spec the clang you want to sent the asm to, I believe. I just haven't found it / tested it / bothered to look for it.

When I want to build new asm with old gcc version (e.g. building TenFourFox for Intel on 10.6.8 using gcc-4.8 with modern libvpx instructions), I just add the mentioned flag and port select a newish clang.

[raimue (Rainer Müller)]

MacPorts forces gcc to use ${prefix}/bin/as instead, which comes from cctools.

/usr/bin/as is already be the LLVM assembler on my system. There is no need to use as -q as far as I can see.

$ sw_vers -productVersion
10.12.6
$ xcodebuild -version
Xcode 9.2
Build version 9C40b
$ /usr/bin/as -v |& head -1
Apple LLVM version 9.0.0 (clang-900.0.39.2)

[kencu (Ken)]

mmmmm. look at the same result on my system, however (10.6.8):

$ which as
/opt/local/bin/as
$ as -version
Apple Inc version cctools-895, GNU assembler version 1.38

looks like the situation depends on which OS you're on -- no surprise :>

[raimue (Rainer Müller)]

Of course /opt/local/bin/as will always be cctools; it is provided by the MacPorts port. I was talking about /usr/bin/as, which is provided by the Xcode toolchain.

[kencu (Ken)]

Oh, OK, you had noted ${prefix}/bin/as ​above.

Turns out /usr/bin/as is even worse on older systems, as it is just an even older version of cctools, although may well provide the exact same assembler:

$ /usr/bin/as -version
Apple Inc version cctools-809~2, GNU assembler version 1.38

Good to know there are different ways of doing things.

You have me thinking now. I'd like to somehow prove that -Wa,-q actually calls through to clang instead of /usr/bin/as or ${prefix}/bin/as just so I know for sure what I'm saying is actually fully correct. I'll see if I can figure out how to do that. -- K

[raimue (Rainer Müller)]

On older systems, /usr/bin/as is an old cctools. That's why we started using ${prefix}/bin/as from the cctools port instead, to get the benefits of the newer version.

That's what I meant above: we should only use --with-as=/usr/bin/as on newer systems on which /usr/bin/as is the newer LLVM assembler. On older systems leave everything as it is.

Your theory on -Wa,-q is easy to prove:

$ /opt/local/bin/as -q
/opt/local/bin/as: assembler (/opt/local/bin/clang) not installed

[kencu (Ken)]

Good! So on some vintage of OS, -Wa,-q is the proper way to go if you want gcc to use a current assembler. That is what I thought, as that is what worked for me.

Now we just need to figure out where the cutoff might be. Likely 10.6.8, I would bet.

[kencu (Ken)]

See also:

36389 51642 54166

and

​https://stackoverflow.com/questions/37855898/macports-gcc-compiler-wa-q-and-opt-local-bin-clang-not-installed

and

​https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80782

[kencu (Ken)]

The guts of this behaviour is controlled by cctools as driver.c.

For practical purposes, /usr/bin/as sends you to /usr/bin/clang for assember work on 10.10 or later, which on my system matched up to Xcode 7. On 10.9 and below, it appears you get the standard GNU assember 1.38 unless you send the -q flag, or have set the AS_INTEGRATED_ASSEMBER environment variable.

On older systems, if you have set AS_INTEGRATED_ASSEMBER, or if you have set the -q flag with -Wa,-q, then cctools as passes off the assembly to clang in the same directory as as is.

At this moment, you can't set a more specific name for clang, and you can't specify a full path to clang. It's just going to look for clang, next to as, wherever the running copy of as is.

So for a MacPorts install of cctools, /opt/local/bin/as which is the first one on the path usually will look for /opt/local/bin/clang. /usr/bin/as will look for /usr/bin/clang.

To make the MacPorts cctools as work to use clang as the assembler, there needs to be an /opt/local/bin/clang and the only way to get that is to sudo port select a clang manually, at present.

So on systems 10.9 or older, you need to select a clang, and then use -Wa,-q to make it work.

[raimue (Rainer Müller)]

Has duplicate #56352.

[dubiousjim]

See also ​https://stackoverflow.com/questions/49981445/gcc-assembler-not-recognizing-tzcntl-or-lzcntl-though-my-processor-does.

[jmroot (Joshua Root)]

In 7aa2ff679e61f923475b6c2a56958da61205430b/macports-ports (master):

cctools: try to use clang in as

The clang integrated assembler is way more up-to-date than the ancient
pre-GPLv3 version of gas that is the alternative. The latter doesn't
know about newer instructions such as AVX. The as(1) shipped with
recent Xcode versions will just run clang behind the scenes under most
circumstances for this reason.

Our cctools can't actually depend on clang because that would create a
circular dependency. So we'll try to run the clang corresponding to the
LLVM being used by cctools, and if that isn't present, try the system
clang in /usr/bin if we're on an OS version that has a clang with a
working integrated assembler. (The ancient gas is still used as a final
fallback.)

Fixes: #37846