Opened 3 years ago

Closed 22 months ago

Last modified 20 months ago

#47035 closed defect (fixed)

openssl: Build failure: invalid instruction mnemonic 'adcxq'

Reported by: frozencemetery (Robbie Harwood) Owned by: larryv (Lawrence Velázquez)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: neverpanic (Clemens Lang), lechris@…, mojca (Mojca Miklavec), su-v, persramd@…, someuser12, McDutchie (Martijn Dekker)
Port: openssl

Description (last modified by ryandesign (Ryan Schmidt))

On 10.9.5 server, on a non-root installation with gcc 4.9.2 and clang 3.5.1, openssl fails to upgrade:

robbie@osxserver:~$ port upgrade openssl
--->  Computing dependencies for openssl
--->  Fetching distfiles for openssl
--->  Verifying checksums for openssl
--->  Extracting openssl
--->  Applying patches to openssl
--->  Configuring openssl
--->  Building openssl
Error: org.macports.build for port openssl returned: command execution failed
Please see the log file for port openssl for details:
    /Network/Servers/osxserver.b-aassoc.edu/Volumes/ServerDrive/NetUsers/robbie/var/macports/logs/_Volumes_ServerDrive_NetUsers_robbie_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_openssl/openssl/main.log
Error: Unable to upgrade port: 1
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
robbie@osxserver:~$ 

I've attached main.log. Thanks!

Attachments (4)

main.log (139.5 KB) - added by frozencemetery (Robbie Harwood) 3 years ago.
main.log - openssl upgrade failure
patch-revert-clang-detection-on-Lion.diff (4.9 KB) - added by su-v 22 months ago.
upstream commit reversed, patch level adjusted
fix-Apple-clang-version-detection.patch (9.2 KB) - added by neverpanic (Clemens Lang) 22 months ago.
fix-Apple-clang-version-detection2.patch (3.3 KB) - added by osth@… 20 months ago.
additional fix needed for mountain lion

Download all attachments as: .zip

Change History (42)

Changed 3 years ago by frozencemetery (Robbie Harwood)

main.log - openssl upgrade failure

comment:1 Changed 3 years ago by frozencemetery (Robbie Harwood)

And I messed up the formatting... we're off to a great start here. Sorry, all.

comment:2 Changed 3 years ago by ryandesign (Ryan Schmidt)

  • Cc larryv@… removed
  • Description modified (diff)
  • Owner changed from macports-tickets@… to larryv@…

What is the output of "/usr/bin/clang -v" and "xcodebuild -version"?

comment:3 Changed 3 years ago by neverpanic (Clemens Lang)

This was also reported in #46874, and the problem apparently were outdated Command Line Tools. Try updating them.

comment:4 Changed 3 years ago by neverpanic (Clemens Lang)

Did updating the Command Line Tools help?

comment:5 follow-up: Changed 3 years ago by frozencemetery (Robbie Harwood)

Hi cal, thanks for the prompt response. Unfortunately, I see no change after updating command line tools.

comment:6 in reply to: ↑ 5 ; follow-up: Changed 3 years ago by lechris@…

Replying to rharwood@…:

Hi cal, thanks for the prompt response. Unfortunately, I see no change after updating command line tools.

I stumbled into the same issue today with Xcode 5 and its command line tools. Upgrading to Xcode 6.2 (and its command line tools) solved the issue in my case.

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9.5
BuildVersion:	13F1066

$ clang --version
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

$ xcodebuild -version
Xcode 6.2
Build version 6C131e

$ sudo port upgrade openssl
--->  Computing dependencies for openssl
--->  Fetching distfiles for openssl
--->  Verifying checksums for openssl
--->  Extracting openssl
--->  Applying patches to openssl
--->  Configuring openssl
--->  Building openssl
--->  Staging openssl into destroot
--->  Installing openssl @1.0.2a_0
--->  Cleaning openssl
--->  Computing dependencies for openssl
--->  Deactivating openssl @1.0.1j_0
--->  Cleaning openssl
--->  Activating openssl @1.0.2a_0
--->  Cleaning openssl
--->  Updating database of binaries
--->  Scanning binaries for linking errors               
--->  No broken files found.  

comment:7 Changed 3 years ago by lechris@…

  • Cc lechris@… added

Cc Me!

comment:8 Changed 2 years ago by GregoryEAllen (Greg Allen)

Updating the command line tools fixed it for me.

$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.8.5
BuildVersion:	12F2501

$ xcodebuild -version
Xcode 5.1.1
Build version 5B1008

$ clang -v
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin12.6.0
Thread model: posix

comment:9 in reply to: ↑ 6 Changed 2 years ago by frozencemetery (Robbie Harwood)

Replying to lechris@…:

I stumbled into the same issue today with Xcode 5 and its command line tools. Upgrading to Xcode 6.2 (and its command line tools) solved the issue in my case.

For me as well. Thanks!

comment:10 Changed 2 years ago by larryv (Lawrence Velázquez)

  • Resolution set to invalid
  • Status changed from new to closed

Well, let’s tentatively close this as “invalid” (since it appears to have been a toolchain issue) and hope no one else runs into it.

comment:11 Changed 22 months ago by mojca (Mojca Miklavec)

  • Cc mojca@… added

Cc Me!

comment:12 Changed 22 months ago by mojca (Mojca Miklavec)

  • Resolution invalid deleted
  • Status changed from closed to reopened
  • Version 2.3.3 deleted

I'm reopening the issue. I never had problems so far, I have

  openssl @1.0.2d_0 (active)

installed, but it now fails to work for me:

:info:build x86_64-mont.s:807:2: error: invalid instruction mnemonic 'adcxq'
:info:build  adcxq %rax,%r13
:info:build  ^~~~~
:info:build x86_64-mont.s:808:2: error: invalid instruction mnemonic 'adcxq'
:info:build  adcxq %rbp,%r14
:info:build  ^~~~~

I have

> sw_vers
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63b

> xcodebuild -version
Xcode 4.6.3
Build version 4H1503

> clang -v
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

However I'm unable to update my toolchain (without changing the OS version as well, that is).

comment:13 Changed 22 months ago by su-v

Build failure of openssl 1.0.2e_0 on OS X 10.7.5 confirmed (openssl 1.0.2d_0 built and installed ok).

comment:14 Changed 22 months ago by su-v

  • Cc suv-sf@… added

Cc Me!

comment:15 follow-ups: Changed 22 months ago by ryandesign (Ryan Schmidt)

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

comment:16 in reply to: ↑ 15 Changed 22 months ago by su-v

Replying to ryandesign@…:

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

I use the same versions as mojca in comment:12 (latest Xcode available for Lion):

$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63b
$ 
$ xcodebuild -version
Xcode 4.6.3
Build version 4H1503
$ 
$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
$ 

comment:17 in reply to: ↑ 15 Changed 22 months ago by McDutchie (Martijn Dekker)

Replying to ryandesign@…:

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63
$ xcodebuild -version
Xcode 4.6.3
Build version 4H1503
$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
$ 
Last edited 22 months ago by McDutchie (Martijn Dekker) (previous) (diff)

comment:18 Changed 22 months ago by persramd@…

  • Cc persramd@… added

Cc Me!

comment:19 Changed 22 months ago by beremour@…

Problem with:

$sw_vers
ProductName:    Mac OS X
ProductVersion: 10.7.5
BuildVersion:   11G63

$xcodebuild -version
Xcode 4.6.3
Build version 4H1503

$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
Last edited 22 months ago by beremour@… (previous) (diff)

comment:20 Changed 22 months ago by andrew.long@…

Here's a 'me too!'

ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
Xcode 4.6.3
Build version 4H1503

comment:21 Changed 22 months ago by someuser12

  • Cc macosx12345@… added

Cc Me!

comment:22 Changed 22 months ago by neverpanic (Clemens Lang)

I think the problem here is Apple's particularly stupid clang --version output. The relevant code is in crypto/bn/asm/x86_64-mont.pl, line 71:

`$ENV{CC} -v 2>&1` =~ /((?:^clang|LLVM) version|based on LLVM) ([3-9])\.([0-9]+)/)

Now, which version number do you think the regex will match… "Apple LLVM version 4.2" or "based on LLVM 3.2svn"?

Can anybody provide me with a MacPorts (or otherwise open source) clang --version output so I can come up with a regex that will avoid this problem?

Last edited 22 months ago by neverpanic (Clemens Lang) (previous) (diff)

comment:23 Changed 22 months ago by mojca (Mojca Miklavec)

> clang-mp-3.4 --version
clang version 3.4.2 (tags/RELEASE_34/dot2-final)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

comment:24 Changed 22 months ago by mojca (Mojca Miklavec)

Upstream report:

The commit that broke the build:

Relevant report that lead to that change:

The reason is Apple since Xcode 7.0 removed string "based on LLVM x.y.z" from version information. Now the clang -v command returns:

Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin15.0.0
Thread model: posix
Last edited 22 months ago by mojca (Mojca Miklavec) (previous) (diff)

comment:25 Changed 22 months ago by mojca (Mojca Miklavec)

A temporary workaround could be to use configure.compiler macports-clang-3.[anything]. But we should try to eventually fix the problem without blacklisting the system compiler (unless we don't want to bother fixing the regexp and want to wait for upstream to fix it; and blacklist the compiler just in the meantime).

comment:26 Changed 22 months ago by su-v

Why not - as temporary workaround in MacPorts - revert that one upstream commit which broke detecting Lion's clang correctly if port is installed on Lion (*), instead of requiring the user to compile a custom clang from source?

(*)

    if {${os.major} == 11} {
        patchfiles-append   patch-revert-clang-detection-on-Lion.diff
    }

Changed 22 months ago by su-v

upstream commit reversed, patch level adjusted

Changed 22 months ago by neverpanic (Clemens Lang)

comment:27 Changed 22 months ago by neverpanic (Clemens Lang)

Please try the attached fix-Apple-clang-version-detection.patch and report back. My build on 10.11 is unchanged with this patch, but the real question is: does it fix your issue?

comment:28 Changed 22 months ago by mojca (Mojca Miklavec)

It compiles fine. (Whether it also works properly I'm unable to tell.)

comment:29 Changed 22 months ago by mojca (Mojca Miklavec)

PS: This is all Perl code. Wouldn't it be better to create a single Perl function that would return the version of clang and then the same code wouldn't have to be repeated/patched at zillion places?

comment:30 Changed 22 months ago by neverpanic (Clemens Lang)

Yes, this is all Perl code that compares compiler version numbers to find out whether AVX or the addx instruction are available. Why the OpenSSL project has literally (at least) 13 copies of the same code is beyond me. I also don't understand why they compare version numbers rather than just, say, run an AVX or addx instruction through the toolchain to see if it works.

However, I'm not in the business of refactoring OpenSSL code in what seems to be rather large endeavor.

You can use the OpenSSL test suite to check whether the code miscompiles with the change. Additionally, compare the output of openssl speed with that of 1.0.2d on 10.7, that should give you a rough idea whether it is using the same assembler optimizations.

Last edited 22 months ago by neverpanic (Clemens Lang) (previous) (diff)

comment:31 Changed 22 months ago by mojca (Mojca Miklavec)

I tried port -v test openssl which completes. There is too much output to check whether everything went fine: I assume that it would complain at the end or even break in case of problems.

comment:32 Changed 22 months ago by McDutchie (Martijn Dekker)

  • Cc martijn@… added

Cc Me!

comment:33 Changed 22 months ago by McDutchie (Martijn Dekker)

I can confirm that applying fix-Apple-clang-version-detection.patch​ to the build directory makes openssl @1.0.2e compile and work fine on Lion. The speed also doesn't seem to have suffered any, judging by the output of "openssl speed".

comment:34 follow-up: Changed 22 months ago by neverpanic (Clemens Lang)

  • Resolution set to fixed
  • Status changed from reopened to closed

Committed the patch in r143276.

comment:35 in reply to: ↑ 34 Changed 20 months ago by osth@…

Replying to cal@…:

Committed the patch in r143276.

RESOLVED: had other clang version in /usr/bin than in XCode 5.1.1 app folder.

Solution: copy contents of /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/* to /usr/ clang -v gives then clang-503.0.40

Same problem on mountain lion (edit: only with outdated clang-500.2.79): sh-3.2# sw_vers ProductName: Mac OS X ProductVersion: 10.8.5 BuildVersion: 12F2560 sh-3.2# clang -v Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn) Target: x86_64-apple-darwin12.6.0 Thread model: posix Will attach my patch which fixes it for me

Last edited 20 months ago by osth@… (previous) (diff)

Changed 20 months ago by osth@…

additional fix needed for mountain lion

comment:36 follow-up: Changed 20 months ago by neverpanic (Clemens Lang)

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

comment:37 in reply to: ↑ 36 Changed 20 months ago by osth@…

Replying to cal@…:

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

Had installed my Apple clang as command line tools package (clang in /usr/bin/). Did not realize, that XCode 5.1.1 app folder contained a newer version of clang. - I did edit my first comment..

comment:38 Changed 20 months ago by neverpanic (Clemens Lang)

Ah, so the check didn't run the compiler that ended up being used? We should probably try to avoid this…

Note: See TracTickets for help on using tickets.