Opened 9 years ago

Closed 8 years ago

Last modified 18 months ago

#47035 closed defect (fixed)

openssl: Build failure: invalid instruction mnemonic 'adcxq'

Reported by: frozencemetery (Robbie Harwood) Owned by: larryv (Lawrence Velázquez)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: neverpanic (Clemens Lang), lechris@…, mojca (Mojca Miklavec), su-v, persramd@…, someuser12, McDutchie (Martijn Dekker), mascguy (Christopher Nielsen)
Port: openssl

Description (last modified by ryandesign (Ryan Carsten Schmidt))

On 10.9.5 server, on a non-root installation with gcc 4.9.2 and clang 3.5.1, openssl fails to upgrade:

robbie@osxserver:~$ port upgrade openssl
--->  Computing dependencies for openssl
--->  Fetching distfiles for openssl
--->  Verifying checksums for openssl
--->  Extracting openssl
--->  Applying patches to openssl
--->  Configuring openssl
--->  Building openssl
Error: org.macports.build for port openssl returned: command execution failed
Please see the log file for port openssl for details:
    /Network/Servers/osxserver.b-aassoc.edu/Volumes/ServerDrive/NetUsers/robbie/var/macports/logs/_Volumes_ServerDrive_NetUsers_robbie_var_macports_sources_rsync.macports.org_release_tarballs_ports_devel_openssl/openssl/main.log
Error: Unable to upgrade port: 1
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
robbie@osxserver:~$ 

I've attached main.log. Thanks!

Attachments (4)

main.log (139.5 KB) - added by frozencemetery (Robbie Harwood) 9 years ago.
main.log - openssl upgrade failure
patch-revert-clang-detection-on-Lion.diff (4.9 KB) - added by su-v 8 years ago.
upstream commit reversed, patch level adjusted
fix-Apple-clang-version-detection.patch (9.2 KB) - added by neverpanic (Clemens Lang) 8 years ago.
fix-Apple-clang-version-detection2.patch (3.3 KB) - added by osth@… 8 years ago.
additional fix needed for mountain lion

Download all attachments as: .zip

Change History (44)

Changed 9 years ago by frozencemetery (Robbie Harwood)

Attachment: main.log added

main.log - openssl upgrade failure

comment:1 Changed 9 years ago by frozencemetery (Robbie Harwood)

And I messed up the formatting... we're off to a great start here. Sorry, all.

comment:2 Changed 9 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: larryv@… removed
Description: modified (diff)
Owner: changed from macports-tickets@… to larryv@…

What is the output of "/usr/bin/clang -v" and "xcodebuild -version"?

comment:3 Changed 9 years ago by neverpanic (Clemens Lang)

This was also reported in #46874, and the problem apparently were outdated Command Line Tools. Try updating them.

comment:4 Changed 9 years ago by neverpanic (Clemens Lang)

Did updating the Command Line Tools help?

comment:5 Changed 9 years ago by frozencemetery (Robbie Harwood)

Hi cal, thanks for the prompt response. Unfortunately, I see no change after updating command line tools.

comment:6 in reply to:  5 ; Changed 9 years ago by lechris@…

Replying to rharwood@…:

Hi cal, thanks for the prompt response. Unfortunately, I see no change after updating command line tools.

I stumbled into the same issue today with Xcode 5 and its command line tools. Upgrading to Xcode 6.2 (and its command line tools) solved the issue in my case.

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.9.5
BuildVersion:	13F1066

$ clang --version
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix

$ xcodebuild -version
Xcode 6.2
Build version 6C131e

$ sudo port upgrade openssl
--->  Computing dependencies for openssl
--->  Fetching distfiles for openssl
--->  Verifying checksums for openssl
--->  Extracting openssl
--->  Applying patches to openssl
--->  Configuring openssl
--->  Building openssl
--->  Staging openssl into destroot
--->  Installing openssl @1.0.2a_0
--->  Cleaning openssl
--->  Computing dependencies for openssl
--->  Deactivating openssl @1.0.1j_0
--->  Cleaning openssl
--->  Activating openssl @1.0.2a_0
--->  Cleaning openssl
--->  Updating database of binaries
--->  Scanning binaries for linking errors               
--->  No broken files found.  

comment:7 Changed 9 years ago by lechris@…

Cc: lechris@… added

Cc Me!

comment:8 Changed 9 years ago by GregoryEAllen (Greg Allen)

Updating the command line tools fixed it for me.

$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.8.5
BuildVersion:	12F2501

$ xcodebuild -version
Xcode 5.1.1
Build version 5B1008

$ clang -v
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin12.6.0
Thread model: posix

comment:9 in reply to:  6 Changed 9 years ago by frozencemetery (Robbie Harwood)

Replying to lechris@…:

I stumbled into the same issue today with Xcode 5 and its command line tools. Upgrading to Xcode 6.2 (and its command line tools) solved the issue in my case.

For me as well. Thanks!

comment:10 Changed 9 years ago by larryv (Lawrence Velázquez)

Resolution: invalid
Status: newclosed

Well, let’s tentatively close this as “invalid” (since it appears to have been a toolchain issue) and hope no one else runs into it.

comment:11 Changed 8 years ago by mojca (Mojca Miklavec)

Cc: mojca@… added

Cc Me!

comment:12 Changed 8 years ago by mojca (Mojca Miklavec)

Resolution: invalid
Status: closedreopened
Version: 2.3.3

I'm reopening the issue. I never had problems so far, I have

  openssl @1.0.2d_0 (active)

installed, but it now fails to work for me:

:info:build x86_64-mont.s:807:2: error: invalid instruction mnemonic 'adcxq'
:info:build  adcxq %rax,%r13
:info:build  ^~~~~
:info:build x86_64-mont.s:808:2: error: invalid instruction mnemonic 'adcxq'
:info:build  adcxq %rbp,%r14
:info:build  ^~~~~

I have

> sw_vers
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63b

> xcodebuild -version
Xcode 4.6.3
Build version 4H1503

> clang -v
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

However I'm unable to update my toolchain (without changing the OS version as well, that is).

comment:13 Changed 8 years ago by su-v

Build failure of openssl 1.0.2e_0 on OS X 10.7.5 confirmed (openssl 1.0.2d_0 built and installed ok).

comment:14 Changed 8 years ago by su-v

Cc: suv-sf@… added

Cc Me!

comment:15 Changed 8 years ago by ryandesign (Ryan Carsten Schmidt)

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

comment:16 in reply to:  15 Changed 8 years ago by su-v

Replying to ryandesign@…:

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

I use the same versions as mojca in comment:12 (latest Xcode available for Lion):

$ sw_vers 
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63b
$ 
$ xcodebuild -version
Xcode 4.6.3
Build version 4H1503
$ 
$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
$ 

comment:17 in reply to:  15 Changed 8 years ago by McDutchie (Martijn Dekker)

Replying to ryandesign@…:

Will those of you experiencing the problem with 1.0.2e please let us know what version of clang and what version of Xcode you have installed?

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63
$ xcodebuild -version
Xcode 4.6.3
Build version 4H1503
$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
$ 
Last edited 8 years ago by McDutchie (Martijn Dekker) (previous) (diff)

comment:18 Changed 8 years ago by persramd@…

Cc: persramd@… added

Cc Me!

comment:19 Changed 8 years ago by beremour

Problem with:

$sw_vers
ProductName:    Mac OS X
ProductVersion: 10.7.5
BuildVersion:   11G63

$xcodebuild -version
Xcode 4.6.3
Build version 4H1503

$ clang --version
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
Last edited 8 years ago by beremour (previous) (diff)

comment:20 Changed 8 years ago by andrew.long@…

Here's a 'me too!'

ProductName:	Mac OS X
ProductVersion:	10.7.5
BuildVersion:	11G63
Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
Target: x86_64-apple-darwin11.4.2
Thread model: posix
Xcode 4.6.3
Build version 4H1503

comment:21 Changed 8 years ago by someuser12

Cc: macosx12345@… added

Cc Me!

comment:22 Changed 8 years ago by neverpanic (Clemens Lang)

I think the problem here is Apple's particularly stupid clang --version output. The relevant code is in {{crypto/bn/asm/x86_64-mont.pl}}, line 71:

`$ENV{CC} -v 2>&1` =~ /((?:^clang|LLVM) version|based on LLVM) ([3-9])\.([0-9]+)/)

Now, which version number do you think the regex will match… "Apple LLVM version 4.2" or "based on LLVM 3.2svn"?

Can anybody provide me with a MacPorts (or otherwise open source) clang --version output so I can come up with a regex that will avoid this problem?

Version 0, edited 8 years ago by neverpanic (Clemens Lang) (next)

comment:23 Changed 8 years ago by mojca (Mojca Miklavec)

> clang-mp-3.4 --version
clang version 3.4.2 (tags/RELEASE_34/dot2-final)
Target: x86_64-apple-darwin11.4.2
Thread model: posix

comment:24 Changed 8 years ago by mojca (Mojca Miklavec)

Upstream report:

The commit that broke the build:

Relevant report that lead to that change:

The reason is Apple since Xcode 7.0 removed string "based on LLVM x.y.z" from version information. Now the clang -v command returns:

Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin15.0.0
Thread model: posix
Last edited 8 years ago by mojca (Mojca Miklavec) (previous) (diff)

comment:25 Changed 8 years ago by mojca (Mojca Miklavec)

A temporary workaround could be to use configure.compiler macports-clang-3.[anything]. But we should try to eventually fix the problem without blacklisting the system compiler (unless we don't want to bother fixing the regexp and want to wait for upstream to fix it; and blacklist the compiler just in the meantime).

comment:26 Changed 8 years ago by su-v

Why not - as temporary workaround in MacPorts - revert that one upstream commit which broke detecting Lion's clang correctly if port is installed on Lion (*), instead of requiring the user to compile a custom clang from source?

(*)

    if {${os.major} == 11} {
        patchfiles-append   patch-revert-clang-detection-on-Lion.diff
    }

Changed 8 years ago by su-v

upstream commit reversed, patch level adjusted

Changed 8 years ago by neverpanic (Clemens Lang)

comment:27 Changed 8 years ago by neverpanic (Clemens Lang)

Please try the attached fix-Apple-clang-version-detection.patch and report back. My build on 10.11 is unchanged with this patch, but the real question is: does it fix your issue?

comment:28 Changed 8 years ago by mojca (Mojca Miklavec)

It compiles fine. (Whether it also works properly I'm unable to tell.)

comment:29 Changed 8 years ago by mojca (Mojca Miklavec)

PS: This is all Perl code. Wouldn't it be better to create a single Perl function that would return the version of clang and then the same code wouldn't have to be repeated/patched at zillion places?

comment:30 Changed 8 years ago by neverpanic (Clemens Lang)

Yes, this is all Perl code that compares compiler version numbers to find out whether AVX or the addx instruction are available. Why the OpenSSL project has literally (at least) 13 copies of the same code is beyond me. I also don't understand why they compare version numbers rather than just, say, run an AVX or addx instruction through the toolchain to see if it works.

However, I'm not in the business of refactoring OpenSSL code in what seems to be rather large endeavor.

You can use the OpenSSL test suite to check whether the code miscompiles with the change. Additionally, compare the output of openssl speed with that of 1.0.2d on 10.7, that should give you a rough idea whether it is using the same assembler optimizations.

Last edited 8 years ago by neverpanic (Clemens Lang) (previous) (diff)

comment:31 Changed 8 years ago by mojca (Mojca Miklavec)

I tried port -v test openssl which completes. There is too much output to check whether everything went fine: I assume that it would complain at the end or even break in case of problems.

comment:32 Changed 8 years ago by McDutchie (Martijn Dekker)

Cc: martijn@… added

Cc Me!

comment:33 Changed 8 years ago by McDutchie (Martijn Dekker)

I can confirm that applying fix-Apple-clang-version-detection.patch​ to the build directory makes openssl @1.0.2e compile and work fine on Lion. The speed also doesn't seem to have suffered any, judging by the output of "openssl speed".

comment:34 Changed 8 years ago by neverpanic (Clemens Lang)

Resolution: fixed
Status: reopenedclosed

Committed the patch in r143276.

comment:35 in reply to:  34 Changed 8 years ago by osth@…

Replying to cal@…:

Committed the patch in r143276.

RESOLVED: had other clang version in /usr/bin than in XCode 5.1.1 app folder.

Solution: copy contents of /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/* to /usr/
clang -v gives then clang-503.0.40

Same problem on mountain lion (edit: only with outdated clang-500.2.79):

sh-3.2# sw_vers
ProductName:	Mac OS X
ProductVersion:	10.8.5
BuildVersion:	12F2560
sh-3.2# clang -v
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin12.6.0
Thread model: posix

Will attach my patch which fixes it for me

Last edited 2 years ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

Changed 8 years ago by osth@…

additional fix needed for mountain lion

comment:36 Changed 8 years ago by neverpanic (Clemens Lang)

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

comment:37 in reply to:  36 Changed 8 years ago by osth@…

Replying to cal@…:

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

Had installed my Apple clang as command line tools package (clang in /usr/bin/). Did not realize, that XCode 5.1.1 app folder contained a newer version of clang. - I did edit my first comment..

comment:38 Changed 8 years ago by neverpanic (Clemens Lang)

Ah, so the check didn't run the compiler that ended up being used? We should probably try to avoid this…

comment:39 Changed 2 years ago by mascguy (Christopher Nielsen)

Cc: mascguy added

comment:40 in reply to:  36 Changed 18 months ago by barracuda156

Replying to neverpanic:

Can anybody confirm this patch is required on Mountain Lion with clang-500.2.79?

Yes, openssl3 is broken on 10.8.5 now.

Note: See TracTickets for help on using tickets.