Opened 5 years ago

Closed 4 years ago

#58832 closed defect (fixed)

OpenBLAS @0.3.7: build error when build_arch != host arch: ../kernel/x86/amax.S:64:2: error: ../kernel/x86/amax.S:instruction requires: Not 64-bit mode

Reported by: kencu (Ken) Owned by: NicosPavlov
Priority: Normal Milestone:
Component: ports Version:
Keywords: lion snowleopard Cc: michaelld (Michael Dickens), SlaunchaMan (Jeff Kelley), fvaccari, mojca (Mojca Miklavec), cooljeanius (Eric Gallager)
Port: openblas

Description

Something not working right with the openblas update - this is on 10.7.5:

/opt/local/bin/clang-mp-5.0 -c -O3 -DMAX_STACK_ALLOC=2048 -DEXPRECISION -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX -DNO_AVX512 -DSMP_SERVER -DMAX_CPU_NUMBER=2 -DMAX_PARALLEL_NUMBER=1 -DVERSION=\"1\" -DASMNAME=_samin_k -DASMFNAME=_samin_k_ -DNAME=samin_k_ -DCNAME=samin_k -DCHAR_NAME=\"samin_k_\" -DCHAR_CNAME=\"samin_k\" -DNO_AFFINITY -I.. -UDOUBLE  -UCOMPLEX -UCOMPLEX -UDOUBLE -DUSE_ABS  -DUSE_MIN ../kernel/x86/amax.S -o samin_k.o
../kernel/x86/amax.S:64:2: error: ../kernel/x86/amax.S:instruction requires: Not 64-bit mode
 pushl %esi
 ^
../kernel/x86/amax.S:65:2: error: instruction requires: Not 64-bit mode
 pushl %ebx
 ^
../kernel/x86/amax.S:311:2: error: instruction requires: Not 64-bit mode
 popl %ebx
 ^
../kernel/x86/amax.S:312:2: error: instruction requires: Not 64-bit mode
 popl %esi
 ^
64:2: error: instruction requires: Not 64-bit mode
 pushl %esi
 ^
../kernel/x86/amax.S:65:2: error: instruction requires: Not 64-bit mode
 pushl %ebx
 ^
../kernel/x86/amax.S:311:2: error: instruction requires: Not 64-bit mode
 popl %ebx
 ^
../kernel/x86/amax.S:312:2: error: instruction requires: Not 64-bit mode
 popl %esi
 ^
make[1]: *** [samin_k.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [samax_k.o] Error 1
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_OpenBLAS/OpenBLAS/work/OpenBLAS-0.3.7/kernel'
make: *** [libs] Error 1

Attachments (2)

openblas-10.7.5-fail-gcc9.log (1.2 MB) - added by kencu (Ken) 5 years ago.
openblas-fail-1068.log (1.1 MB) - added by kencu (Ken) 5 years ago.

Change History (25)

Changed 5 years ago by kencu (Ken)

comment:1 Changed 5 years ago by kencu (Ken)

I have helpfully downloaded the prebuilt binary from the buildbot. However, it appears I still can't build openblas myself on this machine. Always a bit hard to figure those kinds of errors out...

comment:2 Changed 5 years ago by kencu (Ken)

best I can figure at present, it seems to be trying to build the 32bit version of amax.S when it is building the 64bit library on this machine, instead of building the 64bit version.

comment:3 Changed 5 years ago by kencu (Ken)

Same error on 10.6.8.

Changed 5 years ago by kencu (Ken)

Attachment: openblas-fail-1068.log added

comment:4 Changed 5 years ago by kencu (Ken)

Keywords: snowleopard added

comment:6 Changed 5 years ago by jmroot (Joshua Root)

Could be doing something silly like using uname -m to determine the arch?

comment:7 Changed 5 years ago by kencu (Ken)

for now, I pegged openblas at OpenBLAS @0.3.6_1 until I get some time to sort this out.

comment:8 Changed 5 years ago by kencu (Ken)

maybe we need to look at this area in the Portfile:

        if {${build_arch} eq "x86_64" || ${build_arch} eq "ppc64"} {
            puts $makeINC "BINARY = 64"
        } else {
            puts $makeINC "BINARY = 32"
        }

comment:9 Changed 5 years ago by rmottola (Riccardo)

I encountered exactly the same error on 10.7 (obviously 64bit)

comment:10 in reply to:  6 Changed 4 years ago by kencu (Ken)

Replying to jmroot:

Could be doing something silly like using uname -m to determine the arch?

Indeed, clever sir, that is just exactly what it is doing:

# If ARCH is not set, we use the host system's architecture.
ifndef ARCH
ARCH := $(shell uname -m)
endif

and fixing that by specifically hacking in a setting for ARCH to x86_64:

   if { ${os.major} <= 10 } {
     puts $makeINC "NO_AVX2 = 1"
+    puts $makeINC "ARCH = x86_64"
   }

produces the expected happiness:

$ port -v installed openblas
The following ports are currently installed:
  OpenBLAS @0.3.6_1+gcc8+lapack+native platform='darwin 10' archs='x86_64' date='2019-05-26T18:25:37-0700'
  OpenBLAS @0.3.7_0+gcc8+lapack+native (active) platform='darwin 10' archs='x86_64' date='2019-10-02T19:52:08-0700'

Looks like it came in here <https://github.com/xianyi/OpenBLAS/pull/2110> and it may already be fixed in git master <https://github.com/xianyi/OpenBLAS/issues/2231>, so perhaps I'll just try that instead of messing about with the whole build.

comment:11 Changed 4 years ago by kencu (Ken)

Summary: OpenBLAS @0.3.7: ../kernel/x86/amax.S:64:2: error: ../kernel/x86/amax.S:instruction requires: Not 64-bit modeOpenBLAS @0.3.7: build error when build_arch != host arch: ../kernel/x86/amax.S:64:2: error: ../kernel/x86/amax.S:instruction requires: Not 64-bit mode

comment:12 Changed 4 years ago by kencu (Ken)

So yes, this issue is already fixed in the current OpenBLAS-devel port, in which OpenBLAS has more cleverness about ARCH setting.

comment:13 Changed 4 years ago by SlaunchaMan (Jeff Kelley)

Cc: SlaunchaMan added

comment:14 Changed 4 years ago by fvaccari

Cc: fvaccari added

comment:15 Changed 4 years ago by mojca (Mojca Miklavec)

Cc: mojca added

comment:16 Changed 4 years ago by fvaccari

I just tested on 10.6.8 that after adjusting

puts $makeINC "ARCH = x86_64"

another error pops up later in the compilation:

/opt/local/bin/clang-mp-9.0 -c -O3 -DMAX_STACK_ALLOC=2048 -DEXPRECISION -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX2 -DNO_AVX512 -DSMP_SERVER -DMAX_CPU_NUMBER=2 -DMAX_PARALLEL_NUMBER=1 -DVERSION=\"1\" -DASMNAME=_ssymv_U -DASMFNAME=_ssymv_U_ -DNAME=ssymv_U_ -DCNAME=ssymv_U -DCHAR_NAME=\"ssymv_U_\" -DCHAR_CNAME=\"ssymv_U\" -DNO_AFFINITY -I.. -UDOUBLE  -UCOMPLEX -UCOMPLEX -UDOUBLE -ULOWER ../kernel/x86_64/symv_U_sse.S -o ssymv_U.o
../kernel/x86_64/symv_U_sse.S:425:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbx)
 ^
../kernel/x86_64/symv_U_sse.S:444:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%r13)
 ^
../kernel/x86_64/symv_U_sse.S:469:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbx, %rcx, 1)
 ^
../kernel/x86_64/symv_U_sse.S:510:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbp)
 ^
../kernel/x86_64/symv_U_sse.S:529:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCHW PREFETCHSIZE(%r14)
 ^
../kernel/x86_64/symv_U_sse.S:554:2: error: 32-bit absolute addressing is not supported in 64-bit mode
 PREFETCH PREFETCHSIZE(%rbp, %rcx, 1)
 ^
make[1]: *** [ssymv_U.o] Error 1
make[1]: *** Waiting for unfinished jobs....

which actually points to ticket #58528

Last edited 4 years ago by fvaccari (previous) (diff)

comment:17 Changed 4 years ago by kencu (Ken)

Yes, true. I actually have this all sorted out, but just have been too distracted with all the other things going on.

The YONAH processor specified in the Portfile will not work on any 64bit OS, as YONAH is 32 bit only.

So if I manually change the processor to CORE2, or let the software work out the processor, it builds.

So -- this was a bug in OpenBLAS, and then our forcing YONAH is just not going to work. I have a fancier fix coming, using sysctl and testing for a 64-bit capable processor (and if so, forcing CORE2) but in truth, recreating all the OpenBLAS logic to sort out the processor in Portfile logic is a bit of a pain.

comment:18 Changed 4 years ago by rmottola (Riccardo)

I read the portfile and it says:

#OS 10.7-10.11 supports down to iMac 7,1, with Intel Core 2 Duo architecture
#OS 10.12-13 supports down to iMac 10,1, with Intel Core 2 Duo architecture
if {${os.major} >= 11 && ${os.major} <= 17} {
    set blas_arch "CORE2"
}

So, it appears that the CPU is set correctly for 10.7 !

10.6 is:

#OS 10.6 supports down to Intel Core Solo architecture
if {${os.major} == 10} {
    set blas_arch "YONAH"
}

for 10.5 we have:

#Define target based on version of OS
#OS 10.5 supports down to Intel Core Solo (Intel) or PowerPC G4
if {${os.major} == 9} {
    if {${os.arch} eq "i386" || [variant_isset universal]} {
        set blas_arch "YONAH"
    } else {
        set blas_arch "PPCG4"
    }
}

since it is setting "down to" if you are correct that CORE2 is required, it is wrong for 10.6 and 10.5 when in universal mode!

But... why does it fail even on 10.7 with that error, I don't fully understand.

Last edited 4 years ago by rmottola (Riccardo) (previous) (diff)

comment:19 Changed 4 years ago by kencu (Ken)

There are two different errors. See also #58528.

#58528 is fixed in OpenBLAS-devel, I believe (should be, as I saw that commit roll by a while ago), but this one is our bug, and will never be fixed until we fix it.

Here is my WIP <https://github.com/kencu/macports-ports/commit/462e990eb0208187c5abdcfa29d9507945106a81>, not quite done yet.

comment:20 Changed 4 years ago by rmottola (Riccardo)

I just tried openblas-devel on 10.7 and it appears to work. on 10.5 and 10.6 I did not try yet... but I am stll puzzled by YONAH vs CORE2 and forcing it.

comment:21 Changed 4 years ago by kencu (Ken)

we are forcing YONAH on systems that are sometimes 64bit, and it breaks in a 64bit build. We have to force CORE2 on 64bit systems instead.

The trick is to solve this generally for the buildbots.

I have been waiting for a new openblas release, as the other error was in openblas itself, and is fixed in openblas-devel, as you noticed.

comment:22 Changed 4 years ago by cooljeanius (Eric Gallager)

Cc: cooljeanius added

comment:23 Changed 4 years ago by kencu (Ken)

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.