py27-scipy build failure on PowerBookG4 running Leopard 10.5.8; fortran compiler called with bizarre arguments

[zzanderr]

While upgrading py27-scipy on a PowerBook G4 (1.67 Ghz) running Leopard 10.5.8, the build calls the fortran compiler from (macports) gcc47 with completely wrong arguments:

building 'dfftpack' library
compiling Fortran sources
Fortran f77 compiler: /mp2/local/bin/gfortran-mp-4.7 -Os -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2
Fortran f90 compiler: /mp2/local/bin/gfortran-mp-4.7 -Os -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2
Fortran fix compiler: /mp2/local/bin/gfortran-mp-4.7 -Wall -g -ffixed-form -fno-second-underscore -Os -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2
creating build/temp.macosx-10.5-ppc-2.7
creating build/temp.macosx-10.5-ppc-2.7/scipy
creating build/temp.macosx-10.5-ppc-2.7/scipy/fftpack
creating build/temp.macosx-10.5-ppc-2.7/scipy/fftpack/src
creating build/temp.macosx-10.5-ppc-2.7/scipy/fftpack/src/dfftpack
compile options: '-I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -c'
gfortran-mp-4.7:f77: scipy/fftpack/src/dfftpack/dcosqb.f
gfortran-mp-4.7: error: unrecognized command line option '-march=x86-64'
gfortran-mp-4.7: error: unrecognized argument in option '-mtune=generic'
gfortran-mp-4.7: note: valid arguments to '-mtune=' are: 401 403 405 405fp 440 440fp 464 464fp 476 476fp 505 601 602 603 603e 604 604e 620 630 740 7400 7450 750 801 821 823 8540 8548 860 970 G3 G4 G5 a2 cell common e300c2 e300c3 e500mc e500mc64 ec603e native power power2 power3 power4 power5 power5+ power6 power6x power7 powerpc powerpc64 rios rios1 rios2 rs64 rsc rsc1 titan
gfortran-mp-4.7: error: unrecognized command line option '-msse2'
gfortran-mp-4.7: error: unrecognized command line option '-march=x86-64'
gfortran-mp-4.7: error: unrecognized argument in option '-mtune=generic'
gfortran-mp-4.7: note: valid arguments to '-mtune=' are: 401 403 405 405fp 440 440fp 464 464fp 476 476fp 505 601 602 603 603e 604 604e 620 630 740 7400 7450 750 801 821 823 8540 8548 860 970 G3 G4 G5 a2 cell common e300c2 e300c3 e500mc e500mc64 ec603e native power power2 power3 power4 power5 power5+ power6 power6x power7 powerpc powerpc64 rios rios1 rios2 rs64 rsc rsc1 titan
gfortran-mp-4.7: error: unrecognized command line option '-msse2'
error: Command "/mp2/local/bin/gfortran-mp-4.7 -Os -fno-second-underscore -fPIC -O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2 -I/mp2/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -c -c scipy/fftpack/src/dfftpack/dcosqb.f -o build/temp.macosx-10.5-ppc-2.7/scipy/fftpack/src/dfftpack/dcosqb.o" failed with exit status 1
Command failed:  cd "/mp2/local/var/macports/build/_mp2_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_python_py-scipy/py27-scipy/work/scipy-0.16.1" && /mp2/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg config_fc --fcompiler gnu95 --f77exec /mp2/local/bin/gfortran-mp-4.7 --f77flags=' -Os -fno-second-underscore' --f90exec /mp2/local/bin/gfortran-mp-4.7 --f90flags=' -Os -fno-second-underscore' config --cc /mp2/local/bin/gcc-mp-4.7 --include-dirs /mp2/local/include --library-dirs /mp2/local/lib build 
Exit code: 1

The last time I upgraded py27-scipy (as well as py34-scipy) was in August (for a rev-bump from @0.16.0_0 to @0.16.0_1) which built with no problem. The last "true" version upgrade was a few days before the rev-bump when py-scipy went from @0.15.1_0 to @0.16.0_0--again, no problem. I haven't upgraded gcc47 since the end of May, so I don't believe gcc47 is the source of the error (unless scipy now requires a more recent gcc).

Any ideas?

[zzanderr]

py34-scipy also fails with the same weird options, however, the build log gives some additional info that may be of use.

[ryandesign (Ryan Carsten Schmidt)]

It's certainly possible that something changed between scipy 0.16.0 and 0.16.1 that caused it to either require an x86_64 processor or else to detect non-x86_64 processors incorrectly. There are some 180,000 lines of differences between those two versions so it's difficult to look through all that to figure out what might be the cause.

[zzanderr]

I would agree with you immediately if I had reason to suppose this to be a problem with c++ code. But fortran?

[josephsacco]

Same build problem seen when using macports-gcc-4.8 or macports-gcc-5.

-Joseph

[josephsacco]

The culprit is py-numpy. Specifically: the method get_flags_opt(self) in the file

site-packages/numpy/distfiles/fcompiler/gnu.py

The problem is the code in this method was crafted for an intel processor. Below is a reworked version of that method that will work on a PPC or PPC64.

-Joseph

---

   def get_flags_opt(self):
        v = self.get_version()
        if v and v <= '3.3.3':
            # With this compiler version building Fortran BLAS/LAPACK
            # with -O3 caused failures in lib.lapack heevr,syevr tests.
            opt = ['-O2']
        elif v and v >= '4.6.0':
            if is_win32():
                # use -mincoming-stack-boundary=2
                # due to the change to 16 byte stack alignment since GCC 4.6
                # but 32 bit Windows ABI defines 4 bytes stack alignment
                opt = ['-O2 -march=core2 -mtune=generic -mfpmath=sse -msse2 '
                       '-mincoming-stack-boundary=2']
            elif is_win64():
                opt = ['-O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2']
            else:
                opt = ['-O2']
        else:
            opt = ['-O2']

        return opt

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #49606.

[ryandesign (Ryan Carsten Schmidt)]

Replying to drjesacco@…:

The culprit is py-numpy. Specifically: the method get_flags_opt(self) in the file

site-packages/numpy/distfiles/fcompiler/gnu.py

The problem is the code in this method was crafted for an intel processor. Below is a reworked version of that method that will work on a PPC or PPC64.

-Joseph

---

   def get_flags_opt(self):
        v = self.get_version()
        if v and v <= '3.3.3':
            # With this compiler version building Fortran BLAS/LAPACK
            # with -O3 caused failures in lib.lapack heevr,syevr tests.
            opt = ['-O2']
        elif v and v >= '4.6.0':
            if is_win32():
                # use -mincoming-stack-boundary=2
                # due to the change to 16 byte stack alignment since GCC 4.6
                # but 32 bit Windows ABI defines 4 bytes stack alignment
                opt = ['-O2 -march=core2 -mtune=generic -mfpmath=sse -msse2 '
                       '-mincoming-stack-boundary=2']
            elif is_win64():
                opt = ['-O2 -march=x86-64 -DMS_WIN64 -mtune=generic -msse2']
            else:
                opt = ['-O2']
        else:
            opt = ['-O2']

        return opt

Did this fix come from the developers, or someone else? If someone else, please inform the developers of this fix.

[josephsacco]

patch file for numpy/distfiles/fcompiler/gnu.py

[josephsacco]

I rewrote the method. I have added a patch file [gnu.py.diff]. Not sure who/where to send it to.

-Joseph

[michaelld (Michael Dickens)]

The change that caused this issue was committed on Feb 27, 2015: ​https://github.com/numpy/numpy/commit/96abd32de241864ee97f30357234cbc9a96c43ae . So, it was part of the 1.9.2 release. On Oct 20, 2015, this change was reverted (finally): ​https://github.com/numpy/numpy/commit/a08437e12d6b6796f4b0beda49bce9adc37db63d . So, this fix will be part of 1.10.2. Can you try out commit a08437e12d too see if it works for you? If so, then I think that's the better way to go since it's already in the repo.

[seanfarley (Sean Farley)]

This should have been fixed with 1.10.2.