Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#27002 closed defect (duplicate)

py26-numpy @1.5.0, Revision 1 - build failure

Reported by: dannowolk@… Owned by: MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)
Priority: Normal Milestone:
Component: ports Version: 1.9.1
Keywords: tiger Cc: jpmasseria@…, antoine@…, michaelld (Michael Dickens), afisynte@…
Port: py26-numpy

Description

I tried port -vf install py26-numpy, and it failed. Before that I tried several times to upgrade the port, and also tried the +gc44 variant, as recommended somewhere. It failed the same way every time.

Here is some data about my machine:

Machine Name: iBook G4 Machine Model: PowerBook6,7 CPU Type: PowerPC G4 (1.5) Mac OS: 10.4.11 XCode Tools: 2.5

Here is some relevant terminal output:

port -v install py26-numpy--->  Computing dependencies for py26-numpy.
--->  Building py26-numpy
Running from numpy source directory.non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 1
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib
  libraries mkl,vml,guide not found in /usr/lib
  libraries mkl,vml,guide not found in /opt/local/lib
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in /opt/local/lib
  libraries ptf77blas,ptcblas,atlas not found in /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib
  NOT AVAILABLE

atlas_blas_info:
  FOUND:
    libraries = ['f77blas', 'cblas', 'atlas']
    library_dirs = ['/opt/local/lib']
    language = c
    include_dirs = ['/opt/local/include']

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py26-numpy/work/numpy-1.5.0/numpy/distutils/command/config.py:397: DeprecationWarning: 
+++++++++++++++++++++++++++++++++++++++++++++++++
Usage of get_output is deprecated: please do not 
use it anymore, and avoid configuration checks 
involving running executable on the target machine.
+++++++++++++++++++++++++++++++++++++++++++++++++

  DeprecationWarning)
customize NAGFCompiler
Could not locate executable f95
customize AbsoftFCompiler
Could not locate executable f90
Could not locate executable f77
customize IBMFCompiler
Could not locate executable xlf90
Could not locate executable xlf
customize IntelFCompiler
Could not locate executable ifort
Could not locate executable ifc
customize GnuFCompiler
Found executable /opt/local/bin/gfortran-mp-4.4
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
customize Gnu95FCompiler
customize Gnu95FCompiler
customize Gnu95FCompiler using config
compiling '_configtest.c':

[snip]

creating build/temp.macosx-10.4-ppc-2.6/numpy/linalg
compile options: '-DATLAS_INFO="\"3.8.3\"" -I/opt/local/include -Inumpy/core/include -Ibuild/src.macosx-10.4-ppc-2.6/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -Ibuild/src.macosx-10.4-ppc-2.6/numpy/core/src/multiarray -Ibuild/src.macosx-10.4-ppc-2.6/numpy/core/src/umath -c'
gcc-4.0: numpy/linalg/lapack_litemodule.c
gcc-4.0: numpy/linalg/python_xerbla.c
/opt/local/bin/gfortran-mp-4.4 -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.4-ppc-2.6/numpy/linalg/lapack_litemodule.o build/temp.macosx-10.4-ppc-2.6/numpy/linalg/python_xerbla.o -L/opt/local/lib -L/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5 -Lbuild/temp.macosx-10.4-ppc-2.6 -llapack -lf77blas -lcblas -latlas -lgfortran -o build/lib.macosx-10.4-ppc-2.6/numpy/linalg/lapack_lite.so -undefined dynamic_lookup -bundle
/usr/bin/ld: Undefined symbols:
___powisf2 referenced from liblapack expected to be defined in libgcc
___powidf2 referenced from liblapack expected to be defined in libgcc
collect2: ld returned 1 exit status
/usr/bin/ld: Undefined symbols:
___powisf2 referenced from liblapack expected to be defined in libgcc
___powidf2 referenced from liblapack expected to be defined in libgcc
collect2: ld returned 1 exit status
error: Command "/opt/local/bin/gfortran-mp-4.4 -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.4-ppc-2.6/numpy/linalg/lapack_litemodule.o build/temp.macosx-10.4-ppc-2.6/numpy/linalg/python_xerbla.o -L/opt/local/lib -L/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5 -Lbuild/temp.macosx-10.4-ppc-2.6 -llapack -lf77blas -lcblas -latlas -lgfortran -o build/lib.macosx-10.4-ppc-2.6/numpy/linalg/lapack_lite.so -undefined dynamic_lookup -bundle" failed with exit status 1
shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py26-numpy/work/numpy-1.5.0" && /opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 setup.py --no-user-cfg build " returned error 1
Error: Target org.macports.build returned: shell command failed
Warning: the following items did not execute (for py26-numpy): org.macports.activate org.macports.build org.macports.destroot org.macports.install
Log for py26-numpy is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py26-numpy/main.log
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#project.tickets>

Attachments (1)

main.log (188.8 KB) - added by dannowolk@… 13 years ago.
log file for failed py26-numpy build

Download all attachments as: .zip

Change History (20)

Changed 13 years ago by dannowolk@…

Attachment: main.log added

log file for failed py26-numpy build

comment:1 Changed 13 years ago by dannowolk@…

Cc: dannowolk@… added

Cc Me!

comment:2 in reply to:  1 Changed 13 years ago by dannowolk@…

Replying to dannowolk@…:

Cc Me!

cc mcalhoun@…, openmaintainer@…

comment:3 Changed 13 years ago by mf2k (Frank Schima)

Cc: dannowolk@… removed
Keywords: "build failure" removed
Owner: changed from macports-tickets@… to mcalhoun@…

comment:4 in reply to:  3 ; Changed 13 years ago by ccorn@…

Reproduced the failure on my MacOS X 10.4.11 ppc platform.

The logfile attached by dannowalk is consistent with mine.

According to the logfile, the linker line includes the correct -L flag:

  -L${prefix}/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5

Symbols missed by the linker (expected in libgcc):

  ___powisf2
  ___powidf2

(three underscores).

When I do

  ar -t /opt/mp/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5/libgcc.a | grep 'powi[sd]f2'

I get

  _powisf2.o
  _powidf2.o

So this could be an underscore issue.

comment:5 in reply to:  4 Changed 13 years ago by ccorn@…

According to the logfile, the symbols missed by the linker are from liblapack.

So this might be an issue with the atlas port.

comment:6 Changed 13 years ago by jpmasseria@…

Cc: jpmasseria@… added

Cc Me!

comment:7 Changed 13 years ago by antoine@…

Cc: antoine@… added

Cc Me!

comment:8 Changed 13 years ago by jpmasseria@…

Actually, the file names for the routines start with single underscore characters, but the external symbols have three underscores, using "nm" to display the symbol tables reveals the following:

$ nm -m /opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5/libgcc.a | grep 'powi[sd]f2'
/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5/libgcc.a(_powisf2.o):
00000000 (__TEXT,__text) private external ___powisf2
/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5/libgcc.a(_powidf2.o):
00000000 (__TEXT,__text) private external ___powidf2

So I don't believe that this is an atlas issue. I need to redo the build of py26-numpy and check to see how "ld" was being used when these undefined values popped up as they appear to be defined in libgcc.a

comment:9 in reply to:  8 Changed 13 years ago by jpmasseria@…

Replying to jpmasseria@…:

Actually, the file names for the routines start with single underscore characters, but the external symbols have three underscores, using "nm" to display the symbol tables reveals the following:

$ nm -m /opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5/libgcc.a | grep 'powi[sd]f2'
/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5/libgcc.a(_powisf2.o):
00000000 (__TEXT,__text) private external ___powisf2
/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5/libgcc.a(_powidf2.o):
00000000 (__TEXT,__text) private external ___powidf2

So I don't believe that this is an atlas issue. I need to redo the build of py26-numpy and check to see how "ld" was being used when these undefined values popped up as they appear to be defined in libgcc.a

Looking at this again, I'm confused over the fact that these labels are "external" and "private". To me that's almost an oxymoron. However, greping the nm for "external" shows that all of the external labels are private. I have to admit, that I'm not very experienced with UNIX and MAC linking. My background in building executables is on UNISYS mainframe computers, but the concept of a library file containing a table of all the exported (external) symbols that is then used my the linker to resolve the undefined labels of the program being built is the same as on the UNISYS.

comment:10 Changed 13 years ago by jpmasseria@…

The Mac OS reference library definition of private external symbol:

A private external symbol is a defined external symbol that is visible only to other modules within the same object file as the module that contains it. The standard static linker changes private external symbols into private defined symbols unless you specify otherwise (using the -keep_private_externs flag).

You can mark a symbol as private external by using the private_extern keyword (which works only in C) or the visibility("hidden") attribute (which works both in C and C++ with GCC 4.0), as in this example:

comment:11 Changed 13 years ago by yaseppochi (Stephen J. Turnbull)

This bug is still present in py26-numpy @1.5.1_0 (and py25-numpy @1.5.1_0).

I suspect that the problem is that something is forcing the use of /usr/bin/gcc-4.0 for the build, regardless of the setting of configure.cc (and configure.compiler, for that matter). I thought it was Python itself -- I know that Python does care about the core compiler when building C extensions -- but maybe it isn't. Anyway, the compiler controller is visible in the trace above.

I tried rebuilding the python26 port with "configure.compiler macports-gcc-4.4" hacked into the Portfile. It seemed to succeed (at least the compiler used to compile much of Python and its extensions was gcc-mp-4.4) However, in building py26-numpy, once again /usr/bin/gcc-4.0 was forced as the compiler, and later on the build totally blew up with "cannot link a simple C program" (because the Macports gcc doesn't recognize the -arch command-line option, it seems).

The only other thing I can think of to try is to install the apple-gcc-4.2 port, and rebuild python26 and all my py26-* ports with that compiler. But on a slow iBook G4 installing a new gcc will take forever, and rebuilding everything that depends on python26 as well as python itself will take almost that long; I'm not eager to do this.

comment:12 Changed 13 years ago by yaseppochi (Stephen J. Turnbull)

BTW, it blocks upgrade of ufraw (probably others as well).

comment:13 Changed 13 years ago by yaseppochi (Stephen J. Turnbull)

OK (well, not really OK, but I managed to butkus gcc/apple-ld into building):

sudo port -f uninstall py26-numpy
sudo port clean py26-numpy
sudo port install py26-numpy
cd `port work py26-numpy`/numpy-1.5.1
sudo /opt/local/bin/gfortran-mp-4.4 -v -Z -t -Wall -arch ppc build/temp.macosx-10.4-ppc-2.6/numpy/linalg/lapack_litemodule.o build/temp.macosx-10.4-ppc-2.6/numpy/linalg/python_xerbla.o -L/opt/local/lib -L/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5 -Lbuild/temp.macosx-10.4-ppc-2.6 -L/usr/lib -llapack -lf77blas -lcblas -latlas -lgfortran -o build/lib.macosx-10.4-ppc-2.6/numpy/linalg/lapack_lite.so -undefined dynamic_lookup -bundle
sudo port install py26-numpy

I don't know if this build actually *works*, I don't know how to test it offhand. (It's a direct dependency of at least 3 ports I have installed, and is currently blocking upgrade of about 15 ports. But I didn't ask for it personally.)

I don't know if the uninstall is necessary; sometimes that helps so I tried it. It didn't help, but of course then I'm stuck without an installed port until I figure out how to install. I'm in no mood to do further testing right now....

The gfortran invocation is a modification of the one recorded in main.log just before the error message. The additions to the gfortran invocation are the -v -t -Z options near the beginning of the command line and the -L/usr/lib option which is the last of the -L options. Presumably the -v -t can be omitted.

Aha - py25-numpy has the same problems. Yep, "port -u upgrade ...; cd ...; gfortran ...; port -u upgrade" works without uninstalling, and the "-v -t" flags to gfortran are unnecessary. Once again, this build is untested, I don't know if it actually works.

FWIW it seems to me that this is probably Apple ld screwage, not a problem with MacPorts or gcc. Maybe someday I'll try GCC 4.5 (cf #24925, 2d to last comment by vince), but I bet it doesn't help.

Platform much like the OP:

iBook G4 (PPC)
Mac OS X "Tiger" 10.4.11
MacPorts from svn, fresh sync of dports
port build probably 3-4 months old (port -v sez "1.9.1").

comment:14 Changed 13 years ago by jpmasseria@…

/opt/local/bin/gfortran-mp-4.4 -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.4-ppc-2.6/numpy/linalg/lapack_litemodule.o build/temp.macosx-10.4-ppc-2.6/numpy/linalg/python_xerbla.o -L/opt/local/lib -L/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5 -Lbuild/temp.macosx-10.4-ppc-2.6 ""-L/usr/lib"" -llapack -lf77blas -lcblas -latlas -lgfortran -o build/lib.macosx-10.4-ppc-2.6/numpy/linalg/lapack_lite.so -undefined dynamic_lookup -bundle

The above command worked for me. This is the same command that failed in main.log with ld undefined symbols. The only addition to that command was the addition of ""-L/usr/lib"" prior to ""-llapack""

comment:15 Changed 13 years ago by jpmasseria@…

Sorry, I used the incorrect formating for bold, correct is below:

/opt/local/bin/gfortran-mp-4.4 -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.4-ppc-2.6/numpy/linalg/lapack_litemodule.o build/temp.macosx-10.4-ppc-2.6/numpy/linalg/python_xerbla.o -L/opt/local/lib -L/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5 -Lbuild/temp.macosx-10.4-ppc-2.6 -L/usr/lib -llapack -lf77blas -lcblas -latlas -lgfortran -o build/lib.macosx-10.4-ppc-2.6/numpy/linalg/lapack_lite.so -undefined dynamic_lookup -bundle

The above command worked for me. This is the same command that failed in main.log with ld undefined symbols. The only addition to that command was the addition of -L/usr/lib prior to -llapack in the command.

The sudo port install py26-numpy then completed successfully.

Wow, I guess that the port install command is smart enough to see that lapack_lite.so is aleady built and picks up the install from that new point.

comment:16 Changed 13 years ago by michaelld (Michael Dickens)

Cc: michaelld@… added

Cc Me!

comment:17 Changed 13 years ago by jmroot (Joshua Root)

Resolution: duplicate
Status: newclosed

comment:18 Changed 13 years ago by afisynte@…

I do not think this bug is a duplicate of #24925 because it is marked as fixed and I encountered this bug yesterday with the latest MacPorts and ports tree.

The stuff in this comment allowed me to get past this issue, but it would be nice if it was officially fixed: https://trac.macports.org/ticket/27002#comment:13

Specifically I ran this after I got the error:

cd `port work py26-numpy`/numpy-1.5.1
sudo /opt/local/bin/gfortran-mp-4.4 -Wall -arch ppc build/temp.macosx-10.4-ppc-2.6/numpy/linalg/lapack_litemodule.o build/temp.macosx-10.4-ppc-2.6/numpy/linalg/python_xerbla.o -L/opt/local/lib -L/opt/local/lib/gcc44/gcc/ppc-apple-darwin8/4.4.5 -Lbuild/temp.macosx-10.4-ppc-2.6 -L/usr/lib -llapack -lf77blas -lcblas -latlas -lgfortran -o build/lib.macosx-10.4-ppc-2.6/numpy/linalg/lapack_lite.so -undefined dynamic_lookup -bundle

Then reran "sudo port install kdesdk4" (but I suppose "sudo port install py26-numpy" works too) and the installation continued.

comment:19 Changed 13 years ago by afisynte@…

Cc: afisynte@… added

Cc Me!

Note: See TracTickets for help on using tickets.