py27-scikits-learn: Missing ATLAS symbol on sklearn.cluster import

[cdeil (Christoph Deil)]

This issue might be a consequence of #36694 or a separate issue.

I can't use py27-scikit-learn because I get missing ATLAS symbol errors on import of sklearn sub-packages.

This is with OS X 10.8.2 (12C60) and XCode 4.5.1 (4G1004) and Accelerate Framework 1.8.

ChristophMacbook:tmp deil$ python -c 'import sklearn.cluster'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/__init__.py", line 6, in <module>
    from .spectral import spectral_clustering, SpectralClustering
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/spectral.py", line 15, in <module>
    from .k_means_ import k_means
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/k_means_.py", line 29, in <module>
    from . import _k_means
ImportError: dlopen(/Users/deil/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/_k_means.so, 2): Symbol not found: _ATL_ddot
  Referenced from: /Users/deil/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/_k_means.so
  Expected in: flat namespace
 in /Users/deil/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/_k_means.so
$ python -c 'import sklearn.decomposition'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/decomposition/__init__.py", line 10, in <module>
    from .sparse_pca import SparsePCA, MiniBatchSparsePCA
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/decomposition/sparse_pca.py", line 8, in <module>
    from ..linear_model import ridge_regression
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/__init__.py", line 12, in <module>
    from .base import LinearRegression
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/base.py", line 28, in <module>
    from cd_fast import sparse_std
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/cd_fast.so, 2): Symbol not found: _ATL_daxpy
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/cd_fast.so
  Expected in: flat namespace
 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/linear_model/cd_fast.so

The problem is that some ATLAS symbols are not found in Apple's Accelerate library. Here is some info that I think shows that numpy, scipy and sklearn were linked against the Accelerate library:

$ otool -L /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/_k_means.so
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/_k_means.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)

$ otool -L /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/sparse/linalg/eigen/arpack/_arpack.so:
	/opt/local/lib/gcc45/libgfortran.3.dylib (compatibility version 4.0.0, current version 4.0.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1669.0.0)
	/opt/local/lib/gcc45/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

$ otool -L /Users/deil/Library/Python/2.7/lib/python/site-packages/numpy/core/_dotblas.so
/Users/deil/Library/Python/2.7/lib/python/site-packages/numpy/core/_dotblas.so:
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

$ otool -L  /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate
/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate:
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/vImage (compatibility version 1.0.0, current version 204.5.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib (compatibility version 1.0.0, current version 380.6.0)
	/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvDSP.dylib (compatibility version 1.0.0, current version 380.6.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

This shows the existing variants and the ones I have installed:

$ port variants py27-numpy
py27-numpy has the variants:
   atlas: Use the MacPorts' ATLAS libraries  instead of Apple's Accelerate framework
   universal: Build for multiple architectures
$ port variants py27-scipy
py27-scipy has the variants:
   atlas: Use MacPorts ATLAS libraries
   gcc43: Use gcc 4.3 tool chain for building
     * conflicts with gcc44 gcc45 gcc46 gcc47
   gcc44: Use gfortran-mp-4.4 as fortran compiler
     * conflicts with gcc43 gcc45 gcc46 gcc47
[+]gcc45: Use gfortran-mp-4.5 as fortran compiler
     * conflicts with gcc43 gcc44 gcc46 gcc47
   gcc46: Use gfortran-mp-4.6 as fortran compiler
     * conflicts with gcc43 gcc44 gcc45 gcc47
   gcc47: Use gfortran-mp-4.7 as fortran compiler
     * conflicts with gcc43 gcc44 gcc45 gcc46
$ port variants py27-scikits-learn
py27-scikits-learn has the variants:
   universal: Build for multiple architectures
$ port installed py27-numpy
The following ports are currently installed:
  py27-numpy @1.6.2_0 (active)
$ port installed py27-scipy
The following ports are currently installed:
  py27-scipy @0.11.0_0+gcc45 (active)
$ port installed py27-scikits-learn
The following ports are currently installed:
  py27-scikits-learn @0.12.1_0 (active)

The problem of missing ATLAS symbols has been discussed before, see e.g. ​https://github.com/scikit-learn/scikit-learn/issues/1247

[cdeil (Christoph Deil)]

I've mentioned this issue in the scikit-learn tracker: ​https://github.com/scikit-learn/scikit-learn/issues/1247#issuecomment-9697159

[cdeil (Christoph Deil)]

Sorry, some of the output I showed above was from /Users/deil/Library/Python/2.7/lib/python/site-packages where I have an installation of sklearn git master via python setup.py install --user. I believe the problems I describe should be there for Macports-only though, because I see them after setting the PYTHONPATH to the Macports site-packages:

$ export PYTHONPATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
$ python -c 'import sklearn.cluster'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/__init__.py", line 6, in <module>
    from .spectral import spectral_clustering, SpectralClustering
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/spectral.py", line 15, in <module>
    from .k_means_ import k_means
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/k_means_.py", line 29, in <module>
    from . import _k_means
ImportError: dlopen(/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/_k_means.so, 2): Symbol not found: _ATL_ddot
  Referenced from: /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/_k_means.so
  Expected in: flat namespace
 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sklearn/cluster/_k_means.so

[cmutel (Chris Mutel)]

Cc Me!

[petrrr]

Cc Me!

[petrrr]

Cc Me!