Scipy packages not importing ("cannot import" .integrate, .specfun, .special, .sparse)

[alison.mansheim@…]

I have had this problem with every version of scipy I've tried since 0.14.0 (currently am running the latest 15.1). Each time scipy and dependencies installed cleanly using macports for py27 on mac OSX 10.6.8. After upgrading to 0.14.0, some of the functions will not import ( those inside scipy.integrate, .specfun, .special, .sparse). I tried uninstalling and re-installing everything and still no luck. I found some older threads that reported a similar problem for older versions, but then closed because they were fixed in an upgrade.

For example, when I run a function that uses scipy.integrate.quad():

In [10]: cosmo.ProjectedLength(0.5)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-10-38d57df266ec> in <module>()
----> 1 cosmo.ProjectedLength(0.5)
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cluster_tools/cosmo.pyc in ProjectedLength(z, h, Om, Ol)
     58     Assumes small angles (i.e.: Da*Theta = Projected Length, i.e. sin(Theta)=Theta)
     59     """
---> 60     return Da(z,h,Om,Ol)/3437.75 #given that 1 radian = 3437.75 arcminutes
     61 
     62 def lensgeo(zl,zs,h=0.7,Om=0.3,Ol=0.7):
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cluster_tools/cosmo.pyc in Da(z, h, Om, Ol)
     38     Returns the cosmological angular diameter distance (in units of Mpc)
     39     """
---> 40     f0 = scipy.integrate.quad(lambda x: chi(x,Om,Ol),0,z)[0]
     41     #f0 = Integral(chi(x,Om,Ol),(x,0,z))
     42     da = c/(100.0*h)/(1+z)*f0.n()
AttributeError: 'module' object has no attribute 'integrate'

When I try to import one of the packages I also get an error:

In [15]: import scipy.integrate
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-15-4d37fa311919> in <module>()
----> 1 import scipy.integrate
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/integrate/__init__.py in <module>()
     49 from __future__ import division, print_function, absolute_import
     50 
---> 51 from .quadrature import *
     52 from .odepack import *
     53 from .quadpack import *
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/integrate/quadrature.py in <module>()
      4            'cumtrapz','newton_cotes']
      5 
----> 6 from scipy.special.orthogonal import p_roots
      7 from scipy.special import gammaln
      8 from numpy import sum, ones, add, diff, isinf, isscalar, \
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/__init__.py in <module>()
    547 
    548 from .basic import *
--> 549 from . import specfun
    550 from . import orthogonal
    551 from .orthogonal import *
ImportError: cannot import name spec fun

I looked in the scipy directories and the /integrate/quadrature.pyc and special/orthogonal.pyc and /special/specfun.so exist. I had similar problems that I traced to other .so's, so I thought there was an issue with my compiler, but I thought macports would have detected this. I did not have this problem with older versions of scipy.

I posted this problem on scipy git and they replied "You can try asking the macports people for support with their Scipy packaging."

I would appreciate any help you can offer! There are a lot of useful packages that rely on these functions.

Thank you, Alison

[mf2k (Frank Schima)]

In the future, please use WikiFormatting, fill in the Port field with the actual port name and Cc the port maintainers (port info --maintainers py27-scipy), if any.

It all works fine for me in Mavericks and Yosemite:

In [1]: import scipy.integrate

In [2]: 

Looks like it is either a Snow Leopard issue or you are not using Macports python.

[alison.mansheim@…]

Replying to mf2k@…:

Hi! Thank you. Sorry, I read the WikiFormatting page and must have missed that. I am using macports py27, and macports for everything else. I checked my .bash and path and everything looks fine there. Do you have an idea what the root of the problem may be, so I can experiment and find a workaround? Have you encountered any similar problems related to the .so files? If the other similar problems I've read from others were fixed by scipy upgrades I want to first make sure the solution isn't related to scipy. I'm not optimistic about asking the Snow Leopard folks for help on something this specific...

Thank you for your time! Alison

[seanfarley (Sean Farley)]

This also works for me. Can you send the output of port select --list python?

[alison.mansheim@…]

Available versions for python:

none python25-apple python26 python26-apple python27 (active)

[mf2k (Frank Schima)]

A reminder to use WikiFormatting... You can use the Preview button to see if it looks right before posting.

Also, what is the output of the following?

port installed py27-scipy

[michaelld (Michael Dickens)]

Also, what is the output of the following:

ls -lAF /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/

[alison.mansheim@…]

port installed py27-scipy

The following ports are currently installed:
  py27-scipy @0.14.0_0+gcc48
  py27-scipy @0.15.0_0+gcc48
  py27-scipy @0.15.1_0+gcc48
  py27-scipy @0.15.1_0+gfortran (active)

Replying to mf2k@…:

A reminder to use WikiFormatting... You can use the Preview button to see if it looks right before posting.

Also, what is the output of the following?

port installed py27-scipy

[alison.mansheim@…]

total 10384
-rw-r--r--   1 root  wheel    21520 Jan 18 05:18 __init__.py
-rw-r--r--   1 root  wheel    21994 Jan 18 12:18 __init__.pyc
-rwxr-xr-x   1 root  admin  1563548 Apr  3  2013 _cephes.so*
-rw-r--r--   1 root  wheel     5641 Jan 16 14:17 _ellip_harm.py
-rw-r--r--   1 root  wheel     6619 Jan 18 12:18 _ellip_harm.pyc
-rwxr-xr-x   1 root  wheel    98464 Jan 18 12:17 _ellip_harm_2.so*
-rwxr-xr-x   1 root  admin    30088 Apr  3  2013 _logit.so*
-rw-r--r--   1 root  wheel    11445 Jan 16 14:17 _testutils.py
-rw-r--r--   1 root  wheel    10592 Jan 18 12:18 _testutils.pyc
-rwxr-xr-x   1 root  wheel  1846820 Jan 18 12:17 _ufuncs.so*
-rwxr-xr-x   1 root  wheel   137904 Jan 18 12:17 _ufuncs_cxx.so*
-rw-r--r--   1 root  wheel    68807 Jan 18 05:18 add_newdocs.py
-rw-r--r--   1 root  wheel    65769 Jan 18 12:18 add_newdocs.pyc
-rw-r--r--   1 root  wheel    39497 Jan 18 05:18 basic.py
-rw-r--r--   1 root  wheel    51313 Jan 18 12:18 basic.pyc
-rw-r--r--   1 root  admin     1158 Apr  3  2013 gendoc.py
-rw-r--r--   1 root  admin     1355 Apr  3  2013 gendoc.pyc
-rw-r--r--   1 root  wheel    43577 Jan 18 05:18 generate_ufuncs.py
-rw-r--r--   1 root  wheel    42399 Jan 18 12:18 generate_ufuncs.pyc
-rw-r--r--   1 root  wheel     2970 Nov 23 11:09 lambertw.py
-rw-r--r--   1 root  wheel     3440 Jan 18 12:18 lambertw.pyc
-rwxr-xr-x   1 root  admin    42444 Apr  3  2013 lambertw.so*
-rw-r--r--   1 root  wheel    34870 Jan 16 14:17 orthogonal.py
-rw-r--r--   1 root  wheel    49086 Jan 18 12:18 orthogonal.pyc
-rwxr-xr-x   1 root  admin    86876 Apr  3  2013 orthogonal_eval.so*
-rw-r--r--   1 root  wheel     4626 Jan 16 14:17 setup.py
-rw-r--r--   1 root  wheel     3735 Jan 18 12:18 setup.pyc
-rw-r--r--   1 root  admin      502 Apr  3  2013 setupscons.py
-rw-r--r--   1 root  admin      896 Apr  3  2013 setupscons.pyc
-rwxr-xr-x   1 root  wheel   939076 Jan 18 12:17 specfun.so*
-rw-r--r--   1 root  admin      119 Apr  3  2013 special_version.py
-rw-r--r--   1 root  admin      334 Apr  3  2013 special_version.pyc
-rw-r--r--   1 root  wheel     3487 Nov 23 11:09 spfun_stats.py
-rw-r--r--   1 root  wheel     2453 Jan 18 12:18 spfun_stats.pyc
drwxr-xr-x  13 root  wheel      442 Mar 26 14:15 tests/

Replying to michaelld@…:

Also, what is the output of the following:

ls -lAF /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/

[alison.mansheim@…]

I already tried cycling through each of these options to see if activating another would help and it did not solve the problem. Oops sorry I meant to reply with this to the other suggestion mf2k...sorry! Replying to michaelld@…:

Also, what is the output of the following:

ls -lAF /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/

[alison.mansheim@…]

I already tried cycling through each of these options to see if activating another would help and it did not solve the problem.

Replying to mf2k@…:

A reminder to use WikiFormatting... You can use the Preview button to see if it looks right before posting.

Also, what is the output of the following?

port installed py27-scipy

[michaelld (Michael Dickens)]

I don't know if this will change anything, but I'm guessing that all of those files dated "Apr 3 2013" are cruft. You can verify via:

port contents py27-scipy | grep /integrate/

and then comparing the output of that to the output of "ls -lAF".

If this is the case, I'll guess that there are other cruft files around there too. If so, I'd recommend removing them. Doing so won't hurt, and might help.

[michaelld (Michael Dickens)]

What does the following return:

lipo -detailed_info /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/specfun.so

[alison.mansheim@…]

Sounds great, I will try removing all of the April 3 2013 files. There are quite a few .so, .pyc, and .pyc files in the scipy folders which are from 2013. Thanks for teaching me a new word: cruft!

Replying to michaelld@…:

I don't know if this will change anything, but I'm guessing that all of those files dated "Apr 3 2013" are cruft. You can verify via:

port contents py27-scipy | grep /integrate/

and then comparing the output of that to the output of "ls -lAF".

If this is the case, I'll guess that there are other cruft files around there too. If so, I'd recommend removing them. Doing so won't hurt, and might help.

[alison.mansheim@…]

It worked! Yaaaaay! Thanks to your sleuthing!

Alison

Replying to alison.mansheim@…:

Sounds great, I will try removing all of the April 3 2013 files. There are quite a few .so, .pyc, and .pyc files in the scipy folders which are from 2013. Thanks for teaching me a new word: cruft!

Replying to michaelld@…:

I don't know if this will change anything, but I'm guessing that all of those files dated "Apr 3 2013" are cruft. You can verify via:

port contents py27-scipy | grep /integrate/

and then comparing the output of that to the output of "ls -lAF".

If this is the case, I'll guess that there are other cruft files around there too. If so, I'd recommend removing them. Doing so won't hurt, and might help.

[michaelld (Michael Dickens)]

Excellent! You're welcome.