Opened 10 years ago

Closed 9 years ago

Last modified 9 years ago

#43321 closed defect (fixed)

py-scipy @0.13.3_1: builds agains ATLAS (variant +atlas) Seg. fault during testing.

Reported by: petrrr Owned by: seanfarley (Sean Farley)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: seanfarley (Sean Farley), michaelld (Michael Dickens), Veence (Vincent)
Port: py-scipy atlas

Description

I built py-scipy against ATLAS. This version of scipy crashes during testing with a Segmentation fault. The details below are for py27-scipy +atlas, but it was observed before (version @0.13.3) with other python subports as well.

There seem to be some real failures before the crash as well.

In [2]: import scipy

In [3]: scipy.test()
Running unit tests for scipy
NumPy version 1.8.0
NumPy is installed in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy
SciPy version 0.13.3
SciPy is installed in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy
Python version 2.7.6 (default, Nov 22 2013, 13:39:24) [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))]
nose version 1.3.1
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/utils.py:134: DeprecationWarning: `scipy.lib.blas` is deprecated, use `scipy.linalg.blas` instead!
  warnings.warn(depdoc, DeprecationWarning)
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/lib/utils.py:134: DeprecationWarning: `scipy.lib.lapack` is deprecated, use `scipy.linalg.lapack` instead!
  warnings.warn(depdoc, DeprecationWarning)
..............................................................................................................................................................................................................................K..................................................................................................................K.......................................................................................K..K...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F..FF......................................................................................................................................SSSSSS......SSSSSS......SSSS..............................FFF.....................................................0-th dimension must be fixed to 3 but got 15
..0-th dimension must be fixed to 3 but got 5
...F.Segmentation fault

ATLAS is installed as:

atlas @3.10.1_5+mpclang33 (active)

I also attach a crash report in the hope that it might help.

Not sure if this could be due to compiler mismatch. I CC Vince as atlas maintainer as well.

Attachments (1)

Python_2014-04-11-092015_radegast.crash (49.7 KB) - added by petrrr 10 years ago.
crash report

Download all attachments as: .zip

Change History (13)

Changed 10 years ago by petrrr

crash report

comment:1 Changed 10 years ago by seanfarley (Sean Farley)

Owner: changed from macports-tickets@… to sean@…
Status: newassigned

Thanks for the report. I see the failure for me is from test_nonlin.TestJacobianDotSolve.test_broyden1 (ran with scipy.test(verbose=10)). Searching around on the internet makes it seem that this is a known problem with ATLAS:

http://mail.scipy.org/pipermail/scipy-dev/2011-September/016582.html

and that the workaround is just not to use ATLAS.

comment:2 Changed 10 years ago by seanfarley (Sean Farley)

In other words, I am stuck.

comment:3 Changed 10 years ago by Veence (Vincent)

If you deem it useful to you, I can ask Atlas developer (Clint) to know if this is fixed in the latest unstable version.

Last edited 10 years ago by Veence (Vincent) (previous) (diff)

comment:4 in reply to:  3 ; Changed 10 years ago by seanfarley (Sean Farley)

Replying to vince@…:

If you deem it useful to you, I can ask Atlas developer (Clint) to know if this is fixed in the latest unstable version.

Sure, but I suspect this is something in the fortran compiler (maybe misaligning the arrays?). The bug report from scipy is from 2011 and they basically just say, "If this is a problem, then use another lapack package." I'll try to point them to this bug report and see if that gains any traction.

comment:5 in reply to:  4 Changed 10 years ago by seanfarley (Sean Farley)

Replying to sean@…:

Replying to vince@…:

If you deem it useful to you, I can ask Atlas developer (Clint) to know if this is fixed in the latest unstable version.

Sure, but I suspect this is something in the fortran compiler (maybe misaligning the arrays?). The bug report from scipy is from 2011 and they basically just say, "If this is a problem, then use another lapack package." I'll try to point them to this bug report and see if that gains any traction.

I found the issue on their github tracker here:

https://github.com/scipy/scipy/issues/3193

comment:6 Changed 9 years ago by seanfarley (Sean Farley)

Resolution: wontfix
Status: assignedclosed

Looks like this issue gained no traction on the scipy front. I still get the error but don't have the resources to dig into the scipy code. Marking this as 'wontfix' since it seems to be a bug in scipy.

comment:7 Changed 9 years ago by petrrr

I guess it makes perfectly sense not to try to fix bug in upstream software.

However, I wonder if we shouldn't disable variants if these are broken? Or do we judge that this is one too specific case we do not want to bother about?

comment:8 in reply to:  7 Changed 9 years ago by seanfarley (Sean Farley)

Replying to petr@…:

I guess it makes perfectly sense not to try to fix bug in upstream software.

However, I wonder if we shouldn't disable variants if these are broken? Or do we judge that this is one too specific case we do not want to bother about?

That is a great question. I am debating the same thing with the cgal +demo variant (which is still broken for me).

comment:9 Changed 9 years ago by seanfarley (Sean Farley)

Actually, looking into the compiler situation with scipy and numpy, I think I do have a fix for this.

comment:10 Changed 9 years ago by seanfarley (Sean Farley)

Resolution: wontfix
Status: closedreopened

comment:11 Changed 9 years ago by seanfarley (Sean Farley)

Resolution: fixed
Status: reopenedclosed

It seemed that numpy and scipy made incorrect (or outdated) assumptions about atlas (before the clang changes). I've fixed those in r130955. Some of the tests don't pass because numpy being too new.

comment:12 Changed 9 years ago by Veence (Vincent)

Sean, thanks for fixing this. I seem to have written a lot of things on my MacPorts backlog lately. I’ll try to catch up.

Note: See TracTickets for help on using tickets.