Opened 8 years ago

Closed 8 years ago

#35020 closed update (fixed)

py-scipy: work with updated SuiteSparse 4.0.0

Reported by: zan@… Owned by: skymoo (Adam Mercer)
Priority: Normal Milestone:
Component: ports Version:
Keywords: haspatch Cc:
Port: py-scipy

Description

Patch to make Scipy's UMFPack work with updated SuiteSparse.

Related tickets: metis 35018 and SuiteSparse 35019

Attachments (2)

Portfile.diff (1.9 KB) - added by zan@… 8 years ago.
Unified diff of Portfile
umfpack_extra_libs.patch (326 bytes) - added by zan@… 8 years ago.
Patchfile for umfpack

Download all attachments as: .zip

Change History (11)

Changed 8 years ago by zan@…

Attachment: Portfile.diff added

Unified diff of Portfile

Changed 8 years ago by zan@…

Attachment: umfpack_extra_libs.patch added

Patchfile for umfpack

comment:1 Changed 8 years ago by jmroot (Joshua Root)

Cc: jmr@… ram@… removed
Owner: changed from macports-tickets@… to ram@…
Version: 2.1.1

comment:2 Changed 8 years ago by skymoo (Adam Mercer)

Status: newassigned

comment:3 Changed 8 years ago by skymoo (Adam Mercer)

Resolution: fixed
Status: assignedclosed

Patch didn't apply cleanly, there are small changes here so it's not difficult to manually apply the patch.

I do have a question however: Why remove the --no-user-cfg option? It builds cleanly without removing it?

I've pushed the patch minus the removal of the --no-user-cfg option in r94990. If it turns out it needs to be removed that can be addressed later...

comment:4 in reply to:  3 ; Changed 8 years ago by zan@…

--no-user-cfg makes setup.py ignore the site.cfg local configuration file. I used site.cfg to make scipy find the SuiteSparse libraries needed for umfpack.

During compilation without the patch, libumfpack.a and libamd.a are found and so umfpack is built. There are no compilation errors. However, 'nm -u _ _umfpack.so' in scipy/sparse/linalg/dsolve/umfpack shows that several _cholmod functions are undefined. The only way to detect this is to do a complete installation of scipy, and then:

% python
>>> import scipy
>>> t = scipy.test('full')
>>> print t.skipped
(Test(test_umfpack.TestFactorization.test_complex_lu), Exception('Skipping test: 
test_complex_lu\nUMFPACK appears not to be compiled',)).....<more skipped>

This patch tells setup.py to look for SuiteSparse(.dylib) in stead of umfpack and amd.

comment:5 in reply to:  4 ; Changed 8 years ago by skymoo (Adam Mercer)

Resolution: fixed
Status: closedreopened

I don't see that in the reason for the skipped tests, there are 10 failures though:

======================================================================
FAIL: test_asum (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 58, in test_asum
    assert_almost_equal(f([3,-4,5]),12)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 12

======================================================================
FAIL: test_dot (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 67, in test_dot
    assert_almost_equal(f([3,-4,5],[2,5,1]),-9)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: -9

======================================================================
FAIL: test_nrm2 (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/lib/blas/tests/test_blas.py", line 78, in test_nrm2
    assert_almost_equal(f([3,-4,5]),math.sqrt(50))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 7.0710678118654755

======================================================================
FAIL: test_basic.TestNorm.test_overflow
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_basic.py", line 581, in test_overflow
    assert_almost_equal(norm(a), a)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 452, in assert_almost_equal
    return assert_array_almost_equal(actual, desired, decimal, err_msg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 800, in assert_array_almost_equal
    header=('Arrays are not almost equal to %d decimals' % decimal))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 636, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals

(mismatch 100.0%)
 x: array(-0.0)
 y: array([  1.00000002e+20], dtype=float32)

======================================================================
FAIL: test_basic.TestNorm.test_stable
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_basic.py", line 586, in test_stable
    assert_almost_equal(norm(a) - 1e4, 0.5)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: -10000.0
 DESIRED: 0.5

======================================================================
FAIL: test_basic.TestNorm.test_types
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_basic.py", line 568, in test_types
    assert_allclose(norm(x), np.sqrt(14), rtol=tol)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 1168, in assert_allclose
    verbose=verbose, header=header)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 636, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=2.38419e-06, atol=0

(mismatch 100.0%)
 x: array(1.0842021724855044e-19)
 y: array(3.7416573867739413)

======================================================================
FAIL: test_asum (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 99, in test_asum
    assert_almost_equal(f([3,-4,5]),12)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 12

======================================================================
FAIL: test_dot (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 109, in test_dot
    assert_almost_equal(f([3,-4,5],[2,5,1]),-9)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: -9

======================================================================
FAIL: test_nrm2 (test_blas.TestFBLAS1Simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/linalg/tests/test_blas.py", line 127, in test_nrm2
    assert_almost_equal(f([3,-4,5]),math.sqrt(50))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 0.0
 DESIRED: 7.0710678118654755

======================================================================
FAIL: test_iv_cephes_vs_amos_mass_test (test_basic.TestBessel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/special/tests/test_basic.py", line 1656, in test_iv_cephes_vs_amos_mass_test
    assert_(dc[k] < 1e-9, (v[k], x[k], special.iv(v[k], x[k]), special.iv(v[k], x[k]+0j)))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/testing/utils.py", line 34, in assert_
    raise AssertionError(msg)
AssertionError: (189.2947429454936, 3.0238805556481037, 4.089165443940765e-317, 0j)

----------------------------------------------------------------------
Ran 5834 tests in 643.557s

FAILED (KNOWNFAIL=14, SKIP=26, failures=10)

do you see these also?

comment:6 in reply to:  5 Changed 8 years ago by zan@…

You were right, --no-user-cfg does not need to be deleted, it does not do what I thought it would. site.cfg from the patch file gets picked up and umfpack is built correctly.

After building I get 14 knownfail (single precision FFT, zero-order splines, fancy indexing, cephes, test_scxx_dict.TestDictGetItemOp), 18 skipped (all to do wih clapack/flapack), and 1 failure (test_iv_cephes_vs_amos_mass_test). The results are the same for the +atlas variant. Both are built with macports gcc 4.5 (variant +gcc45).

comment:7 Changed 8 years ago by skymoo (Adam Mercer)

Not sure why we're getting different results from the testsuite. What version of OS X are you using? 10.7.4 here.

comment:8 in reply to:  7 Changed 8 years ago by zan@…

I'm on 10.6.8, building for x86_64.

Your errors, except the last one which I also get, are mentioned in the scipy bug tracker. But that is unrelated to this ticket.

comment:9 Changed 8 years ago by skymoo (Adam Mercer)

Resolution: fixed
Status: reopenedclosed

Looks like I filled that ticket, I'd forgotten about that...

Note: See TracTickets for help on using tickets.