New Ticket     Tickets     Wiki     Browse Source     Timeline     Roadmap     Ticket Reports     Search

Ticket #31891 (new defect)

Opened 2 years ago

Last modified 4 months ago

Python loading system numpy in preference to MacPorts version (Lion)

Reported by: jamie@… Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version: 2.0.3
Keywords: Cc: guygurari@…, ptmkenny@…, bardeau@…, russell.jones@…, ttimo@…, jwa@…, dh@…, michaelld@…, tsutterley@…, dersh@…
Port: python27 py-numpy

Description (last modified by ryandesign@…) (diff)

I'm having an issue where python scripts are loading the system numpy version instead of the MacPorts compiled version, this leads to the following error message:

RuntimeError: module compiled against API version 6 but this version of numpy is 4

and in some cases (e.g. scripts that use matplotlib) causes crashes.

This has only started occurring on my system since I upgraded to OS X Lion.

The fix so far seems to be to add the following code to the top of each script before loading numpy:

sys.path.reverse()

but this is extremely kludgy fix, and I never had to previously do this. My PYTHONPATH variable points to my local python repository and to the MacPorts site-packages directory only. Is there somewhere else I can tell python to stop loading stuff from /System/Library/Frameworks/Python.framework?

Change History

comment:1 Changed 2 years ago by ryandesign@…

  • Description modified (diff)

comment:2 Changed 2 years ago by guygurari@…

  • Cc guygurari@… added

Cc Me!

comment:3 Changed 2 years ago by guygurari@…

Same problem here, and the workaround works for me. Thanks!

comment:4 Changed 2 years ago by jamie@…

I should add that another way to fix this issue, which I'm now using (modifying all my python scripts which are used across multiple platforms in a mission critical system is not an option for me), is to simply make the Apple installed python-2.7 the default using:

port select python python27-apple

comment:5 Changed 22 months ago by jmr@…

This is my sys.path in macports python2.7:

>>> import sys
>>> print sys.path
['', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info']

What is in yours? There must be some system paths if reversing it helps? Which python port are you using anyway?

comment:6 Changed 22 months ago by ptmkenny@…

I have the same issue. This is what I get:

>>> import sys
>>> print sys.path
['', '/Library/Python/2.7/site-packages/virtualenv-1.6.4-py2.7.egg', '/Library/Python/2.7/site-packages/CherryPy-3.2.2-py2.7.egg', '/Library/Python/2.7/site-packages/Mnemosyne-2.0_RC4-py2.7.egg', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', '/Library/Python/2.7/site-packages']

comment:7 Changed 22 months ago by ptmkenny@…

  • Cc ptmkenny@… added

Cc Me!

comment:8 in reply to: ↑ description Changed 21 months ago by bardeau@…

I am experiencing the same problem for the software I am in charge (http://www.iram.fr/IRAMFR/GILDAS/).

From what I can understand, it seems that the numpy/core/multiarray.so dynamic library provided by the MacPorts-numpy has been built with the numpy API version 4, while the include files it offers provide the API version 6. API (and ABI) versions can be found in numpy/core/include/numpy/_numpyconfig.h (NPY_API_VERSION and NPY_ABI_VERSION resp).

When you import numpy.core.multiarray (explicitely, or implicitely when importing numpy) from the python interpreter, there is no problem since numpy/core/multiarray.so is self-consistent.

But custom C-based Python-Numpy extensions are built with those lines in particular: #include "numpy/arrayobject.h" import_array();

At load time the function "import_array" checks if the 'NPY_FEATURE_VERSION' (from NPY_API_VERSION in _numpyconfig.h) is consistent with the one numpy/core/multiarray.so has been built with (see PyArray_GetNDArrayCFeatureVersion in numpy/core/code_generators/generate_numpy_api.py in numpy sources). The error message comes from here.

I can also quote those few lines from doc/source/reference/c-api.array.rst: "ABI incompatibility is automatically detected in every numpy's version. API incompatibility detection was added in numpy 1.4.0. If you want to supported many different numpy versions with one extension binary, you have to build your extension with the lowest NPY_FEATURE_VERSION as possible."

So at this stage I do not why the MacPorts-numpy is inconsistent. Is this intended?

comment:9 Changed 21 months ago by bardeau@…

  • Cc bardeau@… added

Cc Me!

comment:10 Changed 17 months ago by russell.jones@…

  • Cc russell.jones@… added

Cc Me!

comment:11 Changed 11 months ago by ttimo@…

  • Cc ttimo@… added

Cc Me!

comment:12 Changed 11 months ago by ttimo@…

Would appreciate a fix for this. The workaround sys.path.reverse still works though.

comment:13 Changed 10 months ago by larryv@…

  • Cc jwa@…, dh@…, michaelld@… added

comment:14 Changed 10 months ago by macsforever2000@…

  • Port set to python27 py-numpy

comment:15 Changed 10 months ago by tsutterley@…

  • Cc tsutterley@… added

Cc Me!

comment:16 Changed 4 months ago by dersh@…

  • Cc dersh@… added

Cc Me!

Note: See TracTickets for help on using tickets.