Opened 8 years ago

Closed 6 years ago

#34763 closed defect (fixed)

python27 @2.7.3_0 Includes system paths in sys.path

Reported by: jameskyle@… Owned by: larryv (Lawrence Velázquez)
Priority: Normal Milestone:
Component: ports Version: 2.1.99
Keywords: Cc: eborisch (Eric A. Borisch), kyounger@…, larryv (Lawrence Velázquez), guygurari@…, ptmkenny@…, bardeau, Russell-Jones-OxPhys (Russell Jones), ttimo@…, jyrkiwahlstedt, dh@…, michaelld (Michael Dickens), tsutterley (Tyler C Sutterley), dershow, petrrr, jamie@…
Port: python27 python31 python32 python33 python34

Description (last modified by jameskyle@…)

Macports installation of python 2.7 includes system paths in sys.path. This causes mucho problemo with module lookups.

Example:

[, '/Library/Python/2.7/site-packages/pip-1.1-py2.7.egg', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/readline', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/aeosa', '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info', '/Library/Python/2.7/site-packages']

The behavior was observed on a fresh install of macports with only a single package installed (a python package) and duplicated across multiple others.

Change History (27)

comment:1 Changed 8 years ago by jameskyle@…

Description: modified (diff)

Ah, one commonality on all systems that I just realized.

They all had pip installed on the system level.

I'll poke around and add to the ticket as needed.

comment:2 Changed 8 years ago by ned-deily (Ned Deily)

Adding /Library/Python/2.7/site-packages is an upstream "feature" as of Python 2.7 for all OS X framework builds; see http://bugs.python.org/issue4865. I must admit that I'm not convinced it is a good idea. Feel free to open an issue about on the Python bug tracker.

comment:3 Changed 8 years ago by eborisch (Eric A. Borisch)

FWIW, this isn't the case in my install:

MacPro:~$ /opt/local/bin/python2.7
Python 2.7.3 (default, Apr 13 2012, 10:00:30) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for n in sys.path:print(n)
... 

/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/readline
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PIL
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/PyObjC
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/gtk-2.0
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/wx-2.8-mac-unicode

10.6.8 / 4.0.2

comment:4 Changed 8 years ago by eborisch (Eric A. Borisch)

Owner: changed from macports-tickets@… to jwa@…

comment:5 Changed 8 years ago by mf2k (Frank Schima)

I get the same result as eborisch. Isn't this just an issue with the py-pip port?

comment:6 in reply to:  5 Changed 8 years ago by ned-deily (Ned Deily)

It's only an issue on 10.7 and beyond since Apple first supplied a system Python 2.7 with 10.7. /Library/Python/2.7 won't exist on earlier systems.

comment:7 Changed 8 years ago by eborisch (Eric A. Borisch)

Cc: eborisch@… added

Cc Me!

comment:8 Changed 8 years ago by jameskyle@…

I'm not so sure this is just a pip thing. I did the following this morning:

  • Uninstalled all python modules
  • Uninstalled all packages depending on python
  • Uninstalled python27 package
  • Verified the /opt/local/Frameworks/Python directory was removed (indicating no left over cruft)

Then I

  • Reinstalled python27 and nothing else

I then dropped into the python shell via:

% /opt/local/bin/python2.7

And printed sys.path, it's the exact same path as above.

I also did a sudo -u root -i for a clean $HOME/SHELL and inspected the path there, same path.

comment:9 Changed 8 years ago by ned-deily (Ned Deily)

Right, it has nothing to do with pip. As I explained above, it's a change in Python 2.7. If you want the behavior changed, you should open an issue on the Python bug tracker.

comment:10 in reply to:  9 Changed 8 years ago by jameskyle@…

Replying to nad@…:

Right, it has nothing to do with pip. As I explained above, it's a change in Python 2.7. If you want the behavior changed, you should open an issue on the Python bug tracker.

I'm not so sure this is an upstream issue.

When I compile my own python 2.7 on the same system with

./configure --prefix=/Users/jkyle/Downloads/Python && make -j8 && make install

And none of the non-install paths are present.

sys.path

[, '/Users/jkyle/Downloads/Python/lib/python27.zip', '/Users/jkyle/Downloads/Python/lib/python2.7', '/Users/jkyle/Downloads/Python/lib/python2.7/plat-darwin', '/Users/jkyle/Downloads/Python/lib/python2.7/plat-mac', '/Users/jkyle/Downloads/Python/lib/python2.7/plat-mac/lib-scriptpackages', '/Users/jkyle/Downloads/Python/lib/python2.7/lib-tk', '/Users/jkyle/Downloads/Python/lib/python2.7/lib-old', '/Users/jkyle/Downloads/Python/lib/python2.7/lib-dynload', '/Users/jkyle/Downloads/Python/lib/python2.7/site-packages']

I suspect the issue may lie in the macports python build itself. I'll look into this and update as needed.

comment:11 Changed 8 years ago by ned-deily (Ned Deily)

Trust me, it's an upstream issue (I am part of upstream). You'll see the same behavior using a python.org OS X installer for 2.7. The reason you didn't see it in your test is that it only applies to OS X framework builds, which you did not select. And only on 10.7 and beyond. The code that does this is in Lib/site.py in getsitepackages, currently around line 313 or so (http://hg.python.org/cpython/file/2.7/Lib/site.py).

comment:12 Changed 8 years ago by jameskyle@…

Thanks for the explanation.

I'll file an upstream bug on this. Pulling in system paths for separate installs would seem to be a bad idea.

comment:13 Changed 8 years ago by jmroot (Joshua Root)

Port: python27 added

comment:14 Changed 8 years ago by kyounger@…

Any legitimate hacks around this?

(Sorry if this might seem obvious, I'm generally pretty new to python and just ran into this issue tonight)

comment:15 Changed 8 years ago by kyounger@…

Cc: kyounger@… added

Cc Me!

comment:16 Changed 8 years ago by ned-deily (Ned Deily)

An upstream bug was filed as http://bugs.python.org/issue15048 and was rejected for a 2.7.x or 3.2.x maintenance release as breaking backward compatibility. If you have specific cases where this behavior causes problems and you can help to make a case that the current behavior should be changed, you could add them to the issue and ask that it be reopened. Otherwise, you could: try to avoid having duplicate distributions/packages installed for both the system Python 2.7 and MacPorts; or you could manually manipulate sys.path within your Python program; or Python's behavior could be patched in site.py by you or by the maintainer of the MacPorts Python ports.

comment:17 Changed 7 years ago by jmroot (Joshua Root)

I vote we patch this behaviour out of the port.

comment:18 Changed 7 years ago by petrrr

+1 for patching this! This is really kind of unexpected behavior. Especially because I always thought, this is why Python packages handling with MacPort would be clean and therefore preferable. But if I cannot rely on being independent from System's package installation this breaks.

I preferable, this could be configurable with a variant, but probably should default to activate the patch. As otherwise we are back here ...

comment:19 Changed 6 years ago by petrrr

Cc: petr@… added

Cc Me!

comment:20 Changed 6 years ago by larryv (Lawrence Velázquez)

Cc: larryv@… added

Cc Me!

comment:21 Changed 6 years ago by smithsp (Sterling Smith)

+1 for a patch, which is the default.

comment:22 Changed 6 years ago by petrrr

By history this ticket is probably a duplicate of #31891. However, this ticket evidences the problem in a much clearer way.

So if we agree that the problems described here and in #31891 are basically equivalent or have the same cause, we may want to close #31891, instead of this one and keep tracking the problem here.

comment:23 Changed 6 years ago by petrrr

Cc: petr@… removed

Cc Me!

comment:24 Changed 6 years ago by petrrr

Cc: guygurari@… ptmkenny@… bardeau@… russell.jones@… ttimo@… jwa@… dh@… michaelld@… tsutterley@… dersh@… petr@… added

Adding CC addresses from #31891 here.

Last edited 6 years ago by petrrr (previous) (diff)

comment:25 Changed 6 years ago by petrrr

Cc: jamie@… added

Adding reporters address of #31891 to CC as well

comment:26 Changed 6 years ago by larryv (Lawrence Velázquez)

Owner: changed from jwa@… to larryv@…
Status: newassigned

comment:27 Changed 6 years ago by larryv (Lawrence Velázquez)

Port: python31 python32 python33 python34 added
Resolution: fixed
Status: assignedclosed

Fixed for python27 and python31 through python34 in r126840.

Note: See TracTickets for help on using tickets.