Error in pygments after upgrade from 2.0.2_0 to 2.1_0

[rpspringuel (Fr. Samuel Springuel)]

I use pygments through the MacPorts installation py-pygments package and after updating from 2.0.2 to 2.1 yesterday (a process that went off without a hitch) have run into an error. When I invoke pygmentize -V at the command-line I get the following:

Traceback (most recent call last):
  File "/opt/local/bin/pygmentize", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3138, in <module>
    @_call_aside
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3124, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3151, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 663, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 676, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'Pygments==2.0.2' distribution was not found and is required by the application

That last line tells me pygmentize is looking for the wrong version. If I revert the py27-pygments (which py-pygments depends on) package to 2.0.2_0 then the error goes away, but of course that's because I'm using the old version (I do not need to revert the py-pygments package).

Using terminal for further investigation I did/got the following:

cd /
sudo find . -name "pygmentize" -print -xdev
./opt/local/bin/pygmentize
./opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/pygmentize
./usr/local/bin/pygmentize

Invoking each of these commands specifically, the first one fails with the above error while the other two work fine. Using which I find that the first and third are in PATH with the first (the one which fails) being first. Further testing also shows that the third is a symlink to the second, but the first is a normal file.

As a final check I do/get the following:

sudo find . -name "pygments" -print -xdev -type d
./Applications/calibre.app/Contents/Resources/Python/site-packages/pygments
./opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pygments

The first pygments site-package is safely tucked away inside an application bundle, so it's shouldn't be causing the problems. The second is the MacPorts installed one. Looking inside it, I can clearly see that the version is 2.1 after the update.

My conclusion from all this is that /opt/local/bin/pygmentize was not updated properly when py-pygments (and py27-pygments) was updated from 2.0.2 to 2.1. What's my best course of action for fixing this problem?

[stromnov (Andrey Stromnov)]

It looks like you have additional Pygmens 2.0.2 installation (/usr/local/bin/pygmentize).

Do you have something customized in PYTHONPATH or PYTHONSTARTUP environment variables?

Do you have something in ~/Library/Python/2.7/lib/python/site-packages?

[rpspringuel (Fr. Samuel Springuel)]

/usr/local/bin/pygmentize is a symlink to /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/pygmentize. I don't recall putting it there, but it actually works, invoking pygments 2.1 after upgrade. It is the /opt/local/bin/pygmentize which is creating the problems. That is not a symlink, and is attempting to invoke pygments 2.0.2 according to the error traceback (posted above).

Both PYTHONPATH and PYTHONSTARTUP are empty on my system.

~/Library/Python does not exist on my system. Neither does ~/Library/Frameworks/Python.framework. As I noted above, I did a system-wide search for folders named pygments and got only two hits (one inside the calibre application bundle, the other the MacPorts installed package). A systeme-wide search for folders called site-packages reveals a few more in application bundles, the OS packaged ones (/Library/Python/2.7/site-packages and /Library/Python/2.6/site-packages, both of which contain only the README), a /opt/local/lib/python2.7/site-packages (which only contains a folder called xcbgen), the 3.4 folder (another MacPorts installation, well populated, but not with pygments), and /opt/X11/lib/python2.6/site-packages (which only contains xcb and xcbgen).

[rpspringuel (Fr. Samuel Springuel)]

As a recent update of my MacPorts packages (which included py27-pygments@2.1.1_0) brought this problem back to the front of my mind, I came up with the following solution:

1. I removed both /opt/local/bin/pygmentize and /usr/local/bin/pygmentize.
2. I added a symlink in /opt/local/bin to /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/pygmentize.

I have no idea how this will behave the next time the package updates, but this does work nicely now and I can cycle through the activation of all three installed versions (2.0.2, 2.1, and 2.1.1) without any problem.

[rpspringuel (Fr. Samuel Springuel)]

Update: I figured out why the symlink in /usr/local/bin was there. As it turns out the pygments package for LaTeX has a hard coded path to pygmentize in that location. The symlink is therefore necessary to use that package. I've had to restore that symlink in order to get that package working again.

[petrrr]

The port py27-pygments does not contain /opt/local/bin/pygmentize, but only /opt/local/bin/pygmentize-2.7. So very probably it was a manual addition, maybe a copy, instead of a symlink.

[petrrr]

Cc Me!