py-virtualenv @20.6.0: pkg_resources considers platformdirs requirement not met

[leofiore (Leonardo)]

i recently updated my macports base from 2.6.4 to 2.7.1 and, with this, my installation of pipenv. In my environment i use python38 as default python3 installation, so all of my virtualenvs are created for this version of python. I ran pipenv and it fails immediatly with this error

leonardo@leda.local$ pipenv
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 568, in _build_master
    ws.require(__requires__)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 886, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 777, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (platformdirs 0.0.0 (/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages), Requirement.parse('platformdirs<3,>=2'), {'virtualenv'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/local/bin/pipenv", line 33, in <module>
    sys.exit(load_entry_point('pipenv==2021.5.29', 'console_scripts', 'pipenv')())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 781, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 700, in make_context
    self.parse_args(ctx, args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1212, in parse_args
    rest = Command.parse_args(self, ctx, args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1044, in parse_args
    parser = self.make_parser(ctx)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 965, in make_parser
    for param in self.get_params(ctx):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 912, in get_params
    help_option = self.get_help_option(ctx)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/cli/options.py", line 27, in get_help_option
    from ..core import format_help
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/core.py", line 33, in <module>
    from .project import Project
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/project.py", line 24, in <module>
    from .environment import Environment
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pipenv/environment.py", line 16, in <module>
    import pkg_resources
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3243, in <module>
    def _initialize_master_working_set():
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3226, in _call_aside
    f(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3255, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 570, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 583, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pkg_resources/__init__.py", line 772, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'platformdirs<3,>=2' distribution was not found and is required by virtualenv

since the error involes virtualenv, i tried to switch to virtualenv39 via sudo port select --set virtualenv virtualenv39. The error persists. Launching virtualenv (3.9) by command line successfully creates a new environment.

I tried to reinstall a previous version of pipenv ( @2020.8.13 ) but it shows me the error message Error: pipenv version 2020.8.13 is not available (current version is 2021.5.29_0)

[jmroot (Joshua Root)]

That's odd, since py-virtualenv does declare its dependency on py-platformdirs.

[reneeotten (Renee Otten)]

strange, for me running just virtualenv-3.9 works fine - that should be the case for you as well... or? It seems to me that the issue is that py-platformdirs is installed as version "0.0.0" at least looking at the directory name (e.g., /opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/platformdirs-0.0.0-py3.9.egg-info/PKG-INFO and it wants platformdirs<3,>=2.

Let me try and fix that and I'll push a change.

[reneeotten (Renee Otten)]

In 79ed23e4e8904cb830f45732c60fb4785c64506d/macports-ports (master):

py-platformdirs: patch version specifier

Closes: #63320

[reneeotten (Renee Otten)]

upstream discussion ​here

[reneeotten (Renee Otten)]

@jmroot: upstream's opinion seems to be "don't use python setup.py install, instead use a PEP-517/PEP-518 compliant builder".

That discussion has come up before I think and at some point you added the python.pep517 option to the python PG. I am not sure whether that will solve this issue here (will try later in the weekend, for now I just patched in the version field to setup.py).

What is your view on a long-term solution for MacPorts with respect to installing Python packages?

[jmroot (Joshua Root)]

There isn't a single solution for all packages, but in most cases it should be fine to use pep517. In this case they're still using setuptools as the backend though, so I wouldn't be surprised if the missing version metadata still caused issues somewhere.

The seemingly widespread belief that "just use pip" is a sufficient solution for packagers is troublesome, but that's a different bug. :)

[ryandesign (Ryan Carsten Schmidt)]

According to the upstream discussion, MacPorts is packaging all python modules wrong and must change in order for anything to work.

[danchr (Dan Villiom Podlaski Christiansen)]

I'll take a look at switching pipenv to use python.pep517; from my understanding, that is part of the bug here? Perhaps python.pep517 should be made the default for 3.10?

[danchr (Dan Villiom Podlaski Christiansen)]

Replying to danchr:

I'll take a look at switching pipenv to use python.pep517; from my understanding, that is part of the bug here? Perhaps python.pep517 should be made the default for 3.10?

I posted a ​PR that makes Pipenv use PEP-517. I noticed that the tool currently used is deprecated, so I fixed that.

[mascguy (Christopher Nielsen)]

There's also semi-related issue:63325.

[jmroot (Joshua Root)]

FTR, the actual problem here was a missing dependency on setuptools_scm.