py-spaCy @2.2.4: the `allow-hosts` option is not supported when using pip to install requirements

[zap150 (Jan Zapletal)]

Installation of py38-spaCy fails, because py38-spaCy-2.2.4_0.darwin_19.x86_64.tbz2 is not available from ​https://packages.macports.org/py38-spaCy (and other servers). The latest one available is py38-spaCy-2.2.3_0.darwin_19.x86_64.tbz2. This affects other spaCy ports as well.

[ryandesign (Ryan Carsten Schmidt)]

Prebuilt archives are not available for py-spaCy 2.2.4 because we are unable to build it. The error from ​our build server log is:

Traceback (most recent call last):
  File "setup.py", line 200, in <module>
    setup_package()
  File "setup.py", line 190, in setup_package
    setup(
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 143, in setup
    _install_setup_requires(attrs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/__init__.py", line 138, in _install_setup_requires
    dist.fetch_build_eggs(dist.setup_requires)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/dist.py", line 695, in fetch_build_eggs
    resolved_dists = pkg_resources.working_set.resolve(
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 781, in resolve
    dist = best[req.key] = env.best_match(
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1066, in best_match
    return self.obtain(req, installer)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1078, in obtain
    return installer(requirement)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/dist.py", line 754, in fetch_build_egg
    return fetch_build_egg(self, req)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/setuptools/installer.py", line 83, in fetch_build_egg
    raise DistutilsError('the `allow-hosts` option is not supported '
distutils.errors.DistutilsError: the `allow-hosts` option is not supported when using pip to install requirements.

It looks like this ​may be because of a change in py-setuptools. Josh already commented on that bug report back in December 2019 about how it broke this feature in MacPorts. Josh, what should we be doing to fix this port? Surely this problem doesn't prevent all correctly-configured python modules from building or it would be a much bigger problem for MacPorts. Is this new error message just the new way that we'll be told that a dependency is missing?

[jmroot (Joshua Root)]

Yes, I would certainly still consider it a bug that setuptools raises an error in this situation when allow-hosts is only set in the account-wide config file. And yes, the real problem that it should be reporting is that the requirements aren't satisfied. Looks like it wants older versions of catalogue and srsly.

[cjones051073 (Chris Jones)]

ah thanks. I had seen this error and your comment in the upstream issue, but hadn't connected the dots to realise what it was telling me w.r.t. the port deps..

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #60483.

[museum-future (Skynet)]

still broken in spacy 3.0.0.dev9

[reneeotten (Renee Otten)]

Has duplicate #60842

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #61009.

[ryandesign (Ryan Carsten Schmidt)]

This is happening because spaCy 2.2.4's requirements include thinc==7.4.0. MacPorts has py-thinc 7.4.1 which does not satisfy that requirement. spaCy 2.3.0 changed the dependency to thinc==7.4.1 so updating spaCy would fix this problem.

However then we see that it also requires catalogue>=0.0.7,<1.1.0 but MacPorts has py-catalogue 2.0.0 which does not satisfy that requirement.

It also requires blis>=0.4.0,<0.5.0 but MacPorts has py-blis 0.7.1.

And it requires srsly>=1.0.2,<1.1.0 but MacPorts has py-srsly 2.3.0.

At least, those are the version ranges of those modules that spaCy 2.3.2 claims to require. I don't know if relaxing those version requirements, as we've done in some other python modules, would work.

[ryandesign (Ryan Carsten Schmidt)]

Upstream has relaxed the blis requirement to blis>=0.4.0,<0.8.0 ​here (on the develop branch where they're working on spaCy 3). This commit only changes that requirement so it may be safe to make on its own.

Upstream has updated the catalogue requirement to catalogue>=2.0.1,<2.1.0 ​here (on the develop branch). This commit comes with a few more changes but they might be unrelated.

Upstream has updated the srsly requirement to srsly>=2.0.0,<3.0.0 ​here (on the develop branch) but this also comes with a bump of the thinc requirement to thinc==8.0.0a0 and comes with a zillion other changes.

[ryandesign (Ryan Carsten Schmidt)]

I'm not sure how we usually handle python module version incompatibilities when multiple different modules depend on another module. But in this case the only port that depends on py-catalogue and py-thinc is py-spaCy, and the only ports that depend on py-blis and py-srsly are py-spaCy and py-thinc, so maybe we should downgrade py-blis, py-catalogue and py-srsly to the versions py-spaCy 2.3.2 wants. I'll see if I can do that.

[ryandesign (Ryan Carsten Schmidt)]

​https://github.com/macports/macports-ports/pull/8651

[ryandesign (Ryan Carsten Schmidt)]

In d12c7d09949b3b154c9d6b648ac53640227e670e/macports-ports (master):

Comment not to update spaCy's deps too far

See: #60474

[ryandesign (Ryan Carsten Schmidt)]

In 23fcf370ebd1da8f3199a264b6fd8f483b2665ba/macports-ports (master):

py-srsly: Downgrade to 1.0.2

Last version supported by py-spaCy 2.3.2.

See: #60474

[ryandesign (Ryan Carsten Schmidt)]

In ec9b1dba0296832c5c14836886cf1dc736200351/macports-ports (master):

py-blis: Downgrade to 0.4.2

Last version supported by py-spaCy 2.3.2.

See: #60474

[ryandesign (Ryan Carsten Schmidt)]

In d7b3123bafc97a55253be166a50fd92aca4a223a/macports-ports (master):

py-catalogue: Downgrade to 1.0.0

Last version supported by py-spaCy 2.3.2.

See: #60474

[ryandesign (Ryan Carsten Schmidt)]

In e500cd298bffb57a775ee9d06a439288f1ae2bef/macports-ports (master):

py-spaCy: Update to 2.3.2

Closes: #60474
Closes: #61010