py-numpy 2.0.0

[Schamschula (Marius Schamschula)]

We are transitioning from py-numpy 1.26.4 to py-numpy 2.0.0, which was officially released on June 16, 2024.

From the numpy documentation: ​https://numpy.org/devdocs/dev/depending_on_numpy.html#numpy-2-0-specific-advice

So far we have run into the following ports that require >= 2.0.0rc2:

py-astropy 6.1.3: Only needed for PyPI wheels. See: ​https://github.com/macports/macports-ports/commit/9c190e872373cb073b7e3168787575944a054ed5

py-bottleneck 1.4.0: Only needed for PyPI wheels. See: ​https://github.com/macports/macports-ports/pull/25662

py-cartopy 0.23.0: Only needed for PyPI wheels. See: ​https://github.com/macports/macports-ports/commit/cdaae5629bec4ee4f5de51701695d28e8ebef86e

py-matplotlib 3.9.0

py-pyerfa 2.0.1.41: Only needed for PyPI wheels. See: ​https://github.com/macports/macports-ports/commit/1cb2757245ac645d5669299a765635cc9337ba35

py-sunpy 6.0.1: Only needed for PyPI wheels. See: ​https://github.com/macports/macports-ports/commit/6c882c5fe7c78d98ff47974931bc5b8a27dbf733

py-vispy 0.14.3: Only needed for PyPI wheels. See: ​https://github.com/macports/macports-ports/commit/dc64d200891ff5e80281097b7cc8c8e310d81457

py-wavelets 1.7.0

Updates have been held back until py-numpy >= 2.0.0 is available.

I will update this ticket as I run into additional ports.

[reneeotten (Renee Otten)]

latest version of py-matplotlib also requires it, and I have held back the update

[Marius Schamschula <mschamschula@…>]

In a55e7512c9aed21eccf0694a57c5d926031da80b/macports-ports (master):

py-tropycal: revert to version 1.2.1

See: #70251

[stromnov (Andrey Stromnov)]

py-bottleneck 1.4.0 (​https://github.com/pydata/bottleneck/blob/v1.4.0/pyproject.toml#L17)

py-wavelets 1.7.0 (​https://github.com/PyWavelets/pywt/blob/v1.7.0/pyproject.toml#L19)

[reneeotten (Renee Otten)]

I am working on this update and will put up a PR once I've finished up a few remaining things. NumPy v2.0.1 supports PY39, the latest upstream version v2.1.0 supports PY310 and above.

I would propose the following:

1. keep the current version as py-numpy-legacy, this has to conflict with the latest versions as Python packages cannot be installed side-by-side.
2. list all dependents and:

• remove all subports below 39 where possible, if needed going down the dependency chain
• if a port depends on Python < 3.9 and is deemed to be valuable and cannot be removed, then it should depend on pyXY-numpy-legacy
• update all ports if a newer upstream version is available, or if not, rebuild the port with NumPy v2.0 and enable/run the test-suite to verify that it still works correctly. If no issues are reported by using ruff check path/to/code/ --select NPY201 I think we assume that things are likely to work.

I *think* this is the only way it can work, but am open to comments/suggestions.

I will update here with a link to the PR once that is all fixed and we can start adding ports/updates there and once we are fairly convinced most of the things will work, then we can merge. Additionally, once that's the case people who are interested in older systems should verify that this all still works and if not provide patches to make that happen if possible. The build system has changed to meson and upstream is using a vendored version with NumPy specific changes. Our meson port has quite a few changes specifically intended to support older OSes, but integrating these modified meson versions sounds complicated and not something I am willing to invest time in.

[reneeotten (Renee Otten)]

Same considerations/issues apply to py-scipy as well. If we want to make any progress in keeping things up-to-date we will have to drop unsupported Python versions. Also here the build system was switched to meson and there is no point patching around that any longer; is someone wants older systems supported, then please do the work on getting meson to do TheRightThing there. I will try to put up an initial PR for NumPy and SciPy - and will try to clean-up Python subports below 3.9 to make things easier.

However, if there are expectations to keep older Python versions/OSes supported then I am not even going to bother; there is no sustainable way to do that given that upstream aggressively drops things.

[Schamschula (Marius Schamschula)]

It turns out that

[Schamschula (Marius Schamschula)]

Even though py-tifffile specifies numpy 2.1.0 in the release notes, it builds correctly with numpy < 2.0.0.

[Schamschula (Marius Schamschula)]

py-tropycal works with numpy < 2.0.0

[Marius Schamschula <mschamschula@…>]

In bbbcd2096b9409e39855703dd663408ddcef618f/macports-ports (master):

py-bottleneck: update to 1.4.0

Work around PyPI py-numpy >= 2.0.0 requirement
See: ​https://github.com/pydata/bottleneck/blob/master/pyproject.toml
See: #70251

[jjstickel (Jonathan Stickel)]

How is the progress going for updating numpy to 2.X?

[reneeotten (Renee Otten)]

the py-numpy and py-scipy ports were recently completely transitioned to the meson build system with proper support for BLAS/LAPACK libraries and such. So in that sense, we should be ready to look at the update to NumPy v2. It's on my to-do-list but it does involve making sure that all dependent ports build against that new version, and, if not, to update/fix those to resolve that.

My plan is to put up a PR with the NumPy update in the next few days and then port maintainers can test "their" ports to see if they build against it and/or fix them and then add them to the PR. If you're willing to help out in that process that would be great!

[jjstickel (Jonathan Stickel)]

Yes, I can test a PR. More specifically, I can test which numpy/scipy ecosystem ports that I use work with numpy-2 and which need attention.

[Schamschula (Marius Schamschula)]

I've just upgraded to numpy 2.1.3 on my main build machine. Remember numpy >= 2.0.0 is supposed to be backwardly compatible.

[reneeotten (Renee Otten)]

Replying to Schamschula:

I've just upgraded to numpy 2.1.3 on my main build machine. Remember numpy >= 2.0.0 is supposed to be backwardly compatible.

No, ​it is not:

This major release includes breaking changes that could not happen in a
regular minor (feature) release - including an ABI break, changes to
type promotion rules, and API changes which may not have been emitting
deprecation warnings in 1.26.x. Key documents related to how to adapt to
changes in NumPy 2.0, in addition to these release notes, include:
....

You are right that building with NumPy v2 will result in packages that still work with NumPy v1 - but that is irrelevant for MacPorts as we will never provide both versions. What needs to be checked is that -especially older- packages do not rely on changes in NumPy’s Python and C APIs. One way to do that is checking source code with: $ ruff check path/to/code/ --select NPY201.

A compatibility table is ​here, which would help to triage what needs to be done for MacPorts.

[reneeotten (Renee Otten)]

I started doing work in this and can put up a work-on-progress PR. There are several packages though that will not (yet) work but people who want to test things and help going through the remaining packages might find it useful.

[jjstickel (Jonathan Stickel)]

Replying to reneeotten:

I started doing work in this and can put up a work-on-progress PR. There are several packages though that will not (yet) work but people who want to test things and help going through the remaining packages might find it useful.

Can you please provide a link to the PR? I know I could search, but I want to make sure I got the right one.

[reneeotten (Renee Otten)]

Replying to jjstickel:

Replying to reneeotten:

I started doing work in this and can put up a work-on-progress PR. There are several packages though that will not (yet) work but people who want to test things and help going through the remaining packages might find it useful.

Can you please provide a link to the PR? I know I could search, but I want to make sure I got the right one.

there is ​a branch on my fork with the latest NumPy version and a collections of commits to rebuild packages that need to be, and some updates for others where upstream now supports the latest version 2.x Please note that it has been a while since I started this so some packages have seen updates again in the meantime; but the ones that are in the branch should work with NumPy v2 (and thus newer ones as well).

Some commits show "WIP" and those are not (yet) supported; I've attached here a list of packages that are known to not work, the ones that I have checked and need an update/revbump, the remainder of the list still needs to be checked.

[eborisch (Eric A. Borisch)]

For tracking: py-h5py 3.14.0 requires Numpy 2.0 to build — but I'm planning to patch around that and exclude the new/pseudo-optional (NumPy variable-width strings) functionality for now.