Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add numpy 2 support #340

Merged
merged 36 commits into from
Apr 16, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
ba5b0d9
Try out numpy 2 beta release
MuellerSeb Mar 14, 2024
3ad422a
Remove py38; use np 2 rc
MuellerSeb Apr 4, 2024
8c52b9a
try running with np 2; remove 32bit wheels
MuellerSeb Apr 4, 2024
7e51f1e
cap black version
MuellerSeb Apr 4, 2024
9618c1c
project: sphinx version fix; black min py version fix
MuellerSeb Apr 5, 2024
4de1557
estimator: use int instead of long
MuellerSeb Apr 5, 2024
7971261
np: use asarray where possible with atleast_Xd to mimic ndmin feature
MuellerSeb Apr 5, 2024
3b35caa
cython: always use 64-bit integers for counts
MuellerSeb Apr 5, 2024
bcf22c0
deps: emcee try patch version
MuellerSeb Apr 5, 2024
f354999
re-enable py38; np2 to build extensions for py>=3.9; check np2 in CI
MuellerSeb Apr 6, 2024
e52cc9b
CI: check several versions of numpy
MuellerSeb Apr 7, 2024
70a59fb
Doc: fix pyvista issue with renaming UniformGrid->ImageData
MuellerSeb Apr 7, 2024
7b8aab3
Doc: use rtd theme >v2
MuellerSeb Apr 7, 2024
b3b4fb5
Doc: fix ref issues
MuellerSeb Apr 7, 2024
a13968e
CI: try fix missing pip step
MuellerSeb Apr 7, 2024
74b130a
CI: try fix missing pip step by separate job step
MuellerSeb Apr 7, 2024
5f37aee
CI: try fix missing pip step by using env var for version
MuellerSeb Apr 7, 2024
ea00a48
CI: try fix missing pip step by using oldest-supported-numpy package
MuellerSeb Apr 7, 2024
ebe8ef6
CI: fix missing pip step by using quotes (>= is evil)
MuellerSeb Apr 7, 2024
9074056
CI: install numpy in test stage to ensure version
MuellerSeb Apr 7, 2024
dacc89a
deps: bump min np ver to 1.20 (meshio needs np.typing)
MuellerSeb Apr 7, 2024
4353f6e
build: use build dep caps from scipy
MuellerSeb Apr 7, 2024
d221248
CI: also fix scipy version
MuellerSeb Apr 7, 2024
ba5bfb1
Doc: fix pyvista deprecation warnings
MuellerSeb Apr 7, 2024
552cb58
CI: fix config
MuellerSeb Apr 7, 2024
728adee
fix typo
MuellerSeb Apr 7, 2024
227efb6
update changelog
MuellerSeb Apr 8, 2024
83f8594
finalize pyproject.toml
MuellerSeb Apr 8, 2024
1124580
CI: install scipy/numpy before test to really check compatibility
MuellerSeb Apr 8, 2024
3bff873
update docs with correct deps versions
MuellerSeb Apr 8, 2024
24fd5ca
update actions versions
MuellerSeb Apr 11, 2024
9a754fa
cython: use np.int64_t
MuellerSeb Apr 12, 2024
7b1fd1e
use black 24 and add black preview job to CI
MuellerSeb Apr 12, 2024
062caa8
apply black 24
MuellerSeb Apr 12, 2024
50e6672
fix black preview complain
MuellerSeb Apr 12, 2024
25e002e
CI: emcee 3.1.5 with numpy 2 support was released
MuellerSeb Apr 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
57 changes: 36 additions & 21 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,14 @@ jobs:
source_check:
name: source check
runs-on: ubuntu-latest
strategy:
fail-fast: false

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: '0'

- name: Set up Python 3.9
uses: actions/setup-python@v2
uses: actions/setup-python@v5
with:
python-version: 3.9

Expand All @@ -38,6 +36,10 @@ jobs:
run: |
python -m black --check --diff --color .

- name: black preview
run: |
python -m black --preview --diff --color .

- name: isort check
run: |
python -m isort --check --diff --color .
Expand All @@ -58,60 +60,73 @@ jobs:
matrix:
cfg:
- { os: ubuntu-latest, arch: x86_64 }
- { os: ubuntu-latest, arch: i686 }
- { os: windows-latest, arch: AMD64 }
- { os: windows-latest, arch: x86 }
- { os: macos-latest, arch: x86_64 }
- { os: macos-latest, arch: arm64 }
- { os: macos-latest, arch: universal2 }

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: '0'

- name: Build wheels
uses: pypa/cibuildwheel@v2.16.2
uses: pypa/cibuildwheel@v2.17.0
env:
CIBW_ARCHS: ${{ matrix.cfg.arch }}
with:
output-dir: dist

- uses: actions/upload-artifact@v2
- uses: actions/upload-artifact@v3
with:
path: ./dist/*.whl

build_sdist:
name: sdist on ${{ matrix.os }} with py ${{ matrix.python-version }}
name: sdist on ${{ matrix.os }} with py ${{ matrix.ver.py }} numpy${{ matrix.ver.np }} scipy${{ matrix.ver.sp }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']

# https://github.com/scipy/oldest-supported-numpy/blob/main/setup.cfg
ver:
- {py: '3.8', np: '==1.20.0', sp: '==1.5.4'}
- {py: '3.9', np: '==1.20.0', sp: '==1.5.4'}
- {py: '3.10', np: '==1.21.6', sp: '==1.7.2'}
- {py: '3.11', np: '==1.23.2', sp: '==1.9.2'}
- {py: '3.12', np: '==1.26.2', sp: '==1.11.2'}
- {py: '3.12', np: '>=2.0.0rc1', sp: '>=1.13.0'}
Comment on lines -85 to +98
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awful! But I guess there is no other way until py 3.11 is deprecated?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What exactly is awful? I think it is pretty neat to test multiple numpy/scipy versions to fix stuff like #246 faster in the future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a lot of manual (and typo prone?) work. Just imagine we have to add another few packages to that version matrix.

And it also makes it more work to locally reproduce specific setups of users.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a new Python version is release we have to add it by hand either way. And then I would just add the first version of numpy and scipy that support that new python version. That is how I selected the numpy and scipy versions for the python version together with a bleeding edge job for the latest versions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to keep it like this for now. If we encounter version incompatibilities in the future, we can rethink this.

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: '0'

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
- name: Set up Python ${{ matrix.ver.py }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
python-version: ${{ matrix.ver.py }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build "coveralls>=3.0.0"

- name: bleeding edge
if: matrix.ver.np == '>=2.0.0rc1'
run: |
python -m pip install git+https://github.com/dfm/emcee.git

- name: Install GSTools
env:
GSTOOLS_BUILD_PARALLEL: 1
run: |
python -m pip install --upgrade pip
pip install build coveralls>=3.0.0
pip install -v --editable .[test]

- name: Run tests
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
pip install "numpy${{ matrix.ver.np }}" "scipy${{ matrix.ver.sp }}"
python -m pytest --cov gstools --cov-report term-missing -v tests/
python -m coveralls --service=github

Expand All @@ -120,8 +135,8 @@ jobs:
# PEP 517 package builder from pypa
python -m build --sdist --outdir dist .

- uses: actions/upload-artifact@v2
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.9'
- uses: actions/upload-artifact@v3
if: matrix.os == 'ubuntu-latest' && matrix.ver.py == '3.9'
with:
path: dist/*.tar.gz

Expand All @@ -130,7 +145,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/download-artifact@v2
- uses: actions/download-artifact@v3
with:
name: artifact
path: dist
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ src/gstools/_version.py
# generated docs
docs/source/examples/
docs/source/api/
docs/source/sg_execution_times.rst

# other settings
.vscode/
Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,25 @@

All notable changes to **GSTools** will be documented in this file.

## [Unreleased] - ?

### Enhancements

- added global variable `config.NUM_THREADS` to select number of threads for parallel computation ([#336](https://github.com/GeoStat-Framework/GSTools/pull/336))
- prepare numpy 2 support ([#340](https://github.com/GeoStat-Framework/GSTools/pull/340))
- numpy 2.0.0rc1 for building extensions (for Python 3.9 and above)
- check multiple numpy and scipy versions in CI
- fixed minimal versions for numpy
- use `np.asarray` everywhere with `np.atleast_(n)d`
- fix long/longlong integer issue in cython on windows by always using 64bit integers

### Bugfixes
- build docs with latest sphinx version ([#340](https://github.com/GeoStat-Framework/GSTools/pull/340))

### Changes
- require pyvista 0.40 at least ([#340](https://github.com/GeoStat-Framework/GSTools/pull/340))


## [1.5.1] - Nifty Neon - 2023-11

### Enhancements
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ in memory for immediate 3D plotting in Python.

## Requirements:

- [NumPy >= 1.14.5](https://www.numpy.org)
- [NumPy >= 1.20.0](https://www.numpy.org)
- [SciPy >= 1.1.0](https://www.scipy.org/scipylib)
- [hankel >= 1.0.0](https://github.com/steven-murray/hankel)
- [emcee >= 3.0.0](https://github.com/dfm/emcee)
Expand All @@ -366,7 +366,7 @@ You can contact us via <info@geostat-framework.org>.

## License

[LGPLv3][license_link] © 2018-2021
[LGPLv3][license_link] © 2018-2024

[pip_link]: https://pypi.org/project/gstools
[conda_link]: https://docs.conda.io/en/latest/miniconda.html
Expand Down
7 changes: 3 additions & 4 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,10 +259,9 @@ def setup(app):
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {
"Python": ("https://docs.python.org/", None),
"NumPy": ("http://docs.scipy.org/doc/numpy/", None),
"SciPy": ("http://docs.scipy.org/doc/scipy/reference", None),
"matplotlib": ("http://matplotlib.org", None),
"Sphinx": ("http://www.sphinx-doc.org/en/stable/", None),
"NumPy": ("https://numpy.org/doc/stable/", None),
"SciPy": ("https://docs.scipy.org/doc/scipy/", None),
"matplotlib": ("https://matplotlib.org/stable/", None),
"hankel": ("https://hankel.readthedocs.io/en/latest/", None),
"emcee": ("https://emcee.readthedocs.io/en/latest/", None),
}
Expand Down
2 changes: 1 addition & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ in memory for immediate 3D plotting in Python.
Requirements
============

- `Numpy >= 1.14.5 <http://www.numpy.org>`_
- `NumPy >= 1.20.0 <http://www.numpy.org>`_
- `SciPy >= 1.1.0 <http://www.scipy.org>`_
- `hankel >= 1.0.0 <https://github.com/steven-murray/hankel>`_
- `emcee >= 3.0.0 <https://github.com/dfm/emcee>`_
Expand Down
1 change: 1 addition & 0 deletions examples/00_misc/00_tpl_stable.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
\sigma^2_{\ell_{\mathrm{up}}} &=
C\cdot\frac{\ell_{\mathrm{up}}^{2H}}{2H}
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/00_misc/01_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

These can be viewed for example with `Paraview <https://www.paraview.org/>`__.
"""

# sphinx_gallery_thumbnail_path = 'pics/paraview.png'
import gstools as gs

Expand Down
1 change: 1 addition & 0 deletions examples/00_misc/02_check_rand_meth_sampling.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
Check Random Sampling
---------------------
"""

import numpy as np
from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
Expand Down
1 change: 1 addition & 0 deletions examples/00_misc/04_herten.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
functions, since the only produce the ``herten_transmissivity.gz``
and ``grid_dim_origin_spacing.txt``, which are already present.
"""

import os

import matplotlib.pyplot as plt
Expand Down
1 change: 1 addition & 0 deletions examples/00_misc/05_standalone_field.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
In the following example we will produce 10000 random points in 4D with
random values and plot them.
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/01_random_field/03_unstr_srf_export.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
Normally, such a grid would be read in, but we can simply generate one and
then create a random field at those coordinates.
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/01_random_field/04_srf_merge.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
to merge two unstructured rectangular fields.

"""

# sphinx_gallery_thumbnail_number = 2
import numpy as np

Expand Down
5 changes: 3 additions & 2 deletions examples/01_random_field/06_pyvista_support.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
The :any:`Field.mesh` method enables easy field creation on PyVista meshes
used by the :any:`SRF` or :any:`Krige` class.
"""

# sphinx_gallery_thumbnail_path = 'pics/GS_pyvista_cut.png'
import pyvista as pv

Expand All @@ -21,8 +22,8 @@
# We create a structured grid with PyVista containing 50 segments on all three
# axes each with a length of 2 (whatever unit).

dim, spacing = (50, 50, 50), (2, 2, 2)
grid = pv.UniformGrid(dim, spacing)
dims, spacing = (50, 50, 50), (2, 2, 2)
grid = pv.ImageData(dimensions=dims, spacing=spacing)

###############################################################################
# Now we set up the SRF class as always. We'll use an anisotropic model.
Expand Down
1 change: 1 addition & 0 deletions examples/02_cov_model/01_basic_methods.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
correlation function as demonstrated in the introductory example.
If one of the above functions is given, the others will be determined:
"""

import gstools as gs

model = gs.Exponential(dim=3, var=2.0, len_scale=10, nugget=0.5)
Expand Down
1 change: 1 addition & 0 deletions examples/02_cov_model/02_aniso_rotation.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
represents the isotropic case for the model.
Nevertheless, you can provide anisotropy ratios by:
"""

import gstools as gs

model = gs.Gaussian(dim=3, var=2.0, len_scale=10, anis=0.5)
Expand Down
1 change: 1 addition & 0 deletions examples/02_cov_model/03_spectral_methods.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@

You can access these methods by:
"""

import gstools as gs

model = gs.Gaussian(dim=3, var=2.0, len_scale=10)
Expand Down
1 change: 1 addition & 0 deletions examples/02_cov_model/04_different_scales.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

You can access it by:
"""

import gstools as gs

model = gs.Stable(dim=3, var=2.0, len_scale=10)
Expand Down
1 change: 1 addition & 0 deletions examples/02_cov_model/05_additional_para.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

This leads to the so called **stable** covariance model and we can define it by
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/02_cov_model/06_fitting_para_ranges.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
variogram data. In the following we will use the self defined stable model
from a previous example.
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/03_variogram/00_fit_variogram.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
Fit Variogram
-------------
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/03_variogram/01_find_best_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
Finding the best fitting variogram model
----------------------------------------
"""

import numpy as np
from matplotlib import pyplot as plt

Expand Down
1 change: 1 addition & 0 deletions examples/03_variogram/02_multi_vario.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
In this example, we demonstrate how to estimate a variogram from multiple
fields on the same point-set that should have the same statistical properties.
"""

import matplotlib.pyplot as plt
import numpy as np

Expand Down
1 change: 1 addition & 0 deletions examples/03_variogram/03_directional_2d.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

Afterwards we will fit a model to this estimated variogram and show the result.
"""

import numpy as np
from matplotlib import pyplot as plt

Expand Down
1 change: 1 addition & 0 deletions examples/03_variogram/04_directional_3d.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

Afterwards we will fit a model to this estimated variogram and show the result.
"""

import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d import Axes3D
Expand Down
1 change: 1 addition & 0 deletions examples/03_variogram/05_auto_fit_variogram.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
Fit Variogram with automatic binning
------------------------------------
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/03_variogram/06_auto_bin_latlon.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

We use a data set from 20 meteo-stations choosen randomly.
"""

import numpy as np

import gstools as gs
Expand Down
1 change: 1 addition & 0 deletions examples/04_vector_field/00_2d_vector_field.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
As a first example we are going to generate a 2d vector field with a Gaussian
covariance model on a structured grid:
"""

import numpy as np

import gstools as gs
Expand Down