Incompatible numpy API versions with Python 3.7 and 3.8

[pazner]

I am using PyMFEM through Google Colab, and it looks like they just updated the Python version to 3.8.16 (I believe 3.7.x was the default before).

Now when I try pip install mfem the PyMFEM package is successfully installed, but import mfem.ser as mfem results in

module compiled against API version 0x10 but this version of numpy is 0xe

The numpy version is 1.21.6, which used to be working with the older version of Python.

Updating numpy with pip install -U numpy fixes the problem for me, but I feel like this should be caught at the pip install mfem stage.

It looks like the issue is that the 3.7 binary is built with numpy 1.21.6 (see here) but the 3.8 binary is built with numpy 1.23.3 (see here).

Maybe the numpy version requirements need to be changed in requirements.txt?

(An ideal solution for me would allow the 3.8 binary to work with numpy 1.21.6, since this is what is available by default in Colab, and upgrading to a newer numpy version requires restarting the runtime)

Thanks!

[sshiraiwa]

Yes.... we could change the requirement, which says numpy>=1.20.0; python_version>="3.7" at moment.
There is a bit of complication here, however. If I do
$ pip install numba-scipy
this package requires scipy<=1.7.3 and scipy 1.7.3 has requirement numpy<1.23.0.
So at moment, a good compromise is change Github action for building the binary to use the old numpy as you suggested, although I am bit reluctant to introduce the upper bound in the version number ;D

[pazner]

Would this change to the action work for now?

[sshiraiwa]

@pazner
I suppose yes as long as Numpy API changes are backward compatible. If not, after binary installing MFEM, it may break if a user update numpy. I will check it using PyPI test

[pazner]

Hi @sshiraiwa, were you able to look into this?

[sshiraiwa]

This is on-going. (Sorry, it took a while to work on PR #159)
Right now, I am stack with a strange error. (https://github.com/mfem/PyMFEM/actions/runs/3835018775/jobs/6527909826)
I will take a look tomorrow.

[sshiraiwa]

Hi @pazner
There was an issue relating to pypa/setuptools#3743.
Looks like the proposed change works file. A test binary is in PyPI test repo.
If possible, please try the following command.

 $ pip3 install -i  https://test.pypi.org/simple  mfem==4.5.0.1rc8  --verbose

Please note that all the dependencies need to be installed beforehand. This is because with -i option, pip try to
download all dependencies, such as numpy, from Test PyPI, not the official PyPI

[pazner]

The test binary works for me, thanks @sshiraiwa!