New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for Array API #3037
Comments
For some more context on this. @Zac-HD and I discussed this a bit at some point last year, when I started working on the array API test suite. At the time Zac was open to the idea, but it hasn't yet been implemented. I have since developed quite a bit of the array API test suite, which uses hypothesis extensively. However, the parts of the suite that generate arrays currently only generate constant arrays, because the arrays() strategy hard-codes NumPy. We do not want NumPy to be a dependency of the test suite (actually, it unfortunately currently is because we use the For those strategies that would be used in the array API test suite ( |
I'd be very happy to ship a (e.g.)
CC @rsokl; I know you're busy but probably also interested. |
Glad to hear this could see a future inside Hypothesis :) Regarding stability, having an external package coexist sounds good. My impression has been that array creation via And yeah the I'll be figuring out the implementation details for now externally but I will get to work on a |
Hello folks, I want to ascertain whether Hypothesis is interested in having generalised "library-agnostic" strategies for the Array API libraries (NumPy, TensorFlow, PyTorch, MXNet, JAX, Dask & CuPy are listed as primary stakeholders). If so I would be up to implement such strategies and open a PR in a few weeks, but I would need guidance.
I've been developing such strategies at honno/hypothesis-array-api with heavy reference to
hypothesis.extra.numpy
and the related internal test suite. These strategies have no dependencies and just assume an Array API-compliant has been monkey-patched to the variablearray_module
.No library 100% adopts the standard right now (NumPy is getting close numpy/numpy#18585) but only using some key parts of the API should get us a powerful
arrays()
strategy. In areas of non-compliance I've been throwing errors on missing required attributes/methods and warning users when we can still generate some things. For example PyTorch doesn't support all the unsigned integers specified in the Array API (onlyuint8
) , so if a user uses theunsigned_integers_dtypes()
strategy (with no arguments) only theuint8
dtype will be generated and the user is warned that the dtypesuint16
,uint32
&uint64
are not available.My biggest concern is how users should tell Hypothesis what Array API library to use. My current plan is to have the array module as an optional kwarg in the strategies, which if not specified defaults to a global variable specified by a
register_array_module()
method.Do note that the limited feature set of the API means functionality from
hypothesis.extra.numpy
could not be achieved through a library-agnostic approach. Additionally helpful properties of the NumPy strategies, such asarray_shapes()
not accepting dimensions above 32 due to NumPy's limits, would either require some checks on runtime or be a nicety scrapped altogether... maybe a purely library-agnostic approach first would let us determine if library-specific checks could be included nicely or not. My first thought is just keeping thenumpy
extra as-is and having anarrays
submodule or something be standalone.So yeah, I'm interested to hear if these strategies could see a future inside Hypothesis, and otherwise I'd generally appreciate input. My priority is to make honno/hypothesis-array-api feature complete and emulate
hypothesis.extra.numpy
concepts like fill values inarrays()
, and then if appropriate I'll work on a PR.Please ask me any questions or if you need clarification on something! I'll cc @asmeurer as they tasked me to create library-agnostic Array API strategies to extend the use of Hypothesis in the Array API's compliance suite data-apis/array-api-tests and may have some ideas.
The text was updated successfully, but these errors were encountered: