[Tracker] Implement all numpy.* APIs in CuPy

[kmaehashi]

Implement GPU version of numpy.* functions in cupy.* namespace.

This is a tracker issue that lists the remaining numpy.* APIs (see also: comparison table). I've categorized them based on difficulty so that new contributors can pick the right task. Your contribution is highly welcomed and appreciated!

List of APIs

Very Easy

Beta Give feedback

1. numpy.asfarray (Add cupy.asfarray #6085)
2. numpy.byte_bounds (hint: ndarray.data.ptr) (Add cupy.byte_bounds #7015)
3. numpy.format_float_positional (added cupy.format_float_positional #6308)
4. numpy.format_float_scientific (Add cupy.format_float_scientific API #6474)
5. numpy.ndarray.searchsorted (added cupy.ndarray.searchsorted #7059)

Easy

Beta Give feedback

1. numpy.apply_over_axes
2. numpy.array_equiv (Add array_equiv API. #6254)
3. numpy.asarray_chkfinite (Add asarray_chkfinite API. #6275)
4. numpy.fabs (Add cupy.fabs API. #6282)
5. numpy.float_power (added api for cupy.float_power #6371)
6. numpy.heaviside (add cupy.heaviside api. #6798)
7. numpy.isneginf (Add cupy.isneginf and cupy.isposinf #6089)
8. numpy.isposinf (Add cupy.isneginf and cupy.isposinf #6089)
9. numpy.mask_indices (Add cupy.mask_indices #6156)
10. numpy.ndarray.itemset (Implement numpy.ndarray.itemset #7459)
11. numpy.real_if_close (Add cupy.real_if_close API #6475)
12. numpy.setdiff1d (Add cupy.setdiff1d api. #6433)
13. numpy.setxor1d (add cupy.setxor1d api #6582)
14. numpy.tril_indices & numpy.tril_indices_from (Add tril_indices and tril_indices_from API. #6305)
15. numpy.triu_indices & numpy.triu_indices_from (Add triu_indices and triu_indices_from API. #6316)
16. numpy.union1d (Add cupy.union1d API. #6357)
17. numpy.lib.format.*
18. numpy.lib.stride_tricks.sliding_window_view (Provide sliding_window_view #6956, Add cupy.lib.stride_tricks.sliding_window_view #7575)
19. numpy.emath.* (numpy.lib.scimath.*) (add cupy.emath module #7295)

Medium

Beta Give feedback

1. numpy.block
2. numpy.delete (Implement delete function, add documentation #7359)
3. numpy.geomspace
4. numpy.insert (add cupy.insert api #6597)
5. numpy.put_along_axis
6. numpy.row_stack (Add cupy.row_stack API. #6312)
7. numpy.spacing
8. numpy.vander (Add cupy.vander API. #6279)

Medium to Hard

Beta Give feedback

1. numpy.histogram_bin_edges
2. numpy.ediff1d (Add cupy.ediff1d API. #6280)
3. numpy.intersect1d (Add cupy.intersect1d API. #6402, Add cupy.intersect1d API #6407)
4. numpy.nanpercentile
5. numpy.nanquantile
6. numpy.ndarray.ctypes (note: needs design discussion)
7. numpy.ndarray.getfield
8. numpy.ndarray.resize
9. numpy.ndarray.setfield
10. numpy.polynomial.*
11. numpy.poly (Add cupy.poly #3547, Add cupy.poly #6697)
12. numpy.polyder (Add cupy.polyder API. #6469)
13. numpy.polydiv (Add cupy.polydiv #3780)
14. numpy.polyint
15. numpy.trapz (Add cupy.trapz #6107)
16. numpy.linalg.cond
17. numpy.linalg.eig
18. numpy.linalg.eigvals
19. numpy.linalg.multi_dot (Add linalg.multi_dot API. #6358)
20. numpy.random.Generator.* (see the dedicated tracker issue for details: [Tracker] Add random distributions to new Generator #4557)

Low priority (iterator functions)

Beta Give feedback

1. numpy.ndenumerate
2. numpy.nditer
3. numpy.nested_iters

Low priority (help functions)

Beta Give feedback

1. numpy.info
2. numpy.lookfor
3. numpy.source

Low priority (internal functions)

Beta Give feedback

1. numpy.deprecate
2. numpy.deprecate_with_doc

Low priority (dtype APIs - need to filter types unsupported by CuPy)

Beta Give feedback

1. numpy.maximum_sctype
2. numpy.cast (undocumented API) (hint: see numpy/core/numerictypes.py)
3. numpy.typecodes (undocumented API)
4. numpy.sctypeDict (undocumented API)
5. numpy.sctypes (undocumented API)
6. numpy.nbytes (undocumented API)

Low priority (rarely used APIs)

Beta Give feedback

1. numpy.einsum_path (Add cupy.einsum_path #6723)
2. numpy.frompyfunc (maybe just call GUFunc?)

Steps to Contribute

1. Fork and star ⭐ the CuPy repository 😉

2. Pick a function you want to work on. You can find the function in the NumPy API Reference to understand what should be implemented.

3. Implement a function in your branch. If you need help, join Gitter or just ask for help in this issue.

4. Don't forget to write a test code!

5. Build CuPy and run tests to confirm that the function runs fine:
   pip install -e . && pytest tests/cupy_tests/PATH_TO_YOUR_TEST
   See the Contribution Guide for details.

6. Submit a pull-request to the main branch.

Note that you will need a GPU environment to develop CuPy.

See also:

• [Tracker] Implement all scipy.* APIs in CuPy #6324
• [Tracker] Fix unimplemented options #6692

[khushi-411]

Hi @kmaehashi,
I am glad to look into the awesome collection of required APIs.

I was wondering if other APIs for constant functions like numpy.inf, numpy.ninf etc can also be added to the cupy library. Curious to know your viewpoints. Thanks!

NumPy Reference: https://numpy.org/devdocs/reference/constants.html.

[kmaehashi]

@khushi-411 Thanks for pull-requests! All constants are imported in CuPy 😄
https://github.com/cupy/cupy/blob/v10.0.0rc1/cupy/__init__.py#L56-L73

[jakirkham]

cc @leofang @grlee77 @rgommers

[rgommers]

There's a whole bunch of very questionable stuff here, including things that are deprecated. It does not make sense to me to add those. But rather than spend time arguing about that, let me just go work on a PR to remove them from NumPy:)

[jakirkham]

Yeah some of the easy and very easy things read that way to me

The medium & medium to hard seem ok though (with the exception of the typing stuff)

The iterating functions (particularly on GPUs) I'm not sure about, but maybe Leo has thoughts there.

Anything else stick out to you, Ralf? Does the above sound roughly correct to you or are there other things that stick out?

[leofang]

@rgommers Could you list what is deprecated/inappropriate? I thought @kmaehashi has avoided listing them.

[rgommers]

loads and ndfromtxt were already removed in numpy/numpy#19615, so can be removed from the list.

The medium & medium to hard seem ok though (with the exception of the typing stuff)

With "typing stuff" do you mean the dtype-related functions (cast, typecodes, etc.)? If so, then I agree - that's questionable. Also xref numpy/numpy#17325 for that.

The iterating functions (particularly on GPUs) I'm not sure about, but maybe Leo has thoughts there.

Anything else stick out to you, Ralf? Does the above sound roughly correct to you or are there other things that stick out?

That sounds about right. The random and linalg stuff is important, as are some of the medium-hard functions like put_along_axis, nanquantile, nanpercentile, etc.

All of the undocumented ones I'd like to get rid of in NumPy.

[jakirkham]

The medium & medium to hard seem ok though (with the exception of the typing stuff)

With "typing stuff" do you mean the dtype-related functions (cast, typecodes, etc.)? If so, then I agree - that's questionable. Also xref numpy/numpy#17325 for that.

Yeah exactly. Sorry that was vague

[rgommers]

alen and asscalar will be removed in 1.23.0 (probably, unless it still gets backported to 1.22.x) by numpy/numpy#20414

[rgommers]

See also:

https://github.com/numpy/numpy/blob/main/numpy/tests/test_public_api.py#L33-L54

[kmaehashi]

@rgommers Thank you for all the information. I've removed deprecated ones from the list:

• numpy.loads
• numpy.ndfromtxt
• numpy.alen
• numpy.asscalar
• numpy.ndarray.tostring
• numpy.set_numeric_ops

Also marked dtype-related functions as low priority as there's an ongoing discussion in NumPy.

[knshkp]

i also want to contribute in this library but when i seen this all issues are resolved so could you please assign me a tasks

[Nordicus]

I've noticed that the automatic domain functions (numpy.emath.* or numpy.lib.scimath.*) seem to be missing from this. Is this a mistake, or are they not supposed to be supported?
https://numpy.org/doc/stable/reference/routines.emath.html

[kmaehashi]

@Nordicus Yes it's just because I overlooked it, thanks for the catch! 😄

[pri1311]

Hi team, I was looking into implementing cupy.emath.* and I believe the implementation is pretty simple. And since all our functions sqrt, log, etc are implemented using ufunc and already support complex numbers it is just a matter of converting any negatives into complex numbers like in numpy with no other changes required. Is my thinking correct? Pardon my negligence if any.

[kmaehashi]

@pri1311 Hi, I agree it should be that simple.

[rajveer43]

hey @kmaehashi , I would like to work on numpy.ndarray.resize

[kmaehashi]

Hi @rajveer43, sure go ahead.

[rajveer43]

Hi @rajveer43, sure go ahead.

I know how to add that function but..can you tell the locations i.e. path..where it is needed to be added? I am getting confused actually!