Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: sparse: Add indexing for 1D arrays #20120

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open

Conversation

dschult
Copy link
Contributor

@dschult dschult commented Feb 20, 2024

Adds support for indexing with 1D arrays.

Builds on #19833 (csr-1d) to provide indexing for 1D CSR arrays, though this code also works for DOK with minor changes after #19715 (dok-1d) gets merged.

The diff will be much easier to read after #19833 is merged. Look at only the last commit to see the diff for just this PR. I will rebase as needed, but I think the CSR PR will be merged soon and the diff here will become cleaner.

  • adds test_indexing1d.py
  • refactors IndexMixin._validate_indices to use an ndim-independent approach for both 1d and 2d.
  • the supported formats will need methods _get_int, _get_slice and _get_array. These are dispatched to after processing the index.
  • this tries to incorporate all improvements from the recently merged defect: sparse: 1d bool mask with wrong shape should raise IndexError #19957 (and doesn't change those tests).
  • the ndim-independent approach means some helper functions are no longer needed and/or have been inlined. The two remaining helper functions are now methods so they have access to self.ndim. They are _as_indices and _compatible_boolean_array.

@github-actions github-actions bot added scipy.sparse Meson Items related to the introduction of Meson as the new build system for SciPy enhancement A new feature or improvement labels Feb 20, 2024
@lucascolley lucascolley removed the Meson Items related to the introduction of Meson as the new build system for SciPy label Feb 20, 2024
Move TestGetSet1d to indexing1d.py. Simplify helper functions.
… fix fancy

This should set us up for nD indexing when the time comes.
Note: ndindex could work for much of this -- but not for sparse array boolean
In the future we could maybe implement sparse integer array indexing.

We should think about whether 0D return should be sparse array or ndarray.
Currently ndarray.

This doesn't make reduction functions return sparse arrays yet -- only indexing
@dschult
Copy link
Contributor Author

dschult commented Mar 18, 2024

I've changed the tests to use np.testing assert_equal and assert_allclose in these new tests. I'll create a separate PR to change the new-but-merged-tests to use them too.

I've also added indexing support for np.newaxis/None (though it can't make 3D+). And that was an excuse to revamp some of the getitem/setitem code to be easier to move to nd. Quanstide's ndindex library would give almost everything we need for validate_indices, but it doesn't support boolean sparse arrays as indexes. Anyway, we can now index 2d and 1d sparse arrays.

I believe the test failures are not related to this PR.

Note: This PR does not change reduction operations to have them return 1D sparse arrays. That will be a separate PR.

Note: Another separate PR: We need to decide how to handle 0D. A[3,4] should return a 0D object as per array_api. But should it return a numpy or scipy.sparse 0D object? In general, an array type "should" return it's own type when indexed. But sparse is not a standard array type -- we don't expect the entire array api to be implemented. But the answer to this question will impact this same code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature or improvement scipy.sparse
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants