Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect shape computation with getitem and structured numpy array #11022

Open
louisletoumelin opened this issue Mar 25, 2024 · 0 comments
Open
Labels

Comments

@louisletoumelin
Copy link

Dear all,

When trying to access an element from a numpy structured array in dask, the dask array shape is not correctly estimated.

On my real world use case, I try to read lazily from a binary file using np.frombuffer/dask and then save my variables in a xarray dataset.

Thanks,

import dask.array as da
import numpy as np

buffer = da.random.randint(10,size=(100,4), dtype=np.uint8)
structured_array = da.map_blocks(np.frombuffer, buffer, ">H, >B, >B", dtype=">H, >B, >B")
print(structured_array["f1"].shape)  # print (100, 4), not ok
print(structured_array["f1"].compute().shape)  # print (100,), ok

Environment:

  • Dask version: '2023.11.0'
  • Python version: 3.12.2
  • Operating System: Ubuntu 22.04.4 LTS
  • Install method (conda, pip, source): conda
@github-actions github-actions bot added the needs triage Needs a response from a contributor label Mar 25, 2024
@phofl phofl added array and removed needs triage Needs a response from a contributor labels Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants