Replies: 1 comment
-
@mishooax Welcome! I'm sorry to redirect you again, Dask uses Discourse for usage questions. So, I've moved this question to Discourse here: https://dask.discourse.group/t/fast-dask-parallel-in-place-modification-of-xr-dataset-data-specific-dimensions/1072 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
[ Re-posted here following advice from the xarray-dev community: https://github.com/pydata/xarray/discussions/6951 ]
Hi there,
I am trying to come up with a fast, Dask-parallel way of "shuffling" the contents of an xr.Dataset (or Datarray) along certain dimensions. An example below:
So the call to map_blocks seems to work as expected but it's likely slower than it has to be - and the manual loop is ugly.
I've also tried to use
xr.apply_ufunc
:This is faster than the
map_blocks
call but i get warnings about dask creating large chunks - likely as a result of thevindex
call.What's the best way to vectorize the "shuffle" (in-place assignment, really) using Dask?
Beta Was this translation helpful? Give feedback.
All reactions