Data Management
#7945
Replies: 2 comments 1 reply
-
Hi Joe, can you say a bit more about what you mean by data management and
data scheduling?
These terms can mean a variety of different things to different people.
…On Mon, Jul 26, 2021 at 10:13 PM Joseph Curtin ***@***.***> wrote:
Hi, I'm curious about the current roadmap for data management in Dask. Is
there a document outlining how data is distributed and what features are
planned for the future of data scheduling?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#7945>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACKZTHLRBSGTKKN43FX5QLTZYP4NANCNFSM5BBKM4YA>
.
|
Beta Was this translation helpful? Give feedback.
1 reply
-
Here is one documentation page that might help?:
https://distributed.dask.org/en/latest/memory.html , but in general the
question "how does dask handle data?" has lots of complex answers. There
isn't a single overarching explanation for everything that it does.
Maybe this page is useful? If not you might want to look through the other
pages at distributed.dask.org (which tends to be more technical than
docs.dask.org) If you have a specific problem that you're running into
then I encourage you to share it. Then folks here will be more effective
at pointing you in the right direction.
…On Tue, Jul 27, 2021 at 9:21 AM Joseph Curtin ***@***.***> wrote:
Hi Matt, yeah. That is part of the question I'm trying to iron out. I'm
not sure how to ask the question correctly and I am trying to identify the
terms I should be using here.
I used the word scatter because I'm aware of Data Scatter
<https://distributed.dask.org/en/latest/locality.html#data-scatter>, but
I'm not aware of the overall architectural design or roadmap to implement
data-transmission to different workers. Is there one available?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#7945 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACKZTFNEPWS7ADUURXPEQ3TZ26FPANCNFSM5BBKM4YA>
.
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi, I'm curious about the current roadmap for data management in Dask. Is there a document outlining how data is distributed and what features are planned for the future of data scheduling/scatter/distribution?
Beta Was this translation helpful? Give feedback.
All reactions