New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AMM: Don't schedule tasks to paused workers #5431
Conversation
5be5309
to
5698018
Compare
05a12e5
to
8bf6b0e
Compare
253d397
to
4c55e9b
Compare
4c55e9b
to
df7c9b0
Compare
I further broke down this PR. It's now ready for review. |
client_msgs: dict = {} | ||
worker_msgs: dict = {} | ||
parent._transitions(recs, client_msgs, worker_msgs) | ||
self.send_all(client_msgs, worker_msgs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
client_msgs: dict = {} | |
worker_msgs: dict = {} | |
parent._transitions(recs, client_msgs, worker_msgs) | |
self.send_all(client_msgs, worker_msgs) | |
self.transitions(recs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aren't client_msgs and worker_msgs filled in place by _transitions()? I fell for this already before.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, if you're using the underscored method, you'll need to call the send yourself. If you use the non-underscored one this will be done. You basically just copied what the non-underscored method does,
distributed/distributed/scheduler.py
Lines 7145 to 7155 in f3aa9d1
def transitions(self, recommendations: dict): | |
"""Process transitions until none are left | |
This includes feedback from previous transitions and continues until we | |
reach a steady state | |
""" | |
parent: SchedulerState = cast(SchedulerState, self) | |
client_msgs: dict = {} | |
worker_msgs: dict = {} | |
parent._transitions(recommendations, client_msgs, worker_msgs) | |
self.send_all(client_msgs, worker_msgs) |
Co-authored-by: Florian Jetter <fjetter@users.noreply.github.com>
In scope
Follow-ups
Out of scope