Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable batch size finder for distributed strategies #19789

Open
clumsy opened this issue Apr 19, 2024 · 1 comment
Open

Enable batch size finder for distributed strategies #19789

clumsy opened this issue Apr 19, 2024 · 1 comment
Labels
feature Is an improvement or enhancement needs triage Waiting to be triaged by maintainers

Comments

@clumsy
Copy link
Contributor

clumsy commented Apr 19, 2024

Description & Motivation

It's not clear why it's currently disabled here.

Pitch

There should not be a big difference in how it works vs. LR finder. E.g. all ranks try the same size under try/catch and reduce 1 or 0 based on whether it was successful. This operation is repeated with a given strategy until all ranks are successful.

Alternatives

Manual HPO

Additional context

No response

cc @Borda

@clumsy clumsy added feature Is an improvement or enhancement needs triage Waiting to be triaged by maintainers labels Apr 19, 2024
@clumsy
Copy link
Contributor Author

clumsy commented Apr 19, 2024

Any context on this is greatly appreciated, @awaelchli @carmocca. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Is an improvement or enhancement needs triage Waiting to be triaged by maintainers
Projects
None yet
Development

No branches or pull requests

1 participant