Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Unscale the gradients before gradient clipping in manual optimization #19536

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

WIP: Unscale the gradients before gradient clipping in manual optimization #19536

wants to merge 1 commit into from
+1 −0

Conversation

awaelchli
Copy link
Member

@awaelchli awaelchli commented Feb 27, 2024 •
edited by github-actions bot

What does this PR do?

Fixes #18089

📚 Documentation preview 📚: https://pytorch-lightning--19536.org.readthedocs.build/en/19536/

@awaelchli awaelchli added bug Something isn't working precision: amp Automatic Mixed Precision labels Feb 27, 2024
@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Feb 27, 2024
@awaelchli awaelchli added fun Staff contributions outside working hours - to differentiate from the "community" label and removed pl Generic label for PyTorch Lightning package labels Feb 27, 2024
@awaelchli awaelchli added this to the 2.2.x milestone Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@williamFalcon williamFalcon Awaiting requested review from williamFalcon williamFalcon will be requested when the pull request is marked ready for review williamFalcon is a code owner

@tchaton tchaton Awaiting requested review from tchaton tchaton will be requested when the pull request is marked ready for review tchaton is a code owner

@carmocca carmocca Awaiting requested review from carmocca carmocca will be requested when the pull request is marked ready for review carmocca is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
bug Something isn't working fun Staff contributions outside working hours - to differentiate from the "community" label precision: amp Automatic Mixed Precision
Projects
None yet
Milestone
2.2.x
Development

Successfully merging this pull request may close these issues.

Example of gradient clipping with manual optimization does not handle gradient unscaling properly
1 participant
@awaelchli