Add function to explicitly mark forward methods in Fabric #19690

awaelchli · 2024-03-24T00:15:41Z

What does this PR do?

Fixes #19608
Fixes #19495

Adds a new method mark_forward_method to the wrapper class:

model = fabric.setup(model)
model.mark_forward_method(model.generate)

# Now can use `model.generate()` in DDP/FSDP without issues:
prediction = model.generate(input)

See the added docs page and linked issues for why this is important.

📚 Documentation preview 📚: https://pytorch-lightning--19690.org.readthedocs.build/en/19690/

cc @Borda @carmocca @justusschock @awaelchli

src/lightning/fabric/wrappers.py

for more information, see https://pre-commit.ci

…/lightning into feature/mark-forward-method

for more information, see https://pre-commit.ci

…/lightning into feature/mark-forward-method

github-actions · 2024-04-04T00:20:56Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 2.0, oldest)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.1)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.2)	success	✅
pl-cpu (macOS-14, lightning, 3.10, 2.3)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 2.0, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.2)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.3)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 2.0, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.1)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.2)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.3)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 2.0)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 2.0)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.1)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.1)	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs) (testing Lightning \| latest)	success	✅
pytorch-lightning (GPUs) (testing PyTorch \| latest)	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py.

🟢 pytorch_lightning: Benchmarks

Check ID	Status
lightning.Benchmarks	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py.

🟢 fabric: Docs

Check ID	Status
docs-make (fabric, doctest)	success	✅
docs-make (fabric, html)	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, docs/source-fabric/api/fabric_methods.rst, docs/source-fabric/api/wrappers.rst, docs/source-fabric/glossary/index.rst.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 2.0, oldest)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
fabric-cpu (macOS-11, lightning, 3.11, 2.1)	success	✅
fabric-cpu (macOS-11, lightning, 3.11, 2.2)	success	✅
fabric-cpu (macOS-14, lightning, 3.10, 2.3)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 2.0, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.2)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.3)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 2.0, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.2)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.3)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 2.0)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 2.0)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.0)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.0)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.1)	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, tests/tests_fabric/test_wrappers.py.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs) (testing Fabric \| latest)	success	✅
lightning-fabric (GPUs) (testing Lightning \| latest)	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py, tests/tests_fabric/test_wrappers.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.11)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.11)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.11)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.11)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.11)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.11)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.11)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.11)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.11)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.11)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.11)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.11)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.11)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.11)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.11)	success	✅

These checks are required after the changes to src/lightning/fabric/wrappers.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

codecov · 2024-04-04T00:41:06Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59%. Comparing base (0c8a193) to head (bb21cc8).

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #19690     +/-   ##
=========================================
- Coverage      84%      59%    -25%     
=========================================
  Files         425      420      -5     
  Lines       34900    34813     -87     
=========================================
- Hits        29275    20448   -8827     
- Misses       5625    14365   +8740

src/lightning/fabric/wrappers.py

docs/source-fabric/api/wrappers.rst

lantiga

LGTM

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>

mark forward method

7425e1e

github-actions bot added the fabric lightning.fabric.Fabric label Mar 24, 2024

awaelchli commented Mar 24, 2024

View reviewed changes

src/lightning/fabric/wrappers.py Outdated Show resolved Hide resolved

awaelchli and others added 2 commits March 23, 2024 20:16

Update src/lightning/fabric/wrappers.py

f40b290

[pre-commit.ci] auto fixes from pre-commit.com hooks

d01babd

for more information, see https://pre-commit.ci

awaelchli added feature Is an improvement or enhancement fun Staff contributions outside working hours - to differentiate from the "community" label labels Mar 24, 2024

awaelchli added this to the 2.3 milestone Mar 24, 2024

awaelchli added 5 commits March 26, 2024 01:52

delete

e31210c

Merge branch 'master' into feature/mark-forward-method

aed73aa

Merge branch 'master' into feature/mark-forward-method

8ce9068

docs

358d725

the long explanation

6a0823b

github-actions bot added the docs Documentation related label Apr 3, 2024

awaelchli and others added 9 commits April 4, 2024 01:52

update

92cdef5

[pre-commit.ci] auto fixes from pre-commit.com hooks

84b11c8

for more information, see https://pre-commit.ci

update

e87916f

Merge branch 'feature/mark-forward-method' of github.com:Lightning-AI…

f8c3084

…/lightning into feature/mark-forward-method

update

0a8ff61

test corner cases

8c3c468

[pre-commit.ci] auto fixes from pre-commit.com hooks

44cfdc1

for more information, see https://pre-commit.ci

chlog

e0cabd2

Merge branch 'feature/mark-forward-method' of github.com:Lightning-AI…

8ebe728

…/lightning into feature/mark-forward-method

awaelchli marked this pull request as ready for review April 4, 2024 00:20

awaelchli requested review from carmocca, justusschock, lantiga and Borda as code owners April 4, 2024 00:20

update test

285a161

update

70cf896

carmocca approved these changes Apr 4, 2024

View reviewed changes

src/lightning/fabric/wrappers.py Show resolved Hide resolved

awaelchli added 2 commits April 5, 2024 03:29

Merge branch 'master' into feature/mark-forward-method

a4154d5

edge case

98967d6

justusschock approved these changes Apr 8, 2024

View reviewed changes

mergify bot added the ready PRs ready to be merged label Apr 8, 2024

Merge branch 'master' into feature/mark-forward-method

a5b12b4

rasbt reviewed May 8, 2024

View reviewed changes

docs/source-fabric/api/wrappers.rst Outdated Show resolved Hide resolved

rasbt reviewed May 8, 2024

View reviewed changes

docs/source-fabric/api/wrappers.rst Outdated Show resolved Hide resolved

lantiga approved these changes May 8, 2024

View reviewed changes

awaelchli and others added 4 commits May 8, 2024 21:44

Merge branch 'master' into feature/mark-forward-method

54ce4d6

Update docs/source-fabric/api/wrappers.rst

219c56a

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>

Update docs/source-fabric/api/wrappers.rst

361fb2c

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>

fix docs render backticks

bb21cc8

awaelchli merged commit e030727 into master May 8, 2024
117 of 118 checks passed

awaelchli deleted the feature/mark-forward-method branch May 8, 2024 20:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add function to explicitly mark forward methods in Fabric #19690

Add function to explicitly mark forward methods in Fabric #19690

awaelchli commented Mar 24, 2024 •

edited by github-actions bot

github-actions bot commented Apr 4, 2024 •

edited

codecov bot commented Apr 4, 2024 •

edited

lantiga left a comment

Add function to explicitly mark forward methods in Fabric #19690

Add function to explicitly mark forward methods in Fabric #19690

Conversation

awaelchli commented Mar 24, 2024 • edited by github-actions bot

What does this PR do?

github-actions bot commented Apr 4, 2024 • edited

⚡ Required checks status: All passing 🟢

Groups summary

codecov bot commented Apr 4, 2024 • edited

Codecov Report

lantiga left a comment

Choose a reason for hiding this comment

awaelchli commented Mar 24, 2024 •

edited by github-actions bot

github-actions bot commented Apr 4, 2024 •

edited

codecov bot commented Apr 4, 2024 •

edited