Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Releasing/1.8.4 #15870

Merged
merged 54 commits into from Dec 7, 2022
Merged

Releasing/1.8.4 #15870

merged 54 commits into from Dec 7, 2022

Conversation

Borda
Copy link
Member

@Borda Borda commented Nov 30, 2022

What does this PR do?

TBD

Before submitting

  • Was this discussed/approved via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

cc @Borda

@Borda Borda marked this pull request as ready for review November 30, 2022 08:06
@Borda Borda added the release label Nov 30, 2022
@Borda Borda added this to the v1.8.x milestone Nov 30, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Nov 30, 2022

⛈️ Required checks status: Has failure 🔴

⚠️ This job will need to be re-run to merge your PR. If you do not have write access to the repository you can ask Lightning-AI/lai-frameworks to re-run it for you. If you push a new commit, all of CI will re-trigger ⚠️ If you have any other questions, you can reach out to carmocca for help.

Groups summary

🔴 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-11, pytorch, 3.8, 1.11) success
pl-cpu (macOS-11, pytorch, 3.9, 1.12) success
pl-cpu (macOS-11, pytorch, 3.10, 1.13) success
pl-cpu (macOS-11, pytorch, 3.8, 1.9, oldest) success
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.10) success
pl-cpu (ubuntu-20.04, pytorch, 3.9, 1.11) success
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.12) success
pl-cpu (ubuntu-20.04, pytorch, 3.10, 1.13) failure
pl-cpu (ubuntu-20.04, pytorch, 3.7, 1.9, oldest) success
pl-cpu (windows-2022, pytorch, 3.9, 1.11) success
pl-cpu (windows-2022, pytorch, 3.10, 1.12) success
pl-cpu (windows-2022, pytorch, 3.10, 1.13) success
pl-cpu (windows-2022, pytorch, 3.7, 1.9, oldest) success
pl-cpu (slow, macOS-11, pytorch, 3.7, 1.11) success
pl-cpu (slow, ubuntu-20.04, pytorch, 3.7, 1.11) success
pl-cpu (slow, windows-2022, pytorch, 3.7, 1.11) success
pl-cpu (macOS-11, lightning, 3.8, 1.13) success
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.13) success
pl-cpu (windows-2022, lightning, 3.8, 1.13) success
🟢 pytorch_lightning: Azure GPU
Check ID Status
pytorch-lightning (GPUs) success
🟢 pytorch_lightning: Benchmarks
Check ID Status
pytorch-lightning.Benchmark success
🟢 pytorch_lightning: Azure HPU
Check ID Status
pytorch-lightning (HPUs) success
🟢 pytorch_lightning: Azure IPU
Check ID Status
pytorch-lightning (IPUs) success
🟢 pytorch_lightning: Docs
Check ID Status
make-doctest (pytorch) success
make-html (pytorch) success
🔴 pytorch_lightning: Docker
Check ID Status
build-cuda (3.9, 1.10, 11.3.1) success
build-cuda (3.9, 1.11, 11.3.1) success
build-cuda (3.9, 1.12, 11.6.1) success
build-cuda (3.9, 1.13, 11.6.1) in_progress
build-hpu (1.5.0, 1.11.0) success
build-ipu (3.9, 1.9) success
build-NGC failure
build-pl (3.9, 1.10, 11.3.1) success
build-pl (3.9, 1.11, 11.3.1) success
build-pl (3.9, 1.12, 11.6.1) success
build-pl (3.9, 1.13, 11.6.1) success
build-xla (3.7, 1.12) success
🟢 lightning_lite: CPU workflow
Check ID Status
lite-cpu (macOS-11, lite, 3.8, 1.11) success
lite-cpu (macOS-11, lite, 3.9, 1.12) success
lite-cpu (macOS-11, lite, 3.10, 1.13) success
lite-cpu (macOS-11, lite, 3.7, 1.9, oldest) success
lite-cpu (ubuntu-20.04, lite, 3.8, 1.10) success
lite-cpu (ubuntu-20.04, lite, 3.9, 1.11) success
lite-cpu (ubuntu-20.04, lite, 3.10, 1.12) success
lite-cpu (ubuntu-20.04, lite, 3.10, 1.13) success
lite-cpu (ubuntu-20.04, lite, 3.7, 1.9, oldest) success
lite-cpu (windows-2022, lite, 3.9, 1.11) success
lite-cpu (windows-2022, lite, 3.10, 1.12) success
lite-cpu (windows-2022, lite, 3.10, 1.13) success
lite-cpu (windows-2022, lite, 3.7, 1.9, oldest) success
lite-cpu (macOS-11, lightning, 3.8, 1.13) success
lite-cpu (ubuntu-20.04, lightning, 3.8, 1.13) success
lite-cpu (windows-2022, lightning, 3.8, 1.13) success
🔴 lightning_lite: Azure GPU
Check ID Status
lightning-lite (GPUs) failure
🔴 lightning_app: Tests workflow
Check ID Status
app-pytest (macOS-11, app, 3.8, latest) failure
app-pytest (macOS-11, app, 3.8, oldest) failure
app-pytest (macOS-11, lightning, 3.9, latest) failure
app-pytest (ubuntu-20.04, app, 3.8, latest) success
app-pytest (ubuntu-20.04, app, 3.8, oldest) success
app-pytest (ubuntu-20.04, lightning, 3.9, latest) success
app-pytest (windows-2022, app, 3.8, latest) success
app-pytest (windows-2022, app, 3.8, oldest) success
app-pytest (windows-2022, lightning, 3.8, latest) cancelled 🚫
🟢 lightning_app: Examples
Check ID Status
app-examples (macOS-11, app, 3.9, latest) success
app-examples (macOS-11, app, 3.9, oldest) success
app-examples (macOS-11, lightning, 3.9, latest) success
app-examples (ubuntu-20.04, app, 3.9, latest) success
app-examples (ubuntu-20.04, app, 3.9, oldest) success
app-examples (ubuntu-20.04, lightning, 3.9, latest) success
app-examples (windows-2022, app, 3.9, latest) success
app-examples (windows-2022, app, 3.9, oldest) success
app-examples (windows-2022, lightning, 3.9, latest) success
🔴 lightning_app: Azure
Check ID Status
App.cloud-e2e failure
🟢 lightning_app: Docs
Check ID Status
make-doctest (app) success
make-html (app) success
🟢 mypy
Check ID Status
mypy success
🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.7) success
install-pkg (ubuntu-22.04, app, 3.10) success
install-pkg (ubuntu-22.04, lite, 3.7) success
install-pkg (ubuntu-22.04, lite, 3.10) success
install-pkg (ubuntu-22.04, pytorch, 3.7) success
install-pkg (ubuntu-22.04, pytorch, 3.10) success
install-pkg (ubuntu-22.04, lightning, 3.7) success
install-pkg (ubuntu-22.04, lightning, 3.10) success
install-pkg (macOS-12, app, 3.7) success
install-pkg (macOS-12, app, 3.10) success
install-pkg (macOS-12, lite, 3.7) success
install-pkg (macOS-12, lite, 3.10) success
install-pkg (macOS-12, pytorch, 3.7) success
install-pkg (macOS-12, pytorch, 3.10) success
install-pkg (macOS-12, lightning, 3.7) success
install-pkg (macOS-12, lightning, 3.10) success
install-pkg (windows-2022, app, 3.7) success
install-pkg (windows-2022, app, 3.10) success
install-pkg (windows-2022, lite, 3.7) success
install-pkg (windows-2022, lite, 3.10) success
install-pkg (windows-2022, pytorch, 3.7) success
install-pkg (windows-2022, pytorch, 3.10) success
install-pkg (windows-2022, lightning, 3.7) success
install-pkg (windows-2022, lightning, 3.10) success

This comment was automatically generated and updates for 60 minutes every 180 seconds.

Thank you for your contribution! 💜

Borda and others added 10 commits November 30, 2022 09:10
* rename _examples dir

* refactor

* clean

* path

* add inits

* skip

* e2e

* azure

* e2e

* rev

* unify single depth for ignore docs req.

* group

(cherry picked from commit 59fa320)
split examples and pytests

(cherry picked from commit 952b64b)
…Flow in the cloud (#15800)

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* updte

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update src/lightning_app/CHANGELOG.md

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* Update src/lightning_app/utilities/port.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
(cherry picked from commit 8ca6dfe)
…quirements (#15420)

Update lightning-utilities requirement in /requirements

Updates the requirements on [lightning-utilities](https://github.com/Lightning-AI/utilities) to permit the latest version.
- [Release notes](https://github.com/Lightning-AI/utilities/releases)
- [Changelog](https://github.com/Lightning-AI/utilities/blob/main/CHANGELOG.md)
- [Commits](Lightning-AI/utilities@v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: lightning-utilities
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit e150d08)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
(cherry picked from commit 8ee889b)
ethanwharris and others added 5 commits November 30, 2022 09:12
…ice such as GPU CUDA / MPS (#15813)

(cherry picked from commit 4e64391)
(cherry picked from commit 1e56b75)
(cherry picked from commit cca3432)
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
(cherry picked from commit 6a2a83a)
Borda and others added 6 commits December 2, 2022 11:12
* prune dependency for benchmarks
* drop

(cherry picked from commit 993bd67)
* fixing legacy checkpoints
* Apply suggestions from code review

Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
(cherry picked from commit fee52f9)
(cherry picked from commit f4fcad3)
* waiting
* builds

(cherry picked from commit a86584d)
* drop name column from cluster list

* change create cluster to accept id as well

* rename validator

* remove cluster name from logs

* fix merge with master

* more merge with master issues

(cherry picked from commit a82be2f)
@Borda Borda changed the title Releasing/1.8.4 [wip] Releasing/1.8.4 Dec 5, 2022
@Borda
Copy link
Member Author

Borda commented Dec 5, 2022

seem the failing PL-cpu test are green but the CI/check ends-up red 🦦

rlizzo and others added 9 commits December 6, 2022 03:33
* initial work on deleting apps

* after PR review

* delete CLI working

* restructred to make tests easier

* revert manifest changes

* added changelog, fix mypy issue

* updates

* Update src/lightning_app/cli/cmd_apps.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Sherin Thomas <sherin@lightning.ai>

* Update src/lightning_app/cli/lightning_cli_delete.py

Co-authored-by: Sherin Thomas <sherin@lightning.ai>

* import typing

* adding tests

* finished adding tests

* addressed code review comments

* fix mypy error

* make mypy happy

* make mypy happy

* make mypy happy

* make mypy happy

* fix windows cli

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: Sherin Thomas <sherin@lightning.ai>

(cherry picked from commit b4d99e3)
* Add `is_headless` when dispatching in the cloud

* Bump cloud version

* Add tests

* Dont open app page for headless apps locally

* Refactor

* Update CHANGELOG.md

* Support dynamic UIs at runtime

* Comments

* Fix

* Updates

* Fixes and cleanup

* Fix tests

* Dont open view page for headless apps

* Fix test, resolve URL the right way

* Remove launch

* Clean

* Cleanup tests

* Fixes

* Updates

* Add test

* Increase app cloud tests timeout

* Increase timeout

* Wait for running

* Revert timeouts

* Clean

* Dont update if it hasnt changed

* Increase timeout

(cherry picked from commit 32cf1fa)
(cherry picked from commit ab022ac)
* Direct support for compiled models

* Update test

* Update src/pytorch_lightning/core/module.py

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
(cherry picked from commit 2992002)
* update
* param
* Apply suggestions from code review

(cherry picked from commit 77006a2)
@Borda
Copy link
Member Author

Borda commented Dec 6, 2022

@tchaton see like all e2e tests are failing after picking your last commit, mind having a look?

tchaton and others added 4 commits December 7, 2022 10:04
(cherry picked from commit 6f54a82)
* Enable back inference mode support with hpu
* Remove unused
* Update document link and address comment

Signed-off-by: Jerome <janand@habana.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

(cherry picked from commit 6aaac8b)
* Exlucde __pycache__ in setuptools

* Add load balancer example

* wip

* Update example

* rename

* remove prints

* _LoadBalancer -> LoadBalancer

* AutoScaler(work)

* change var name

* remove locust

* Update docs

* include autoscaler in api ref

* docs typo

* docs typo

* docs typo

* docs typo

* remove unused loadtest

* remove unused device_type

* clean up

* clean up

* clean up

* Add docstring

* type

* env vars to args

* expose an API for users to override to customise autoscaling logic

* update example

* comment

* udpate var name

* fix scale mechanism and clean up

* Update exampl

* ignore mypy

* Add test file

* .

* update impl and update tests

* Update changlog

* .

* revert docs

* update test

* update state to keep calling 'flow.run()'

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>

* Add aiohttp to base requirements

* Update docs

Co-authored-by: Luca Antiga <luca.antiga@gmail.com>

* Use deserializer utility

* fake trigger

* wip: protect /system/* with basic auth

* read password at runtime

* Change env var name

* import torch as optional

* Don't overcreate works

* simplify imports

* Update example

* aiohttp

* Add work_args work_kwargs

* More docs

* remove FIXME

* Apply Jirka's suggestions

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clean example device

* add comment on init threshold value

* bad merge

* nit: logging format

* {in,out}put_schema -> {in,out}put_type

* lowercase

* docs on seconds

* process_time -> processing_time

* Dont modify work state from flow

* Update tests

* worker_url -> endpoint

* fix exampl

* Fix default scale logic

* Fix default scale logic

* Fix num_pending_works

* Update num_pending_works

* Fix bug creating too many works

* Remove up/downscale_threshold args

* Update example

* Add typing

* Fix example in docstring

* Fix default scale logic

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* rename method

* rename locvar

* Add todo

* docs ci

* docs ci

* asdfafsdasdf pls docs

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* .

* doc

* Update src/lightning_app/components/auto_scaler.py

Co-authored-by: Noha Alon <nohalon@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"

This reverts commit 24983a0.

* Revert "Update src/lightning_app/components/auto_scaler.py"

This reverts commit 56ea78b.

* Remove redefinition

* Remove load balancer run blocker

* raise RuntimeError

* remove has_sent

* lower the default timeout_batching from 10 to 1

* remove debug

* update the default timeout_batching

* .

* tighten condition

* fix endpoint

* typo in runtimeerror cond

* async lock update severs

* add a test

* {in,out}put_type typing

* Update examples/app_server_with_auto_scaler/app.py

Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>

* Update .actions/setup_tools.py

Co-authored-by: Aniket Maurya <theaniketmaurya@gmail.com>
Co-authored-by: Luca Antiga <luca.antiga@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Noha Alon <nohalon@gmail.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Akihiro Nitta <aki@pop-os.localdomain>
Co-authored-by: thomas chaton <thomas@grid.ai>

(cherry picked from commit 64b19fb)
rlizzo and others added 3 commits December 7, 2022 17:08
fixed conflicts

(cherry picked from commit 2041908)
* Fix LRScheduler import for PyTorch 2.0
* Add comment for posterity

(cherry picked from commit de93167)
Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@lantiga lantiga merged commit 0ba28c8 into release/stable Dec 7, 2022
@lantiga lantiga deleted the releasing/1.8.4 branch December 7, 2022 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet