Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Releasing: 1.8.6 #16134

Merged
merged 36 commits into from Dec 21, 2022
Merged

Releasing: 1.8.6 #16134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
3652718
chlog continued
Dec 16, 2022
17e15af
[App] PoC: Add support for Request (#16047)
tchaton Dec 16, 2022
d1ca635
Have checkgroup pull the latest runs (#16033)
carmocca Dec 16, 2022
0db6b50
Update Multinode Warning (#16091)
justusschock Dec 16, 2022
87dd800
[App] Serve datatypes with better client code (#16018)
Dec 16, 2022
b519c59
docs: add PT version (#16010)
Borda Dec 17, 2022
ecaac18
add 1.13.1 to adjust versions (#16099)
Borda Dec 17, 2022
1d8ba52
[App] Add display name property to the work (#16095)
tchaton Dec 18, 2022
a9346c8
Fix detection of whether app is running in cloud (#16045)
awaelchli Dec 19, 2022
6996dc8
[App] Add work.delete (#16103)
tchaton Dec 19, 2022
a0f8f70
[App] Improve the autoscaler UI (#16063)
akihironitta Dec 19, 2022
c57af59
[App] Min replica=0 would break autoscaler component (#16092)
Dec 19, 2022
3efeaaa
[App] Scale out/in interval for autoscaler (#16093)
Dec 19, 2022
be4129c
Set the default work start method to spawn on MacOS (#16089)
awaelchli Dec 19, 2022
fb4d01c
[App] Add status endpoint, enable `ready` (#16075)
ethanwharris Dec 19, 2022
0c80e66
Clarify `work.stop()` limitation (#16073)
carmocca Dec 19, 2022
9a08330
Update torchvision requirement from <=0.14.0,>=0.11.1 to >=0.11.1,<0.…
dependabot[bot] Dec 19, 2022
0171f8a
CI: settle file names (#16098)
Borda Dec 19, 2022
2710b4d
[App] Resolve some bugs from the Training Studio scaling (#16114)
tchaton Dec 19, 2022
2376960
Update scikit-learn requirement from <1.1.3,>0.22.1 to >0.22.1,<1.2.1…
dependabot[bot] Dec 20, 2022
716abc9
[App] update app testing (#16000)
tchaton Dec 20, 2022
da1133d
[App] Change overwrite to True (#16009)
tchaton Dec 20, 2022
90822bc
[App] Improve PythonServer info message on startup (#15989)
akihironitta Dec 20, 2022
6697832
[App] Cold start proxy in autoscaler (#16094)
Dec 20, 2022
0800063
Fix endpoint information tab not showing up in AutoScaler UI (#16128)
akihironitta Dec 20, 2022
b0369a1
[App] Implement `ready` for components (#16129)
ethanwharris Dec 20, 2022
86d52df
[App] Fix e2e tests (#16146)
ethanwharris Dec 20, 2022
4475a53
[App] Remove outdated warning from cloud requirements (#16140)
ethanwharris Dec 20, 2022
bce3a15
[App] Fix support for streamlit > 1.14 (#16139)
ethanwharris Dec 20, 2022
677cf93
Fix broken link in "Build a Model" section of docs (#16025)
awaelchli Dec 21, 2022
f5c7b87
drop colossalai from testing as no stable release yet (#16122)
Borda Dec 21, 2022
445404e
[App] Simplify messaging in cloud dispatch (#16160)
ethanwharris Dec 21, 2022
18df651
[App] Add annotations endpoint (#16159)
ethanwharris Dec 21, 2022
16f66e1
Small fix in test_cli.py to avoid failure with future version of json…
mauvilsa Dec 21, 2022
86e69f0
Fix test failing on master due to bad auto-merge (#16118)
awaelchli Dec 19, 2022
384dffa
ver1.8.6
Borda Dec 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/checkgroup.yml
View file
Open in desktop
Expand Up @@ -6,7 +6,7 @@ subprojects:
- id: "pytorch_lightning: Tests workflow"
paths:
- ".actions/**"
- ".github/workflows/ci-pytorch-tests.yml"
- ".github/workflows/ci-tests-pytorch.yml"
- "requirements/lite/**"
- "src/lightning_lite/**"
- "requirements/pytorch/**"
Expand Down Expand Up @@ -178,7 +178,7 @@ subprojects:
- "src/lightning_lite/**"
- "tests/tests_lite/**"
- "setup.cfg" # includes pytest config
- ".github/workflows/ci-lite-tests.yml"
- ".github/workflows/ci-tests-lite.yml"
- "!requirements/*/docs.txt"
- "!*.md"
- "!**/*.md"
Expand Down Expand Up @@ -221,7 +221,7 @@ subprojects:
- id: "lightning_app: Tests workflow"
paths:
- ".actions/**"
- ".github/workflows/ci-app-tests.yml"
- ".github/workflows/ci-tests-app.yml"
- "src/lightning_app/**"
- "tests/tests_app/**"
- "requirements/app/**"
Expand All @@ -243,7 +243,7 @@ subprojects:
- id: "lightning_app: Examples"
paths:
- ".actions/**"
- ".github/workflows/ci-app-examples.yml"
- ".github/workflows/ci-examples-app.yml"
- "src/lightning_app/**"
- "tests/tests_examples_app/**"
- "examples/app_*/**"
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/README.md
View file
Open in desktop
Expand Up @@ -4,10 +4,10 @@

## Unit and Integration Testing

| workflow name | workflow file | action | accelerator\* |
| -------------------------- | ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
| Test PyTorch full | .github/workflows/ci-pytorch-tests.yml | Run all tests except for accelerator-specific, standalone and slow tests. | CPU |
| Test PyTorch slow | .github/workflows/ci-pytorch-tests-slow.yml | Run only slow tests. Slow tests usually need to spawn threads and cannot be speed up or simplified. | CPU |
| workflow name | workflow file | action | accelerator\* |
| ----------------- | -------------------------------------- | ------------------------------------------------------------------------- | ------------- |
| Test PyTorch full | .github/workflows/ci-tests-pytorch.yml | Run all tests except for accelerator-specific, standalone and slow tests. | CPU |

| pytorch-lightning (IPUs) | .azure-pipelines/ipu-tests.yml | Run only IPU-specific tests. | IPU |
| pytorch-lightning (HPUs) | .azure-pipelines/hpu-tests.yml | Run only HPU-specific tests. | HPU |
| pytorch-lightning (GPUs) | .azure-pipelines/gpu-tests-pytorch.yml | Run all CPU and GPU-specific tests, standalone, and examples. Each standalone test needs to be run in separate processes to avoid unwanted interactions between test cases. | GPU |
Expand Down
0 .github/workflows/ci-pytorch-dockers.yml → .github/workflows/ci-dockers-pytorch.yml
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion .github/workflows/ci-app-examples.yml → .github/workflows/ci-examples-app.yml
View file
Open in desktop
Expand Up @@ -9,7 +9,7 @@ on:
types: [opened, reopened, ready_for_review, synchronize] # added `ready_for_review` since draft is skipped
paths:
- ".actions/**"
- ".github/workflows/ci-app-examples.yml"
- ".github/workflows/ci-examples-app.yml"
- "src/lightning_app/**"
- "tests/tests_examples_app/**"
- "examples/app_*/**"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-app-tests.yml → .github/workflows/ci-tests-app.yml
View file
Open in desktop
Expand Up @@ -9,7 +9,7 @@ on:
types: [opened, reopened, ready_for_review, synchronize] # added `ready_for_review` since draft is skipped
paths:
- ".actions/**"
- ".github/workflows/ci-app-tests.yml"
- ".github/workflows/ci-tests-app.yml"
- "src/lightning_app/**"
- "tests/tests_app/**"
- "requirements/app/**"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-lite-tests.yml → .github/workflows/ci-tests-lite.yml
View file
Open in desktop
Expand Up @@ -13,7 +13,7 @@ on:
- "src/lightning_lite/**"
- "tests/tests_lite/**"
- "setup.cfg" # includes pytest config
- ".github/workflows/ci-lite-tests.yml"
- ".github/workflows/ci-tests-lite.yml"
- "!requirements/*/docs.txt"
- "!*.md"
- "!**/*.md"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-pytorch-tests.yml → .github/workflows/ci-tests-pytorch.yml
View file
Open in desktop
Expand Up @@ -14,7 +14,7 @@ on:
- "tests/tests_pytorch/**"
- "tests/legacy/back-compatible-versions.txt"
- "setup.cfg" # includes pytest config
- ".github/workflows/ci-pytorch-tests.yml"
- ".github/workflows/ci-tests-pytorch.yml"
- "requirements/lite/**"
- "src/lightning_lite/**"
- "!requirements/pytorch/docs.txt"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/probot-check-group.yml
View file
Open in desktop
Expand Up @@ -14,7 +14,7 @@ jobs:
if: github.event.pull_request.draft == false
timeout-minutes: 61 # in case something is wrong with the internal timeout
steps:
- uses: Lightning-AI/probot@v5.1
- uses: Lightning-AI/probot@v5.3
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
Expand Down
4 changes: 2 additions & 2 deletions .gitignore
View file
Open in desktop
Expand Up @@ -109,8 +109,8 @@ celerybeat-schedule

# dotenv
.env
.env_staging
.env_local
.env.staging
.env.local

# virtualenv
.venv
Expand Down
18 changes: 9 additions & 9 deletions README.md
View file
Open in desktop
Expand Up @@ -90,15 +90,15 @@ Lightning is rigorously tested across multiple CPUs, GPUs, TPUs, IPUs, and HPUs

<center>

| System / PyTorch ver. | 1.9 | 1.10 | 1.12 (latest) |
| :------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| Linux py3.7 \[GPUs\*\*\] | - | - | - |
| Linux py3.7 \[TPUs\*\*\*\] | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/tpu-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/tpu-tests.yml) | - | - |
| Linux py3.8 \[IPUs\] | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(IPUs)?branchName=master>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=25&branchName=master) | - | - |
| Linux py3.8 \[HPUs\] | - | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(HPUs)?branchName=master>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=26&branchName=master) | - |
| Linux py3.{7,9} | - | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml) |
| OSX py3.{7,9} | - | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml) |
| Windows py3.{7,9} | - | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-pytorch-tests.yml) |
| System / PyTorch ver. | 1.10 | 1.12 |
| :------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| Linux py3.7 \[GPUs\*\*\] | - | - |
| Linux py3.7 \[TPUs\*\*\*\] | - | - |
| Linux py3.8 \[IPUs\] | - | - |
| Linux py3.8 \[HPUs\] | [![Build Status](<https://dev.azure.com/Lightning-AI/lightning/_apis/build/status/pytorch-lightning%20(HPUs)?branchName=master>)](https://dev.azure.com/Lightning-AI/lightning/_build/latest?definitionId=26&branchName=master) | - |
| Linux py3.{7,9} | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |
| OSX py3.{7,9} | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |
| Windows py3.{7,9} | - | [![Test](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml/badge.svg?branch=master&event=push)](https://github.com/Lightning-AI/lightning/actions/workflows/ci-tests-pytorch.yml) |

- _\*\* tests run on two NVIDIA P100_
- _\*\*\* tests run on Google GKE TPUv2/3. TPU py3.7 means we support Colab and Kaggle env._
Expand Down
3 changes: 2 additions & 1 deletion docs/source-app/api_references.rst
View file
Open in desktop
Expand Up @@ -45,7 +45,8 @@ ___________________
~multi_node.lite.LiteMultiNode
~multi_node.pytorch_spawn.PyTorchSpawnMultiNode
~multi_node.trainer.LightningTrainerMultiNode
~auto_scaler.AutoScaler
~serve.auto_scaler.AutoScaler
~serve.auto_scaler.ColdStartProxy

----

Expand Down
2 changes: 2 additions & 0 deletions docs/source-pytorch/index.rst
View file
Open in desktop
Expand Up @@ -64,6 +64,8 @@ Conda users

Or read the `advanced install guide <starter/installation.html>`_

We are fully compatible with any stable PyTorch version v1.10 and above.

.. raw:: html

<hr class="docutils" style="margin: 50px 0 50px 0">
Expand Down
2 changes: 1 addition & 1 deletion docs/source-pytorch/model/build_model.rst
View file
Open in desktop
Expand Up @@ -23,7 +23,7 @@ Build a Model
:header: 2: Validate and test a model
:description: Add a validation and test data split to avoid overfitting.
:col_css: col-md-4
:button_link: validate_model_basic.html
:button_link: ../common/evaluation_basic.html
:height: 150
:tag: basic

Expand Down
4 changes: 4 additions & 0 deletions examples/app_boring/app.py
View file
Open in desktop
Expand Up @@ -43,6 +43,10 @@ def __init__(self):
raise_exception=True,
)

@property
def ready(self) -> bool:
return self.dest_work.is_running

def run(self):
self.source_work.run()
if self.source_work.has_succeeded:
Expand Down
1 change: 1 addition & 0 deletions examples/app_display_name/.lightningignore
View file
Open in desktop
@@ -0,0 +1 @@
venv/
25 changes: 25 additions & 0 deletions examples/app_display_name/app.py
View file
Open in desktop
@@ -0,0 +1,25 @@
import lightning as L


class Work(L.LightningWork):
def __init__(self, start_with_flow=True):
super().__init__(start_with_flow=start_with_flow)

def run(self):
pass


class Flow(L.LightningFlow):
def __init__(self):
super().__init__()
self.w = Work()
self.w1 = Work(start_with_flow=False)
self.w.display_name = "My Custom Name" # Not supported yet
self.w1.display_name = "My Custom Name 1"

def run(self):
self.w.run()
self.w1.run()


app = L.LightningApp(Flow())
17 changes: 7 additions & 10 deletions examples/app_server_with_auto_scaler/app.py
View file
Open in desktop
@@ -1,5 +1,5 @@
# ! pip install torch torchvision
from typing import Any, List
from typing import List

import torch
import torchvision
Expand All @@ -8,16 +8,12 @@
import lightning as L


class RequestModel(BaseModel):
image: str # bytecode


class BatchRequestModel(BaseModel):
inputs: List[RequestModel]
inputs: List[L.app.components.Image]


class BatchResponse(BaseModel):
outputs: List[Any]
outputs: List[L.app.components.Number]


class PyTorchServer(L.app.components.PythonServer):
Expand Down Expand Up @@ -79,10 +75,11 @@ def scale(self, replicas: int, metrics: dict) -> int:
# autoscaler specific args
min_replicas=1,
max_replicas=4,
autoscale_interval=10,
scale_out_interval=10,
scale_in_interval=10,
endpoint="predict",
input_type=RequestModel,
output_type=Any,
input_type=L.app.components.Image,
output_type=L.app.components.Number,
timeout_batching=1,
max_batch_size=8,
)
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Expand Up @@ -79,8 +79,8 @@ module = [
"lightning_app.components.serve.types.image",
"lightning_app.components.serve.types.type",
"lightning_app.components.serve.python_server",
"lightning_app.components.serve.auto_scaler",
"lightning_app.components.training",
"lightning_app.components.auto_scaler",
"lightning_app.core.api",
"lightning_app.core.app",
"lightning_app.core.flow",
Expand Down
2 changes: 0 additions & 2 deletions requirements/app/cloud.txt
View file
Open in desktop
@@ -1,5 +1,3 @@
# WARNING: this file is not used directly by the backend
# any dependency here needs to be shipped with the base image
redis>=4.0.1, <=4.2.4
docker>=5.0.0, <6.0.2
s3fs>=2022.5.0, <2022.8.3
Expand Down
1 change: 1 addition & 0 deletions requirements/app/test.txt
View file
Open in desktop
Expand Up @@ -4,6 +4,7 @@ pytest==7.2.0
pytest-timeout==2.1.0
pytest-cov==4.0.0
pytest-doctestplus>=0.9.0
pytest-asyncio==0.20.3
playwright==1.28.0
httpx
trio<0.22.0
Expand Down
2 changes: 1 addition & 1 deletion requirements/lite/base.txt
View file
Open in desktop
Expand Up @@ -2,7 +2,7 @@
# in case you want to preserve/enforce restrictions on the latest compatible version, add "strict" as an in-line comment

numpy>=1.17.2, <1.23.1
torch>=1.9.0, <=1.13.0
torch>=1.9.0, <=1.13.1
fsspec[http]>2021.06.0, <2022.6.0
packaging>=17.0, <=21.3
typing-extensions>=4.0.0, <=4.4.0
Expand Down
2 changes: 1 addition & 1 deletion requirements/lite/examples.txt
View file
Open in desktop
@@ -1,4 +1,4 @@
# NOTE: the upper bound for the package version is only set for CI stability, and it is dropped while installing this package
# in case you want to preserve/enforce restrictions on the latest compatible version, add "strict" as an in-line comment

torchvision>=0.10.0, <=0.13.0
torchvision>=0.10.0, <=0.14.1