Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

⬆️🔒️ Maintenance/vulnerability upgrade for ujson, upgrade fastapi+starlette #3112

Merged
merged 9 commits into from Sep 20, 2022
Merged

⬆️🔒️ Maintenance/vulnerability upgrade for ujson, upgrade fastapi+starlette #3112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ade9393
upgrades fastapi -> starlette/ujson
pcrespov Jun 16, 2022
d2799f8
adds ujson repo-wide constraints
pcrespov Jul 12, 2022
7516ff2
doc
pcrespov Jul 12, 2022
8ff64eb
make reqs-all upgrade=fastapi
pcrespov Sep 18, 2022
9c18e1a
make reqs-all upgrade=ujson
pcrespov Sep 18, 2022
c5c832c
returns empty strings that cannot be json.loaded
pcrespov Sep 19, 2022
d493137
modified openapi.json: fastapi extended definition of ValididationError
pcrespov Sep 19, 2022
5a1e33d
typo
pcrespov Sep 19, 2022
b49c4cf
@GitHK review: doc
pcrespov Sep 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/service-library/requirements/_fastapi.txt
View file
Open in desktop
Expand Up @@ -14,7 +14,7 @@ certifi==2022.6.15
# httpx
click==8.1.3
# via uvicorn
fastapi==0.82.0
fastapi==0.85.0
# via
# -r requirements/_fastapi.in
# fastapi-contrib
Expand Down Expand Up @@ -54,7 +54,7 @@ sniffio==1.3.0
# anyio
# httpcore
# httpx
starlette==0.19.1
starlette==0.20.4
# via fastapi
threadloop==1.0.2
# via jaeger-client
Expand Down
1 change: 1 addition & 0 deletions requirements/constraints.txt
View file
Open in desktop
Expand Up @@ -18,6 +18,7 @@ pyyaml>=5.4 # https://github.com/advisories/GH
rsa>=4.1 # https://github.com/advisories/GHSA-537h-rv9q-vvph
sqlalchemy[postgresql_psycopg2binary]>=1.3.3 # https://nvd.nist.gov/vuln/detail/CVE-2019-7164
sqlalchemy>=1.3.3 # https://nvd.nist.gov/vuln/detail/CVE-2019-7164
ujson>=5.4.0 # https://github.com/advisories/GHSA-fh56-85cw-5pq6, https://github.com/advisories/GHSA-wpqr-jcpx-745r
urllib3>=1.26.5 # https://github.com/advisories/GHSA-q2q7-5pp4-w6pg

#
Expand Down
21 changes: 17 additions & 4 deletions services/api-server/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -95,7 +95,7 @@ email-validator==1.2.1
# via
# fastapi
# pydantic
fastapi==0.75.0
fastapi==0.85.0
# via
# -r requirements/../../../packages/service-library/requirements/_fastapi.in
# -r requirements/_base.in
Expand Down Expand Up @@ -285,7 +285,7 @@ sqlalchemy==1.4.37
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/postgres-database/requirements/_base.in
# aiopg
# alembic
starlette==0.17.1
starlette==0.20.4
# via fastapi
tenacity==8.0.1
# via
Expand Down Expand Up @@ -317,8 +317,21 @@ typing-extensions==4.3.0
# via
# aiodebug
# pydantic
ujson==4.3.0
# via fastapi
# starlette
ujson==5.5.0
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/./../../../requirements/constraints.txt
# -c requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../requirements/constraints.txt
# -c requirements/../../../requirements/constraints.txt
# fastapi
urllib3==1.26.9
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
3 changes: 3 additions & 0 deletions services/api-server/tests/unit/test__fastapi.py
View file
Open in desktop
Expand Up @@ -114,6 +114,9 @@ def test_fastapi_route_paths_in_paths(client: TestClient, faker: Faker):


def test_fastapi_route_name_parsing(client: TestClient, faker: Faker):
#
# Ensures ':' is allowed in routes
# SEE https://github.com/encode/starlette/pull/1657

solver_key = Solver.Config.schema_extra["example"]["id"]
version = Solver.Config.schema_extra["example"]["version"]
Expand Down
4 changes: 2 additions & 2 deletions services/autoscaling/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -33,7 +33,7 @@ dnspython==2.2.1
# via email-validator
email-validator==1.2.1
# via pydantic
fastapi==0.82.0
fastapi==0.85.0
# via
# -r requirements/../../../packages/service-library/requirements/_fastapi.in
# -r requirements/_base.in
Expand Down Expand Up @@ -114,7 +114,7 @@ sniffio==1.3.0
# anyio
# httpcore
# httpx
starlette==0.19.1
starlette==0.20.4
# via fastapi
tenacity==8.0.1
# via
Expand Down
16 changes: 12 additions & 4 deletions services/catalog/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -51,7 +51,7 @@ email-validator==1.2.1
# via
# fastapi
# pydantic
fastapi==0.71.0
fastapi==0.85.0
# via
# -r requirements/../../../packages/service-library/requirements/_fastapi.in
# -r requirements/_base.in
Expand Down Expand Up @@ -191,7 +191,7 @@ sqlalchemy==1.4.37
# -r requirements/../../../packages/postgres-database/requirements/_base.in
# -r requirements/_base.in
# alembic
starlette==0.17.1
starlette==0.20.4
# via fastapi
tenacity==8.0.1
# via
Expand All @@ -217,8 +217,16 @@ typing-extensions==4.3.0
# aiodebug
# aioredis
# pydantic
ujson==4.3.0
# via fastapi
# starlette
ujson==5.5.0
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/./../../../requirements/constraints.txt
# -c requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../requirements/constraints.txt
# fastapi
urllib3==1.26.9
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
5 changes: 3 additions & 2 deletions services/datcore-adapter/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -54,7 +54,7 @@ ecdsa==0.14.1
# via python-jose
email-validator==1.2.1
# via pydantic
fastapi==0.75.1
fastapi==0.85.0
# via
# -r requirements/../../../packages/service-library/requirements/_fastapi.in
# -r requirements/_base.in
Expand Down Expand Up @@ -196,7 +196,7 @@ sniffio==1.2.0
# anyio
# httpcore
# httpx
starlette==0.17.1
starlette==0.20.4
# via fastapi
tenacity==8.0.1
# via
Expand All @@ -220,6 +220,7 @@ typing-extensions==4.3.0
# via
# aiodebug
# pydantic
# starlette
urllib3==1.26.9
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
2 changes: 1 addition & 1 deletion services/datcore-adapter/src/simcore_service_datcore_adapter/modules/remote_debug.py
View file
Open in desktop
Expand Up @@ -38,4 +38,4 @@ def setup_remote_debugging(force_enabled=False, *, boot_mode=None):
)


__all__ = ["setup_remote_debugging"]
__all__ = ("setup_remote_debugging",)
28 changes: 15 additions & 13 deletions services/datcore-adapter/tests/unit/test_route_files.py
View file
Open in desktop
@@ -1,8 +1,9 @@
# pylint:disable=unused-variable
# pylint:disable=unused-argument
# pylint:disable=redefined-outer-name
# pylint: disable=redefined-outer-name
# pylint: disable=unused-argument
# pylint: disable=unused-variable

from typing import Dict
from typing import Iterator
from unittest.mock import Mock

import httpx
import pytest
Expand All @@ -14,7 +15,9 @@


@pytest.fixture
async def pennsieve_files_mock(pennsieve_subsystem_mock, pennsieve_file_id: str):
async def pennsieve_files_mock(
pennsieve_subsystem_mock: Mock, pennsieve_file_id: str
) -> Iterator[Mock]:
mock = pennsieve_subsystem_mock
if mock:
FAKE_FILE_ID = "123434"
Expand All @@ -39,9 +42,9 @@ async def pennsieve_files_mock(pennsieve_subsystem_mock, pennsieve_file_id: str)

async def test_download_file_entrypoint(
async_client: httpx.AsyncClient,
pennsieve_subsystem_mock,
pennsieve_files_mock,
pennsieve_api_headers: Dict[str, str],
pennsieve_subsystem_mock: Mock,
pennsieve_files_mock: Mock,
pennsieve_api_headers: dict[str, str],
pennsieve_file_id: str,
):
file_id = pennsieve_file_id
Expand All @@ -57,9 +60,9 @@ async def test_download_file_entrypoint(

async def test_delete_file_entrypoint(
async_client: httpx.AsyncClient,
pennsieve_subsystem_mock,
pennsieve_files_mock,
pennsieve_api_headers: Dict[str, str],
pennsieve_subsystem_mock: Mock,
pennsieve_files_mock: Mock,
pennsieve_api_headers: dict[str, str],
pennsieve_file_id: str,
):
file_id = pennsieve_file_id
Expand All @@ -68,5 +71,4 @@ async def test_delete_file_entrypoint(
headers=pennsieve_api_headers,
)
assert response.status_code == status.HTTP_204_NO_CONTENT
data = response.json()
assert not data
assert response.num_bytes_downloaded == 0
23 changes: 19 additions & 4 deletions services/director-v2/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -121,7 +121,7 @@ email-validator==1.2.1
# via
# fastapi
# pydantic
fastapi==0.71.0
fastapi==0.85.0
# via
# -r requirements/../../../packages/service-library/requirements/_fastapi.in
# -r requirements/_base.in
Expand Down Expand Up @@ -370,7 +370,7 @@ sqlalchemy==1.4.37
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/postgres-database/requirements/_base.in
# aiopg
# alembic
starlette==0.17.1
starlette==0.20.4
# via fastapi
tblib==1.7.0
# via
Expand Down Expand Up @@ -416,8 +416,23 @@ typing-extensions==4.3.0
# aiodocker
# aioredis
# pydantic
ujson==4.3.0
# via fastapi
# starlette
ujson==5.5.0
# via
# -c requirements/../../../packages/dask-task-models-library/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/dask-task-models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/./../../../requirements/constraints.txt
# -c requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/simcore-sdk/requirements/../../../requirements/constraints.txt
# -c requirements/../../../requirements/constraints.txt
# fastapi
urllib3==1.26.9
# via
# -c requirements/../../../packages/dask-task-models-library/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
9 changes: 8 additions & 1 deletion services/dynamic-sidecar/openapi.json
View file
Open in desktop
Expand Up @@ -954,7 +954,14 @@
"title": "Location",
"type": "array",
"items": {
"type": "string"
"anyOf": [
{
"type": "string"
},
{
"type": "integer"
}
]
}
},
"msg": {
Expand Down
5 changes: 3 additions & 2 deletions services/dynamic-sidecar/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -124,7 +124,7 @@ docopt==0.6.2
# via docker-compose
email-validator==1.2.1
# via pydantic
fastapi==0.71.0
fastapi==0.85.0
# via
# -r requirements/../../../packages/service-library/requirements/_fastapi.in
# -r requirements/_base.in
Expand Down Expand Up @@ -311,7 +311,7 @@ sqlalchemy==1.4.37
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/postgres-database/requirements/_base.in
# aiopg
# alembic
starlette==0.17.1
starlette==0.20.4
# via fastapi
tenacity==8.0.1
# via
Expand Down Expand Up @@ -344,6 +344,7 @@ typing-extensions==4.3.0
# aiodebug
# aiodocker
# pydantic
# starlette
urllib3==1.26.9
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
11 changes: 9 additions & 2 deletions services/storage/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -217,8 +217,15 @@ typing-extensions==4.3.0
# pydantic
# types-aiobotocore
# types-aiobotocore-s3
ujson==5.3.0
# via aiohttp-swagger
ujson==5.5.0
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../packages/service-library/requirements/./../../../requirements/constraints.txt
# -c requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../requirements/constraints.txt
# aiohttp-swagger
urllib3==1.26.9
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
16 changes: 14 additions & 2 deletions services/web/server/requirements/_base.txt
View file
Open in desktop
Expand Up @@ -335,8 +335,20 @@ typing-extensions==4.3.0
# via
# aiodebug
# pydantic
ujson==5.3.0
# via aiohttp-swagger
ujson==5.5.0
# via
# -c requirements/../../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/service-library/requirements/./../../../requirements/constraints.txt
# -c requirements/../../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/simcore-sdk/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/simcore-sdk/requirements/../../../packages/postgres-database/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/simcore-sdk/requirements/../../../packages/settings-library/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../packages/simcore-sdk/requirements/../../../requirements/constraints.txt
# -c requirements/../../../../requirements/constraints.txt
# aiohttp-swagger
urllib3==1.26.11
# via
# -c requirements/../../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down