⬆️🔒️ Maintenance/vulnerability upgrade for ujson, upgrade fastapi+starlette

[pcrespov]

What do these changes do?

• 🔒️⬆️ Upgrades ujson to overcome vulnerabilities CVE-2021-45958, GHSA-fh56-85cw-5pq6
• ⬆️ Affected ujson dependencies were introduced byfastapi. By upgrading the latter, it pushed mainly the versions of starlette and ujson. The problem is that starlette broke parsing : in routes Fix regression on route paths with colons encode/starlette#1675 (see failing tests in services/api-server/tests/unit/test__fastapi.py). That issue was fixed in starlette and released in starlette==0.20.2, BUT fastapi still has starlette==0.19.1 which should be resolved with When a url contains colon, it makes 404 response. After I upgrade fastapi from 0.75.0 to 0.77.1. tiangolo/fastapi#4892.

ON HOLD -> recently released https://github.com/tiangolo/fastapi/releases/tag/0.85.0

Related issue/s

• maintenance/scaling of the platform osparc-issues#428
• M1–9 Cybersecurity and privacy osparc-issues#509

How to test

• services/api-server/tests/unit/test__fastapi.py

Checklist

• wait for fastapi to release with starlette>=0.20.2 -> https://github.com/tiangolo/fastapi/releases/tag/0.85.0
• add repo-wide constraint
• upgrade director? won't do
• resolve this vulnerability alarm in project panel (once merged)

Highlights on updated libraries (only updated libraries are included)

• #packages before: 3
• #packages after : 3

#	name	before	after	upgrade	count	packages
1	fastapi	0.71.0, 0.75.0, 0.75.1, 0.82.0	0.85.0	minor	7	api-server⬆️ autoscaling⬆️ catalog⬆️ datcore-adapter⬆️ director-v2⬆️ dynamic-sidecar⬆️ service-library🧪
2	starlette	0.19.1, 0.17.1	0.20.4	minor	7	api-server⬆️ autoscaling⬆️ catalog⬆️ datcore-adapter⬆️ director-v2⬆️ dynamic-sidecar⬆️ service-library🧪
3	ujson	4.3.0, 5.3.0	5.5.0	minor	5	api-server⬆️ catalog⬆️ director-v2⬆️ storage⬆️ web⬆️

Legend:

• ⬆️ base dependency (only services because packages are floating)
• 🧪 test dependency
• 🔧 tool dependency

Repo-wide overview of libraries

• #reqs files parsed: 60

#	name	versions-base	versions-test	versions-tool
1	aio-pika	6.8.0, 7.2.0	6.8.0, 8.2.0
2	aioboto3	9.6.0	10.0.0
3	aiobotocore	2.3.0, 2.3.3	2.3.4
4	aiocache	0.11.1	0.11.1
5	aiodebug	2.3.0	2.3.0
6	aiodocker	0.19.1, 0.21.0	0.21.0
7	aiofiles	0.8.0, 22.1.0	22.1.0
8	aiohttp	3.8.1	3.8.1
9	aiohttp-jinja2	1.5
10	aiohttp-security	0.4.0
11	aiohttp-session	2.11.0
12	aiohttp-swagger	1.0.16
13	aioitertools	0.10.0	0.10.0
14	aiopg	1.3.3, 1.3.4	1.3.4
15	aioredis	2.0.1
16	aioresponses		0.7.3
17	aiormq	3.3.1, 6.2.3	3.3.1, 6.4.1
18	aiosignal	1.2.0	1.2.0
19	aiosmtplib	1.1.6
20	aiozipkin	1.1.1
21	alembic	1.8.1	1.8.1
22	anyio	3.5.0, 3.6.1	3.5.0, 3.6.1
23	argon2-cffi	20.1.0
24	asgi-lifespan		1.0.1
25	asgiref	3.5.0, 3.5.2
26	astroid		2.12.9	2.12.9
27	async-asgi-testclient		1.4.11
28	async-generator	1.10
29	async-timeout	4.0.2	4.0.2
30	asyncpg	0.25.0
31	attrs	21.4.0, 22.1.0	21.4.0, 22.1.0
32	aws-sam-translator		1.50.0
33	aws-xray-sdk		2.10.0
34	bcrypt	3.2.0	4.0.0
35	beautifulsoup4	4.10.0
36	black			22.8.0
37	bleach	3.3.0
38	blosc	1.10.6
39	bokeh	2.4.3	2.4.3
40	boto3	1.21.21, 1.21.33	1.21.21, 1.24.67
41	boto3-stubs		1.24.67
42	botocore	1.24.21, 1.24.33	1.24.21, 1.27.67
43	botocore-stubs	1.27.17	1.27.67
44	build			0.8.0
45	bump2version			1.0.1
46	certifi	2021.10.8, 2022.5.18.1, 2022.6.15, 2022.6.15.1	2021.10.8, 2022.5.18.1, 2022.6.15, 2022.6.15.1
47	cffi	1.15.0	1.15.0, 1.15.1
48	cfgv			3.3.1
49	cfn-lint		0.64.1
50	change-case			0.5.2
51	charset-normalizer	2.0.12, 2.1.1	2.0.12, 2.1.1
52	click	8.1.3	8.1.3	8.1.3
53	cloudpickle	2.0.0, 2.1.0
54	codecov		2.1.12
55	colorlog		6.7.0
56	configparser	5.2.0
57	coverage		6.4.4
58	coveralls		3.3.1
59	cryptography	3.4.7, 36.0.2, 37.0.2	36.0.2, 38.0.0
60	cytoolz	0.11.0
61	dask	2022.6.0, 2022.9.0
62	dask-gateway	2022.6.1
63	dask-gateway-server		2022.6.1
64	decorator	4.4.2
65	defusedxml	0.7.1
66	deprecated	1.2.13	1.2.13
67	dill		0.3.5.1	0.3.5.1
68	distlib			0.3.6
69	distributed	2022.6.0, 2022.9.0
70	distro	1.5.0
71	dnspython	2.0.0, 2.1.0, 2.2.1	2.2.1
72	docker	5.0.3, 6.0.0	6.0.0
73	docker-compose	1.29.1
74	dockerpty	0.4.1
75	docopt	0.6.2	0.6.2
76	ecdsa	0.14.1	0.18.0
77	email-validator	1.2.1	1.2.1
78	entrypoints	0.3
79	et-xmlfile	1.1.0
80	exceptiongroup		1.0.0
81	execnet		1.9.0
82	expiringdict	1.2.1
83	faker		14.2.0
84	fastapi	0.85.0
85	fastapi-contrib	0.2.11
86	fastapi-pagination	0.9.1
87	fastjsonschema	2.15.3
88	filelock			3.8.0
89	flaky		3.7.0
90	flask		2.1.3
91	flask-cors		3.0.10
92	frozenlist	1.3.0, 1.3.1	1.3.0, 1.3.1
93	fsspec	2022.5.0, 2022.8.2
94	future	0.18.2
95	futures	3.0.5
96	graphql-core		3.2.1
97	greenlet	1.1.2, 1.1.3	1.1.2, 1.1.3
98	gunicorn	20.1.0
99	h11	0.12.0	0.12.0
100	h2	4.1.0
101	heapdict	1.0.1
102	hpack	4.0.0
103	httpcore	0.15.0	0.15.0
104	httptools	0.2.0, 0.4.0
105	httpx	0.23.0	0.23.0
106	hyperframe	6.0.1
107	hypothesis		6.54.5
108	icdiff		2.0.5
109	identify			2.5.5
110	idna	2.10, 3.3	2.10, 3.3
111	importlib-metadata		4.12.0
112	iniconfig	1.1.1	1.1.1
113	inotify			0.2.10
114	isodate	0.6.1
115	isort		5.10.1	5.10.1
116	itsdangerous	1.1.0, 2.1.2	2.1.2
117	jaeger-client	4.8.0
118	jinja-app-loader	1.0.2
119	jinja2	3.1.2	3.1.2	3.1.2
120	jmespath	1.0.0	1.0.0, 1.0.1
121	jschema-to-python		1.2.3
122	json2html	1.3.0
123	jsondiff	2.0.0	2.0.0
124	jsonpatch		1.32
125	jsonpickle		2.2.0
126	jsonpointer		2.3
127	jsonschema	3.2.0, 4.15.0	3.2.0, 4.15.0
128	junit-xml		1.9
129	jupyter-client	6.1.12
130	jupyter-core	4.7.1
131	jupyter-server	1.18.1
132	jupyter-server-proxy	3.2.1
133	jupyterlab-pygments	0.1.2
134	lazy-object-proxy	1.7.1	1.7.1	1.7.1
135	locket	1.0.0
136	lz4	4.0.0
137	mako	1.1.5, 1.2.0, 1.2.2	1.1.5, 1.2.0, 1.2.2
138	markupsafe	2.1.1	2.1.1	2.1.1
139	mccabe		0.7.0	0.7.0
140	minio		7.0.4
141	mistune	0.8.4
142	moto		4.0.1, 4.0.2
143	msgpack	1.0.3, 1.0.4
144	multidict	6.0.2	6.0.2
145	mypy-extensions			0.4.3
146	nbclient	0.5.3
147	nbconvert	6.4.5
148	nbformat	5.3.0
149	nest-asyncio	1.5.1
150	networkx	2.5.1	2.8.6
151	nodeenv			1.7.0
152	nose			1.3.7
153	numpy	1.22.3	1.23.2
154	openapi-core	0.12.0
155	openapi-schema-validator	0.2.3	0.2.3
156	openapi-spec-validator	0.4.0	0.4.0
157	openpyxl	3.0.9
158	opentracing	2.4.0
159	orjson	3.7.2
160	packaging	21.3	21.3	21.3
161	pamqp	2.3.0, 3.1.0	2.3.0, 3.2.0
162	pandas	1.2.4	1.4.4
163	pandocfilters	1.4.3
164	paramiko	2.11.0
165	parfive	1.5.1
166	partd	1.2.0, 1.3.0
167	passlib	1.7.4	1.7.4
168	pathspec			0.10.1
169	pbr		5.10.0
170	pennsieve	6.2.0
171	pep517			0.13.0
172	pillow	9.0.1	9.2.0
173	pint	0.19.2	0.19.2
174	pip-tools			6.8.0
175	platformdirs		2.5.2	2.5.2
176	pluggy	1.0.0	1.0.0
177	pprintpp		0.4.0
178	pre-commit			2.20.0
179	prometheus-client	0.14.1
180	protobuf	3.20.0
181	psutil	5.9.0, 5.9.1, 5.9.2
182	psycopg2-binary	2.9.3	2.9.3
183	ptvsd		4.3.2
184	ptyprocess	0.7.0
185	py	1.11.0	1.11.0
186	py-cpuinfo		8.0.0
187	pyasn1	0.4.8	0.4.8
188	pycparser	2.20, 2.21	2.20, 2.21
189	pydantic	1.9.0, 1.10.2	1.10.2
190	pyftpdlib		1.5.6
191	pygments	2.9.0
192	pyinstrument	3.4.2, 4.1.1, 4.3.0	4.3.0
193	pyinstrument-cext	0.2.4
194	pyjwt	2.4.0
195	pylint		2.15.0, 2.15.2	2.15.0
196	pynacl	1.4.0
197	pyopenssl		22.0.0
198	pyparsing	3.0.9	3.0.9	3.0.9
199	pyrsistent	0.18.1	0.18.1
200	pytest	7.1.3	7.1.3
201	pytest-aiohttp		1.0.4
202	pytest-asyncio		0.19.0
203	pytest-benchmark		3.4.1
204	pytest-cov		3.0.0
205	pytest-docker		1.0.0
206	pytest-forked		1.4.0
207	pytest-icdiff		0.6
208	pytest-instafail		0.4.2
209	pytest-lazy-fixture		0.6.3
210	pytest-localftpserver		1.1.3
211	pytest-mock		3.8.2
212	pytest-runner		6.0.0
213	pytest-sugar		0.9.5
214	pytest-xdist		2.5.0
215	python-dateutil	2.8.1, 2.8.2	2.8.1, 2.8.2
216	python-dotenv	0.20.0	0.20.0, 0.21.0
217	python-engineio	3.14.2
218	python-jose	3.2.0	3.3.0
219	python-magic	0.4.25
220	python-multipart	0.0.5
221	python-socketio	4.6.1
222	pytz	2020.1, 2022.1	2022.2.1
223	pyyaml	5.4.1, 6.0	5.4.1, 6.0	5.4.1, 6.0
224	pyzmq	22.1.0
225	redis	4.3.1, 4.3.4	4.3.1
226	requests	2.27.1, 2.28.1	2.27.1, 2.28.1
227	responses		0.21.0
228	respx		0.19.2
229	rfc3986	1.4.0, 1.5.0	1.4.0, 1.5.0
230	rsa	4.9	4.9
231	s3fs	2022.5.0
232	s3transfer	0.5.2	0.5.2, 0.6.0
233	sarif-om		1.0.4
234	semantic-version	2.9.0
235	semver	2.13.0
236	send2trash	1.7.1
237	setproctitle	1.2.3
238	simpervisor	0.4
239	six	1.15.0, 1.16.0	1.15.0, 1.16.0
240	sniffio	1.2.0, 1.3.0	1.2.0, 1.3.0
241	sortedcontainers	2.4.0	2.4.0
242	soupsieve	2.3.2
243	sqlalchemy	1.4.37, 1.4.41	1.4.37, 1.4.41
244	sshpubkeys		3.3.1
245	starlette	0.20.4
246	strict-rfc3339	0.7
247	tblib	1.7.0
248	tenacity	8.0.1	8.0.1
249	termcolor		1.1.0
250	terminado	0.10.1
251	testpath	0.5.0
252	texttable	1.6.3
253	threadloop	1.0.2
254	thrift	0.16.0
255	toml			0.10.2
256	tomli	2.0.1	2.0.1	2.0.1
257	tomlkit		0.11.4	0.11.4
258	toolz	0.11.1, 0.12.0
259	tornado	6.1, 6.2	6.1
260	tqdm	4.64.0, 4.64.1	4.64.1
261	traitlets	5.1.1	5.3.0
262	twilio	7.12.0
263	typer	0.4.1, 0.6.1	0.6.1	0.6.1
264	types-aiobotocore	2.3.3
265	types-aiobotocore-s3	2.3.3
266	types-aiofiles		0.8.11
267	types-awscrt		0.14.5
268	types-boto3		1.0.2
269	types-pkg-resources		0.1.3
270	types-pyyaml		6.0.11
271	types-s3transfer		0.6.0.post4
272	typing-extensions	4.3.0	4.3.0	4.3.0
273	ujson	5.5.0
274	urllib3	1.26.9, 1.26.11, 1.26.12	1.26.9, 1.26.11, 1.26.12
275	uvicorn	0.15.0, 0.17.0, 0.17.6, 0.18.3
276	uvloop	0.16.0
277	virtualenv			20.16.4, 20.16.5
278	watchdog	2.1.5		2.1.9
279	watchgod	0.8.2
280	webencodings	0.5.1
281	websocket-client	0.59.0, 1.3.2, 1.4.1	0.59.0, 1.4.1
282	websockets	10.1, 10.2	10.3
283	werkzeug	2.0.3, 2.1.2	2.0.3, 2.1.2
284	wheel			0.37.1
285	wrapt	1.14.0, 1.14.1	1.14.0, 1.14.1	1.14.1
286	xmltodict		0.13.0
287	yarl	1.5.1, 1.7.2, 1.8.1	1.5.1, 1.7.2, 1.8.1
288	zict	2.2.0
289	zipp		3.8.1

[codecov]

Codecov Report

Merging #3112 (b49c4cf) into master (988cc25) will increase coverage by 0.0%.
The diff coverage is 100.0%.

@@          Coverage Diff           @@
##           master   #3112   +/-   ##
======================================
  Coverage    82.9%   83.0%           
======================================
  Files         808     808           
  Lines       34261   34261           
  Branches     1358    1358           
======================================
+ Hits        28419   28450   +31     
+ Misses       5661    5630   -31     
  Partials      181     181           

Flag	Coverage Δ
integrationtests	68.5% <ø> (+0.1%)	⬆️
unittests	79.8% <100.0%> (+<0.1%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...re_service_datcore_adapter/modules/remote_debug.py	56.2% <100.0%> (ø)
...webserver/computation_comp_tasks_listening_task.py	81.9% <0.0%> (-3.2%)	⬇️
...rc/simcore_service_catalog/db/repositories/dags.py	44.4% <0.0%> (-2.8%)	⬇️
...simcore_service_director_v2/modules/node_rights.py	98.1% <0.0%> (-1.0%)	⬇️
...ore_service_director_v2/utils/dask_client_utils.py	80.8% <0.0%> (-0.9%)	⬇️
.../director/src/simcore_service_director/producer.py	67.5% <0.0%> (+0.2%)	⬆️
...simcore_service_director_v2/modules/dask_client.py	93.4% <0.0%> (+0.5%)	⬆️
...tor_v2/modules/dynamic_sidecar/docker_api/_core.py	95.7% <0.0%> (+0.9%)	⬆️
...tor_v2/modules/dynamic_sidecar/scheduler/_utils.py	90.8% <0.0%> (+1.1%)	⬆️
...rector_v2/modules/comp_scheduler/base_scheduler.py	92.4% <0.0%> (+1.2%)	⬆️
... and 8 more

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[GitHK]

👍

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication