chore: bump werkzeug and Flask #23965

dpgaspar · 2023-05-08T10:08:28Z

SUMMARY

Bump Flask and Werkzeug

https://nvd.nist.gov/vuln/detail/CVE-2023-30861

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

EugeneTorap · 2023-05-08T13:31:17Z

Hi @dpgaspar! We have a Mypy configuration problem in #23927
Do you know how to fix it?

Basically the problem is that some dependencies (e.g. Flask since version 2.0) have stubs include in their main distribution, while other dependencies require stubs to be installed vie the typeshed distribution. (i.e. the 'types-*' packages from PyPI).

dpgaspar · 2023-05-08T13:37:21Z

Hi @dpgaspar! We have a Mypy configuration problem in #23927 Do you know how to fix it?

Basically the problem is that some dependencies (e.g. Flask since version 2.0) have stubs include in their main distribution, while other dependencies require stubs to be installed vie the typeshed distribution. (i.e. the 'types-*' packages from PyPI).

not yet

EugeneTorap · 2023-05-08T13:38:20Z

requirements/base.txt

@@ -309,7 +310,7 @@ vine==5.0.0
    #   kombu
 wcwidth==0.2.5
    # via prompt-toolkit
-werkzeug==2.1.2
+werkzeug==2.3.3


I suggest you to pin werkzeug in setup.py because we use this lib in superset project directly.

done, makes sense

codecov · 2023-05-08T13:57:58Z

Codecov Report

Merging #23965 (d76eabc) into master (4a828f5) will decrease coverage by 11.20%.
The diff coverage is n/a.

❗ Current head d76eabc differs from pull request most recent head 5f80dcf. Consider uploading reports for the commit 5f80dcf to get more accurate results

@@             Coverage Diff             @@
##           master   #23965       +/-   ##
===========================================
- Coverage   68.18%   56.98%   -11.20%     
===========================================
  Files        1941     1941               
  Lines       75261    75261               
  Branches     8168     8168               
===========================================
- Hits        51317    42888     -8429     
- Misses      21855    30284     +8429     
  Partials     2089     2089

Flag	Coverage Δ
hive	`53.18% <ø> (ø)`
mysql	`?`
presto	`53.10% <ø> (ø)`
python	`59.60% <ø> (-23.11%)`	⬇️
sqlite	`?`
unit	`53.05% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 306 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

EugeneTorap · 2023-05-08T14:36:36Z

@cwegener @dpgaspar I found that mypy PR python/mypy#10652 which resolves flask & werkzeug typing problems.
@dpgaspar Can you try to upgrade the mypy version?

EugeneTorap · 2023-05-08T15:17:22Z

@cwegener @dpgaspar Ohh we already has v1.0.1 mypy but it downloads flask-stubs because we use additional_dependencies: [types-all] in .pre-commit-config.yaml file

cwegener · 2023-05-08T21:37:44Z

Hi @dpgaspar! We have a Mypy configuration problem in #23927 Do you know how to fix it?

Basically the problem is that some dependencies (e.g. Flask since version 2.0) have stubs include in their main distribution, while other dependencies require stubs to be installed vie the typeshed distribution. (i.e. the 'types-*' packages from PyPI).

I'm actually not very concerned about the static type checking at the moment. I think it's a lower priority.
Getting the Async Query Runner working with Flask 2.2 is currently the main blocker for migrating to Flask 2.2

cwegener · 2023-05-08T21:49:25Z

@cwegener @dpgaspar Ohh we already has v1.0.1 mypy but it downloads flask-stubs because we use additional_dependencies: [types-all] in .pre-commit-config.yaml file

Yes. The solution for this particular mypy issue should be to remove the types-all dependency in the pre-commit config and replace it with the curated, explicit list of all desired python type stubs packages that we know are required to successfully run static type checks.

cwegener · 2023-05-08T22:04:25Z

Would it be better to go to Flask 2.2 before migrating to Flask 2.3?

dpgaspar · 2023-05-11T15:14:34Z

tests/integration_tests/async_events/api_tests.py

@@ -33,6 +33,7 @@ def fetch_events(self, last_id: Optional[str] = None):

    @mock.patch("uuid.uuid4", return_value=UUID)
    def test_events(self, mock_uuid4):
+        app._got_first_request = False


a bit of an hack, async_query_manager.init_app(app) registers a Flask app.before_request. New Flask will only allow registering before request hooks if no request was yet processed by the app.

dpgaspar · 2023-05-11T15:17:13Z

tests/integration_tests/charts/data/api_tests.py

@@ -1188,10 +1193,10 @@ def test_data_cache_default_timeout(


 def test_chart_cache_timeout(
+    load_energy_table_with_slice: List[Slice],


This is an interesting case that surfaced on this new Flask version. Flask now is stricter, so when popping a new context from the app stack it checks if the context is the one we are expecting. load_energy_table_with_slice creates a new app_context so it needs to be first

requirements/integration.txt

dpgaspar · 2023-05-12T08:03:42Z

Would it be better to go to Flask 2.2 before migrating to Flask 2.3?

Flask is on 2.2

dpgaspar · 2023-05-12T08:05:06Z

Wow, that's impressive work! Now we need to fix the flask-stubs issue. We use additional_dependencies: [types-all] in .pre-commit-config.yaml file which downloads the old flask-stubs.

yep, thank you once more for doing that on: #24033

villebro

LGTM, pretty big changes needed, thanks for all the work here 👍

chore: bump werkzeug and Flask

7d3f7d1

pull-request-size bot added the size/S label May 8, 2023

dpgaspar added 5 commits May 8, 2023 11:13

fix

187aae1

fix

7a76a3e

debug

4e34493

fix test

56c28f8

fix send_file

f3ed08e

pull-request-size bot added size/M and removed size/S labels May 8, 2023

dpgaspar added 2 commits May 8, 2023 14:12

fix test

358b4b4

debug

ee743e1

EugeneTorap mentioned this pull request May 8, 2023

chore: relax Flask version constraint #23927

Closed

EugeneTorap reviewed May 8, 2023

View reviewed changes

debug

da866ce

debug

a805236

dpgaspar added 4 commits May 9, 2023 12:00

fix test

93db414

fix test

cd953b0

fix test

eeb0ef2

fix test

443473b

pull-request-size bot added size/L and removed size/M labels May 9, 2023

fix tests

493c078

dpgaspar commented May 11, 2023

View reviewed changes

dpgaspar added 2 commits May 11, 2023 16:26

disable mypy on flask send_file

8ec6738

merge master in

22d579c

EugeneTorap reviewed May 12, 2023

View reviewed changes

requirements/integration.txt Outdated Show resolved Hide resolved

dpgaspar requested review from villebro, geido, eschutho and betodealmeida May 12, 2023 09:13

new pip-compile-multi

5f80dcf

villebro approved these changes May 12, 2023

View reviewed changes

dpgaspar merged commit 97482c5 into apache:master May 12, 2023
26 checks passed

dpgaspar deleted the chore/flask-vulnerability branch May 12, 2023 14:01

EugeneTorap mentioned this pull request May 12, 2023

CVE-2023-25577 #23555

Closed

eschutho added v2.1 2.1.1 labels May 12, 2023

eschutho pushed a commit that referenced this pull request May 25, 2023

chore: bump werkzeug and Flask (#23965)

a564bd2

eschutho removed v2.1 2.1.1 labels Jun 7, 2023

eschutho added v2.1 2.1.3 labels Dec 1, 2023

eschutho pushed a commit that referenced this pull request Dec 2, 2023

chore: bump werkzeug and Flask (#23965)

dabf7cb

villebro mentioned this pull request Feb 20, 2024

fix: unlock and bump werkzeug #27164

Merged

9 tasks

mistercrunch added 🍒 2.1.3 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 3.0.0 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: bump werkzeug and Flask #23965

chore: bump werkzeug and Flask #23965

dpgaspar commented May 8, 2023 •

edited

EugeneTorap commented May 8, 2023

dpgaspar commented May 8, 2023

EugeneTorap May 8, 2023 •

edited

dpgaspar May 12, 2023

codecov bot commented May 8, 2023 •

edited

EugeneTorap commented May 8, 2023 •

edited

EugeneTorap commented May 8, 2023

cwegener commented May 8, 2023

cwegener commented May 8, 2023

cwegener commented May 8, 2023

dpgaspar May 11, 2023 •

edited

dpgaspar May 11, 2023

dpgaspar commented May 12, 2023

dpgaspar commented May 12, 2023

villebro left a comment

		@@ -1188,10 +1193,10 @@ def test_data_cache_default_timeout(


		def test_chart_cache_timeout(
		load_energy_table_with_slice: List[Slice],

chore: bump werkzeug and Flask #23965

chore: bump werkzeug and Flask #23965

Conversation

dpgaspar commented May 8, 2023 • edited

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

EugeneTorap commented May 8, 2023

dpgaspar commented May 8, 2023

EugeneTorap May 8, 2023 • edited

Choose a reason for hiding this comment

dpgaspar May 12, 2023

Choose a reason for hiding this comment

codecov bot commented May 8, 2023 • edited

Codecov Report

EugeneTorap commented May 8, 2023 • edited

EugeneTorap commented May 8, 2023

cwegener commented May 8, 2023

cwegener commented May 8, 2023

cwegener commented May 8, 2023

dpgaspar May 11, 2023 • edited

Choose a reason for hiding this comment

dpgaspar May 11, 2023

Choose a reason for hiding this comment

dpgaspar commented May 12, 2023

dpgaspar commented May 12, 2023

villebro left a comment

Choose a reason for hiding this comment

dpgaspar commented May 8, 2023 •

edited

EugeneTorap May 8, 2023 •

edited

codecov bot commented May 8, 2023 •

edited

EugeneTorap commented May 8, 2023 •

edited

dpgaspar May 11, 2023 •

edited