Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve migration performance #169

Merged
merged 4 commits into from
May 25, 2024
Merged

Improve migration performance #169

merged 4 commits into from
May 25, 2024

Conversation

codingjoe
Copy link
Owner

  • perform delete IO async
  • create only newly added sizes
  • delete only obsolete file sizes
  • create sizes from an in-memory copy of the file

@codingjoe codingjoe self-assigned this May 5, 2024
@codingjoe codingjoe force-pushed the async-migrations branch 3 times, most recently from 117b591 to aae65ef Compare May 5, 2024 14:20
@codingjoe codingjoe marked this pull request as draft May 5, 2024 14:37
@codingjoe codingjoe force-pushed the async-migrations branch 2 times, most recently from d5bb268 to 23ace56 Compare May 5, 2024 16:33
@codingjoe codingjoe marked this pull request as ready for review May 5, 2024 16:35
file_name: str,
new: list[tuple[str, list, dict]] | None = None,
old: list[tuple[str, list, dict]] | None = None,
) -> None: ...

Check notice

Code scanning / CodeQL

Statement has no effect Note

This statement has no effect.
Copy link

codecov bot commented May 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (f361b9c) to head (0a676e2).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #169   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           13        13           
  Lines          466       488   +22     
=========================================
+ Hits           466       488   +22     
Flag Coverage Δ
celery 84.42% <90.41%> (+0.94%) ⬆️
cleanup 87.90% <91.78%> (+3.78%) ⬆️
dj4.1 87.90% <91.78%> (+3.78%) ⬆️
dj4.2 87.90% <91.78%> (+3.78%) ⬆️
django-rq 84.42% <90.41%> (+0.94%) ⬆️
dramatiq 84.42% <90.41%> (+0.94%) ⬆️
drf 96.31% <91.78%> (+3.39%) ⬆️
macos-latest 88.11% <91.78%> (+3.77%) ⬆️
py3.10 87.90% <91.78%> (+3.78%) ⬆️
py3.12 87.90% <91.78%> (+3.78%) ⬆️
py3.8 87.90% <91.78%> (+3.78%) ⬆️
py3.9 87.90% <91.78%> (+3.78%) ⬆️
ubuntu-latest 87.90% <91.78%> (+3.78%) ⬆️
windows-latest 81.01% <86.66%> (+5.20%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* perform delete IO async
* create only newly added sizes
* delete only obsolete file sizes
* create sizes from an in-memory copy of the file
tests/test_models.py Dismissed Show dismissed Hide dismissed
@codingjoe codingjoe requested a review from amureki May 5, 2024 17:25
Copy link
Collaborator

@amureki amureki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-wise it looks good to me, however I did not test it. Not sure I'd manage testing before going offline for some time.

tests/test_models.py Show resolved Hide resolved
@codingjoe codingjoe merged commit ec91252 into main May 25, 2024
24 checks passed
@codingjoe codingjoe deleted the async-migrations branch May 25, 2024 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants