Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow compression with many files - add skip patterns settings #279

Open
PetrDlouhy opened this issue Mar 18, 2021 · 1 comment · May be fixed by #354
Open

Slow compression with many files - add skip patterns settings #279

PetrDlouhy opened this issue Mar 18, 2021 · 1 comment · May be fixed by #354

Comments

@PetrDlouhy
Copy link

I have many static files and the compression is very slow (~5 minutes). There are many files collected, that would be probably never used by website users.

If there was some settings similar to WHITENOISE_SKIP_COMPRESS_EXTENSIONS which would allow to add file patterns to skip compression, that would solve my problem.

@LucidDan
Copy link
Contributor

LucidDan commented Nov 8, 2021

This is a good idea IMO...
There is a should_compress() function on the Compressor class, which can let you have complete control over the compression decision.

Rather than just setting a file pattern, a really flexible way to handle this in Whitenoise could be to have a setting that allows configuring the Compressor class in settings, and then you provide your own (where you override the should_compress() function). That would let you select based on filename pattern but also on other things like a lookup or file size etc.

settings.py:

WHITENOISE_COMPRESSOR_CLASS="myapp.apps.CustomCompressor"

myapp/apps.py:

from whitenoise.compress import Compressor
class CustomCompressor(Compressor):
    def should_compress(self, filename: str) -> bool:
        if filename.startswith("big_assets/"):
            return False
        return True

For WSGI apps, this could also work with an extra command line option to the whitenoise.compress CLI, that instantiates your custom class instead of the normal one:

python -m whitenoise.compress --compressor-class mywsgi.compress.CustomCompress

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants