Skip post process if already exists

[thenewguy]

I am working on integration testing and run collectstatic before testing so that the same static backend can be used during the tests.

This is fast with the default storage because it skips files that are already in the static root and have not changed. However, it takes some time to make all of the files by this backend and it slows down iterative testing and is somewhat of a pain point.

It seems like it should be possible to only rebuild if needed, otherwise just emit the name and path to go in the manifest. Perhaps via a setting?

[thenewguy]

@evansd Would you accept a PR with this for immutable files? I.e. if admin/js/vendor/jquery/jquery.min.220afd743d9e.js exists, do not re-upload it?

[evansd]

Is this issue specific to Whitenoise or do you get the same behaviour with Django's ManifestStaticFilesStorage?

[thenewguy]

@evansd This is a feature request I would like to implement for Whitenoise. Sorry for the poor wording

It offers similar benefit to WHITENOISE_KEEP_ONLY_HASHED_FILES. This would greatly speed up the collectstatic part of whitenoise's storage backend for us.

I have not investigated how the actual approach would work because the concept has not been approved. But the pseudo code would be something like this:

• if WHITENOISE_SKIP_EXISTING_IMMUTABLE_FILES, then check to see if the hashed file already exists before copying it. If it does exist, add file to manifest and continue to the next file. If it does not exist, proceed as normal.

Ideally this would let us skip recreating the .gz and .br files and assuming the compression routine has already done its job. So we would just detect if the .gz file and the .br file exist, and if so, add them to the manifest but skip recreating them

[evansd]

Ah OK, I understand. I can see that a feature to skip recompressing files might be useful.

I'd say the best thing would be to implement this for your own use-case (either as a subclass of the Whitenoise backend or a forked copy) and check that it solves your problem and works as intended. And then we can think about the best way of getting that feature merged upstream.

[thenewguy]

@evansd good news. will implement in a way that makes it possible to contribute