Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] Controlling compression of wheels built in a unified way between the build backends #2610

Open
1 task done
KOLANICH opened this issue Mar 18, 2021 · 5 comments
Open
1 task done
Labels
enhancement Needs Discussion Issues where the implementation still needs to be discussed.

Comments

@KOLANICH
Copy link
Contributor

KOLANICH commented Mar 18, 2021

What's the problem this feature will solve?

Controlling compression of wheels built in a unified way between the build backends.

Describe the solution you'd like

The arguments to build backends controlling wheel compression (store/deflate/lzma) being unified among all the impls.

Alternative Solutions

No response

Additional context

pypa/build#258

Code of Conduct

  • I agree to follow the PSF Code of Conduct
@KOLANICH KOLANICH added Needs Triage Issues that need to be evaluated for severity and status. enhancement labels Mar 18, 2021
@FRidh
Copy link

FRidh commented Jul 17, 2021

As discussed in pypa/build#258 the format is standardized already, but the compression is indeed not. Best to bring it up at https://discuss.python.org/c/packaging and propose a PEP.

@abravalheri
Copy link
Contributor

Thank you very much @KOLANICH for bringing this up, and thank you @FRidh for giving us some pointers on where the discussions were happening.

So it seems that some part of this issue is being discussed in #2491 and the other part should be taken to https://discuss.python.org/c/packaging, before we can proceed right?

@abravalheri abravalheri added blocker Needs Discussion Issues where the implementation still needs to be discussed. and removed blocker Needs Triage Issues that need to be evaluated for severity and status. labels Jan 28, 2022
@layday
Copy link
Member

layday commented Jan 28, 2022

The wheel standard has nothing to say about compression. The wheel standard would need to be amended before PEP 517 can be amended (presumably with the addition of an optional parameter to build_wheel). This is the wrong venue to pursue either change.

@layday
Copy link
Member

layday commented Jan 28, 2022

Of relevance: https://discuss.python.org/t/making-the-wheel-format-more-flexible-for-better-compression-speed/3810.

@KOLANICH
Copy link
Contributor Author

KOLANICH commented Jan 28, 2022

I'm mostly interested in editing zip files in-place without unpacking and repacking and moving parts. I have half-created ctypes-based bindings to libzip and plan to reimplement ZipFile on top of it.

Currently wheels have the metadata compressed. I'd like to have them always uncompressed, because it'd allow me editing it in-place by offset as if I have used a hex-editor without any changes in the format, given that I pad the contents with non-semantically meaningful bytes, like whitespaces (if compression is used, I cannot easily pad the contents to the needed size).

I would also like to have ZipFile refactored. I mean there is a lot of functionality just inlined and cannot be overriden in subclasses or just reused. I.e conversion of dates. I had to reimplement it instead of reusing it. It is very inflexible.

I also think ZipArchive should support accepting datetime objects for dates and times. Also I absolutely hate raw numeric constants in modules, they are pain to deal with. I'd like more if ziparchive defined IntEnums and IntFlags, and then assigned their values to the legacy variables.

And, yes, every constant known must be defined, even if it is not used in ziparchive itself. Because it spares the ones creating modules extending ZipArchive from reinventing wheel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Needs Discussion Issues where the implementation still needs to be discussed.
Projects
None yet
Development

No branches or pull requests

4 participants