[question] compressing packages before exporting is taking forever with the CPU doing nothing

[elvisdukaj]

What is your question?

Before uploading the packages on the server they are compressed by conan. The issue is that this compressing it's just taking forever with the machine having a very low cpu usage.
Is there any plan to make the compression more efficiently?

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @elvisdukaj

For understanding what is happening, it would be necessary to have more data, and something closer to a benchmark that can be reproduced accurately in different machines. We are aware that zipping Conan packages, but it is not that

Coming up with such a benchmark should be relatively easy, a simple recipe that creates some large files, and conan create then time the conan upload --dry-run, and provide the full specifications of the machine running it, including important things like harddrive specifications, memory and CPU

I have been playing a bit with packaging the android_ndk.
It is true that with the default compress_level=9, the speed of zipping the binary in Windows takes around 5 minutes, resulting in a binary around 600Mb, while the system zipping takes only around 1 minute, but creates a larger binary of around 650Mbs. But lowering the compress_level=6 in Conan, will achieve similar results than the system zipping, around 650Mbs in around 1 minute.

Conan uses a higher compress level, even if it is slower, because packages are downloaded more times than zipped, so saving in storage and download transfer is worth taking a bit more time to zip the package when created.