-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[deliver] Implement parallel screenshot downloads #16654
Conversation
This comment has been minimized.
This comment has been minimized.
CLAs look good, thanks! ℹ️ Googlers: Go here for more info. |
This comment has been minimized.
This comment has been minimized.
fb3f2a0
to
341cee4
Compare
Wow, this changes the game quite a lot. Are you sure this would not have any possible negative effects? Is it safe to just spawn that many threads (for some apps this will be quite a lot, right?) |
Hi @janpio!
The only negative effect that I see is that the processor and network channel will be more busy. The processor will be more loaded. And network failures with a weak connection are slightly more likely. Also on a single-core processor, this will slightly slow down the download due to additional resources for creating threads. The number of images in the store, the size of the images, and the number of languages supported are limited. Apple supports 40 localizations, 10 screenshots per device, 4 types of devices, 1600 images in total. On a multi-core processor, creating threads, even if there are more threads than cores, will not lead to negative overhead. I create threads according to the number of localizations, maximum 40, this is not a lot. So it should be fine |
@CognitiveDisson @janpio This is going to conflict with changes I have going on in the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@CognitiveDisson There are some conflicting changes now due to the recent App Store Connect API updates. Do you mind rebasing and fixing this? 😇
@joshdholtz Of course, I will do it soon. Thanks for letting me know. |
341cee4
to
5abf1d5
Compare
5abf1d5
to
0b342d3
Compare
@joshdholtz Done and checked |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is so quick! Thanks for rebasing this 😊 Really appreciate it! ❤️
Hey @CognitiveDisson 👋 Thank you for your contribution to fastlane and congrats on getting this pull request merged 🎉 Please let us know if this change requires an immediate release by adding a comment here 👍 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Congratulations! 🎉 This was released as part of fastlane 2.151.2 🚀
Checklist
bundle exec rspec
from the root directory to see all new and existing tests passbundle exec rubocop -a
to ensure the code style is validMotivation and Context
I am working on an application that supports many languages. Therefore, loading and unloading screenshots for all of them takes a lot of time. To get started, I would like to add a parallel download of screenshots. I know that Fastlane is not designed for parallel operations, but in this case I think this is a suitable solution.
Description
I added the creation of a thread for downloading screenshots for each language, this should be the optimal solution (the optimal number of threads and file descriptors) and should not create collisions.
Testing Steps
Execute
fastlane deliver download_screenshots
on the current release version of Fastlane and on version from the current branch.Averaged local measurement results:
Hardware: MacBook Pro (15-inch, 2019) 2.4 GHz Intel Core i9 32 GB 2400 MHz DDR4
Input: 31 language * 4 devices * 9 screenshots = 1116 images 421.6 MB
Connection: WiFi 46.9 Mbps
Result before: ~ 375s
Result after: ~ 99s