Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH Actions make /usr/local cache for mac deps #2762

Merged
merged 1 commit into from Oct 16, 2021
Merged

Conversation

ankith26
Copy link
Contributor

Cache the deps built in /usr/local because they can take really long build times

@ankith26 ankith26 force-pushed the ankith26-gh-actions-cache branch 18 times, most recently from db2e8fc to 317d900 Compare October 12, 2021 04:10
@ankith26 ankith26 added this to the 2.0.3 milestone Oct 12, 2021
@ankith26 ankith26 force-pushed the ankith26-gh-actions-cache branch 9 times, most recently from 711dbc6 to 5113d33 Compare October 12, 2021 08:12
@ankith26
Copy link
Contributor Author

After like a million force pushes, this PR is finally ready for review now!

Also to explain a bit on what this PR does, mac deps take a long while to build from source. This PR adds caching of the built deps so that we don't waste build time building these again and again. The deps are built only when something in the buildconfig/macdependences or buildconfig/manylinux-build dir changes, and this dep build runs on a separated job, because we don't want the deps to be built over and over again in the build matrices running on parallel.

The original idea of this PR was to cache the entire /usr/local dir, but it quickly became clear that it would be problematic. So this PR takes a slightly different approach, the manylinux build scripts also install the built files onto a separate cached dir. This is better than caching /usr/local because /usr/local has tons of stuff we don't need to cache, whereas this cached dir will only have the built deps. Also, this way the cache is more resilient, and would not break when mac versions change (because cached deps are built to work on any 10.9+ mac)

This PR saves about 25-35 minutes of build time on mac runners

@ankith26 ankith26 requested a review from illume October 12, 2021 11:13
@ankith26 ankith26 force-pushed the ankith26-gh-actions-cache branch 7 times, most recently from 7639951 to 122b60f Compare October 13, 2021 05:42
Copy link
Member

@illume illume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks :) 🎉 👍

Very nice improvement from (60mins, 68mins) -> (25mins, 43mins).

@illume illume merged commit 68de48c into main Oct 16, 2021
@illume illume deleted the ankith26-gh-actions-cache branch October 16, 2021 09:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants