-
Notifications
You must be signed in to change notification settings - Fork 289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add repository consolidation #1150
base: main
Are you sure you want to change the base?
Conversation
@pda Would you mind reviewing this? |
Sorry it's taken us so long to get back to you @pyasi! My main concern with this is that it limits concurrent access by lots of pipelines to the one checkout 🤔 Given that the |
@lox all pipelines using the same repo will use the same checkout, correct, but since it's still The We've solved this internally through a hook so no rush on this. But let me know if the above makes sense as a response to the concurrency issue you spotted. |
Interesting, you have a repo that is 1gb checkout out? (e.g not including the git history) |
We used reference repos and would pull from them instead of from remote. This was to reduce traffic on GHE and speed up clone times, so it's not identical to the |
Consolidating jobs using the same repository into single build directory
The idea here is that with large repositories, we waste a lot of space on our nodes when we have numerous pipelines. This is because the agent creates build dirs based on the
pipeline-slug
. We could save a lot of space and not need to limit the amount of pipelines we make by having all builds of the same repo use the same build directoryThis would be really helpful for our org so I imagine other teams with large repositories could benefit as well
Things I'd like feedback on
Next steps