Shipit: I'd like to use this in my monorepo. What should I know?

[jaredpalmer]

[michalsanger]

Hi Jared, it's cool that you like Shipit! Maybe start a discussion in our monorepo if you have any questions:
https://github.com/adeira/universe/discussions

What should you know? Well it works fine for us:
https://github.com/adeira/universe/blob/master/.github/workflows/shipit.yml

It could be useful for you to look at the configs of projects we're exporting with this tool:
https://github.com/adeira/universe/tree/master/src/monorepo-shipit/config

[mrtnzlml]

Just a quick note: I transferred this question to adeira/universe discussions where it belongs.

[mrtnzlml]

Thanks @jaredpalmer for your interest. I'd like to add one more note to @michalsanger comment. Shipit is currently not very reusable because nobody showed interest yet (until now). But I am thinking about making it reusable for a long time (which technically means only moving away our configs and exporting it via NPM for example). This would be a great opportunity to do so.

Would you please let us know whether you are interested in this effort and you'd help us to report any issues with it? I have some capacity to do the necessary changes to Shipit to fit your use-case.

[itsdouges]

Hey folks looks like this hasn't had much traction in a while. @jaredpalmer has referenced this on Twitter recently however: https://twitter.com/jaredpalmer/status/1499496884880261122?s=20

For myself I have a desire to keep my personal private monorepo and sync packages inside it to other public repositories. I've been looking at a few different options:

• adeira/shipit
• fbshipit
• git-repo-sync

Currently none are ideal. Yours isn't published. Fbshipit has a steep getting started (and I'm currently stuck with hhvm errors in my CI), and git-repo-sync doesn't fufill the use case of private -> publish with a filter, 2 way sync with pull requests.

I believe a lot of folks in oss end up just creating multiple repos, some private some public, and then doing a painful release + upgrade process. I'm hoping to instead orchestrate all releases from the private repo but have everything 2 way sync as appropriate. Keen for any insight, it's a gnarly problem and one I'm hoping to not have to write a solution for.

[mrtnzlml]

Hi @itsdouges, I can speak only for adeira/shipit that I use daily, and it works great (at least for my use-case).

What would make it ideal solution for you? Are you willing to help and submit a PR modifying some parts of adeira/shipit to make it reusable? I am still open to this contribution, and I believe it could be quite easy if we think it through. 😎

[itsdouges]

Thanks for the fast response mate!

So my ideal solution is:

• Github only
• Rebase workflow
• Commits only enter a repository via pull requests (or sync)
• I want to sync commits persisting author + email + commit message from a source repository to a destination repository and apply a filter to exclude paths on each commit
• I want to rewrite paths to dest
• I want to sync commits from destination to source, persisting author + email + commit message
• I want git conflict resolution
• I want commits to flow through ASAP after they're committed, github action on push to main might be the play?
• The x-comment lines are interesting and something I can imagine using.

I'd be happy to contribute, let me know areas of interest. What would an MVP look like? Very happy to focus on source to dest sync for now.

[itsdouges]

Stretch goal later would be bitbucket source to GitHub dest.

[mrtnzlml]

@itsdouges I think pretty much everything in the list adeira/shipit already does. One thing that stands out is the "git conflict resolution". That is a tough thing to do since it runs on CI with every commit to master automatically. In reality, I've never needed this since I keep my extracted repositories strictly read-only (that is, no PRs or changes are made to the exported repositories, everything goes through the monorepo export).

Stretch goal later would be bitbucket source to GitHub dest.

I am pretty confident this is already mostly supported. The core of adeira/shipit doesn't depend on GitHub; it depends on Git only (and it's possible to expand it to support other VCS such as Mercurial as well). There might be some places here and there mentioning GitHub but nothing too dramatic. The main thing is probably the import part which currently works only with GitHub URLs.

@itsdouges I realize it's a niche project, and everyone is busy with their things. However, in case you'd like to make it work, I wrote a simple description of what's needed here: #6096 🚀

I am happy to assist, be it via GitHub Discussions or GitHub Issues. 😎

[itsdouges]

I'll have a look.

[mrtnzlml]

It is now possible to use adeira/shipit in external repos. See Shipit readme: https://github.com/adeira/shipit

Kudos to @itsdouges for pushing it through!