What do you think about deduping only for the dependencies that are updated in that PR? Essentially, over time you would end up in a dedupe'd state without a giant diff of unrelated changes on a single (first) PR, and no new duplicates would be introduced, so if you're already in a deduped state, Dependabot doesn't mess that up.

^ would be very simple to implement and I think it could be a good compromise?

That seems reasonable assuming the deduping applies transitively since most duplicates aren't direct dependencies.

That seems reasonable assuming the deduping applies transitively since most duplicates aren't direct dependencies.

Yeah that's a good question, I'll dig into that

Doesn't seem like that's the case :(

Can you walk me through why Dependabot should do this dedupe step in your opinion? The Yarn docs have a disclaimer on the command, which makes me think it might not be a great default?

Note: Even though it never produces a wrong dependency tree, this command should be used with caution, as it modifies the dependency tree, which can sometimes cause problems when packages don't strictly follow semver recommendations. Because of this, it is recommended to also review the changes manually.

Would it maybe make sense for this to be an Action that folks can set up to run whenever the yarn.lock has changed when they feel it's appropriate?

I wonder if this can be implemented upstream, i.e. in Yarn itself: yarnpkg/berry#4976

Yarn has already declined this feature request. They are open to a plugin that does what you propose, but that wouldn't help Dependabot.

They haven't closed yarnpkg/berry#4976 yet depsite pointing to an older discussion (yarnpkg/berry#2770). IMHO Dependabot is a pretty convincing case to tip the scales, so it’d be great if someone from the team could contribute to the discussion.

We'd like this too for foxglove/studio. The LFS change has gotten us almost there but we still need a manual step to run yarn dedupe: #5830

Here's what I came up with.

I run Github Actions workflow on dependabot/** branches, detect working directory, run yarn dedupe on it, commit the changes as GitHub Actions bot.

https://github.com/wojtekmaj/react-date-picker/blob/v10.0.3/.github/workflows/dependabot-dedupe.yml

Does dependabot run configured precommit hooks for a repository? If so, you could just run yarn dedupe whenever yarn.lock is staged.

I can't imagine that Dependabot deliberately executes arbitrary user-provided code.

I can't imagine that Dependabot deliberately executes arbitrary user-provided code.

Why not? GitHub does it all the time inside the safety of action runners.

For our CI job we run yarn install --immutable which fails on dependabot PRs that have not been properly de-duped.

As a result, we setup a similar workflow to what @wojtekmaj described: https://github.com/foxglove/studio/blob/main/.github/workflows/dependabot-fix.yml

We also grab the dependabot PR, run dedupe and commit those changes to the dependabot branch. We'd find the dependabot UX improved if we could avoid having to do this manual fixup action and instead could configure dependabot to run the dedupe command since it has already run the yarn commands to do the upgrades.

Why not? GitHub does it all the time inside the safety of action runners.

I suspect that Dependabot has more permissions than a typical GitHub Action.

@wojtekmaj @defunctzombie How do you handle the problem of Dependabot refusing to update a PR that has been deduped like that? I now have dozens of open Dependabot PRs that are in conflict and abandoned.

@oliversalzburg I linked the workflow we use: https://github.com/foxglove/studio/blob/main/.github/workflows/dependabot-fix.yml This runs on all PRs made by dependabot to dedupe.

Yeah, that's what I copied, but I adjusted the Name/Email of the bot. Maybe that's the problem. I didn't assume you could just imitate Dependabot that easily.

I changed the name and email to the information I got from previous Dependabot commits in our git history now. Hopefully, that resolves the conflicts I was seeing. Thanks :)

@oliversalzburg see https://docs.github.com/en/code-security/dependabot/working-with-dependabot/managing-pull-requests-for-dependency-updates#allowing-dependabot-to-rebase-and-force-push-over-extra-commits

It seems like this issue has been addressed. Will close it pending confirmation.

@Kurt-von-Laven Do you have a link to any PR or docs that indicate this is built-in now?

No, sorry. Just an empirical observation based on a private repository since we use Forking Renovate for our open-source projects.

Apologies; it appears I was wrong. We recently encountered a counterexample where the Yarn dependencies were not deduplicated.

I can confirm it does not dedupe.

It does sometimes but not always, which is what threw me off.

This issue should be fixed in this PR: #9388