Use a single PR for updates with the same GroupId and Version

[larsgrefer]

This is a special case of:

• Add grouped updates #1190

I have multiple dependencies pointing to different submodules of the different overall project:

dependencies {
    compile "org.springframework:spring-beans:4"
    compile "org.springframework:spring-context:4"
    compile "org.springframework:spring-web:4"
}

When Version 5 becomes available all of these dependencies should be updated in one atomic PR.

Some real world examples:

• Updating the group com.squareup.retrofit2 from 2.5.0 to 2.6.1
  • Bump retrofit from 2.5.0 to 2.6.1 freefair/gradle-plugins#58
  • Bump converter-gson from 2.5.0 to 2.6.1 freefair/gradle-plugins#54
• Updating the group io.freefair.gradle from 3.7.4 to 3.8.1
  • Bump lombok-plugin from 3.7.4 to 3.8.1 freefair/okhttp-spring-boot#15
  • Bump okhttp-plugin from 3.7.4 to 3.8.1 freefair/okhttp-spring-boot#16
  • Bump io.freefair.git-version from 3.7.4 to 3.8.1 freefair/okhttp-spring-boot#17
  • Bump io.freefair.aggregate-javadoc-jar from 3.7.4 to 3.8.1 freefair/okhttp-spring-boot#18
  • Bump maven-plugin from 3.7.4 to 3.8.1 freefair/okhttp-spring-boot#19
  • Bump io.freefair.javadoc-links from 3.7.4 to 3.8.1 freefair/okhttp-spring-boot#20

[greysteil]

Interesting. Any reason you're not using a property for these?

[larsgrefer]

I can try using a property for these, but until now it wasn't necessary to do so.

For manual dependency updates it was sufficient to just have the dependencies in question declared adjacent to each other so it would be obvious to a human that these belong together and should be updated simultaneously.

What kinds of properties and property replacement does dependabot support for gradle?
Would all of these work?

dependencies {
    compile "org.springframework:spring-beans:$springVersion"
    compile "org.springframework:spring-context:${springVersion}"
    compile "org.springframework:spring-web:" + springVersion
}

[greysteil]

Dependabot can support the top two formats but not the bottom one (it's regex based for Gradle).

I think having Dependabot try to automatically determine groups based on groupId and version would be nice, but could only happen after it's configurable more generally, otherwise it could cause problems!

[larsgrefer]

I think introducing a property for a version number shared by multiple dependencies only makes sense if the dependency declarations are spread over different places and/or if there are many of them.

For small groups of dependencies (2 or 3) which are declared directly adjacent to each other it seems exaggerated to me to introduce a separate property just to hold the version number.

Dependabot can support the top two formats but not the bottom one (it's regex based for Gradle).

Is there any way I can see which exact dependencies have been found in projects so I can make sure dependabot monitors all of them?

[greysteil]

Is there any way I can see which exact dependencies have been found in projects so I can make sure dependabot monitors all of them?

Not at the moment, but we're planning to use Dependabot's parsers to power parsing for GitHub's dependency insights page. Once we start doing that you'll be able to view the results within GitHub.

[burdoto]

The Issue's name catches me here.
I have some projects that are laying in background, paused ATM, yet their dependencies get updated, of course.

I've hat this scenario:

• jackson-databind dependency gets updated to 2.10.0pr1, #60 was opened
• PR 60 is now getting ignored, hence the project is paused and releasing for this dependency update is overkill in this case
• jackson-databind dependency gets updated to 2.10pr2, previous request #60 gets closed and #65 is opened with the new patch
• Now we have two PRs in total, #60 and #65

I kinda figure it to be way much more nicer to just re-use the old PR and update that and it's name to match the updated version.

[infin8x]

Similar to:

• Add grouped updates #1190

[missedone]

pls check this #1618 (comment) which works for me

[mwaddell]

Similar to:

• Support multiple directories to bump the same package across dependency files #3856

[jeffwidman]

Solving this for all edge cases really requires leveraging a native helper:

• Fully Featured Gradle Support with the Gradle Tooling API #1164

[jeffwidman]

I get that you want them to all be bumped in a single PR, but which of the following is true:

1. They are force-linked together in some way such that Gradle will realize they always must be bumped in lockstep?
2. Or technically is it allowed by the Gradle tooling that only some of them bump and some others don't?

1️⃣ is tracked in the generic sense in:

• Support grouping for "must-be-updated-together" dependencies #1296

2️⃣ is tracked in:

• Add grouped updates #1190

Also possibly related:

• Create 1 PR for the same update across update configs #1595

[mwaddell]

@jeffwidman There are a lot of different issues related to this, but they all basically come down to the same underlying frustration with dependabot - it creates a lot of "noise" because virtually every version of every dependency gets its own individual PR. The value of dependabot is that it automates the process of checking for available updates and creating PRs when new updates are available. However, those PRs are not grouped or managed the way a human would do it if given the same job, so dependabot ends up adding a lot of additional overhead in now managing all of those individual PRs. This problem is much worse in ecosystems like javascript where even relatively small projects have a large number of dependencies which get updated very regularly and where many are logically grouped together. In addition to being frurstrating to manage them all, it can quickly blow up your CI costs if you run CI on your dependabot PRs.

Currently, the only solutions to this problem are to (a) manage all of these various PRs manually, or (b) to set up custom scripts to manage all the dependabot PRs after-the-fact (e.g. https://github.com/github/combine-prs). Managing them all manually gets frustrating pretty quickly. Using custom scripts works okay, but adds another layer of code to maintain and demotes dependabot to an alerting system only and can break frequently because of merge conflicts.

In order to solve this more fundamental problem, dependabot requires 2 major changes. The first is to allow multiple directories for the same ecosystem. The second is to add a new groups: section of the config that would contain one or more group names which each follow the same format as the allow: section does now. For example:

version: 3
updates:
  - package-ecosystem: "npm"
    directories: 
      - "/web"
      - "/api"
      - "/docs"
    schedule:
      interval: "weekly"
    groups:
      react:
        - dependency-name: "react*"
      eslint:
        - dependency-name: "eslint*"
        - dependency-name: "@typescript-eslint*"
      other-development:
        - dependency-type: "development"

The above would put any dependency (across all three directory) which starts with "react" in the same PR and reuse that PR as other dependencies/versions are added (as long as it is still open). Matching can be done in order (so "other-development" only matches if the first 2 didn't match) and anything which doesn't match any of the defined groups would work the same way it does now.

I know that this would be significant change to some core dependabot functionality but it would singlehandedly address a large number of existing issues and make dependabot more flexible and configurable.

[0xdevalias]

I really like how @mwaddell's suggestion above (Ref) puts the 'control of grouping' back on us as individual developers, which IMO would remove a lot of the 'effort' of trying to figure out the correct groupings for everyone that uses dependabot in a more generic case.

And then later, any public repo's can be data-mined for their dependabot grouping configs to build up a 'database' of the most common groupings, that could then be used to improve the 'default smarts' of dependabot (or suggest groupings for users to include in their own config; which itself could potentially be implemented as a dependabot PR suggesting changes to it's own config, to still 'keep a human in the loop')

[larsgrefer]

I get that you want them to all be bumped in a single PR, but which of the following is true:

1. They are force-linked together in some way such that  Gradle will realize they _always_ must be bumped in lockstep?

2. Or technically is it allowed by the Gradle tooling that only some of them bump and some others don't?

I'ts mostly 2, but a bit of 1 could happen.

Technically, Gradle would be fine with them being declared with different versions. But dependencies which are essentially different modules published by the same project have a high chance of being interdependent.
So updating only one module could lead to compilation failures if there is a breaking API change between the modules.

There is no easy way to 100% correctly determine, which dependencies should be bumped together, but multiple dependencies having the same groupId, the same current (old) version and the same new target version should be a reasonably good heuristic.

I'm not sure if I like the idea of having to declare all dependency groups manually. First, because I'm lazy and don't want to spend my time configuring dependabot for all my repositories, and second because potentially thousands of repositories would need to declare the same groupings over and over.

If you don't like to use a heuristic here it might be the better option to have a static list of groups inside dependabot itself.
Having the ~100 most common groupings declared in some sort of semi-official list, should help to reduce the PR count for many projects.

[abdulapopoola]

Update: We've started doing some grouped updates work! This particular issue might not be part of the first ship but if you want to track our updates, do follow #1190.

[carogalvin]

Just wanted to pop in from the Dependabot team to mention that we have configurable grouping rules in public beta, which should solve this problem! You can see the docs on how to set these up here: https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#groups

If you have any feedback, please feel free to email me at carogalvin@github.com or set up a meeting with me by picking a convenient time in my calendar here.

[mwaddell]

Just wanted to pop in from the Dependabot team to mention that we have configurable grouping rules in public beta, which should solve this problem! You can see the docs on how to set these up here: https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#groups

If you have any feedback, please feel free to email me at carogalvin@github.com or set up a meeting with me by picking a convenient time in my calendar here.

@carogalvin It would be nice if dependabot closed the old individual PRs when it created the combined one. It's a little confusing that when you close them manually you get the "I won't notify you" alert but it is still tracking them in the other PR...

[abdulapopoola]

Closing this out as we've officially released grouped version updates; please feel free to reopen or reach out if there are more questions or feedback.