dependabot doesn't work well for multi modules repository

[ahrtr]

Is there an existing issue for this?

• I have searched the existing issues

Package ecosystem

gomod

Package manager version

No response

Language version

golang 1.19.5

Manifest location and content before the Dependabot update

• https://github.com/etcd-io/etcd/blob/main/go.mod
• https://github.com/etcd-io/etcd/blob/main/server/go.mod
• https://github.com/etcd-io/etcd/blob/main/api/go.mod
• https://github.com/etcd-io/etcd/blob/main/pkg/go.mod
• https://github.com/etcd-io/etcd/blob/main/client/v3/go.mod
• https://github.com/etcd-io/etcd/blob/main/client/pkg/go.mod
• https://github.com/etcd-io/etcd/blob/main/etcdctl/go.mod
• https://github.com/etcd-io/etcd/blob/main/etcdutl/go.mod
• https://github.com/etcd-io/etcd/blob/main/tests/go.mod

dependabot.yml content

https://github.com/etcd-io/etcd/blob/main/.github/dependabot.yml

Updated dependency

No response

What you expected to see, versus what you actually saw

etcd is a multi-modules repository. The problem is each time dependabot automatically creates a PR for each module for exactly the same dependency. Let's work with an example. Multiple modules (e.g. client/v3, etcdctl, etcdutl, pkg, server, test ) depend on github.com/dustin/go-humanize, so dependabot automatically creates the following PRs,

• build(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 in /server etcd-io/etcd#15120
• build(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 in /client/v3 etcd-io/etcd#15119
• build(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 in /etcdutl etcd-io/etcd#15117
• build(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 in /pkg etcd-io/etcd#15114
• build(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 in /etcdctl etcd-io/etcd#15113
• build(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 in /tests etcd-io/etcd#15122

Obviously it doesn't make sense. It makes more sense to do all of them in one PR.

Note the go modules are interrelated, and each module depends on the local version of other modules using the replace directive, e.g. go.mod#L6-L13

replace (
	go.etcd.io/etcd/api/v3 => ../api
	go.etcd.io/etcd/client/pkg/v3 => ../client/pkg
	go.etcd.io/etcd/client/v2 => ../client/v2
	go.etcd.io/etcd/client/v3 => ../client/v3
	go.etcd.io/etcd/etcdctl/v3 => ../etcdctl
	go.etcd.io/etcd/etcdutl/v3 => ../etcdutl
	go.etcd.io/etcd/pkg/v3 => ../pkg
	go.etcd.io/etcd/server/v3 => ../server
)

When you bump a dependency for one of the module (assuming A), and then execute go mod tidy for another module (assming B), the go.mod and go.sum of B may also change. So usually the PR submitted by dependabot will never pass the github workflow's test/validation.

Is there any solution for such situation? Or what's the best practice for dependency management for multi-module (go module) repository?

We also attempt to remove non-root gomod dependabot rules as to as remove most of the spam. But the problem is some direct dependencies in sub-directories' go.mod files are marked as indirect dependencies in the top level go.mod file; and it seems that dependabot will not bump version for indirect dependencies. I did a simple experiment in ahrtr/deps, and confirmed that dependabot indeed doesn't bump indirect dependency ( github.com/stretchr/testify v1.7.1 in this experiment). Did I miss anything? Could anyone confirm this?

Native package manager behavior

No response

Images of the diff or a link to the PR, issue, or logs

No response

Smallest manifest that reproduces the issue

No response

[GerardGarcia]

I can confirm I'm having the same issue in a project with multiple modules and I'm also seeing that dependabot is not updating the indirect dependencies of nested modules.

[jeffwidman]

Thanks, and sorry for the frustrating experience here. However, this is already tracked under:

• Create 1 PR for the same update across update configs #1595

So I'm going to close as a duplicate.

Regarding your other questions:

When you bump a dependency for one of the module (assuming A), and then execute go mod tidy for another module (assming B), the go.mod and go.sum of B may also change. So usually the PR submitted by dependabot will never pass the github workflow's test/validation.

Is there any solution for such situation? Or what's the best practice for dependency management for multi-module (go module) repository?

In the abstract, you basically need custom post-processing of the PR based on the structure of your repo... that's probably best accomplished by using a custom GitHub action that watches for Dependabot PR's and then runs go mod tidy on them. Here's an example of how to wire something like this up: https://github.com/dependabot/dependabot-actions-workflow

But the problem is some direct dependencies in sub-directories' go.mod files are marked as indirect dependencies in the top level go.mod file; and it seems that dependabot will not bump version for indirect dependencies.

We actually just added support for bumping indirect deps in go at the beginning of January:

• [gomod] Support updating indirect dependencies #6361

However, bumping transitive deps is the non-default behavior because it's noisy, so you'll need to specifically allow indirect deps in the dependabot.yml config.

Hopefully this helps, and if it's still confusing let us know so we can improve our docs.

[ahrtr]

However, bumping transitive deps is the non-default behavior because it's noisy, so you'll need to specifically allow indirect deps in the dependabot.yml config.

Thanks for the feedback. So we need to configure the dependabot.yml something like below if we want to bump both direct and indirect dependencies?

  - package-ecosystem: gomod
    directory: /
    schedule:
      interval: weekly
    allow:
      - dependency-type: direct
      - dependency-type: indirect

[jeffwidman]

Pretty sure you can use dependency-type: all to get both direct + indirect, and that matches what the docs say. If that doesn't work please let me know so we can fix the docs.

[ahrtr]

Pretty sure you can use dependency-type: all to get both direct + indirect, and that matches what the docs say. If that doesn't work please let me know so we can fix the docs.

Thanks for the quick response. But it seems that the doc isn't correct. Anyway, let me try it out.

All explicitly defined dependencies. For bundler, pip, composer, cargo, gomod, also the dependencies of direct dependencies.

[ahrtr]

Confirmed that dependency-type: all works. Thanks.