Fetch Metadata action returns null update-type output for pull requests

[oguzhanmeteozturk]

Description

I am using the dependabot/fetch-metadata action (v1.3.6) in my workflow to handle Dependabot pull requests. However, I am encountering an issue where the update-type output is returning null, even though the pull request is created by Dependabot.

Repository Configuration

Here's the dependabot.yml configuration for the repository:

version: 2

updates:
  - package-ecosystem: pip
    directory: /dependabot/
    schedule:
      interval: "weekly"
      day: "sunday"
    commit-message:
      prefix: "build: "
      prefix-development: "build: "
      include: "scope"
    rebase-strategy: "auto"
    target-branch: "develop"
    labels:
      - "build"
      - "dependencies"
    versioning-strategy: auto
    allow:
      - dependency-type: "all"
    open-pull-requests-limit: 10

The pyproject.toml file is located in the /dependabot/ directory, which is specified in the dependabot.yml configuration

Workflow Configuration

The following is the configuration for the workflow that uses the dependabot/fetch-metadata action:

on:
  pull_request_target:
    types:
      - opened
      - synchronize
permissions:
    pull-requests: write
    contents: write
jobs:
  review-dependabot-pr:
    runs-on: ubuntu-latest
    if: ${{ github.event.pull_request.user.login == 'dependabot[bot]' }}
    outputs:
      approved: ${{ steps.set-output.outputs.approved }}
    steps:

        # ... (other steps)

        - name: 📛 Dependabot metadata
            id: dependabot-metadata
            uses: dependabot/fetch-metadata@v1.3.6 # https://github.com/dependabot/fetch-metadata
            with:
                github-token: "${{ steps.generate_token.outputs.token }}"

        # ... (subsequent steps using the outputs)

Issue Details

When the workflow runs for a pull request created by Dependabot, the action returns a null update-type output, which causes issues in the subsequent steps that rely on this output.

Here's an example of the outputs returned by the dependabot/fetch-metadata action:

##[group]Run dependabot/fetch-metadata@v1.3.6
with:
  github-token: ***
  skip-commit-verification: false
##[endgroup]
Parsing Dependabot metadata
##[group]Outputting metadata for 1 updated dependency
outputs.dependency-names: pytest
outputs.dependency-type: direct:development
outputs.update-type: null
outputs.directory: /dependabot/develop
outputs.package-ecosystem: pip
outputs.target-branch: develop
outputs.previous-version: 
outputs.new-version: 
outputs.compatibility-score: 0
outputs.alert-state: 
outputs.ghsa-id: 
outputs.cvss: 0
##[endgroup]

The issue persists even after verifying that the pull request is created by Dependabot and that the workflow is triggered by the correct event (pull_request_target with opened and synchronize types).

Request for Assistance

I would appreciate any help in diagnosing and resolving this issue. If there's any additional information or logs that would be useful, please let me know, and I will provide them.

[jeffwidman]

Can you link to the PR that is generating the failure?

If it's not public, it's hard for us to diagnose, you'd need to run the action locally yourself and step through to figure out why it's breaking. Or create a super simple repo that triggers a PR for that dep bump, and then we can see why Fetch-metadata is breaking against it ourselves.

See also #338, which just landed and hasn't been released yet, may be a potential cause.

[yeikel]

Alternatively, you can extract and attach the patch as it will probably include everything we need to start debugging

You can get it adding .patch as a suffix to the pull request url

Example : https://patch-diff.githubusercontent.com/raw/dependabot/fetch-metadata/pull/278.patch

[oguzhanmeteozturk]

The version bump dependabot is trying is actually fairly simple:

From d9ab05edcb36f0086ba97b187a13c30822efa735 Mon Sep 17 00:00:00 2001
From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com>
Date: Sun, 16 Apr 2023 05:43:13 +0000
Subject: [PATCH] build: (deps-dev): update pytest requirement in /dependabot

Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.2.0...7.3.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
---
 dependabot/pyproject.toml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/dependabot/pyproject.toml b/dependabot/pyproject.toml
index 0bad554..dde5ebc 100644
--- a/dependabot/pyproject.toml
+++ b/dependabot/pyproject.toml
@@ -28,7 +28,7 @@ commitizen = "2.39.1"
 coverage = { extras = ["toml"], version = "7.0.3" }
 mypy = "1.2.0"
 pre-commit = "2.21.0"
-pytest = "7.2.0"
+pytest = "7.3.1"
 pytest-clarity = "1.0.1"
 pytest-mock = "3.10.0"
 pytest-xdist = "3.1.0"

One thing I might have missed when creating the issue is that this version bump is attempted before a poetry.lock file is created. Could that be the issue?

[yeikel]

Would it be possible to share the workflow logs as well?

[oguzhanmeteozturk]

I apologize I posted the wrong log file. The log below pertains to the same action, but with null metadata.

Requested labels: ubuntu-latest
Job defined at: redacted/redacted-repo/.github/workflows/dependabot_reviewer.yml@refs/heads/develop
Waiting for a runner to pick up this job...
Job is waiting for a hosted runner to come online.
Job is about to start running on the hosted runner: GitHub Actions 2 (hosted)
Current runner version: '2.303.0'
##[group]Operating System
Ubuntu 22.04.2 LTS
##[endgroup]
##[group]Runner Image
Image: ubuntu-22.04
Version: 20230409.1
Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230409.1/images/linux/Ubuntu2204-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230409.1
##[endgroup]
##[group]Runner Image Provisioner
2.0.139.1
##[endgroup]
##[group]GITHUB_TOKEN Permissions
Contents: write
Metadata: read
PullRequests: write
##[endgroup]
Secret source: Actions
Prepare workflow directory
Prepare all required actions
Getting action download info
Download action repository 'tibdex/github-app-token@v1' (SHA:b62528385c34dbc9f38e5f4225ac829252d1ea92)
Download action repository 'actions/checkout@v3' (SHA:8e5e7e5ab8b370d6c329ec480221332ada57f0ab)
Download action repository 'dependabot/fetch-metadata@v1.3.6' (SHA:4de7a6c08ce727a42e0adbbdc345f761a01240ce)
Complete job name: review-dependabot-pr
##[group]Run tibdex/github-app-token@v1
with:
  app_id: ***
  private_key: ***
  github_api_url: https://api.github.com
  repository: redacted/redacted-repo
##[endgroup]
Token generated successfully!
##[group]Run actions/checkout@v3
with:
  repository: redacted/redacted-repo
  path: redacted-repo
  token: ***
  ssh-strict: true
  persist-credentials: true
  clean: true
  fetch-depth: 1
  lfs: false
  submodules: false
  set-safe-directory: true
##[endgroup]
Syncing repository: redacted/redacted-repo
##[group]Getting Git version info
Working directory is '/home/runner/work/redacted-repo/redacted-repo/redacted-repo'
[command]/usr/bin/git version
git version 2.40.0
##[endgroup]
Temporarily overriding HOME='/home/runner/work/_temp/ec6851f5-0803-45fe-a587-8dc9a2779829' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/usr/bin/git config --global --add safe.directory /home/runner/work/redacted-repo/redacted-repo/redacted-repo
##[group]Initializing the repository
[command]/usr/bin/git init /home/runner/work/redacted-repo/redacted-repo/redacted-repo
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint: 	git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint: 	git branch -m <name>
Initialized empty Git repository in /home/runner/work/redacted-repo/redacted-repo/redacted-repo/.git/
[command]/usr/bin/git remote add origin https://github.com/redacted/redacted-repo
##[endgroup]
##[group]Disabling automatic garbage collection
[command]/usr/bin/git config --local gc.auto 0
##[endgroup]
##[group]Setting up auth
[command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
[command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
[command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
##[endgroup]
##[group]Fetching the repository
[command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +7f82a6ac524148ad1f0439128d6fdeadad4b58ee:refs/remotes/origin/develop
remote: Enumerating objects: 74, done.        
remote: Counting objects:   1% (1/74)        
 ...[truncated]...
remote: Counting objects: 100% (74/74), done.        
remote: Compressing objects:   1% (1/64)        
 ...[truncated]...
remote: Compressing objects: 100% (64/64), done.        
remote: Total 74 (delta 11), reused 45 (delta 4), pack-reused 0        
From https://github.com/redacted/redacted-repo
 * [new ref]         7f82a6ac524148ad1f0439128d6fdeadad4b58ee -> origin/develop
##[endgroup]
##[group]Determining the checkout info
##[endgroup]
##[group]Checking out the ref
[command]/usr/bin/git checkout --progress --force -B develop refs/remotes/origin/develop
Switched to a new branch 'develop'
branch 'develop' set up to track 'origin/develop'.
##[endgroup]
[command]/usr/bin/git log -1 --format='%H'
'7f82a6ac524148ad1f0439128d6fdeadad4b58ee'
##[group]Run dependabot/fetch-metadata@v1.3.6
with:
  github-token: ***
  skip-commit-verification: false
##[endgroup]
Parsing Dependabot metadata
##[group]Outputting metadata for 1 updated dependency
outputs.dependency-names: pytest
outputs.dependency-type: direct:development
outputs.update-type: null
outputs.directory: /dependabot/develop
outputs.package-ecosystem: pip
outputs.target-branch: develop
outputs.previous-version: 
outputs.new-version: 
outputs.compatibility-score: 0
outputs.alert-state: 
outputs.ghsa-id: 
outputs.cvss: 0
##[endgroup]
##[group]Run gh pr merge --auto --merge "$PR_URL"
"gh pr merge --auto --merge "$PR_URL""
shell: /usr/bin/bash -e {0}
env:
  PR_URL: https://github.com/redacted/redacted-repo/pull/29
  GH_TOKEN: ***
##[endgroup]
##[group]Run echo "APPROVED=$APPROVED" >> $GITHUB_OUTPUT
"echo "APPROVED=$APPROVED" >> $GITHUB_OUTPUT"
shell: /usr/bin/bash -e {0}
env:
  APPROVED: 
##[endgroup]
Post job cleanup.
[command]/usr/bin/git version
git version 2.40.0
Temporarily overriding HOME='/home/runner/work/_temp/f72043ac-62de-49a1-b4c1-be04b289ebd9' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/usr/bin/git config --global --add safe.directory /home/runner/work/redacted-repo/redacted-repo/redacted-repo
[command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
[command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
http.https://github.com/.extraheader
[command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
[command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Evaluate and set job outputs
Cleaning up orphan processes

[oguzhanmeteozturk]

Is there anything else I can supply to make the debugging process easier?

[yeikel]

Is there anything else I can supply to make the debugging process easier?

A Reproducer would be ideal, but I don't know if you'll be able to set that up

Where a Reproducer is a public repository that shows the same problem

[svenjr]

We are seeing this as well for internal Docker image update tags (either incoming or outgoing) have a prefixing v then the outputs.update-type is null

We have tried every variant of this and we have found (using internal docker builds) the following:

v1.5.1 to v1.5.2 -> outputs.update-type: null
v1.5.1 to 1.5.2 -> outputs.update-type: null
1.5.2 to v1.5.3 -> outputs.update-type: null
1.5.2 to 1.5.3 -> outputs.update-type: patch

While it is true that this sort of tag is not strictly semver compliant, I would expect Dependabot to be able to infer this sort of thing with a non-complex tag.

I did just create an issue in Dependabot Core but I looked here to see this and thought it was out issue so I closed it.

[kachick]

This does not seem to be just an update-type problem, but a whole version extraction problem.

outputs.previous-version: 
outputs.new-version: 

This action extracts versions from the default commit message, but your custom configuration looks like it removes the versions or changes the format here.

    commit-message:
      prefix: "build: "
      prefix-development: "build: "
      include: "scope"

Subject: [PATCH] build: (deps-dev): update pytest requirement in /dependabot

Updates the requirements on [pytest](https://github.com/pytest-dev/pytest) to permit the latest version.

fetch-metadata/src/dependabot/update_metadata.ts

Lines 30 to 32 in d1defa4

	export async function parse (commitMessage: string, body: string, branchName: string, mainBranch: string, lookup?: alertLookup, getScore?: scoreLookup): Promise<Array<updatedDependency>> {
	const bumpFragment = commitMessage.match(/^Bumps .* from (?<from>v?\d[^ ]*) to (?<to>v?\d[^ ]*)\.$/m)
	const updateFragment = commitMessage.match(/^Update .* requirement from \S*? ?(?<from>v?\d[^ ]*) to \S*? ?(?<to>v?\d[^ ]*)$/m)

(But I also don't fully understand what https://github.com/dependabot/dependabot-core/blob/8fee559e211d8292aad095256e300d02d9ba451f/common/lib/dependabot/pull_request_creator/message_builder.rb#L151 does either.)

As I understand it, there is no perfect solution for deciding the versions from the commit message with providing the user customization feature.

But I think by providing fixed metadata fields in all commit messages with the dependabot-core, the parser could be made simpler and more stable.

[haoliangyu]

This is a reproducible PR haoliangyu/dependabot-minor-version-update-demo#1 for this problem. The workflow run is here. It was initially used to test a bug in fastify/github-action-merge-dependabot and that bug originates from this issue. If you check the workflow run raw log, you can see

2023-08-03T00:44:36.9920160Z ##[group]Run dependabot/fetch-metadata@v1
2023-08-03T00:44:36.9920532Z with:
2023-08-03T00:44:36.9920860Z   skip-commit-verification: false
2023-08-03T00:44:36.9921240Z   skip-verification: false
2023-08-03T00:44:36.9921820Z   github-token: ***
2023-08-03T00:44:36.9922112Z ##[endgroup]
2023-08-03T00:44:37.7113504Z Parsing Dependabot metadata
2023-08-03T00:44:37.7232030Z ##[group]Outputting metadata for 2 updated dependencies
2023-08-03T00:44:37.7233040Z outputs.dependency-names: webpack, @nestjs/cli
2023-08-03T00:44:37.7233576Z outputs.dependency-type: direct:production
2023-08-03T00:44:37.7234039Z outputs.update-type: null
2023-08-03T00:44:37.7234498Z outputs.directory: /webpack-and-nestjs
2023-08-03T00:44:37.7234998Z outputs.package-ecosystem: npm_and_yarn
2023-08-03T00:44:37.7235448Z outputs.target-branch: main
2023-08-03T00:44:37.7235906Z outputs.previous-version: 
2023-08-03T00:44:37.7236352Z outputs.new-version: 
2023-08-03T00:44:37.7236777Z outputs.compatibility-score: 0
2023-08-03T00:44:37.7237245Z outputs.maintainer-changes: true
2023-08-03T00:44:37.7237693Z outputs.dependency-group: 
2023-08-03T00:44:37.7238079Z outputs.alert-state: 
2023-08-03T00:44:37.7238441Z outputs.ghsa-id: 
2023-08-03T00:44:37.7238764Z outputs.cvss: 0
2023-08-03T00:44:37.7239757Z ##[endgroup]

This is the initial state of the repo https://github.com/haoliangyu/dependabot-minor-version-update-demo/tree/4ceb6428814f98a70bc836537eebe21558fbdf5f if you want to setup your test.

[jhonrocha]

I have added this test on my local, using the commit message from the above PR, and it doesn't set the update-type correctly:

test.only('it properly handles dependencies which contain slashes', async () => {
  const commitMessage =
    'Bumps [webpack](https://github.com/webpack/webpack) to 5.88.1 and updates ancestor dependency [@nestjs/cli](https://github.com/nestjs/nest-cli). These dependencies need to be updated together.\n' +
    '\n' +
    '\n' +
    'Updates `webpack` from 5.73.0 to 5.88.1\n' +
    '- [Release notes](https://github.com/webpack/webpack/releases)\n' +
    '- [Commits](webpack/webpack@v5.73.0...v5.88.1)\n' +
    '\n' +
    'Updates `@nestjs/cli` from 8.2.8 to 10.1.11\n' +
    '- [Release notes](https://github.com/nestjs/nest-cli/releases)\n' +
    '- [Changelog](https://github.com/nestjs/nest-cli/blob/master/.release-it.json)\n' +
    '- [Commits](nestjs/nest-cli@8.2.8...10.1.11)\n' +
    '\n' +
    '---\n' +
    'updated-dependencies:\n' +
    '- dependency-name: webpack\n' +
    '  dependency-type: indirect\n' +
    '  update-type: minor\n' +
    '- dependency-name: "@nestjs/cli"\n' +
    '  dependency-type: direct:production\n' +
    '  update-type: major\n' +
    '...\n' +
    '\n' +
    'Signed-off-by: dependabot[bot] <support@github.com>';

  const getAlert = async () => Promise.resolve({ alertState: '', ghsaId: '', cvss: 0 })
  const getScore = async () => Promise.resolve(0)
  const updatedDependencies = await updateMetadata.parse(commitMessage, '', 'dependabot/nuget/api/rails/coffee', 'main', getAlert, getScore)

  console.error(updatedDependencies);
})

To fix it, we could change the updateMetadata.parse function to parse the commit message for multiple dependencies or fix dependabot-core to set the update-type on the yaml part of the commit. I think the latter is the best.

[FlorianLeChat]

I started having this issue when I started using the grouped updates feature on some of my private repositories and on a public repository today.

Run dependabot/fetch-metadata@v1
  with:
    github-token: ***
    skip-commit-verification: false
    skip-verification: false
Parsing Dependabot metadata
Outputting metadata for [2](https://github.com/FlorianLeChat/Domego/actions/runs/6033966165/job/16371490114?pr=232#step:2:2) updated dependencies
  outputs.dependency-names: mongodb, mongoose
  outputs.dependency-type: direct:production
  outputs.update-type: null
  outputs.directory: /
  outputs.package-ecosystem: npm_and_yarn
  outputs.target-branch: master
  outputs.previous-version: 
  outputs.new-version: 
  outputs.compatibility-score: 0
  outputs.maintainer-changes: false
  outputs.dependency-group: 
  outputs.alert-state: 
  outputs.ghsa-id: 
  outputs.cvss: 0