Skip to content

Latest commit

 

History

History
530 lines (402 loc) · 16.3 KB

README.md

File metadata and controls

530 lines (402 loc) · 16.3 KB
Raw

Incrementals

See JEP-305 for context.

Usage in plugin POMs

Since most Jenkins repositories host plugins, this use case will be documented first. You must be using parent POM version 3.10 or later.

Enabling incrementals (the easy way)

Just run

mvn io.jenkins.tools.incrementals:incrementals-maven-plugin:incrementalify -DgenerateBackupPoms=false

or if your POM is already new enough (3.10+)

mvn incrementals:incrementalify

Check the usual build

mvn clean package

and if all is well,

git add .mvn pom.xml
git checkout -b incrementals
git commit -m Incrementalified.

and file as a pull request.

Enabling incrementals (the hard way)

Enabling consumption of incrementals

If your plugin has (or may have) dependencies on incremental versions, run:

mkdir -p .mvn
echo -Pconsume-incrementals >> .mvn/maven.config
git add .mvn

(See this guide for details on the .mvn directory.)

This profile merely activates access to the Incrementals repository.

Enabling production of incrementals

To produce incremental artifacts from your plugin, first edit your pom.xml. If your plugin declares

<version>1.23-SNAPSHOT</version>

then replace that with

<version>${revision}${changelist}</version>

and then in the <properties> section add

<revision>1.23</revision>
<changelist>-SNAPSHOT</changelist>

If you have a multimodule reactor build, the new properties need be defined only in the root POM, but every child POM should use the edited version to refer to its parent. (It should not override the version.) Intermodule dependencyes may use ${project.version} to refer to the version of the sibling.

Also change

<scm>
  <!-- … -->
  <tag>HEAD</tag>
</scm>

to

<scm>
  <!-- … -->
  <tag>${scmTag}</tag>
</scm>

Also replace jenkinsci/your-repo occurrences in the scm section with a POM property named gitHubRepo. This will ensure that deployments from forked pull requests will give correct checkout information, used for example by plugin-compat-tester.

Now run

mkdir -p .mvn
echo -Pmight-produce-incrementals >> .mvn/maven.config

Finally, configure git-changelist-maven-extension in .mvn/extensions.xml. (Update the version to the latest version for this tool.):

<extensions xmlns="http://maven.apache.org/EXTENSIONS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0 http://maven.apache.org/xsd/core-extensions-1.0.0.xsd">
  <extension>
    <groupId>io.jenkins.tools.incrementals</groupId>
    <artifactId>git-changelist-maven-extension</artifactId>
    <version>1.0-beta-7</version>
  </extension>
</extensions>

You may now

git add .mvn pom.xml

and commit and push your edits.

Production and consumption

A single plugin may both consume Incrementals releases, and produce its own. Just make both kinds of edits. (.mvn/maven.config may have multiple lines.)

Producing incrementals

Assumes you have set up the might-produce-incrementals as above, either by hand or using the incrementalify goal.

If you file a pull request built on ci.jenkins.io, and the pull request is up to date with its target branch, and the build is stable, the artifact will be automatically deployed to the Incrementals repository. Your plugin will need to have a github field in repository-permissions-updater if it does not already.

To check from a shell if your deployment is ready, get the gh CLI and try:

$ gh pr checks | fgrep incrementals | awk '{print $4}'
https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/ldap/1.27-rc474.9577fe6b8ea6/

To produce equivalent artifacts in your local repository while working offline:

mvn -Dset.changelist -DskipTests clean install

If you do not select the -Dset.changelist option, you will create a regular *-SNAPSHOT artifact. (And that is what you must do if you have any local modifications or untracked files.)

Updating dependencies

Once you have some dependencies on incremental versions in your POM, you can

mvn incrementals:update

to get a newer version of some dependencies, if merged; or

mvn incrementals:update -Dbranch=yourghacct:experiments-JENKINS-12345

to get the most recent versions from some set of unmerged PRs. Then commit and push the resulting pom.xml edits.

You will need GitHub credentials for this: instructions

Updating versions for Jenkins Docker images

Official Jenkins Docker images offer plugins.txt which supports Incrementals. See this page for more information. Incrementals maven plugin can be used to update plugin versions there. Currently, only Incrementals version update is supported.

mvn incrementals:updatePluginsTxt -DpluginsFile=plugins.txt

When plugins.txt format is used, it is also possible to pass the branch name to the incrementals definition so that the Incrementals version is updated from a particular branch.

Example of the file with incrementals:

scm-api:latest
script-security:latest
workflow-aggregator:2.5
# https://github.com/jenkinsci/workflow-api-plugin/pull/17
workflow-api:incrementals;org.jenkins-ci.plugins.workflow;2.30-rc-802-fa-5-alpha-94-c-8-alpha-65;jglick;logs-JENKINS-38381
workflow-basic-steps:latest
# https://github.com/jenkinsci/workflow-durable-task-step-plugin/pull/21
workflow-durable-task-step:incrementals;org.jenkins-ci.plugins.workflow;jglick;2.20-rc333.74dc7c303e6d
# https://github.com/jenkinsci/workflow-job-plugin/pull/27
workflow-job:incrementals;org.jenkins-ci.plugins.workflow;2.25-rc-824.49-c-91-cd-14666;jglick;logs-JENKINS-38381
workflow-multibranch:latest
# https://github.com/jenkinsci/workflow-support-plugin/pull/15
workflow-support:incrementals;org.jenkins-ci.plugins.workflow;2.21-rc-617.27-alpha-34-dc-2-c-64-c;jglick;logs-JENKINS-38381
# Incrementals from the master branch
artifact-manager-s3:incrementals;io.jenkins.plugins;1.2-rc82.a1e113b09b19

Running Maven releases

You may still use the Maven release plugin (MRP) when might-produce-incrementals is activated:

mvn -B release:{prepare,perform}

The released artifacts should have sensible metadata. (You may notice that they deploy a “flattened” POM file, but this should not break anything.)

Sufficiently recent parent POMs (3.18+) also include a incrementals:reincrementalify mojo run as part of completion goals, so you will notice that the [maven-release-plugin] prepare for next development iteration commit brings your source tree back to a state where the plugin is ready to produce Incrementals. To verify that this is working, after running a release try running

git diff HEAD^^

You should see something like

--- a/pom.xml
+++ b/pom.xml
   <version>${revision}${changelist}</version>
   <properties>
-    <revision>1.1</revision>
+    <revision>1.2</revision>
     <changelist>-SNAPSHOT</changelist>
   </properties>

indicating that the net effect of the [maven-release-plugin] prepare release something-1.1 commit and the commit after it is to change the plugin from 1.1-SNAPSHOT to 1.2-SNAPSHOT. If this failed and your <version> was still a number, you can manually run

mvn incrementals:reincrementalify

to fix it up.

Superseding Maven releases

If you want to use Incrementals instead of MRP, you can override changelist.format in your project (the default value is -rc%d.%s).

For a regular component whose version number is not intrinsically meaningful:

--- a/.mvn/extensions.xml
+++ b/.mvn/extensions.xml
@@ -2,6 +2,6 @@
   <extension>
     <groupId>io.jenkins.tools.incrementals</groupId>
     <artifactId>git-changelist-maven-extension</artifactId>
-    <version>1.0-beta-7</version>
+    <version>1.1</version>
   </extension>
 </extensions>
--- a/.mvn/maven.config
+++ b/.mvn/maven.config
@@ -1,2 +1,3 @@
 -Pconsume-incrementals
 -Pmight-produce-incrementals
+-Dchangelist.format=%d.v%s
--- a/pom.xml
+++ b/pom.xml
@@ -7,7 +7,7 @@
-    <version>${revision}${changelist}</version>
+    <version>${changelist}</version>
     <packaging>hpi</packaging>
@@ -26,8 +26,7 @@
     <properties>
-        <revision>1.23</revision>
-        <changelist>-SNAPSHOT</changelist>
+        <changelist>999999-SNAPSHOT</changelist>
         <jenkins.version>2.176.4</jenkins.version>
         <java.level>8</java.level>
     </properties>

Here a CI/release build (-Dset.changelist specified) will be of the form 123.vabcdef456789. A snapshot build will be 999999-SNAPSHOT: arbitrary but treated as a snapshot by Maven and newer than any release.

For a component whose version number ought to reflect a release version of some wrapped component:

--- a/.mvn/extensions.xml
+++ b/.mvn/extensions.xml
@@ -2,6 +2,6 @@
   <extension>
     <groupId>io.jenkins.tools.incrementals</groupId>
     <artifactId>git-changelist-maven-extension</artifactId>
-    <version>1.0-beta-7</version>
+    <version>1.1</version>
   </extension>
 </extensions>
--- a/.mvn/maven.config
+++ b/.mvn/maven.config
@@ -1,2 +1,3 @@
 -Pconsume-incrementals
 -Pmight-produce-incrementals
+-Dchangelist.format=%d.v%s
--- a/pom.xml
+++ b/pom.xml
@@ -10,12 +10,12 @@
   <artifactId>some-library-wrapper</artifactId>
-  <version>${revision}${changelist}</version>
+  <version>${revision}-${changelist}</version>
   <packaging>hpi</packaging>
   <properties>
-    <revision>4.0.0-1.3</revision>
-    <changelist>-SNAPSHOT</changelist>
+    <revision>4.0.0</revision>
+    <changelist>999999-SNAPSHOT</changelist>
     <jenkins.version>2.176.4</jenkins.version>
     <java.level>8</java.level>

Here the version numbers will look like 4.0.0-123.vabcdef456789 or 4.0.0-999999-SNAPSHOT, respectively. When you pick up a new third-party component like 4.0.1, your version numbers will match; to refer to the third-party component, just use

<version>${revision}</version>

Since inadvertently running MRP on such a project would result in a mess, it is best to explicitly prevent that. This is done automatically by the plugin and general component parent POMs when you define the changelist.format property.

Automatic deployment

If you have configured changelist.format as above, using JEP-229 you can then set up automatic publishing of artifacts.

First, file a patch to your component’s entry(ies) in RPU to add

cd:
  enabled: true

Once that has been merged, start checking https://github.com/jenkinsci/your-plugin/settings/secrets/actions and you should soon see MAVEN_TOKEN and MAVEN_USERNAME appear under Repository secrets.

Next, if you already have Release Drafter configured, remove any tag-template override in .github/release-drafter.yml, and delete .github/workflows/release-drafter.yml if using GitHub Actions (or remove the app from the repo otherwise). If you have not yet configured Release Drafter, just create .github/release-drafter.yml containing only

_extends: .github

Now create .github/workflows/cd.yaml as follows:

name: cd
on:
  workflow_dispatch:
  check_run:
    types:
    - completed
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - name: Verify CI status
      uses: jenkins-infra/verify-ci-status-action@v1.1.0
      with:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    - name: Release Drafter
      uses: release-drafter/release-drafter@v5.13.0
      with:
        name: next
        tag: next
        version: next
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    - name: Check out
      uses: actions/checkout@v2.3.4
      with:
        fetch-depth: 0
    - name: Set up JDK 8
      uses: actions/setup-java@v1
      with:
        java-version: 1.8
    - name: Release
      uses: jenkins-infra/jenkins-maven-cd-action@v1.1.0
      with:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        MAVEN_USERNAME: ${{ secrets.MAVEN_USERNAME }}
        MAVEN_TOKEN: ${{ secrets.MAVEN_TOKEN }}

Now whenever Jenkins reports a successful build of your default branch, and at least one pull request had a label indicating it was of interest to users (e.g. enhancement rather than chore), your component will be released to Artifactory and release notes published in GitHub. You do not need any special credentials or local checkout; just merge pull requests with suitable titles and labels.

You can also trigger a deployment explicitly, if the current commit has a passing check from Jenkins. Visit https://github.com/jenkinsci/your-plugin/actions?query=workflow%3Acd and click Run workflow. If you prefer to only deploy explicitly, not on every push, just comment out the check_run section above.

(Do not be alarmed to see numerous failed cd workflows besides the passing ones that actually deployed: every check posted other than the top-level Jenkins check triggers a workflow which should quickly fail.)

As a fallback, you can also release manually if you have personal deployment credentials. To cut a release:

git checkout master
git pull --ff-only
mvn -Dset.changelist -DaltDeploymentRepository=maven.jenkins-ci.org::default::https://repo.jenkins-ci.org/releases/ clean deploy

Usage in other POMs

From repositories with POMs not inheriting from org.jenkins-ci.plugins:plugin you can follow similar steps to use Incrementals. If you inherit from org.jenkins-ci:jenkins, the same profiles are available; otherwise you will need to copy the definitions of the consume-incrementals, might-produce-incrementals, and produce-incrementals profiles from org.jenkins-ci:jenkins, as well as the incrementals.url and scmTag properties, into your parent POM or directly into your repository POM. Some adjustment of maven-enforcer-plugin configuration may also be necessary.

Publishing

Once Incrementals is enabled in a plugin repository, the stock buildPlugin method takes care of publishing artifacts from stable builds up to date with the base branch. For libraries or other components with custom Jenkinsfiles, you will need to set this up manually:

node('maven') {
  checkout scm
  sh 'mvn -Dset.changelist install'
  infra.prepareToPublishIncrementals()
}
infra.maybePublishIncrementals()

Offline testing

If you wish to test usage offline, run

docker run --rm --name nexus -p 8081:8081 -v nexus-data:/nexus-data sonatype/nexus3

Log in to http://localhost:8081/ and pick an admin password as per instructions, then add to your ~/.m2/settings.xml:

<servers>
  <server>
    <id>incrementals</id>
    <username>admin</username>
    <password>admin123</password>
  </server>
</servers>

and then add to command lines consuming or producing incremental versions:

-Dincrementals.url=http://localhost:8081/repository/maven-releases/

or define an equivalent profile in local settings.

Changelog

1.1 and later

See GitHub releases.

1.0-beta-7

2018 Sep 04

  • mvn incrementals:update mishandled property expressions.

1.0-beta-6

2018 Aug 30

  • New mvn incrementals:updatePluginsTxt goal.

1.0-beta-5

2018 Jul 24

  • Support mvn incrementals:incrementalify on projects using the org.jenkins-ci:jenkins parent.

1.0-beta-4

2018 Jul 19

  • JENKINS-51869: no longer using --first-parent in revision count.
  • Match indentation in mvn incrementals:incrementalify.
  • Make mvn incrementals:reincrementalify fail comprehensibly on a non-incrementalified repository.

1.0-beta-3 and earlier

Not recorded.