Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Copy PGP signature to get.jenkins.io #462

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Copy PGP signature to get.jenkins.io #462

wants to merge 1 commit into from

Conversation

MarkEWaite
Copy link
Contributor

@MarkEWaite MarkEWaite commented Apr 30, 2024 •
edited

Copy PGP signature of war file to get.jenkins.io

Fixes jenkins-infra/helpdesk#4055

Since get.jenkins.io already includes the SHA-256 checksum file for the war file and it is copied to two destinations in this script, it seems like a good place to copy the ASCII-armored PGP signatures for the war at the same time.

The sha256 file shows that the file downloaded by the user is the same file that was uploaded.

The asc file shows that the uploaded file was signed by the Jenkins PGP signing key.

Testing done

Confirmed that the 2.456 asc matches the war file with:

$ wget https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
$ gpg --import jenkins.io-2023.key
$ wget https://get.jenkins.io/war/2.456/jenkins.war
$ wget https://repo.jenkins-ci.org/artifactory/releases/org/jenkins-ci/main/jenkins-war/2.456/jenkins-war-2.456.war.asc
$ mv jenkins-war-2.456.war.asc jenkins.war.asc
$ gpg --verify jenkins.war.asc

Confirmed that the script changes worked as expected with the following commands:

MY_WORKDIR=$(mktemp -d)
echo My work directory is $MY_WORKDIR

AGENT_WORKDIR=${MY_WORKDIR}/agent-workdir
SRCDIR=${MY_WORKDIR}/src-dir
WARDIR=${MY_WORKDIR}/dest-war-dir
WAR_WEBDIR=${MY_WORKDIR}/dest-war-webdir
export AGENT_WORKDIR SRCDIR WARDIR WAR_WEBDIR
mkdir -p ${AGENT_WORKDIR} ${SRCDIR} ${WARDIR} ${WAR_WEBDIR}

ARTIFACTNAME=jenkins
BASE=$(pwd)
SSH_OPTS=
VERSION=2.456
WAR=${SRCDIR}/jenkins.war
WAR_SHASUM=${SRCDIR}/jenkins.war.sha256
export ARTIFACTNAME BASE VERSION SSH_OPTS WAR WAR_SHASUM

PKGSERVER=localhost
export PKGSERVER

(cd $SRCDIR && wget https://get.jenkins.io/war/${VERSION}/jenkins.war)
(cd $SRCDIR && wget https://get.jenkins.io/war/${VERSION}/jenkins.war.sha256)
(cd $SRCDIR && wget https://repo.jenkins-ci.org/artifactory/releases/org/jenkins-ci/main/jenkins-war/${VERSION}/jenkins-war-${VERSION}.war.asc && mv jenkins-war-${VERSION}.war.asc jenkins.war.asc)

echo "WARDIR contents before publish:" && ls ${WARDIR}
echo
bash -v war/publish/publish.sh
echo "WARDIR contents after publish:" && ls ${WARDIR}/
echo "WARDIR/VERSION contents after publish:" && ls ${WARDIR}/${VERSION}/

Submitter checklist

Edit tasklist title
Beta Give feedback Tasklist Submitter checklist, more options

Delete tasklist

Delete tasklist block?
Are you sure? All relationships in this tasklist will be removed.
  1. Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    Options
  2. Ensure that the pull request title represents the desired changelog entry
    Options
  3. Please describe what you did
    Options
  4. Link to relevant issues in GitHub or Jira
    Options
  5. Link to relevant pull requests, esp. upstream and downstream changes
    Options

@MarkEWaite
Copy ascii-armored PGP signature of jenkins.war to get.jenkins.io
7474c24 
Fixes jenkins-infra/helpdesk#4055

Since get.jenkins.io already includes the SHA-256 checksum file for the
war file and it is copied to two destinations in this script, it seems
like a good place to copy the ASCII-armored PGP signatures for the war
at the same time.

The sha256 file shows that the file downloaded by the user is the same
file that was uploaded.

The asc file shows that the uploaded file was signed by the Jenkins PGP
signing key.

Confirmed that the 2.456 asc matches the war file with:

$ wget https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
$ gpg --import jenkins.io-2023.key
$ wget https://get.jenkins.io/war/2.456/jenkins.war
$ wget https://repo.jenkins-ci.org/artifactory/releases/org/jenkins-ci/main/jenkins-war/2.456/jenkins-war-2.456.war.asc
$ mv jenkins-war-2.456.war.asc jenkins.war.asc
$ gpg --verify jenkins.war.asc

Confirmed that the script changes worked as expected with the following
commands:

MY_WORKDIR=$(mktemp -d)
echo My work directory is $MY_WORKDIR

AGENT_WORKDIR=${MY_WORKDIR}/agent-workdir
SRCDIR=${MY_WORKDIR}/src-dir
WARDIR=${MY_WORKDIR}/dest-war-dir
WAR_WEBDIR=${MY_WORKDIR}/dest-war-webdir
export AGENT_WORKDIR SRCDIR WARDIR WAR_WEBDIR
mkdir -p ${AGENT_WORKDIR} ${SRCDIR} ${WARDIR} ${WAR_WEBDIR}

ARTIFACTNAME=jenkins
BASE=$(pwd)
SSH_OPTS=
VERSION=2.456
WAR=${SRCDIR}/jenkins.war
WAR_SHASUM=${SRCDIR}/jenkins.war.sha256
export ARTIFACTNAME BASE VERSION SSH_OPTS WAR WAR_SHASUM

PKGSERVER=localhost
export PKGSERVER

(cd $SRCDIR && wget https://get.jenkins.io/war/${VERSION}/jenkins.war)
(cd $SRCDIR && wget https://get.jenkins.io/war/${VERSION}/jenkins.war.sha256)
(cd $SRCDIR && wget https://repo.jenkins-ci.org/artifactory/releases/org/jenkins-ci/main/jenkins-war/${VERSION}/jenkins-war-${VERSION}.war.asc && mv jenkins-war-${VERSION}.war.asc jenkins.war.asc)

echo "WARDIR contents before publish:" && ls ${WARDIR}
echo
bash -v war/publish/publish.sh
echo "WARDIR contents after publish:" && ls ${WARDIR}/
echo "WARDIR/VERSION contents after publish:" && ls ${WARDIR}/${VERSION}/
@MarkEWaite MarkEWaite requested a review from a team as a code owner April 30, 2024 21:55
basil
basil requested changes Apr 30, 2024
View reviewed changes
Copy link
Member

@basil basil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you verified the two callers (https://github.com/jenkinsci/packaging/blob/0f615197fb8721509f673f8d7d60d91d073e639e/prep.sh and the corresponding code in jenkins-infra/release pending

packaging/prep.sh

Lines 5 to 9 in 0f61519

# TODO jenkins-infra/release performs similar preparatory actions: downloading
# the WAR and importing the GPG key. A common interface for the preparatory
# actions should be designed that meets the needs of both local testing and
# releases, ideally implemented in the Makefile. Then both this repository and
# jenkins-infra/release should be refactored to consume the new functionality.
) are passing in the .asc file as input to this script?

@MarkEWaite
Copy link
Contributor Author

Have you verified the two callers

I attempted to verify the utils/release.bash caller by downloading the 2.456 console output from the release job. It includes the following output lines:

[2024-04-30T13:16:35.006Z] + : latest
[2024-04-30T13:16:35.006Z] + : /home/jenkins/agent/workspace/core_package_master/release/war/target/jenkins.war
[2024-04-30T13:16:35.006Z] + : /home/jenkins/agent/workspace/core_package_master/release/war/target/jenkins.war.asc
[2024-04-30T13:16:35.007Z] + : mirrorbrain@pkg.origin.jenkins.io

I believe those lines correspond to the definition of JENKINS_VERSION, JENKINS_WAR, JENKINS_ASC, and PKGSERVER in https://github.com/jenkins-infra/release/blob/33600880649d02153e4526c8281937f22f5d3792/utils/release.bash#L554-L557

I haven't confirmed that the prep.sh script provides the .asc file. That will need to happen tomorrow or the next day.

@basil
Copy link
Member

basil commented May 1, 2024 •
edited

But those lines you referenced are executed by the release job in the release repository, not the package job in the release repository. The release job in the release repository does indeed build the .war and .war.asc file, and uploads them to Artifactory, and even verifies the signature, but it does not invoke the publish scripts in the packaging repository.

Those publish scripts in the packaging repository (which you are modifying in this PR to require a new input) are rather invoked by the package job in the release repository. And since that job didn't just run Maven, it has to download all the inputs to the publish scripts in the packaging repository before invoking them, which is done in https://github.com/jenkins-infra/release/blob/b71d70d7996c85662a24358d0270b671d652e594/Jenkinsfile.d/core/package#L153-L163. So you are mistaken about the entrypoint from the release repository.

The actual entrypoint, like the one in the packaging repository itself, uses jv to download only the .war file, not the .war.asc file. This PR makes the publish scripts in the packaging repository require both a .war file and a .war.asc file, without updating either entrypoint to provide that file. Thus, if this PR was merged, the next release would fail.

For the above reason, my blocking review remains.

@basil
Copy link
Member

basil commented May 1, 2024

Suggested solution is to replace the call to jv with two calls to wget (one to download the .war file from https://repo.jenkins-ci.org/releases/org/jenkins-ci/main/jenkins-war/ and one to download the .war.asc file from https://repo.jenkins-ci.org/releases/org/jenkins-ci/main/jenkins-war/). This would need to be done in both places where we are currently invoking jv: from the package job in the release repository, and from prep.sh in the packaging repository.

As an alternative strategy, a prerequisite refactoring could be first performed to factor the jv invocation into a single Makefile target in the packaging repository, invoked by both prep.sh and the package job in the release repository. With that prerequisite refactoring out of the way, the conversion from jv into two wget calls only needs to happen a single time, rather than two times, and this task becomes easier.

However, since the duplicate code already exists today, I wouldn't insist on that refactoring as part of my blocking review. It would certainly suffice to change the duplicate code in both places, and postpone the refactoring to a future time.

@MarkEWaite
Copy link
Contributor Author

As an alternative strategy, a prerequisite refactoring could be first performed to factor the jv invocation into a single Makefile target in the packaging repository, invoked by both prep.sh and the package job in the release repository. With that prerequisite refactoring out of the way, the conversion from jv into two wget calls only needs to happen a single time, rather than two times, and this task becomes easier.

I like that approach. I'll plan to implement that approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@basil basil basil requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add .war.asc to get.jenkins.io
2 participants
@MarkEWaite @basil