Skip to content

Release automation Analysis for proposal

Jump to bottom
Lorenzo Natali edited this page Mar 15, 2022 · 11 revisions

Overview

The goal of this improvement is to improve and automate the release procedure in MapStore

Proposed By

  • Lorenzo Natali

Assigned to Release

The proposal is for 2022.02.00 or future.

State

  • Under Discussion
  • In Progress
  • Completed
  • Rejected
  • Deferred

Motivation

Creating a mapstore release takes a lot of time and resources, trying to automate at least some steps of it can help to be more productive and remove work load from the MapStore team.

Proposal

Initial status

Given that the #7165 will been completed, this document what to analyze the current release procedure in order to identify the points that can be automated.

The release is made of 7 phases.

  1. Initialization
    1. create the Issue
    2. create the branch 1
    3. on master branch Increase versions (package.json verson , pom.xml snapshot for mapstore-services and other) 1
    4. On release branch Fix dependencies to not use -SNAPSHOT (geostore, http-proxy, mapfish-print, ...)1
    5. Setup jenkins for stable branch on QA
    6. Create read the docs for the release branch 1
  2. Testing / Backports / Fixes
  3. Release Preparation
    1. Fix versions for mapstore-services and other modules (pom.xml)
    2. Update template pom.xml
    3. Update changelog 3
    4. commit all these changes and the stable branch.
  4. Release
    1. Trigger mapstore releaser
    2. Release mapstore-services and other modules on maven repo (this should be moved to the mapstore-releaser task)
    3. Download the artifacts (binary, war, printing-module) and upload them in the release.
    4. Upload artifacts to a new draft release, pointing to the latest commit of the branch (with fixed versions)
    5. publish the release (this creates a new tag)
    6. Create readthedocs for the release tag
    7. Create MapStoreExtension release on Github (it has it's own steps, upload artifacts etc.). 3
    8. Publish mapstore demo site running Jenkins
  5. Post release
    1. Reset versions to SNAPSHOT, increase package.json to next minor.
    2. Create
    3. on master branch:
      1. Commit changelog 3
      2. Update release procedure if needed
  6. Public relations
    1. Blog post
    2. mailing list
  7. Close
    1. Close the issue
    2. Close the milestone (or ZenHub release)

notes

  • 1 Only for major release.
  • 2 The changelog have to be changed to work on new releases for github (actually they are based on milestones).
  • 3 The creation of the new sample extensions release start before, for testing.

Analysis

Points 1 have to be manual because the dependencies versions have to be set to fixed accordigly with our updates to the various libraries. At least until geostore and proxy are outside the application. There are some commands to not use snapshot in maven, but we don't have knowledge of it and automating it may be sometimes dangerous. We can ask to the GeoServer team if they use these commands or what to update GeoServer across releases.

Point 2 have to be manual bacause we have manual tests.

Point 3.1 could be automated (it is the execution of one command mvn versions:set -DnewVersion=<VERSION> -DprocessAllModules -DgenerateBackupPoms=false. )3.2 and 3.3 maybe automated but we have to investigate. We may not need to update the template if we adopt the new project system.

Point 4 is already mostly automatic. Publishing of the packages and creation of the release can be automated. Anyway is not that work. Moreover we want to manually edit the github release, so this is not needed.

Point 5.1 can be automated (it is one command that can be parametrized), but 5.2 and 5.3 have to be done manually in any case.

Point 6 and 7 are human driven.

So basically the only procedures of points 3, 4 and 5 can be automated, at least partially.

about automating Release Preparation (Point 3)

The phase 3 (apart from changelog that needs to be managed separately ) includes essentially the fix of the version on pom.xml files. This can be done using a command. Anyway some template pom.xml for the project generator have to be updated too.

In particular, the version of mapstore-services should be consistent with the dependency of the generated project's pom.xml. We can automate this with a string replacement, but it can be hard and may be difficult to maintain.

about automating Release (Point 4)

The release is partially automatic, we can improve it by:

  • automate release of mapstoreExtension (maybe github actions or jenkins?)
  • automatic upload and release creation on github (I don't think is so useful).

Avoiding to use jenkins for release cration and try to do it fully with github actions may simplify the things, automatically publishing artifacts, and the same can be done on MapStore extension.

https://trstringer.com/github-actions-create-release-upload-artifacts/

In this case we have only to create the release and the artifacts will be published automatically by github.

Another task to take in consideration is the deploy of mapstore-services and other packages on maven repo. This can still be done by jenkins (but in this case it should not skip any commit, sometimes it do one build that includes many commits).

Also mvn deploy could be delegated to github, that can monitor multiple branches and do the upload.

about automating Post release (Point 5)

Same as Point 3, but restoring the snapshot. The tasks on master could be removed (changelog, release procedure updates).

Ideas

  • Ask the GeoServer team if they already have some release procedure that automates some parts, for taking inspiration on it.
  • We could remove the publishing of the changelog in order to simplify a little more. We can rely on release notes and the automatic features of release of github (compare, etc...) to get the list of things. **We need to see how to review changelog before 2022.01.01 because current changelogs rely on milestones, but we are not using anymore, in favor of ZenHub releases.
  • We could improve our CI/CD to deploy dependencies and maven packages that are missing to simplify the amount of manual work and avoid mistakes. In particular we should make jenkins deploy java artifacts for all the branches (stable, master) and also mapstore-releaser (for fixed versions of the artifacts).

Conclusions

The procedure will be simplified a lot by #7165. Other improvements can come from:

  • Automating release files creation and maven deploy using github actions. We delegate to jenkins only the role of deploying stuff our infrastructure
  • Other automations can be done for things that have to be committed, but I don't think they will add more complexity without benefits.

So point 4 will become simply:

  1. Create release on Github for MapStore
  2. Create release on Github for MapStoreExtension (this have to be reviewed anyway to anticipate the branching and testing etc... in testing)
  3. Publish mapstore demo by running jenkins ( we may be able to automate also this triggering the build from github action )
  4. Create readthedocs for the release tag (we may be able automate also this)

Moreover, some maintaince overload is due to:

  • migration guidelines: new project system: it may improve the migration guideline work for the future. ( after a first difficulty of migrating to new project system, should benefit of the new work).
  • management of dependencies across projects ( geostore, http-proxy, mapfish-print and mapstore). This can be helped both by using the new project system and / or include them in MapStore github repo
  • mapstore extension with the new project system may make easier to test and automate.
Clone this wiki locally