Fix 6403 - add an automatic generated maven deployable p2 site

[laeubi]

See #6403 for a full description of what this PR is about.

I create this as a draft, as the corresponding feature is not (yet) released, but to allow a preview/discussion of this request.

[janbartel]

I think it's a good idea if this can automatically make a p2 repo, it would save some folks a lot of time from curating the hand-crafted deployment scripts etc.

Which jetty jars will be included? Everything? Some of them don't really make sense to deploy in osgi, eg jetty-runner, the demos, the jetty-maven-plugins, jetty-start, the test-osgi wars and jars.

[janbartel]

Some curation of which jars are p2-ified may be necessary.

[laeubi]

Which jetty jars will be included? Everything? Some of them don't really make sense to deploy in osgi

Currently every project that produces a valid OSGi-Bundle as its main artifact is included. This does not mean the user has to use them though as it is possible to select each bundle separately. While this might include more than necessary its the most automatic way with no maintenance (e.g. if new modules are added) no complains that "something" is missing...

As an alternative currently it is possible to limit the set of included items to explicitly declared dependencies, this requires some maintenance if modules are added/removed and a decision what should be included.

This is how it looks like inside eclipse at the moment:

if even more control is needed, one could also provide a category.xml to e.g. categorize items in different ways, add descriptions and so on. This all depend on how much effort is acceptable for the jetty project to supply such a site.

[janbartel]

@laeubi I just checked maven central, and it appears that we do now release all the demo jetty modules (we never used to). So, I it's probably fine if we put into p2 what we put into maven central - in other words, we wouldn't be putting more into p2 than central.

However, another question for you, from the consuming from p2 point of view: as the jetty p2 repository has never contained any of the dependencies bundles, when installing the jetty bundles, one had to first ensure that the appropriate dependencies were installed, otherwise the installation could not resolve. That will still be the case here, right? So I'm wondering if we pack even more jetty jars than before into the p2 repo, won't the user suddenly have to install a lot of dependencies that they're never going to use - here I'm thinking of all the of jetty jars that are integrations, such as with hazelcast, infinispan, gcloud, mongo, fastcgi etc etc?

[laeubi]

The direct dependencies of what is installed need to be present or be provided by other sources, but a consumer of the repository (I won't expect actual users to use these repository directly to install something into eclipse) would usually only select a subset for installation.
e.g eclipse platform currently uses this subset of bundles and dependencies: https://git.eclipse.org/r/c/platform/eclipse.platform.releng.buildtools/+/176736/17/jetty.repository/category.xml

That said, it is also possible to provide additional (maven) dependencies with the site if they are already proper OSGi-Bundles, e.g. the servlet api, the slf4j api and so on to make this more convenient, as these are also only references to maven as well, this does not add any additional cost beside some few extra bytes in the meta-data.

[joakime]

The result of a deployment of this branch looks like this ...

Tree

$ tree org/eclipse/jetty/jetty-p2/
org/eclipse/jetty/jetty-p2/
├── 10.0.5-SNAPSHOT
│   ├── jetty-p2-10.0.5-20210616.115455-1-p2site.zip
│   ├── jetty-p2-10.0.5-20210616.115455-1-p2site.zip.md5
│   ├── jetty-p2-10.0.5-20210616.115455-1-p2site.zip.sha1
│   ├── jetty-p2-10.0.5-20210616.115455-1.pom
│   ├── jetty-p2-10.0.5-20210616.115455-1.pom.md5
│   ├── jetty-p2-10.0.5-20210616.115455-1.pom.sha1
│   ├── maven-metadata.xml
│   ├── maven-metadata.xml.md5
│   └── maven-metadata.xml.sha1
├── maven-metadata.xml
├── maven-metadata.xml.md5
└── maven-metadata.xml.sha1

Deployed File Sizes

$ ls -la org/eclipse/jetty/jetty-p2/10.0.5-SNAPSHOT/
total 72
drwxrwxr-x 2 joakim joakim  4096 Jun 16 06:54 ./
drwxrwxr-x 3 joakim joakim  4096 Jun 16 06:54 ../
-rw-rw-r-- 1 joakim joakim 29893 Jun 16 06:54 jetty-p2-10.0.5-20210616.115455-1-p2site.zip
-rw-rw-r-- 1 joakim joakim    32 Jun 16 06:54 jetty-p2-10.0.5-20210616.115455-1-p2site.zip.md5
-rw-rw-r-- 1 joakim joakim    40 Jun 16 06:54 jetty-p2-10.0.5-20210616.115455-1-p2site.zip.sha1
-rw-rw-r-- 1 joakim joakim  1743 Jun 16 06:54 jetty-p2-10.0.5-20210616.115455-1.pom
-rw-rw-r-- 1 joakim joakim    32 Jun 16 06:54 jetty-p2-10.0.5-20210616.115455-1.pom.md5
-rw-rw-r-- 1 joakim joakim    40 Jun 16 06:54 jetty-p2-10.0.5-20210616.115455-1.pom.sha1
-rw-rw-r-- 1 joakim joakim   818 Jun 16 06:54 maven-metadata.xml
-rw-rw-r-- 1 joakim joakim    32 Jun 16 06:54 maven-metadata.xml.md5
-rw-rw-r-- 1 joakim joakim    40 Jun 16 06:54 maven-metadata.xml.sha1

Contents of Zip File

$ unzip -l org/eclipse/jetty/jetty-p2/10.0.5-SNAPSHOT/jetty-p2-10.0.5-20210616.115455-1-p2site.zip 
Archive:  org/eclipse/jetty/jetty-p2/10.0.5-SNAPSHOT/jetty-p2-10.0.5-20210616.115455-1-p2site.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
   426144  2021-06-16 06:54   content.xml
    84817  2021-06-16 06:54   artifacts.xml
---------                     -------
   510961                     2 files

[janbartel]

@laeubi I'm fine with this addition to jetty, just need the final release of the tycho plugin to give it the final thumbs up.

[olamy]

@laeubi is there any plan to have soon a release 2.4.0 of the plugin? Because we are in the process of cutting soon a release of Jetty and we are not able to merge this PR.
Thanks

[laeubi]

I'll bring this question to the mailinglist and let you know, if a release is scheduled it mostly takes 1-2 weeks for staging and test until the release is finally available, would this be sufficient? If not is there already a plan for the following release so we can plan to release at least for that?

[janbartel]

@laeubi I've moved this from our imminent release (10.0.6) to our next release (10.0.7) - we're supposedly in code-freeze now, so I don't want to try and shoe-horn in something that isn't quite ready yet.

[laeubi]

I've moved this from our imminent release (10.0.6) to our next release (10.0.7)

Thanks for the information, is there already a plan when 10.0.7 will be released?

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[laeubi]

not stale... tycho release is in progress...

[olamy]

@laeubi no worries. we have a bot tagging old PR without activity but we do not close them.

[laeubi]

At the momment you need to checkout the jetty-repo and run a build with

mvn clean install -DskipTest on the root of the project.

You can then use file:/your/chekout/location/jetty/jetty-p2/target/repository or mvn:org.eclipse.jetty:jetty-p2:10.0.7-SNAPSHOT:zip:p2site as an URL.

If you like to sign unsigned artifacts in an update-site produced by any platform-project it is demonstrated here https://github.com/eclipse-m2e/m2e-core/blob/master/org.eclipse.m2e.site/pom.xml#L41 for m2e signing artifacts that are not form an already signed source (bnd project in this case).

[jonahgraham]

@jonahgraham and @mknauer have claimed interest in the issue, maybe they can tell if anything is missing from their side?

My interest is that the flow improve and that there is a single site to pull jetty from. The way the simrel how to respin for 2021-06 was not ideal. I appreciate the progress being made. The IDE WG steering committee has the signing issue at the top of the agenda for today's meeting, so hopefully we can progress there.

[janbartel]

@laeubi if you can resolve the conflicts, I think we can merge this PR.

[laeubi]

That's great, as next week there is a new P2 release I think I'll wait for this an dthe new tycho version so we are up-to date everywhere.

[joakime]

@laeubi this PR cannot be merged, it has conflicts.

[laeubi]

@laeubi this PR cannot be merged, it has conflicts.

Thanks for the hint I'll update it as soon as the updated tycho is released.

[joakime]

@laeubi this PR cannot be merged, it has conflicts.

Thanks for the hint I'll update it as soon as the updated tycho is released.

I was able to fix the merge conflict using the github web ui, see commit 870fbee

[joakime]

@akurtakov and @laeubi

The pack200 plugins have now been removed from branches : jetty-9.4.x, jetty-10.0.x, and jetty-11.0.x
Along with this, the existing ${tycho.version} property has been removed.

[joakime]

@laeubi we want to get Jetty 10.0.7 and 11.0.7 releases out in the next week (or two).

If you can get us a non-snapshot plugin to use, then we can merge this PR and use it for these upcoming releases.

[laeubi]

@joakime we are currently prepare a new Tycho release that will be out (if no regressions are found) at end of next week, as soon as it is out I'll update the PR (most likely around 3. October) if thats not to late...

[joakime]

We'll wait till Oct 1st for our Jetty 10.x and 11.x releases.
If we don't have progress in this PR by then, we'll have to release without this PR.

[laeubi]

Thanks for your patience, we are working hard to get the release out as soon as possible 👍

[laeubi]

We'll wait till Oct 1st for our Jetty 10.x and 11.x releases. If we don't have progress in this PR by then, we'll have to release without this PR.

I have adjusted to use the latest tycho release and integrated the latest changes now.

[janbartel]

Thanks @laeubi for this PR, it's ready to merge.

[olamy]

@laeubi thanks a lot for your hard work

[wimjongman]

The next question is: how we get our hands on the p2 repo?

[laeubi]

@olamy thanks for adjusting the code, I was away for a few days... and of course thanks to all reviewers as well. Good to the see first pure maven-p2 updatesite will be available soon 🥇

@wimjongman for general usage information/URL layout you can take a look at the assemble-maven-repository description.

For this particular case, if the jetty release is out you can access it with the following URL (given you use tycho or m2e): mvn:org.eclipse.jetty:jetty-p2:10.0.7:zip:p2site

Just keep in mind that due to current PDE restrictions such an update site might require you to add at least one "traditional" update-site (e.g. the eclipse sdk) if you are using the planner mode, but this should be the usual case already for any non-trivial target-file. Let me know if you are facing any issues or like to suggest improvements.