Enhancements for new LAYERED_JAR layout in spring-boot-maven-plugin

[aritzbastida]

I've read the blog post regarding the new Docker-related features in the upcoming Spring Boot 2.3 Maven plugin. They are very interesting!

In our company, we are using Openshift, so I guess we cannot use Buildpacks. As far as I know, these are for Cloud Foundry and Heroku mainly. Please correct me if I'm wrong.

So, we would use Layered Jars. For these, I'd like to propose some enhancements, if you are open to suggestions (considering it's still in M2):

• The layers should be configurable. In our company, there is a different update cycle for third-party dependencies (i.e. infrastructure) and organisation dependencies (REST clients and such). So the latter should be positioned in an upper layer.
• I think it's a bit weird to package a Jar in LAYERED_JAR format, because it will be used for building images, not for execution; previous uber-jar format was fine enough for that. It's quite convolute to package it as a Jar, then extract it with "layertools" and finally build the image. We could avoid packaging altogether!
• If we launch the command "mvn deploy", this Layered Jar will be deployed onto the Maven repo, which probably is undesired. Normally, we would want to push the resulting image onto the Docker registry (Openshift in our case), and the attached artifacts (Spring Cloud Contract stubs, Javadoc, etc.) to Maven repo, for dependency resolution.
• Note that spring-boot:build-image can be launched standalone; but Layered Jar depends on the package phase, and is executed after maven-jar-plugin or maven-war-plugin. As a result, even if we configure "attach=false", the original jar/war will be deployed on the repo; no way to avoid that.
• As a result, I think that the Layered Jar format makes more sense in the "build-image" goal, or in a new "prepare-image" goal. This goal would structure the layers in "exploded" form (no packaging), under "target" folder. It could even generate the Dockerfile or use one provided via configuration.
• The way to provide the Dockerfile could be similar to that in maven-assembly-plugin: either specify the path for the Dockerfile (like ) or reference a Maven artifact that contains it (like ).
• The default Dockerfile now would be simpler, no need for multi-stage:

FROM adoptopenjdk:11-jre-hotspot
WORKDIR target/application-layers
COPY dependencies/ ./
COPY snapshot-dependencies/ ./
COPY resources/ ./
COPY application/ ./
ENTRYPOINT ["java", "org.springframework.boot.loader.JarLauncher"]

• If the number of layers is configurable, the default Dockerfile would contain as many COPY instructions as layers.

All in all, I think this behavior would be more intuitive and more aligned with the potential use cases. What do you think?

[mbhave]

@aritzbastida Thank you for trying out the Spring Boot milestone and providing feedback. I'll try to speak to some of the points you've mentioned above.

For layer customization, this is something we are planning to support. It is listed as one of the items in the original issue.

Regarding skipping packaging of the fat jar for layered support, this is something we had considered. This reply from @wilkinsona on the blog post addresses why we think layering the jar has advantages. Indeed the concise docker file is a benefit to the approach you suggested.

When it comes to generating a Dockerfile, we don't think we want to do this in Spring Boot because we think the Dockerfile will be customized more often than not and there are things to consider such as choosing the right base image. This is something we think is a use case more suited to start.spring.io.

Flagging for team attention so that we can discuss this on our team call since it is something that has come up before.

[scottfrederick]

In our company, we are using Openshift, so I guess we cannot use Buildpacks. As far as I know, these are for Cloud Foundry and Heroku mainly. Please correct me if I'm wrong.

In the context of Spring Boot features, "buildpacks" refers to Cloud Native Buildpacks. This is very different from the "legacy" buildpack support in Cloud Foundry and Heroku you might be thinking of, as it is designed around container standards and OCI images. While Cloud Foundry and Heroku initiated and support CNB, Kubernetes-native tools like Tekton are building support for CNB also.

[aritzbastida]

First of all, thank you for the quick reply and attention to this issue.

I now understand the benefits pointed out by @wilkinsona, regarding the LAYERED_JAR format. I think that the confusion with Layered Jars comes from the fact that it might be used in two different use cases: builder images (such as Openshift S2I) and custom Dockerfiles.

So, to sum it up, there are three different use cases:

• Build image (Buildpack): Not supported in Openshift, as far as I know. I will ask Red Hat to know whether it's in the roadmap.
• Package as Layered Jar and provide that to a "builder image" on the cloud platform that understands such format, which is in charge of building the image. I guess this is the benefit pointed out by @wilkinsona. In Openshift I could create a custom S2I (source-to-image) builder image; I'll investigate that.

$ oc new-build layered-jar-java --name=hello-boot --binary=true
$ oc start-build bc/hello-boot --from-file=./target/hello-boot-0.1.0.jar

• Create the Layered Jar in exploded format, and execute the Dockerfile provided by the user to build the image. The "instructions" to build the image are not known by the cloud platform, but by the user. This is the method explained in the blog post. In this case, the JAR itself is useless, because it is not deployed anywhere.

$ cat build.Dockerfile | oc new-build --name hello-boot --dockerfile='-'`

For this 3rd use case, maybe it would suffice to provide a "exploded" attribute in "repackage" goal in order to have a simpler Dockerfile. I think that the default generation of Dockerfile would still be convenient, which could be customized or even provided by the user as in maven-assemply-plugin . After all, Spring Boot is opinionated and with the Buildpack it is no different, which uses "cloudfoundry/cnb:0.0.43-bionic" as base image.

[snicoll]

Thanks for the feedback. We've discussed some more today.

If we launch the command "mvn deploy", this Layered Jar will be deployed onto the Maven repo, which probably is undesired.

Not necessarily. If your arrangement is to only use the image, you can configure the plugin's attach property.

As a result, even if we configure "attach=false", the original jar/war will be deployed on the repo; no way to avoid that.

You can configure the maven-deploy-plugin to not deploy individual modules.

For this 3rd use case, maybe it would suffice to provide a "exploded" attribute in "repackage" goal in order to have a simpler Dockerfile.

We've discussed the option of having an "in place" goal that would generate an exploded structure rather than creating an archive. We've decided not to explore this route at this point as dealing with incremental changes of an exploded structure is quite tricky to get right. And if it doesn't work properly and you have to invoke clean anyway, the benefit is not quite obvious to me. At this point, we prefer
a consistent input source, i.e. the repackged jar

I think that the default generation of Dockerfile would still be convenient,

It would when you are getting started yes. Considering we're not keen to offer options to customize this "default" dockerfile, I don't think we should do that. Once you have your docker file in your project with your customizations, having this default generated is more annoying than anything else. Maybe there are other options we can investigate to help the getting started experience though.

We are going to create separate issues for the layer customizations.

[aritzbastida]

Ok, understood your points. I guess we can close the issue. Just some final observations about "mvn deploy":

Not necessarily. If your arrangement is to only use the image, you can configure the plugin's attach property.

Yes, but with attach=false, Maven would still deploy the primary artifact (JAR/WAR produced by default-jar/default-war executions).

You can configure the maven-deploy-plugin to not deploy individual modules.

Yes, you can skip deploying individual modules within a multi-module, but not individual artifacts within the same module. In this use case, we would deploy Spring Cloud Contract stubs to the Maven repo (used as dependencies by other services), but not the "main" artifact, which is built as an image and deployed to the Docker registry.

Not a big issue, really. I just noticed the difference between build-image and layered jars (first one is independent and second one depends on previous package), even if the intent is the same.

[snicoll]

Thanks for the feedback.

Yes, but with attach=false, Maven would still deploy the primary artifact (JAR/WAR produced by default-jar/default-war executions).

Yes, you mentioned that already and there isn't anything we can do about that.

not the "main" artifact, which is built as an image and deployed to the Docker registry.

It doesn't have to be the main artifact and the image isn't a maven artifact at all at the moment. We have plans to include a way to deploy to a docker registry but it's not implemented yet. As for the repackage goal, it isn't mandatory as the build image goal will repackage things on the fly if necessary (i.e. you can skip the execution with id repackage if you're using our parent).

I am going to close this now as you suggested. We have created issues to custimize layers in both Maven and Gradle. Thanks again for the feedback.

[snicoll]

Closing in favour of #20295 and #20296