Allow building executable semi-fat-jar without dependencies with spring-boot-maven-plugin

[fprochazka]

I want to be able to cache the dependencies as a separate docker layer, so deployments are faster and the resulting image is smaller. But I don't want to give-up the benefits of using spring-boot-maven-plugin. An option to not include the dependencies when repackaging would allow this.

Details

1. I have multi-module setup
   • bussines logic is in core and there is another core-http (depends on core)
   • cli which is a cli app, that handles tasks that don't require UI (depends on core)
   • rest-api and web-admin are separate modules (depend on core-http)
2. I build three executable jars - cli.jar, rest-api.jar, web-admin.jar
3. they're packed in a docker image for deployment on AWS ECS
4. thanks to a custom entrypoint, I can choose which executable is executed

Now this works fine, but there are problems

1. every jar has almost exactly the same dependencies so they're three times in the resulting docker images which increases it's size
2. big docker image is slower to upload even on a fast connection

What I would ideally like is to have an option to exclude lib/ from resulting fat-jar, which would allow me to do

1. configure the maven-dependency-plugin to copy everything into dependency directory
   • I have maven-enforcer-plugin configured to check there is no lib installed with two different versions
2. aggregate jars from all three modules into a single lib/ directory that is copied into the docker image as a separate layer
3. copy the semi-fat-jars into the docker image
4. modify entrypoint to add the lib/ as a classpath option
5. profit

This would result in a separate layer with dependencies that rarely changes and separate layer with executable semi-fat-jars. The dependencies layer won't have to be uploaded with every deployment, but only when the dependencies change.

---

I'm currently trying to configure this without spring-boot-maven-plugin but it turns out it's a lot of work and the option would make it a lot easier :)

[fprochazka]

Just to provide full context, here is how I've made it work (took me a few hours):

parent/pom.xml

This module has spring-boot-starter-parent as parent. Every other module has this as a parent.

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-dependency-plugin</artifactId>
                    <version>${maven-dependency-plugin.version}</version>
                    <executions>
                        <execution>
                            <id>copy-dependencies</id>
                            <phase>package</phase>
                            <goals>
                                <goal>copy-dependencies</goal>
                            </goals>
                            <configuration>
                                <prependGroupId>true</prependGroupId>
                                <stripVersion>true</stripVersion>
                                <overWriteReleases>false</overWriteReleases>
                                <overWriteSnapshots>false</overWriteSnapshots>
                                <overWriteIfNewer>true</overWriteIfNewer>
                                <includeScope>runtime</includeScope>
                                <silent>true</silent>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </pluginManagement>

rest-api/pom.xml

thanks to the parent definition, I don't have to repeat the dependency copying configuration

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>${maven-jar-plugin.version}</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <classpathLayoutType>custom</classpathLayoutType>
                            <!--suppress UnresolvedMavenProperty -->
                            <customClasspathLayout>lib/$${artifact.groupId}.$${artifact.artifactId}.$${artifact.extension}</customClasspathLayout>
                            <mainClass>com.cogvio.RestApiApplication</mainClass>
                        </manifest>
                    </archive>
                    <finalName>api</finalName>
                </configuration>
            </plugin>

pom.xml

Root pom, it inherints spring-boot-starter-parent.

1. It aggregates external dependencies into target/lib/
2. it aggregates project-provided dependencies such as core and core-http into target/project/

This is because core and core-http change regularly and when creating docker caching strategy, you don't want to mix these with other dependencies, otherwise it would bust the cache on every build.

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-resources-plugin</artifactId>
                <executions>
                    <execution>
                        <id>merge-dependencies</id>
                        <phase>generate-resources</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/lib</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>${project.basedir}/cli/target/dependency</directory>
                                    <excludes>
                                        <exclude>org.springframework.boot.spring-boot-devtools.jar</exclude>
                                        <exclude>com.cogvio.pm.*</exclude>
                                    </excludes>
                                </resource>
                                <resource>
                                    <directory>${project.basedir}/rest-api/target/dependency</directory>
                                    <excludes>
                                        <exclude>org.springframework.boot.spring-boot-devtools.jar</exclude>
                                        <exclude>com.cogvio.pm.*</exclude>
                                    </excludes>
                                </resource>
                                <resource>
                                    <directory>${project.basedir}/web-admin/target/dependency</directory>
                                    <excludes>
                                        <exclude>org.springframework.boot.spring-boot-devtools.jar</exclude>
                                        <exclude>com.cogvio.pm.*</exclude>
                                    </excludes>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                    <execution>
                        <id>copy-project-jars</id>
                        <phase>generate-resources</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/project</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>${project.basedir}/cli/target/dependency</directory>
                                    <includes>
                                        <include>com.cogvio.pm.*</include>
                                    </includes>
                                </resource>
                                <resource>
                                    <directory>${project.basedir}/rest-api/target/dependency</directory>
                                    <includes>
                                        <include>com.cogvio.pm.*</include>
                                    </includes>
                                </resource>
                                <resource>
                                    <directory>${project.basedir}/web-admin/target/dependency</directory>
                                    <includes>
                                        <include>com.cogvio.pm.*</include>
                                    </includes>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

.docker/pm/entrypoint

#!/bin/sh
set -e

case "$1" in
    rest-api)
        sh -c "java $JAVA_OPTS -Dserver.port=80 -Djava.security.egd=file:/dev/./urandom -jar /srv/api.jar"
        ;;
    web-admin)
        sh -c "java $JAVA_OPTS -Dserver.port=80 -Djava.security.egd=file:/dev/./urandom -jar /srv/admin.jar"
        ;;
    *)
        sh -c "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /srv/cli.jar $@"
        ;;
esac

Docker.app

1. creates pretty standard docker image with server java
2. includes custom entrypoint
3. copies external libs as one layer
4. copies project-generated libs as another layer
5. copies executables as another layer

FROM anapsix/alpine-java:8u172b11_server-jre_unlimited

EXPOSE 80

ENV JAVA_OPTS=""

ADD ./.docker/pm/entrypoint /cogvio-pm-entrypoint
ENTRYPOINT ["/cogvio-pm-entrypoint"]

WORKDIR /srv

COPY ./target/lib/ /srv/lib/
COPY ./target/project/ /srv/lib/
COPY ./cli/target/cli.jar ./web-admin/target/admin.jar ./rest-api/target/api.jar /srv/

CMD ["rest-api"]

---

before:

(I had a bug there, above those highlighted lines, please ignore that)

after:

---

Hope I'm not forgeting something... and I hope this gives you the full idea what I was trying to achieve.

Turns out, simply having an option for not including the dependencies is not nearly enough... so I'll leave it to you to consider if this approach might have any value for other spring boot users.

[wilkinsona]

Thanks for sharing your findings. We already have #12545 tracking this. You may also be interested in the Maven plugin’s suppprt for custom layouts.

[fprochazka]

@wilkinsona great! thanks for the info :)