Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

buildTar fails with latest Google distroless Java 17 images #3958

Closed
ghost opened this issue Mar 15, 2023 · 5 comments · Fixed by #3974
Closed

buildTar fails with latest Google distroless Java 17 images #3958

ghost opened this issue Mar 15, 2023 · 5 comments · Fixed by #3974
Labels
area/jib-gradle-plugin area/jib-maven-plugin priority:p3 type:bug

Comments

@ghost
Copy link

ghost commented Mar 15, 2023

Environment:

  • Jib version: 3.3.1
  • Build tool: maven, 3.6.6
  • OS: Linux, Ubuntu
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 17.0.6, vendor: Private Build, runtime: /usr/lib/jvm/java-17-openjdk-amd64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.15.0-67-generic", arch: "amd64", family: "unix"

Description of the issue:

We updated our base images (hash is the difference) from

gcr.io/distroless/java17-debian11:nonroot@sha256:3481476327255f2e553415945d9c774dc5936273bf94cfc01509f044da7b3ed5

to

gcr.io/distroless/java17-debian11:nonroot@sha256:5154596f58d81687d66c8302face284198caf65033cde6bf6667c488b21745e6

The maven build fails with

[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.3.1:buildTar (default-cli) on project spring-boot-k8s-example: class com.google.cloud.tools.jib.image.json.OciIndexTemplate cannot be cast to class com.google.cloud.tools.jib.image.json.V22ManifestListTemplate (com.google.cloud.tools.jib.image.json.OciIndexTemplate and com.google.cloud.tools.jib.image.json.V22ManifestListTemplate are in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @2262d6d5) -> [Help 1]

Expected behavior:

Build passes as before and our application images are build successfully.

Steps to reproduce:

  1. Download the spring boot example from this repo
  2. Add the below pom.xml snippet to the jib plugin definition
  3. Run ./mvnw -DskipTestspackage jib:buildTar

jib-maven-plugin Configuration: 

<plugin>
  <groupId>com.google.cloud.tools</groupId>
  <artifactId>jib-maven-plugin</artifactId>
  <version>3.3.1</version>
  <configuration>
    <from>
      <image>gcr.io/distroless/java17-debian11:nonroot@sha256:5154596f58d81687d66c8302face284198caf65033cde6bf6667c488b21745e6</image>
    </from>
  </configuration>
</plugin>

Log output: 

$ ./mvnw -e package jib:buildTar
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------< example:spring-boot-k8s-example >-------------------
[INFO] Building spring-boot-k8s-example 0.1.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ spring-boot-k8s-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/fziegler/3rd-party/jib/examples/spring-boot/src/main/resources
[INFO] skip non existing resourceDirectory /home/fziegler/3rd-party/jib/examples/spring-boot/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ spring-boot-k8s-example ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ spring-boot-k8s-example ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/fziegler/3rd-party/jib/examples/spring-boot/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ spring-boot-k8s-example ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ spring-boot-k8s-example ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-jar-plugin:3.1.2:jar (default-jar) @ spring-boot-k8s-example ---
[INFO] 
[INFO] --- jib-maven-plugin:3.3.1:buildTar (default-cli) @ spring-boot-k8s-example ---
[INFO] Tagging image with generated image reference spring-boot-k8s-example:0.1.0. If you'd like to specify a different tag, you can set the <to><image> parameter in your pom.xml, or use the -Dimage=<MY IMAGE> commandline flag.
[WARNING] 'mainClass' configured in 'maven-jar-plugin' is not a valid Java class: ${start-class}
[INFO] 
[INFO] Containerizing application to file at '/home/fziegler/3rd-party/jib/examples/spring-boot/target/jib-image.tar'...
[INFO] Executing tasks:
[INFO] [============                  ] 40.0% complete
[INFO] > building image to tar file
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.439 s
[INFO] Finished at: 2023-03-15T16:08:53+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.3.1:buildTar (default-cli) on project spring-boot-k8s-example: class com.google.cloud.tools.jib.image.json.OciIndexTemplate cannot be cast to class com.google.cloud.tools.jib.image.json.V22ManifestListTemplate (com.google.cloud.tools.jib.image.json.OciIndexTemplate and com.google.cloud.tools.jib.image.json.V22ManifestListTemplate are in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @670f2466) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.3.1:buildTar (default-cli) on project spring-boot-k8s-example: class com.google.cloud.tools.jib.image.json.OciIndexTemplate cannot be cast to class com.google.cloud.tools.jib.image.json.V22ManifestListTemplate (com.google.cloud.tools.jib.image.json.OciIndexTemplate and com.google.cloud.tools.jib.image.json.V22ManifestListTemplate are in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @670f2466)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:39)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:122)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:55)
Caused by: org.apache.maven.plugin.MojoExecutionException: class com.google.cloud.tools.jib.image.json.OciIndexTemplate cannot be cast to class com.google.cloud.tools.jib.image.json.V22ManifestListTemplate (com.google.cloud.tools.jib.image.json.OciIndexTemplate and com.google.cloud.tools.jib.image.json.V22ManifestListTemplate are in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @670f2466)
    at com.google.cloud.tools.jib.maven.BuildTarMojo.execute (BuildTarMojo.java:159)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:39)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:122)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:55)
Caused by: java.lang.ClassCastException: class com.google.cloud.tools.jib.image.json.OciIndexTemplate cannot be cast to class com.google.cloud.tools.jib.image.json.V22ManifestListTemplate (com.google.cloud.tools.jib.image.json.OciIndexTemplate and com.google.cloud.tools.jib.image.json.V22ManifestListTemplate are in unnamed module of loader org.codehaus.plexus.classworlds.realm.ClassRealm @670f2466)
    at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.getCachedBaseImages (PullBaseImageStep.java:470)
    at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call (PullBaseImageStep.java:134)
    at com.google.cloud.tools.jib.builder.steps.PullBaseImageStep.call (PullBaseImageStep.java:71)
    at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly (TrustedListenableFutureTask.java:131)
    at com.google.common.util.concurrent.InterruptibleTask.run (InterruptibleTask.java:74)
    at com.google.common.util.concurrent.TrustedListenableFutureTask.run (TrustedListenableFutureTask.java:82)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:635)
    at java.lang.Thread.run (Thread.java:833)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Additional Information:

The behavior does not occur with the previously mentioned image (gcr.io/distroless/java17-debian11:nonroot@sha256:3481476327255f2e553415945d9c774dc5936273bf94cfc01509f044da7b3ed5) .

There was a change to the image build that moved from docker manifest format to OCI format

docker buildx imagetools inspect gcr.io/distroless/java17-debian11:nonroot@sha256:3481476327255f2e553415945d9c774dc5936273bf94cfc01509f044da7b3ed5
Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:3481476327255f2e553415945d9c774dc5936273bf94cfc01509f044da7b3ed5
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:3481476327255f2e553415945d9c774dc5936273bf94cfc01509f044da7b3ed5
           
Manifests: 
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:8d3f355ee92b8da899db8739ad994740cace766fa5c22abce49ba242657157f2
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64
             
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:01022fc221d78827d9d2f7d021eaad6705e50b1983fe04e45606c75758b72ed0
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64
             
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:b4de50145c10fed2067f1125ab7bdc0dd5468afa0e27807e01c5e86145d1289c
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/ppc64le
             
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:b337804c4852f00ff55277c44c889bcf352384082e61bb557bfb61f0d56c8f68
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/s390x

docker buildx imagetools inspect gcr.io/distroless/java17-debian11:nonroot@sha256:5154596f58d81687d66c8302face284198caf65033cde6bf6667c488b21745e6
Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:5154596f58d81687d66c8302face284198caf65033cde6bf6667c488b21745e6
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:5154596f58d81687d66c8302face284198caf65033cde6bf6667c488b21745e6
           
Manifests: 
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:4f1f85307f592125bae41a366e62e9c0e7c916e053854004b3acf1cbf7c726e4
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/amd64
             
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:b3fb74abd29a0a15b6a96f0ce3b1b10fe8cd659f5b3a34a68eda7940901a7cc2
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/arm64
             
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:33fdfda1fb7c109ea3cfa239ad37a1d4545e7fc3ce08a8d9592f5054882640b5
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/s390x
             
  Name:      gcr.io/distroless/java17-debian11:nonroot@sha256:0616be507c506457ba61a5204595860a4709b9c7e665d35173172d4bbe7cf8bc
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/ppc64le

See this commit:

GoogleContainerTools/distroless@fb94acc
@cqwense
Copy link

cqwense commented Mar 20, 2023

also confirmed with:

jib: 2.8.0
gradle: 6.9.3
os: linux/ubuntu

and

gcr.io/distroless/java11-debian11:debug

@emmileaf
Copy link
Contributor

Thank you for reporting this! I think the observed limitation is coming from Jib’s base image caching mechanism currently only expecting Docker and not OCI format for manifest lists.

jib/jib-core/src/main/java/com/google/cloud/tools/jib/builder/steps/PullBaseImageStep.java

Line 471 in e50b825

lookUpPlatformSpecificImageManifest((V22ManifestListTemplate) manifestList, platform);

To work around this issue in the meanwhile, you may find this discussion in GoogleContainerTools/distroless#1259 helpful. For your use case, would it be possible to switch to a platform-specific tag for the image (which does not use manifest list), or an older version that still uses the docker format?

@chanseokoh
Copy link
Member

chanseokoh commented Mar 23, 2023 •
edited

Yeah, #3715 add the capability to pull an OCI index, but it missed loading cached images. So, Jib works the first time it pulls an OCI image, but subsequent runs fail because PullBaseImageStep.getCachedBaseImages() assumes that the cached manifest is always the Docker format.

I guess it won't be that difficult to change PullBaseImageStep.getCachedBaseImages() to fix this.

@rvaidya
Copy link

rvaidya commented Mar 26, 2023 •
edited

I get similar behavior but with a different error message, with jib:build

mvn --batch-mode clean compile jib:build $MAVEN_CLI_OPTS

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:48 min
[INFO] Finished at: 2023-03-26T06:18:20Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.2.1:build (default-cli) on project pocketbook-java: Tried to pull image manifest for gcr.io/distroless/java17-debian11:latest but failed because: Manifest with tag 'latest' has media type 'application/vnd.oci.image.index.v1+json', but client accepts 'application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json'. 404 Not Found
Error: [ERROR] ***"errors":[***"code":"MANIFEST_UNKNOWN","message":"Manifest with tag 'latest' has media type 'application/vnd.oci.image.index.v1+json', but client accepts 'application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json'."***]***
Error:  -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error: Process completed with exit code 1.

@rquinio rquinio mentioned this issue Mar 30, 2023
Merged
4 tasks
@chanseokoh
Copy link
Member

@rvaidya @cqwense to clarify, you're using an old version of Jib. The OCI image index support was added in Jib 3.3.0. (But of course, you'll hit this issue even when using 3.3.0+.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/jib-gradle-plugin area/jib-maven-plugin priority:p3 type:bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix V22ManifestListTemplate cast when pulling an oci index manifest rquinio/jib
5 participants
@rvaidya @cqwense @chanseokoh @emmileaf @ddixit14