Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Maven jib:build fails if <outputPaths> contain non-existing directories #3789

Open
etzelc opened this issue Sep 22, 2022 · 3 comments · May be fixed by #4208
Open

Maven jib:build fails if <outputPaths> contain non-existing directories #3789

etzelc opened this issue Sep 22, 2022 · 3 comments · May be fixed by #4208
Labels
area/jib-maven-plugin enhancement priority:p3 type:documentation

Comments

@etzelc
Copy link

etzelc commented Sep 22, 2022

Environment:

  • Jib version: 3.3.0
  • Build tool: Maven
  • OS: Linux

Description of the issue:
jib:build fails if you specify <outputPaths> which contain non-existing directories.
For example the POM contains the following jib-maven-pluging configuration:

...
<!-- Jib -->
      <plugin>
        <groupId>com.google.cloud.tools</groupId>
        <artifactId>jib-maven-plugin</artifactId>
        <version>${jib-maven-plugin.version}</version>
        <configuration>
          <to>
            <image>gcr.io/REPLACE-WITH-YOUR-GCP-PROJECT/image-built-with-jib</image>
          </to>
          <outputPaths>
            <digest>${project.build.directory}/target/nonExistingDirectory/image.digest</digest>
            <imageId>${project.build.directory}/target/nonExistingDirectory/image.id</imageId>
            <imageJson>${project.build.directory}/target/nonExistingDirectory/image.json</imageJson>
        </outputPaths>
        </configuration>
        ...

It fails with
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.3.0:build (default-cli) on project helloworld: /workspace/jib/examples/helloworld/target/target/nonExistingDirectory/image.digest

Expected behavior:
jib:build creates directories specified in outputPaths and completes successful.

Steps to reproduce:

  1. use https://github.com/GoogleContainerTools/jib/tree/master/examples/helloworld
  2. Add <outputPaths> config from above and replace <to><image> accordingly
  3. Run mvn compile jib:build

Log output:

[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------------< example:helloworld >-------------------------
[INFO] Building helloworld 1
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ helloworld ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ helloworld ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- jib-maven-plugin:3.3.0:build (default-cli) @ helloworld ---
[INFO] 
[INFO] Containerizing application to registry.xyz...
[WARNING] Base image 'eclipse-temurin:8-jre' does not use a specific image digest - build may not be reproducible
[INFO] Using credentials from <to><auth> for registry.xyz
[INFO] The base image requires auth. Trying again for eclipse-temurin:8-jre...
[INFO] Using base image with digest: sha256:a3327e73ad173120e7a7cb21de45f4d50545efecc9969291c69a8095fa9d05c0
[INFO] 
[INFO] Container entrypoint set to [java, -cp, /app/resources:/app/classes:/app/libs/*, example.HelloWorld]
[INFO] 
[INFO] Built and pushed image as registry.xyz
[INFO] Executing tasks:
[INFO] [==============================] 100.0% complete
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24.566 s
[INFO] Finished at: 2022-09-22T14:05:45Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:3.3.0:build (default-cli) on project helloworld: /workspace/jib/examples/helloworld/target/target/nonExistingDirectory/image.digest -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Additional information:
Running mvn -e compile jib:build traces down the exception to JibBuildRunner.java:237. Here a Files.write(imageDigestOutputPath, imageDigest.getBytes(StandardCharsets.UTF_8)); is executed, which fails if a directory in the imageDigestOutputPath does not exist.

...
Caused by: java.nio.file.NoSuchFileException: /workspace/jib/examples/helloworld/target/target/nonExistingDirectory/image.digest
    at sun.nio.fs.UnixException.translateToIOException (UnixException.java:92)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:111)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:116)
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel (UnixFileSystemProvider.java:219)
    at java.nio.file.spi.FileSystemProvider.newOutputStream (FileSystemProvider.java:478)
    at java.nio.file.Files.newOutputStream (Files.java:220)
    at java.nio.file.Files.write (Files.java:3425)
    at com.google.cloud.tools.jib.plugins.common.JibBuildRunner.runBuild (JibBuildRunner.java:237)
    at com.google.cloud.tools.jib.maven.BuildImageMojo.execute (BuildImageMojo.java:117)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
@emmileaf
Copy link
Contributor

emmileaf commented Sep 22, 2022 •
edited

Currently outputPaths’s supported usage is either the default (project-dir)/target/jib-image or a custom-specified directory that is a valid (existing) path.

As a workaround (if this setup within target is needed by your project), you could also try using something like the maven-antrun-plugin to define mkdir tasks to set up the sub-directories within target as required.

@etzelc
Copy link
Author

etzelc commented Sep 22, 2022

Thanks for the clarification and the workaround. I hadn't read it that way from the documentation.

I came across the issue because when you specify a custom directory for the tarball and run jib:buildTar, non-existing (sub)directories are created (presumably during the tarball creation?). That's why I expected the same for jib:build. For example run jib:buildTar with the following configuration and the build succeeds:

          <outputPaths>
            <tar>${project.build.directory}/nonExistingDirectory/image.tar</tar>
            <digest>${project.build.directory}/nonExistingDirectory/image.digest</digest>
            <imageId>${project.build.directory}/nonExistingDirectory/image.id</imageId>
            <imageJson>${project.build.directory}/nonExistingDirectory/image.json</imageJson>
        </outputPaths>

BTW: In the jib-maven-plugin/README.md is a minor typo in the text about 'Additional Build Artifacts'. It talks about <outputFiles> instead of <outputPaths>.

jib/jib-maven-plugin/README.md

Line 236 in eccebd0

As part of an image build, Jib also writes out the _image digest_ and the _image ID_. By default, these are written out to `target/jib-image.digest` and `target/jib-image.id` respectively, but the locations can be configured using the `<outputFiles><digest>` and `<outputFiles><imageId>` configuration properties. See [Extended Usage](#outputpaths-object) for more details.

@emmileaf
Copy link
Contributor

I came across the issue because when you specify a custom directory for the tarball and run jib:buildTar, non-existing (sub)directories are created (presumably during the tarball creation?).

Ah, you are right - writing to tarball does have an additional step here to create parent directories if they do not exist. I don’t think this is currently supported for jib:build, but any PR contributions for either this enhancement or adding a note in the README would be welcomed.

BTW: In the jib-maven-plugin/README.md is a minor typo in the text about 'Additional Build Artifacts'. It talks about <outputFiles> instead of <outputPaths>.

Yes - thank you for catching this!

@emmileaf emmileaf mentioned this issue Sep 23, 2022
Merged
This was referenced Mar 6, 2024
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/jib-maven-plugin enhancement priority:p3 type:documentation
Projects
None yet
Milestone
No milestone
3 participants
@emmileaf @etzelc @alicejli