Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

com.google.cloud.tools.jib.plugins.common.PluginConfigurationProcessor#createModificationTimeProvider does not support USE_CURRENT_TIMESTAMP #4071

Open
rmannibucau opened this issue Jul 9, 2023 · 11 comments · May be fixed by #4072

Comments

@rmannibucau
Copy link
Contributor

Hi,

I thought the issue was already created but github didnt popped it up so creating this issue hoping it is not a duplicated.
Long story short, there are numerous cases (think serving static content for ex) where we want to get the build time as file time for extra dir and today it requires build hacks whereas USE_CURRENT_TIMESTAMP is a neat way for the classpath/classes, would be great to add it in the switch for createModificationTimeProvider too if possible.

rmannibucau added a commit to rmannibucau/jib that referenced this issue Jul 9, 2023
rmannibucau added a commit to rmannibucau/jib that referenced this issue Jul 9, 2023
@chanseokoh
Copy link
Member

chanseokoh commented Jul 9, 2023

There is a Jib extension for Maven that can selectively set file timestamps to build time. Please consider using that if you are on Maven. Not sure if there's a Gradle extension, but you can create one if there isn't.

Intentionally, we don't allow setting file modification time to the current timestamp, as it completely nullifies layer caching, which is a huge deal. We strongly discourage using the current timestamp for files for this reason. Even if we decided to provide this feature, we would do so via a Jib extension, never the main Jib plugins.

If you use the build time for files, I strongly recommend limiting the scope to a small layer, since such layer must be uploaded or downloaded every time. You may consider breaking down a large layer into multiple by using the Jib Layer Filter extension first (Maven / Gradle).

@chanseokoh
Copy link
Member

Actually, the user-contributed Layer With Modification Time Jib Extension that I mentioned does a very nice thing for you in that it conveniently creates a special layer for the files you want to set the last modified time to build time for. I strongly recommend this kind of preparation.

@rmannibucau
Copy link
Contributor Author

@chanseokoh why is it allowed for the classpath 🤔 ? Think both should be allowed or forbidden. On a concrete point I never have a build which is able to not use current time cause artifacts always have to be up to date. It would be ok to use the git time but it would require the PR I did which enables to use a custom format cause iso time is not always the one you can expect depending how you inject the time. I am also ok-ish to use an extension while owned by jib project but not so much to get a one man impl today for something so critical. Last thing is that ideally I don't want build time but file time to be respected, I can enhance my PR to do so if you think it would be a good replacement of current time.

@rmannibucau
Copy link
Contributor Author

My current workaround:

<plugin>
        <groupId>io.github.git-commit-id</groupId>
        <artifactId>git-commit-id-maven-plugin</artifactId>
        <version>6.0.0</version>
        <executions>
          <execution>
            <id>get-the-git-infos</id>
            <phase>initialize</phase>
            <goals>
              <goal>revision</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <injectAllReactorProjects>true</injectAllReactorProjects>
          <generateGitPropertiesFile>false</generateGitPropertiesFile>
          <dateFormatTimeZone>UTC</dateFormatTimeZone>
          <dateFormat>yyyy-MM-dd'T'HH:mm:ss'Z'</dateFormat>
          <includeOnlyProperties>
            <includeOnlyProperty>^git.commit.time$</includeOnlyProperty>
          </includeOnlyProperties>
        </configuration>
      </plugin>

and I set

<creationTime>${git.commit.time}</creationTime>
<filesModificationTime>${git.commit.time}</filesModificationTime>

works well enough even if a bit verbose if you have git setup.

@chanseokoh
Copy link
Member

chanseokoh commented Jul 10, 2023

why is it allowed for the classpath 🤔 ?

Sorry, I don't follow. Are you talking about the small text file where Jib records the computed classpath? If the last modification time of that classpath file is set to the current timestamp, it would be a bug.

If you are talking about USE_CURRENT_TIMESTAMP for creationTime, it is about the image metadata, not files in an image, so at least image layers (which are for files) are still reproducible.

In any case, we won't allow the user to use the current timestamp for files. If you really have to do so, please write an extension. Using a git commit timestamp is what people often do and should be much better than using build time.

@rmannibucau
Copy link
Contributor Author

rmannibucau commented Jul 10, 2023

@chanseokoh you are right so let's move this ticket to "support such an extension out of the box in jib and not a random github repo which can disappear" (and hopefully make such an extension better integration by default to avoid to be too verbose). Any chance it gets integrated in jib-core - even if packaged as an extension and not built-in? Alternatively using git as built-in sounds okish but would bring a ton of deps

@chanseokoh
Copy link
Member

I'll let the maintainer (@GoogleContainerTools/cloud-java-team-teamsync) decide if they will accept your contribution as a first-party extension in the jib-extensions repo.

For jib-core, as a library, you can easily set whatever file timestamp you want. For example, you can provide a ModificationTimeProvider that works for you.

@chanseokoh
Copy link
Member

chanseokoh commented Jul 10, 2023

And to reiterate, we strongly discourage setting file timestamps to the current time, and if someone wants to do so, they should limit the scope to a small and separate layer. As you may know, Jib is highly opinionated, and we deliberately make it difficult for the user difficult to use the current timestamp for files. If they were allowed, it would be too easy for unsuspecting users to create very inefficient images, since layer caching is completely lost. An extension is the only compromise we can accept at best.

@Donorlin
Copy link

We are using jib to build a webserver that servers some static data - js, css, html which constantly chagnes and some backend endpoints. As someone else mentioned, this conflicts with computation of eTag header for resources which by default uses mTime.
Main issue is then browser will not download requested resource because eTag is not changed.
I understand the argument @chanseokoh made about inefficiency of images and layers. But since we know how are layers built, we can make it that layer with resources - mostly javascript sources that are changed everytime - is the only one that has timestamps preserved - and please forgive me i dont know the right order - we can make it the last (top in stack) so no other layer depends on it. Then all layers are always reused except this one... which changes always all the time anyway.

@chanseokoh
Copy link
Member

@Donorlin if you are willing to use the community-contributed Layer With Modification Time Jib Extension, you can achieve what you described.

we can make it the last (top in stack) so no other layer depends on it. Then all layers are always reused except this one... which changes always all the time anyway.

BTW, it's a common misconception that the order of layers matters, but that's not the case. Any layers will be reused as long as they don't change, regardless of the order.

@Donorlin
Copy link

BTW, it's a common misconception that the order of layers matters, but that's not the case. Any layers will be reused as long as they don't change, regardless of the order.

@chanseokoh Thank you ! I was kinda hoping this is the case! Heh you learn everyday something new. Thank you

Actually I am using JIB-CLI and i just fixed it with timestamp property for that specific layer.
Example for future generations:

jib.yaml config file based on Fully Annotated Build File (jib.yaml)

...
layers:
  entries:
    - ... other layers
    - name: frontend
      properties:
        timestamp: ${timestamp}
      files:
        - src: ../dist
          dest: /opt/app-root/app

and by running

jib build --parameter=timestamp="DATE.NOW()" ... rest of the params

u can replace ${timestamp} placeholder.. more at JIB CLI options

since this layer that contains built SPA application is changed everytime anyway, it wont matter to set timestamp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants