-
Notifications
You must be signed in to change notification settings - Fork 18.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Add support for build-time environment variables to the 'build' API #9176
Conversation
d041e52
to
ea496b4
Compare
5658c13
to
83ac17d
Compare
the |
6b648af
to
cdb2f78
Compare
@jfrazelle, thanks for the heads up. I have updated the tests. |
I presume that All up, nice! - that would allow me to do ...
which would be useful to add to the apt-cacher-ng article. |
In our case we are looking at using Docker to build images of our Web services, which need to obtain JARs from a private (but Internet-accessible) Maven (Artifactory) repository at build time. This PR would allow us to set our auth credentials for that Maven repo at build time, using environment variables which only exist at build time, which would be very useful to us! Additionally, if the Docker Hub Web UI was extended with a little GUI to set these build-time variables, that would allow us to convert manual builds which our devs must currently run on their boxes to automated builds. |
Yes, this PR will allow build env value for BANANA to be overriden by the value provided by ENV statement ('asd' in this case).
I am not sure if I understand this presumption. Orthogonal to changes in this PR, I think issuing 'export' during a RUN will not persist the exported environment in the final container itself as it will just be run as a shell command in the context of that RUN statement. And this PR doesn't change anything around that behavior. Am I missing something?
Yes, I think this should be addressed by this PR. Were you planning to use persisted environment with 'RUN export' to achieve this? @themasterchef I have mostly worked with cli interface till now and haven't gotten to familiarizing myself with Docker Hub Web UI. So I just enhanced the cli to start making use of this feature. But I could definitely look into enhancing the web UI as well. Is it ok, if I look into it as a separate issue/PR? |
d5fb15c
to
3816882
Compare
@mapuri I was wondering mostly about the intended function - and I like it - it would be worth adding a little more of what you've answered here into the documentation. please also note that the wrt Hub UI - its not open source but your work will enable things that @themasterchef, I and others would like :) next up - @shykes @crosbymichael ? design review |
@SvenDowideit thanks for clarifying and pointers on documentation! I have updated the cli.md to document more details on the changes. I have also updated docker_remote_api.md and docker_remote_api_v1.16.md to document the optional header, introduced to pass the variables to builder. The hub UI and other remote clients will form and pass this header in order to use this feature. I wasn't sure if this change demands bumping up the API version though, so I just updated v1.16 doc file for now. |
Looking good, do you have an estimate of when this will be making its way into master? We've just hit the point today where this is feature a really big deal for us. |
`POST /build` | ||
|
||
**New!** | ||
This endpoint now optinally takes a serialized array of build-time environment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spelling mistake here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for reviewing @EronWright. I have fixed the spelling and updated the diff
402fca3
to
5c1bb40
Compare
+1 I'm in much need of this feature to support privately forked repositories. |
+1 |
+1 |
e840408
to
24a8bcf
Compare
+1 |
1 similar comment
+1 |
…text - The build-time environment variables are passed as environment-context for command(s) run as part of the RUN primitve. These variables are not persisted in environment of intermediate and final images when passed as context for RUN. The build environment is prepended to the intermediate continer's command string for aiding cache lookups. It also helps with build traceability. But this also makes the feature less secure from point of view of passing build time secrets. - The build-time environment variables also get used to expand the symbols used in certain Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a ENV primitive. These variables get persisted in the intermediate and final images whenever they are expanded. Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
+1 the Docker build is totally dependent on the state of the files it's working with, so if those files weren't placed predictably then the build will result in a different output. That argument is a strawman. Many, many users including myself are using hacks to pipe in information into the build process... we need this PR. |
+1 |
3 similar comments
+1 |
+1 |
+1 |
@mapuri The signal to noise ratio in this thread has reach a point where I just cannot keep up. Please come ping me on IRC or send me a mail, and we can discuss the way to move forward into a solution that we will merge. Thank you for your work and time. |
A build-time environment variable is passed to the builder (as part of build API) and made available to the Dockerfile primitives for use in variable expansion and setting up the environment for the RUN primitive (without modifying the Dockerfile and persisting them as environment in the final image).
Following simple example illustrates the feature:
Some of the use cases this PR enables are listed below (captured from comments in the PR's thread).
[Edit: 05/22/2015]
A build-time environment variable gets used only while processing the 'RUN' primitive of a DockerFile. Such a variable is only accessible during 'RUN' and is 'not' persisted with the intermediate and final docker images, thereby addressing the portability concerns of the images generated with 'build'.This addresses issue #4962
+++++++++
Edit: 05/21/2015, 06/26/2015
This PR discussion thread has grown, bringing out use cases that this PR serves and doesn't serves well. Below I consolidate a list of those use cases that have emerged from the comments for ease of reference.
There are two broad usecases that this feature enables:
but this can be any other environment variable as wellbut there are other cases as well like this one docker build should provide an environment variable of the git commit if applicable #14191 (comment).Proposal: Add support for build-time environment variables to the 'build' API #9176 (comment)
Proposal: Add support for build-time environment variables to the 'build' API #9176 (comment)
Proposal: Add support for build-time environment variables to the 'build' API #9176 (comment)
Proposal: Add support for build-time environment variables to the 'build' API #9176 (comment)
Proposal: Add "--from" to docker build for overriding base images to better support hotfix builds #9731 (comment)
The following use-case is not served well by this feature and hence not recommended to be used such:
Proposal: Add support for build-time environment variables to the 'build' API #9176 (comment)
Proposal: Add support for build-time environment variables to the 'build' API #9176 (comment)
Following use-cases might still be suitable with caching turned off:
#9176 (comment)
#9176 (comment)