Proposal: INCLUDE syntax for specifying multiple images per context

[aigarius]

Multiple use cases call for an ability to build several Docker images from a single context. An example would be a set of microservices that reside in a shared repository, because they also share common libraries. The common folder layout looks like this:

/README.md
/bar-service/...
/foo-service/...
/common/...

An ideal solution would:

• allow to specify one (or more) image to be built for each service
• co-locate the Dockerfile(s) of the service inside the folder of the corresponding service
• have a default build action that would build all images
• have an option to only build one specific image

This proposal is to introduce a new verb in the Dockerfile that would look like this:
INCLUDE /bar-service/Dockerfile.foo AS bar-service and a CONTEXT verb to be used in the included Dockerfiles.

The INCLUDE verb can be used multiple times. If there is an INCLUDE in a Dockerfile, it must come before the FROM verb in that file and FROM verb becomes optional.

If such line is included in the root Dockerfile and a user runs docker build -t aigarius/xyz . then this verb would have the following effects:

• docker build specified in the file /bar-service/Dockerfile.foo would be build and tagged as aigarius/xyz/bar-service.foo;
• the included Dockerfile may optionally use the new CONTEXT verb to shift the context up to the parent folder, such as if /bar-service/Dockerfile.foo includes a line CONTEXT .., then the ADD/COPY commands will work in the context of the parent folder. If such verb is not used, then the context of the included build is the folder containing the included file;
• after this line in the root Dockerfile or in any subsequently included Dockerfiles the invocation of FROM bar-service:included would use the newly built bar-service image as a base.

In addition it should be possible for a user to run docker build -t aigarius/xyz --images=bar-service,foo-service . to only build images of those two services.

Tag ":included" should be considered to be equivalent to ":latest" (possibly with a warning) if there is no build with such name in the current build set.

CONTEXT verb shifts the context for a Docker build relative to the Dockerfile. This can be used to only have a subfolder of the original context be uploaded to the daemon. It can also be used to shift the context folder up, but only if the Dockerfile is included in another Dockerfile higher in the path and only up to the folder of the top level Dockerfile.

[cyphar]

Why wouldn't you just make a shell script to run the different builds? This seems to be a very specific use-case (and is filled with strange functionality, because the context of each Dockerfile build is no longer where the Dockerfile is [which means that Dockerfiles no longer are reproducible as an INCLUDE could break them]).

Using shared libraries is a bit of a pain, and the builder-runner pattern can help with this (you have a build of the builders of the shared libraries, then a build of the builders each app and then a build of the runners for each app).

Also, the whole recipe thing can be done with a Makefile. So you could make a Makefile which has the different services as recipes and the shared dependencies can be built and added as volumes first.

[aigarius]

I agree that there is a context confusion when looking at individual Dockerfiles. To alleviate that I propose adding another command - CONTEXT:
CONTEXT ..
CONTEXT src
This command would have to be set before any of the ADD/COPY commands could be used. The top level folder that may be referenced is the folder containing the top level Dockerfile.
I will then modify my proposal to include this in the way that if there is no such command, then context of the build is the same folder where the included Dockerfile is located.

[aigarius]

Updated the proposal.

Also as a comment - builder/runner pattern is very painful to use. And it also breaks the pattern of "docker build ." doing the right thing. This #7115 has a quite heavy syntax in the file. I would prefer the combination of this with #6906 (comment) this to solve such issues.

It does miss a few nice things from #7115 , namely the ability to build in one environment (such as full Debian) and then run in something completely different (like busybox) or to combine multiple outputs, but the syntax is much simpler. And it could actually be merged with #7115 in a way to allow the use of separate images and separate Dockerfiles to define the sub-images.

So the example from #7115 could be reformulated as:
INCLUDE Dockefile.runtime IN /var/build
or even possible as:
INCLUDE Dockerfile.runtime IN /var/build AS runtime
thus removing the need for the PUBLISH directive altogether.

[jessfraz]

closed as duplicate of #7115, trying to consolidate so we can get cool stuff like this in