Proposal - Allow for different types of volumes

[icecrime]

Issue statement

Volumes today can come in different flavors, but always with the underlying idea of persistent data. The recent submission of a --tmpfs PR (#9586) raised interesting questions:

• Is tmpfs meaningful on other systems?
• Should this be related to volumes somehow?

This proposal only aims at offering a place to discuss this particular aspect of volumes management and hopefully providing UX input for #8484.

Proposal

A volume should be about granting special properties to a directory inside a container. That special meaning depends on the nature of the volume being mounted:

• Persistent
• Bound to a host directory
• Ephemeral
• ...

The key aspect of the proposal is that it allows to discuss properties rather than particular implementation: tmpfs doesn't exist on some systems while the notion of ephemerality is universal.

Proposed UX

What is true for all volumes types:

• Volumes are managed with the set of top-level commands specified in Proposal: Add volume ls/inspect/rm/create commands #8484
• The docker volume create command has a --type=(persistent|host|ephemeral|...) argument
• Volumes are mounted using docker run -v <volume_name_or_id>:/path/in/container/[:mode]
• Volumes can be created at container run time using docker run command line arguments:
  • Type of the new volume is specified with --volume-type=(persistent|host|ephemeral|...)
  • Name of the new volume is specified with the optional --volume-name flag

Each type of volume might require specific parameters to be passed at creation time (such as the host path in case of a host-bound volume):

• For docker volume create args are appended as colon separated values to the --type flag
• For docker run command args are appended as colon separated values to the --volume-type flag
• In both cases, this list of positional arguments is passed to the volume strategy for interpretation

Examples

The two following examples are equivalent:

$ docker volume create --type=host --host-path=/path/to/host/dir/ --name bidule
$ docker run -v bidule:/path/in/container:ro -ti busybox

$ docker run -v /path/in/container:ro --volume-name=bidule --volume-type host:/path/to/host/dir/ -ti busybox

Ping @cpuguy83 @crosbymichael @shykes

[icecrime]

Given how GitHub makes it difficult to follow long running discussions, I'm going to try something out and use this first comment to consolidate feedbacks already taken into account. I hope people won't mind me deleting comments: I'll copy them verbatim, I just want to avoid comments count explosion.

---

From @thaJeztah:

I think your last example docker create volume should be docker volume create. Or was the proposed interface changed?

Fixed, thanks!

---

From @tianon:

What does "Shared with host" mean/imply?

When I first read "--type=shared", I thought it was about sharing a volume between containers, so I think that's a bit of a confusing choice of word. Regardless, the same permissions issues really apply to all kinds of volume sharing -- is a workaround for those types of issues part of this proposal?

If we're going to use a word like "share" that implies that this is explicitly meant for "sharing" files/folders, I think our users will expect it to solve these problems somehow. Right now, we paper over the problem by explaining that "-v" isn't really sharing -- it's "just" a bind mount, with all the associated warts that come with bind mounts (which are reasonably well documented around the internet at large).

From @tianon:

Yeah, IMO --type=host would definitely be more clear, and have less implications regarding permissions handling.

Renamed --type=shared to --type=host. Regarding your other remarks: I'm just proposing to review UX, not the volumes feature itself.

[thaJeztah]

Some more thoughts. Sorry for the length, I'm also not sure all of these comments should be put here or in the other proposal, so correct me if I should put them over there.

Is name required?

I don't know if name should be a required option. As long as docker returns the ID of the created volume, we could use the same random naming as is used for containers?

type "host"
If I understand correctly, host is basically what's currently called a "bind mount" now. I don't see that being a "special" type of volume, apart from the location for the files being specified.

I think the same can be achieved by specifying (or skipping) --host-path;

# this creates a "host" volume ("bind mount")
docker volume create --host-path=/path/to/shared/dir/ --name bidule

# this creates a "regular" volume. A random "host-path" is created by docker
docker volume create --name bidule

Possibly, --type persistent should be specified here?

Host path syntax for docker-run

For docker run command args are appended as colon separated values to the --volume-type flag

I think I'd prefer having this more explicit, and not combine the options in a single flag.
Perhaps --volume-host-path?

$ docker run -v /path/in/container:ro \
   --volume-name=bidule \
   --volume-type=host \
   --volume-host-path=/path/to/shared/dir/ \
   -ti busybox

Alternatively, use a more generic flag to specify options for the volume to create (--volume-options="host-path=/foo/bar").

Handling conflicting options

What should happen if I specify different options during docker run for a volume that already exists?

$ docker volume create --type=host \
    --host-path=/path/to/shared/dir/ \
    --name=bidule

# perform docker "run" with a different host-path;
$ docker run -v /path/in/container:ro \
    --volume-name=bidule \
    --volume-type=shared:/some/other/path/ \
    -ti busybox

Specifying multiple volumes on docker run

How would creating multiple volumes during docker run work? How will docker know which flags should be for which volume?

Should "docker run" support all these?

Not sure if docker run should support all these options. If volumes get more options in the future, if will really be a lot of options to pass to docker run. Responsibility isn't clear and handling errors etc will be difficult (i.e. volume is created, but container fails to start or container can start, but volume fails, etc etc.)

I think it will be harder to maintain as well, both from a "technical" viewpoint and from a "project organisation" viewpoint, because adding options to docker volumes also requires docker run to be changed.

Personally, I'd only make minor changes to docker run. Only allow the current behavior (-v [/host/path:]/container/path[:mode]) and add the new option to use a named volume (-v [<volume_name_or_id>]/container/path[:mode]), nothing beyond that. If people want to use "advanced" volumes, they have to use docker volume create.

Related proposal and "pluggable"

I think this proposal; #9250 is also related here. I think the "engine" used to create a volume should be pluggable and be able to have its own options. What kind of driver and where the actual data is stored should be transparent to the container.

[icecrime]

Is name require? I don't think --name is required as we should be able to generate default random names.

Type host I believe our idea that mounting a host directory as a volume is only a particular case of data volumes comes from the fact that a data volume is currently implemented as a directory on the host. They are conceptually quite different, but the current implementation make them very similar. What if tomorrow we support alternative volume drivers and I decide to store my data volume in a database?

Host path syntax for docker-run That's indeed a difficult debate... I think we need something generic, the question is: do we want named arguments (a=1, b=2, c=3), or positional arguments (1, 2, 3)? Given the lack of precedent in Docker, I went for the simplest.

Conflicting options Because --volume-type in a docker run invocation expresses that you want to create a new volume, the fact that the chosen name is taken should result in an error. So I'd say there's no issue here.

Specifying multiple volumes in a docker run This is indeed an issue that isn't addressed here, good point.

Should "docker run" support all these? Well that's the point of using --volume-type to hold positional arguments: you don't make any assumptions on the number or types of arguments expected by a particular driver (and the docker run syntax doesn't have to evolve for each new driver).

Related proposal and "pluggable" I'm discovering that one: added to reading list :-)

[rhatdan]

I would suggest that we have at least one hard coded name tmpfs

docker run -v tmpfs:/tmp ...

Although I would like tmpfs to tar all content of the target directory from the image into memory and then untar it onto the tmpfs.

[trentm]

Throwing out a possible -v ... syntax:

-v volume-name-or-path:container-path[:mode-and-volume-params]

The "mode-and-volume-params" is the new suggestion. Volume params would
be key=value pairs where the keys match the "--key" options to
docker volume create .... Keys other than the common set (name, type, mode)
could be passed to the volume strategy. Your first example becomes:

docker run -v /path/to/host/dir/:/path/in/container:ro,name=bidule,type=host -ti busybox

This could solve the Specifying multiple volumes in a docker run question.

docker run \
    -v /path/to/host/dir/:/path/in/container:ro,name=bidule,type=host \
    -v /blah:/path/in/container:type=ephemeral \
    -ti busybox

[SvenDowideit]

@icecrime can we combine this proposal with my #9250 ? I think they're probably duplicates UX wise - mine goes on to suggest we expand the horizons of the vfs drivers :)

[icecrime]

Sorry, I still didn't take the time to look into the last comments and related PRs... It's on my TODO list, promise!

[ibuildthecloud]

It seems that volumes from a driver should be prepopulated as Docker does that with -v /foo today. This would also address @rhatdan tmpfs comment of prepopulating.

[rhatdan]

It is a real shame that we don't have something here yet. We were working on and proposing this months ago. It actually is a nice secure way to run read/only containers, but here we are 6 months later and still no way to do it. Sad.

[cpuguy83]

@rhatdan We can move forward with this once the top-level volumes command is there. All is getting easier here.
Not sure if this should be an option on the local driver, or a driver that is for tmpfs (for the tmpfs case).

[thaJeztah]

Now that --tempfs is merged in #13587, and top-level volume management implemented in #14242, I think it's safe to close this one.

@ibuildthecloud I agree with pre-populating volumes, I think we can keep that for a separate discussion (I think there are some existing ones)