New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Whiteouts in layers #2
Comments
There might be (at least) one scenario where you want to have whiteouts: if you ship a layer including some sample code, you might want to remove it (or overwrite it with something else) when you derive the layer. On the current platform, we don't use whiteouts in that case, because this data is on a separate volume (therefore not requiring a whiteout). There are also special cases of whiteouts: permission changes. Those can be useful when you want to give access to some files, or remove access. This was a frequent problem on the current platform (e.g. readability of logs when daemons are installed using system packages instead of the default user). There is no convention in AUFS—just an opaque format. I'd personally advocate against making that part of the API, since it effectively ties the implementation with AUFS, which is probably a Very Bad Thing, but YMMV. Last but not least, in some cases (mail servers), you might also want to preserve inode numbers—and in that case, you would have to also preserve the xino translation file of AUFS. ... Unless we want to support bind-mounted volumes; then those concerns are waived. |
On Sun, Jan 20, 2013 at 10:57 PM, jpetazzo notifications@github.com wrote:
Thanks! |
On Sun, Jan 20, 2013 at 10:57 PM, jpetazzo notifications@github.com wrote:
We are in agreement here. Frankly, sticking to vanilla tarballs as the |
|
Ok, after a few off-band discussions: the consensus seems to be that whiteouts and layers are a (very) useful tool, but should not be exposed as a top-level object to the end-user. The main reason being that it would make things unnecessarily complicated (every user needs to acquire the off-band knowledge of which layer works with which, and it is very easy to shoot yourself in the foot by combining layers that were not meant to be combined). At the same time, there is not much benefit to allowing arbitrary combination of layers: even if you are 99% sure your layer can be safely "rebased" to a new base layer, why not run that build script again just to make sure you're right? So, instead, let's use layers (+ their associated whiteouts) as a optimization for storing and transferring images. In other words, the payload for docker is always a tarball. But it may optionally be a sparse tarball, which will reference the bottom layers by ID, and only store changes. Of course these sparse tarballs will only be usable to a receiver capable of (a) decoding the sparse format and (b) retrieving the bottom layers from their ID. |
Hack: Add lvm2 static compilation to Dockerfile
add a little bit mentioning commandline option combinations Docker-DCO-1.1-Signed-off-by: Victor Vieux <victor.vieux@docker.com> (github: vieux)
Miscellaneous fixes and update for Remote API v1.10
Signed-off-by: Malte Janduda <mail@janduda.net>
[18.06] Update Microsoft/go-winio to 0.4.8
When go-1.11beta1 is used for building, the following error is reported: > 14:56:20 daemon\graphdriver\lcow\lcow.go:236: Debugf format %s reads > arg #2, but call has 1 arg While fixing this, let's also fix a few other things in this very function (startServiceVMIfNotRunning): 1. Do not use fmt.Printf when not required. 2. Use `title` whenever possible. 3. Don't add `id` to messages as `title` already has it. 4. Remove duplicated colons. 5. Try to unify style of messages. 6. s/startservicevmifnotrunning/startServiceVMIfNotRunning/ ... In general, logging/debugging here is a mess and requires much more love than I can give it at the moment. Areas for improvement: 1. Add a global var logger = logrus.WithField("storage-driver", "lcow") and use it everywhere else in the code. 2. Use logger.WithField("id", id) whenever possible (same for "context" and other similar fields). 3. Revise all the errors returned to be uniform. 4. Make use of errors.Wrap[f] whenever possible. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Introduce a Makefile for building on FreeBSD
60437cbde7 Merge pull request #2 from WAGO/feat/oelh/fix-version 69c1d2a556 Change component actions to release version v1.0 934e84a197 Merge pull request #1 from WAGO/feat/oelh/extracted a04bdaa494 Extract common docker actions git-subtree-dir: .github/docker-actions git-subtree-split: 60437cbde75455f2248a81de61dc32264aacca26
[20.10] Lock down docker root dir perms.
moby#2) * fix issue that push empty logs to logstore and change vendor updating method * update vendor
…f v1.5.4 full diffs: - protocolbuffers/protobuf-go@v1.31.0...v1.33.0 - golang/protobuf@v1.5.3...v1.5.4 From the Go security announcement list; > Version v1.33.0 of the google.golang.org/protobuf module fixes a bug in > the google.golang.org/protobuf/encoding/protojson package which could cause > the Unmarshal function to enter an infinite loop when handling some invalid > inputs. > > This condition could only occur when unmarshaling into a message which contains > a google.protobuf.Any value, or when the UnmarshalOptions.UnmarshalUnknown > option is set. Unmarshal now correctly returns an error when handling these > inputs. > > This is CVE-2024-24786. In a follow-up post; > A small correction: This vulnerability applies when the UnmarshalOptions.DiscardUnknown > option is set (as well as when unmarshaling into any message which contains a > google.protobuf.Any). There is no UnmarshalUnknown option. > > In addition, version 1.33.0 of google.golang.org/protobuf inadvertently > introduced an incompatibility with the older github.com/golang/protobuf > module. (golang/protobuf#1596) Users of the older > module should update to github.com/golang/protobuf@v1.5.4. govulncheck results in our code: govulncheck ./... Scanning your code and 1221 packages across 204 dependent modules for known vulnerabilities... === Symbol Results === Vulnerability #1: GO-2024-2611 Infinite loop in JSON unmarshaling in google.golang.org/protobuf More info: https://pkg.go.dev/vuln/GO-2024-2611 Module: google.golang.org/protobuf Found in: google.golang.org/protobuf@v1.31.0 Fixed in: google.golang.org/protobuf@v1.33.0 Example traces found: #1: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Peek #2: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls json.Decoder.Read #3: daemon/logger/gcplogs/gcplogging.go:154:18: gcplogs.New calls logging.Client.Ping, which eventually calls protojson.Unmarshal Your code is affected by 1 vulnerability from 1 module. This scan found no other vulnerabilities in packages you import or modules you require. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Will layers store whiteouts (a list of files to remove) in addition to added files?
If I remove /foo in a container, and export those changes into a new layer, what happens when I run a new container using that layer? Either a) /foo will be present (whiteouts are not stored in the layer), or b) /foo will not be present (whiteouts are stored in the layer).
Question 1: are whiteouts needed? Are certain use cases only possible with whiteouts?
Question 2: if we do need whiteouts in layer, what format should we use? We could use AUFS's convention - effectively making aufs part of the api, instead of just an implementation detail. Do we want that?
The text was updated successfully, but these errors were encountered: