Daemon config validation #42393
Merged
Daemon config validation #42393
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thaJeztah
requested changes
May 19, 2021
cmd/dockerd/daemon.go
Outdated
|
|
||
| if opts.Validate { | ||
| // If config wasn't OK we wouldn't have made it this far. | ||
| logrus.Infof("Config OK") |
There was a problem hiding this comment.
mkdir /foo
echo '{' > /foo/malformed-config.json
echo '' > /foo/empty-config-1.json
echo '{}' > /foo/empty-config-2.json
echo '{"unknown-option": true}' > /foo/invalid-config-1.json
echo '{"debug": true}' > /foo/valid-config-1.jsonThen trying them;
for file in /foo/*.json; do sh -xc "dockerd --validate --config-file=${file}"; done
+ dockerd --validate --config-file=/foo/empty-config-1.json
unable to configure the Docker daemon with file /foo/empty-config-1.json: EOF
+ dockerd --validate --config-file=/foo/empty-config-2.json
INFO[2021-05-19T10:31:30.747037342Z] Config OK
+ dockerd --validate --config-file=/foo/invalid-config-1.json
unable to configure the Docker daemon with file /foo/invalid-config-1.json: the following directives don't match any configuration option: unknown-option
+ dockerd --validate --config-file=/foo/malformed-config.json
unable to configure the Docker daemon with file /foo/malformed-config.json: unexpected EOF
+ dockerd --validate --config-file=/foo/valid-config-1.json
INFO[2021-05-19T10:31:30.883917472Z] Config OKLooking at the output:
- For a follow-up: I think we should ignore the "empty" file. It's probably ok to ignore an empty
daemon.json(same as how we ignore empty json ({}) - Looking at it now; using
logrus.Infof()adds a lot of noise; I think we should just printConfig OK. This also allows the output to be more easily used in scripts (check if the output is "Config OK"
I think we should also have a simple integration test for this. I see there's a TestConfigDaemonLibtrustID test in
moby/integration/config/config_test.go
Lines 382 to 400 in 7b9275c
Some notes on that test though;
- it looks like it's "actually" in the wrong place (that test suite is testing
docker config(swarm configs), so perhaps should be moved to a new suite, but we can look at that later - the "test utils" we have to start the daemon may be a bit complicated for this (they're mostly to setup a daemon and then interact with the running daemon, but not so much to test the daemon CLI); it would need a bit looking into (but happy to help if needed). This particular would probably have been easier to implement in the old
integration-clitests, but those were marked "deprecated", so we can't add new tests there. So instead, perhaps just usingos/execto start the binary and check stdout/stderr would work.
|
Failure is unrelated, but .. interesting? edit: opened #42416 for tracking |
aiordache
force-pushed
the
daemon_config
branch
2 times, most recently
from
e54bcb8
to
c905a87
Compare
thaJeztah
requested changes
May 20, 2021
integration/daemon/daemon_test.go
Outdated
Comment on lines
58
to
65
| validOut := "Config OK" | ||
| failedOut := "unable to configure the Docker daemon with file" |
There was a problem hiding this comment.
nit: could make these cons to be explicit they're "immutable";
Suggested change
| validOut := "Config OK" | |
| failedOut := "unable to configure the Docker daemon with file" | |
| const ( | |
| validOut = "Config OK" | |
| failedOut = "unable to configure the Docker daemon with file" | |
| ) |
integration/daemon/daemon_test.go
Outdated
Comment on lines
71
to
78
| out := runCmd([]string{dockerdBinary, "--validate", "--config-file", config}) | ||
|
|
There was a problem hiding this comment.
I think this could use CombinedOutput(), which both executes the command, and collects the output;
cmd := exec.Command(dockerdBinary, "--validate", "--config-file", config)
out, err := cmd.CombinedOutput()However, it would be useful to verify that in the "fail" case, it exists with an error code, and in the "non-fail" case exits with a "zero" exit status; perhaps something like;
if expectedOut == failedOut {
assert.ErrorContains(t, err, "", "expected an error, but got none")
} else {
assert.NilError(t, err)
}There was a problem hiding this comment.
For better debugging which of the tests failed, you could use sub-tests here; basically, that's wrapping everything inside the loop in a t.Run(name, func(){});
for filename, expectedOut := range tests {
t.Run(filename, func(t *testing.T) {
config := filepath.Join("testdata", filename)
cmd := exec.Command(dockerdBinary, "--validate", "--config-file", config)
out, err := cmd.CombinedOutput()
assert.Check(t, is.Contains(string(out), expectedOut))
if expectedOut == failedOut {
assert.ErrorContains(t, err, "", "expected an error, but got none")
} else {
assert.NilError(t, err)
}
})
}I just checked out the branch to test that (with my other suggestions), and; with that;
make DOCKER_GRAPHDRIVER=vfs BIND_DIR=. TESTDIRS='github.com/docker/docker/integration/daemon' TESTFLAGS='-test.run ^TestDaemonConfigValidation$' test-integration
Running /go/src/github.com/docker/docker/integration/daemon (amd64.integration.daemon) flags=-test.v -test.timeout=5m -test.run ^TestDaemonConfigValidation
INFO: Testing against a local daemon
=== RUN TestDaemonConfigValidation
=== RUN TestDaemonConfigValidation/valid-config-1.json
=== RUN TestDaemonConfigValidation/empty-config-1.json
=== RUN TestDaemonConfigValidation/empty-config-2.json
=== RUN TestDaemonConfigValidation/invalid-config-1.json
=== RUN TestDaemonConfigValidation/malformed-config.json
--- PASS: TestDaemonConfigValidation (0.40s)
--- PASS: TestDaemonConfigValidation/valid-config-1.json (0.08s)
--- PASS: TestDaemonConfigValidation/empty-config-1.json (0.07s)
--- PASS: TestDaemonConfigValidation/empty-config-2.json (0.08s)
--- PASS: TestDaemonConfigValidation/invalid-config-1.json (0.07s)
--- PASS: TestDaemonConfigValidation/malformed-config.json (0.07s)
PASS
be26d52
to
19976a0
Compare
thaJeztah
reviewed
May 20, 2021
|
Rootless is failing (which we somewhat expected could happen); I think it's ok to skip the test on rootless, because it's not testing anything that's specific to rootless / non-rootless, so we'd have enough coverage for the other situations. |
thaJeztah
reviewed
May 20, 2021
thaJeztah
requested changes
May 21, 2021
thaJeztah
requested changes
May 25, 2021
|
@AkihiroSuda @cpuguy83 PTAL @aiordache Could you also prepare a pull request in the docker/cli repository to;
|
|
RS5 failure is unrelated, but posting here in case it becomes an issue; |
ebriney
approved these changes
May 27, 2021
thaJeztah
reviewed
Jun 3, 2021
cmd/dockerd/daemon.go
Outdated
|
|
||
| if opts.Validate { | ||
| // If config wasn't OK we wouldn't have made it this far. | ||
| fmt.Println("Config OK") |
There was a problem hiding this comment.
Discussing with @tonistiigi @tianon and (for comparison), NGINX prints this on stderr
nginx -t > /dev/null
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
samuelkarp
reviewed
Jun 4, 2021
integration/daemon/daemon_test.go
Outdated
| testdata := filepath.Join(dest, "..", "..", "integration", "daemon", "testdata") | ||
|
|
||
| const ( | ||
| validOut = "Config OK" |
There was a problem hiding this comment.
It looks like this is now out of date:
[2021-06-04T18:45:32.940Z] --- FAIL: TestDaemonConfigValidation (0.01s)
[2021-06-04T18:45:32.940Z] --- PASS: TestDaemonConfigValidation/malformed_config (0.11s)
[2021-06-04T18:45:32.940Z] --- FAIL: TestDaemonConfigValidation/config_with_no_content (0.11s)
[2021-06-04T18:45:32.940Z] daemon_test.go:96: assertion failed: string "the configuration file /go/src/github.com/docker/docker/integration/daemon/testdata/empty-config-1.json is valid\n" does not contain "Config OK"
[2021-06-04T18:45:32.940Z] --- PASS: TestDaemonConfigValidation/invalid_config (0.08s)
[2021-06-04T18:45:32.940Z] --- FAIL: TestDaemonConfigValidation/config_with_{} (0.10s)
[2021-06-04T18:45:32.940Z] daemon_test.go:96: assertion failed: string "the configuration file /go/src/github.com/docker/docker/integration/daemon/testdata/empty-config-2.json is valid\n" does not contain "Config OK"
[2021-06-04T18:45:32.940Z] --- FAIL: TestDaemonConfigValidation/valid_config (0.06s)
[2021-06-04T18:45:32.940Z] daemon_test.go:96: assertion failed: string "the configuration file /go/src/github.com/docker/docker/integration/daemon/testdata/valid-config-1.json is valid\n" does not contain "Config OK"
[2021-06-04T18:45:32.940Z] FAIL
[2021-06-04T18:45:32.940Z]
[2021-06-04T18:45:32.940Z] === Failed
[2021-06-04T18:45:32.940Z] === FAIL: amd64.integration.daemon TestDaemonConfigValidation/config_with_no_content (0.11s)
[2021-06-04T18:45:32.940Z] --- FAIL: TestDaemonConfigValidation/config_with_no_content (0.11s)
[2021-06-04T18:45:32.940Z] daemon_test.go:96: assertion failed: string "the configuration file /go/src/github.com/docker/docker/integration/daemon/testdata/empty-config-1.json is valid\n" does not contain "Config OK"
[2021-06-04T18:45:32.940Z]
[2021-06-04T18:45:32.940Z] === FAIL: amd64.integration.daemon TestDaemonConfigValidation/config_with_{} (0.10s)
[2021-06-04T18:45:32.940Z] --- FAIL: TestDaemonConfigValidation/config_with_{} (0.10s)
[2021-06-04T18:45:32.940Z] daemon_test.go:96: assertion failed: string "the configuration file /go/src/github.com/docker/docker/integration/daemon/testdata/empty-config-2.json is valid\n" does not contain "Config OK"
[2021-06-04T18:45:32.940Z]
[2021-06-04T18:45:32.940Z] === FAIL: amd64.integration.daemon TestDaemonConfigValidation/valid_config (0.06s)
[2021-06-04T18:45:32.940Z] --- FAIL: TestDaemonConfigValidation/valid_config (0.06s)
[2021-06-04T18:45:32.940Z] daemon_test.go:96: assertion failed: string "the configuration file /go/src/github.com/docker/docker/integration/daemon/testdata/valid-config-1.json is valid\n" does not contain "Config OK"
[2021-06-04T18:45:32.940Z]
[2021-06-04T18:45:32.940Z] === FAIL: amd64.integration.daemon TestDaemonConfigValidation (0.01s)
|
@samuelkarp or @tonistiigi ptal 🤗 |
tonistiigi
reviewed
Jun 7, 2021
daemon/config/config.go
Outdated
| @@ -398,7 +398,7 @@ func getConflictFreeConfiguration(configFile string, flags *pflag.FlagSet) (*Con | |||
| if flags != nil { | |||
| var jsonConfig map[string]interface{} | |||
| reader = bytes.NewReader(b) | |||
| if err := json.NewDecoder(reader).Decode(&jsonConfig); err != nil { | |||
| if err := json.NewDecoder(reader).Decode(&jsonConfig); err != nil && err != io.EOF { | |||
There was a problem hiding this comment.
what change caused this? Also, this ignores the errors after first json block. As the data is already in byte slice then Unmarshal is cleaner.
There was a problem hiding this comment.
I think this was added to allow for an empty daemon.json file (empty file, not empty {})
There was a problem hiding this comment.
Not sure if this is a good idea. But if added should be an explicit empty check then. Tbh I'm surprised this returns io.EOF and not ErrUnexpectedEOF.
There was a problem hiding this comment.
I think it would return ErrUnexpectedEOF if the JSON is incomplete ({), and EOF if the file is empty (or only whitespace);
https://play.golang.org/p/1_N1QidcM7T
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
)
func main() {
var reader io.Reader
var jsonConfig map[string]interface{}
for _, config := range []string{"{", "[", `""`, "", " ", `[]`, `{}`} {
reader = bytes.NewReader([]byte(config))
if err := json.NewDecoder(reader).Decode(&jsonConfig); err != nil {
fmt.Println(err)
continue
}
fmt.Println(config, "is ok")
}
}Prints:
unexpected EOF
unexpected EOF
json: cannot unmarshal string into Go value of type map[string]interface {}
EOF
EOF
json: cannot unmarshal array into Go value of type map[string]interface {}
{} is ok
Not sure if this is a good idea
I thought slightly relaxing would be ok; we already did allow an empty {}; considering an empty file to be the equivalent (config with nothing set).
Do you have specific concerns?
There was a problem hiding this comment.
Yeah, idk why empty and spaces are considered as valid json for Decode. JSON.parse in js in unexpectedeof for both cases.
I'm not strictly against allowing empty file, if detection for it is clear in code. It might lead to error going unnoticed though. If Desktop fills this file from a field I do think correct behavior would be to not create a file if the field is empty, not to create an empty file.
aiordache
force-pushed
the
daemon_config
branch
3 times, most recently
from
367683c
to
136dbed
Compare
thaJeztah
reviewed
Jun 17, 2021
daemon/config/config.go
Outdated
Comment on lines
394
to
401
| // trim leading and trailing spaces | ||
| b = bytes.TrimSpace(b) | ||
| // accept empty config files, set content to {} | ||
| if len(b) == 0 { | ||
| b = []byte("{}") | ||
| } | ||
|
|
||
| var config Config |
There was a problem hiding this comment.
I'm inclined to do the reverse here; not convert "empty" to {}, but check if it's empty, and if so, return early:
Suggested change
| // trim leading and trailing spaces | |
| b = bytes.TrimSpace(b) | |
| // accept empty config files, set content to {} | |
| if len(b) == 0 { | |
| b = []byte("{}") | |
| } | |
| var config Config | |
| var config Config | |
| b = bytes.TrimSpace(b) | |
| if len(b) == 0 { | |
| // empty config file | |
| return &config, nil | |
| } | |
Looks like all the code below should only be for loading the JSON file, and for validating if there's conflicts, so we wouldn't have to run that code afaics (yeah, this code and logic needs a rewrite, as it's quite messy)
aiordache
force-pushed
the
daemon_config
branch
2 times, most recently
from
8c7c9bd
to
d5d3231
Compare
|
@tonistiigi PTAL |
tonistiigi
reviewed
Jun 17, 2021
integration/daemon/main_test.go
Outdated
| fmt.Println(err) | ||
| os.Exit(1) | ||
| } | ||
| err = environment.EnsureFrozenImagesLinux(testEnv) |
There was a problem hiding this comment.
How do these tests use frozen images? Or any of this environment/protect calls actually. The added tests do not seem to use shared daemon at all.
There was a problem hiding this comment.
Yes, looks like we should be able to remove for now as the current tests don't use the shared daemon.
w.r.t. the testEnv, perhaps we can just skip on ! linux instead.
@aiordache could you have a look at removing the unneeded setup code?
aiordache
force-pushed
the
daemon_config
branch
4 times, most recently
from
02bbfa5
to
e39f972
Compare
|
Just realised a minor inconsistency: If no config-file is configured, it will use the default location, but the default is "optional" (as in: Perhaps we should change the message back to the original |
aiordache
force-pushed
the
daemon_config
branch
2 times, most recently
from
d608765
to
8f80e55
Compare
Fixes moby#36911 If config file is invalid we'll exit anyhow, so this just prevents the daemon from starting if the configuration is fine. Mainly useful for making config changes and restarting the daemon iff the config is valid. Signed-off-by: Rich Horwood <rjhorwood@apple.com> Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Signed-off-by: Anca Iordache <anca.iordache@docker.com>
thaJeztah
approved these changes
Jun 23, 2021
|
@tonistiigi PTAL: the test setup was updated ( |
|
CI failure is unrelated (a windows machine running out of disk space) |
tonistiigi
approved these changes
Jun 23, 2021
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On
dockerd --validate, return after the daemon config has been loaded and validated.Continuation of #38138, avoids the
os.Exit.closes #38138
fixes #36911