docker build command outputs general information lines to stderr instead of stdout when buildKit is enabled. #1186
Comments
|
Related issue - microsoft/DockerTools#213 |
|
Treating prints to stderr as error is not correct. Stdout is used for actual build results (eg. you can pipe a tarball of the built files) or for id with |
|
In what scenario is that a thing for Docker? The build output isn't files, it's images, and trying to cram an image into STDOUT...I don't even think that would be possible. Using STDERR for all logging output, while arguably technically correct, is highly irregular, and likely to cause more problems than solve. |
|
It would be nice to be able to build a container and capture the image ID at the end (like |
|
@mdonoughe
Build output is whatever you define in |
How would one do that? Seems not to be possible, normal docker build always writes everything to stdout, and buildkit doesn't write anything to stdout. And |
|
The opinions here by the maintainers are categorically wrong. Unix tools all throughout the ecosystem utilize I've run into this issue so many times now, where I would like to capture the resulting ID from stdout but still show the build progress on stderr. Docker tools have always been scripting-hostile and this is no exception - please improve this. |
|
@Qix- I'm not sure who you arguing here with. Buildkit does use stderr and only stderr for progress. |
|
The ultimate comment still stands - there is no way to extract a resulting image sha from the output of EDIT: Without brittle text processing commands, which completely disable or break the compact and neatly formatted ANSI-escaped output. |
|
Yes, there is no In buildx I would recommend that nobody should use |
|
Hello, What happened is that, everything went fine at first, but if the build was emitting too many logs, due to unfairness, the process handling the output of buildkit was not picking up line on stderr fast enough. Up to a certain point, the pipe of stderr got full and buildkit build was stuck on write syscall waiting for this pipe to get some room. So after this threshold, everything was working, but was extremly slow... I double down on moving logs output to stdout, it is not very common to send everything to stderr |
|
Let's say I have a bash script that wants to build an image with With non-BuildKit, the command Pragmatically speaking, this is completely useless. You might could parse the image id out but that's probably not an ideal way to go. The only way to do it currently is to apply the With BuildKit, the command It outputs nothing. The logs and progress are output to stderr so redirecting stderr to /dev/null silences the logs. This is still useless because it doesn't output the image id. Ironically, if you give it So, with the above, the problem is there is no way to reliably have both the docker build log / progress output and also obtain the image id in a meaningful way. What I think should change:
If people want the logs output to stdout, they can redirect stderr to stdout. If they don't want the logs at all, they can redirect it to null or somewhere it's not seen. This gives us the most flexibility. |
Not if you set |
|
Got hit by this today, after enabling BuildKit in an Azure DevOps pipeline, as AzDo fails tasks by default if anything is written to stderr. AzDo's default has never been an issue for me in years of working with it - it's absolutely bizarre that BuildKit writes stuff to stderr that are not even error messages!! 😕 I get BuildKit wants some means to redirect image output, but surely that unusual case could be behind a flag? |
Historically, stderr has been used for logging, so much so that some languages and libraries have called it the log stream. The whole point is that when meaningful output is possible, you use stderr as a means of human-readable output and stdout as a means for pragmatic output. Frankly, causing anything to fail if something is written to stderr is downright dumb as a default. That behavior should be fixed rather than expecting every application that uses stderr for disposable logs to stop using stderr. I find it more bizarre that this default hasn't caused you more issues. In addition, it's trivial to redirect stderr to stdout if that's what's wanted. |
Then why is there no meaningful output on stdout? Like, for example, the image sha? Which would allow to do something simple like this, which would be incredibly unbelievably useful: Instead I'm required to... , well, actually I don't even know what I'm required to do, all I know is that something which should be easy and obvious will probably take me at least 30min to figure out, and will require juggling and cleaning up temporary files or some complicated shell redirection magic with non standard file descriptors. Guys and gals, all I want is to be able to type |
That's the whole point of the issue, if you would read my previous reply explaining the changes I think should be made. For reference, other tools like buildah do exactly that. EDIT: Also, you can already do that with BuildKit. The problem here is you lose diagnostics. |
|
Yes, I absolutely agree. The question is, we have the year 2022, why are easy and obvious changes like this not being made? EDIT: and of course I would rather prefer to see the diagnostic output... |
Because half the people in the thread appear to think shoving everything into stdout is how things should be done. For the other half, while my suggestion is simple, it has a compatibility break as shown here: #1186 (comment) EDIT: And mine isn't the only suggestion and it differs slightly from others. There probably has to be some consensus which appears to be hard to come to. |
|
As an alternative, something like this would also work: Just remember to cleanup the temp file. |
|
Sorry for being verbose on the thread (I swear I'm not usually this chatty), but another thing that could be done is just keep |
|
Thanks for saving me the 30min. ;) Yea, that's what I meant by require juggling and cleaning up temporary files. Probably something like: |
Image ID is what the user needs only for specific types of builds: when build exported an image (into Docker image store), that image was not multi-arch and that image was not exported(eg. in OCI format). Even with all of these constraints most people just want to see the best progress information about their build, not a line in hex.
This is how it is today.
I agree that we should deprecate it and suggest people to use the new flags that are single purpose and don't do multiple things at once.
This claim sounded so bizarre I needed to set up Azure DevOps for the first time. No surprise that it is not the case.
How do you think the most common unix tools like
Stdout is used for
I'm not OK with removing the deletion logic as that is clearly the expected behavior of |
Is it the file deletion logic that's expected or just that the file is cleared before writing to it? For example, in python (since I don't know how to emulate it in bash), you should be able to open a /dev/stdout device file with truncate mode like so it should work for regular files as well: This won't work if the filesystem is set to read-only though and other issues. |
|
@computerquip-work Truncate mode is not ideal as it would leave an empty file if there is a crash. It should either clear the file or switch it with an atomic rename(there might be some tricky cases for unprivileged users). |
It certainly is the case with the SSH task (we SSH into a VM to build containers) |
|
I think we need to go back to conventions and what developers expect of build tools.
All of these build tools (I would guess every other language's build tool) output progress to stdout and errors to stderr. To break these long-standing conventions is a rather brave and pointless decision. |
|
What problem is being solved by doing that? I'd also argue that's not a long-standing convention, I've seen more projects output to stderr for progress than I have to stdout. If I were to proclaim the opposite as the long-standing convention and say |
|
I haven't just proclaimed it, I've given concrete examples of other build
tools that all conform to the convention, including Dockers own build tool.
Let me repeat that: including Dockers own build tool.
This reason this thread exist is that this change is a breaking change to
docker, a change that appears to provide no benefits and which broke my
code and clearly others.
…On Fri, 17 June 2022, 1:49 am computerquip-work, ***@***.***> wrote:
What problem is being solved by doing that? I'd also argue that's not a
long-standing convention, I've seen more projects output to stderr for
progress than I have to stdout. If I were to proclaim the opposite as the
long-standing convention and say To break these long-standing conventions
is a rather brave and pointless decision., there isn't really much
discussion to be had and it's not very productive.
—
Reply to this email directly, view it on GitHub
<#1186 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAG32OB3UHMLT6DFDZQ6DK3VPNEJZANCNFSM4I43Z46A>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
The "gotcha" arguments are really not productive. The benefit of it is that it allows one to do something like If you can make a simple example where you can pragmatically run an image that you just built without race conditions or temporary files, you would have a stronger argument. |
So because there is a tool on the internet that doesn't have
Your Github bio says you work with C/C++/Java . What stdio stream do gcc, clang or javac print their progress output? |
|
It's this sort of philosophic nonsense from the Moby contributors that has forced me to switch to something else entirely. I've advocated against Docker for a few years now due to these bizarre "dying on a hill" stances they take about how tools should behave. Docker does not behave like any other tool I know of. Its constant misuse of streams, bizarre output formats, and general inoperability with larger systems has rendered it entirely obsolete in my book. Podman is a step in the right direction, and I urge everyone to focus their time switching to it instead. I'm tired of fighting with the Moby devs over these things. It's been years without any budging. |
So you don't know gcc, clang, javac, wget, curl, time (only a sample I listed in the last 2 previous comments)? Please read through the issue and comments before going for the noise. In your previous comment #1186 (comment) you already wrote
.. without realizing that buildkit does use stderr for progress, and this issue was created by the user who found it confusing and suggested we should change it(what we have refused because it is wrong).
Unfortunately, in both of your last comments, the only person you are fighting with is yourself. You are furiously requesting a behavior that is already there. |
|
I think you have misunderstood my logic.
So because there is a tool on the internet that doesn't have -o,
If you think there is a use case for a -o switch that outputs the
artifacts to stdout then go ahead and do it.
The difference here is that the -o switch is under my control and I choose
to use it or not.
Let me detail the problem I'm trying to solve.
Essentially we run a CI/CD environment that builds a no. of docker
containers.
When reviewing logs from those CI/CD runs the docker builds generate
thousands of lines of output, none of which I care about.
Being able to suppress stdout and only outputting stderr lets me
efficiently review the logs as I'm only seeing the errors.
This is a daily use case and I suspect a common use case for others.
There seems to be a lot of talk about getting the id out of the build.
Yes, this is useful but there are lots of ways that this can be done and
creating a temp file with the id is a perfectly acceptable method.
My rule when developing CLI scripts is that the user owns the console and I
should only output to the console what the user has asked me to.
General progress messages go to stdout, errors go to stderr.
If there is an error, the app returns a non-zero exit code and I expect to
see a descriptive message in stderr.
I consider any CLI that doesn't follow these simple rules to be a badly
behaving app.
For what it's worth, I'm the author of DCLI the Dart Console SDK so I feel
I have some experience in this area and with what end-users want from a CLI
app.
dcli.onepub.dev
S. Brett Sutton
Noojee Contact Solutions
03 8320 8100
…On Fri, 17 Jun 2022 at 07:52, Tõnis Tiigi ***@***.***> wrote:
@bsutton <https://github.com/bsutton>
The content of build artifacts are never output to either stdout
So because there is a tool on the internet that doesn't have -o, that
means that buildkit should not have this feature either?
All of these build tools (I would guess every other language's build tool
Your Github bio says you work with C/C++/Java . What stdio stream do gcc,
clang or javac print their progress output?
—
Reply to this email directly, view it on GitHub
<#1186 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAG32OCP7CAKUSHIIJ3NB4LVPOO27ANCNFSM4I43Z46A>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
You can literally already do that, described in this very thread, with |
|
Ran into this issue today. In our functional testing suite, we use general information lines to assert tests. Workaround was to use stderr stream. I find it strange that |
|
Diagnostics/progress/logs go to stderr. Values to be used with pipes, |
|
I just wanted to point out that this is still a bit problematic a year later. While there are workarounds for getting things to work with |
|
Is there any progress on this issue? we are also still facing the same in an Azure DevOps pipeline, where the docker build is called in a bash task. Please see these screenshots:
|
|
|
Since Docker 23.0.0, Buildx and BuildKit are the default builder on Linux, and `docker build` is just an alias to `docker buildx build` https://docs.docker.com/engine/release-notes/23.0/#new We could keep using legacy builder by setting `DOCKER_BUILDKIT=0`, but that isn't long term solution. The failing test case is invoking `docker build`, by using `node:child_process` package and thus the test itself has zero control over Docker version, as it only depends on host machine's Docker, so Docker breaking changes can also break this test. With switch of default Docker builder to Buildx, two changes affected the test case: 1) The output during build no longer goes to stdout, but stderr. See moby/buildkit#1186 We can redirect stderr to stdout by putting `2>&1` at the end of `docker build` command, but that would affect production code as well. Doing it only for tests would require parametrization and unecessary complication of method signatures. 2) Output text has changed significantly, but since we're only matching for final success message, we only need slight adjustments
This is also our use case, and why I ended up on this page. Since |
Ditto, the problem with redirecting is that it requires more effort to detect when a build has actually failed (you have to store the output somewhere and then parse it again to see if anything starts with ERROR, if so, fail the task compared to just failing if errors are written stderr). |
|
This is quite irritating - I faced this issue when recently deploying Docker on a fresh Ubuntu install. Had to revert back to older version of Docker. |
Are you able to use the exit code of the build command? For example, when I build an image with an error, I'll get an exit code of 1. A successful build returns exit code 0. If you're on some bash / zsh / similar, you can use You can also explicitly read the exit code as in You can also inline it |
Description
docker build commands output general information lines to standard error instead of standard output when buildKit is enabled.
This is particularly problematic for extra tooling we have around docker which depend on lines on stderror to determine if the command was successful.
Following is the output of
docker buildcommand which is returned on the stderr instead of stdout. The process has exitcode=0 so, it shouldn't ideally put these lines on stderr.Steps to reproduce the issue:
DOCKER_BUILDKIT=1docker buildcommand on any project.stderrinstead ofstdout.Describe the results you received:
General information lines are present on
stderr.Describe the results you expected:
General information lines should be present on
stdoutwhen there is no error.Additional information you deem important (e.g. issue happens only occasionally):
Output of
docker version:Output of
docker info:Additional environment details (AWS, VirtualBox, physical, etc.):
Physical machine.
Running docker commands through Containers Tools for Visual Studio.
The text was updated successfully, but these errors were encountered: