Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log container init when max verbosity is set #3575

Open
maxgio92 opened this issue Apr 10, 2024 · 8 comments · May be fixed by #3576
Open

Log container init when max verbosity is set #3575

maxgio92 opened this issue Apr 10, 2024 · 8 comments · May be fixed by #3576
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@maxgio92
Copy link

maxgio92 commented Apr 10, 2024

What would you like to be added:

Hello, I propose to log all the container logs from before init, when setting the maximum verbosity supported.

Why is this needed:

This is extremely helpful to debug at runtime with logs from init and before, without the need to read individual logs afterward with export logs.

A lot of points can go wrong e.g. when setting up cgroups and in that case logs from Systemd are essential.
Combining them with logs from kind itself first and from Kubernetes then in a single output can be vital to better understand at runtime the initialization sequence of the containerized nodes, and possibly debug and resolve issues.

As of now only the log of the multi-user systemd target reached event is filtered, leading to ambiguous outputs like this one.

@maxgio92 maxgio92 added the kind/feature Categorizes issue or PR as related to a new feature. label Apr 10, 2024
@aojea
Copy link
Contributor

aojea commented Apr 10, 2024

naive question, how this will solve this issue #3423 (comment) ?
what different will this make?

@maxgio92
Copy link
Author

Hi @aojea, I didn't mean that it solves that issue.
I meant that there are cases where init logs at runtime, like the case related to that issue, are useful for debug.

@aojea
Copy link
Contributor

aojea commented Apr 10, 2024

sure, just trying to understand what are those cases, on my experience kind export logs is always enough, so I'm trying to understand what I'm missing

@maxgio92
Copy link
Author

I get what you mean, I agree that the same logs are available also with export logs, and at the end you can use both ways to debug, but in this case without needing to save all available logs on disk.
I'm not saying that one way is better than another but that sometimes you don't need all the logs to be exported and just the init log at runtime can provide enough information to debug issues like the one linked.

@BenTheElder
Copy link
Member

BenTheElder commented Apr 15, 2024

Usually the container log isn't where the problem is, and we wind up needing the component logs. (IE export logs)

@maxgio92
Copy link
Author

maxgio92 commented Apr 15, 2024

That's why I'm proposing to add the ability to make it print them (eg systemd logs) when maximum verbosity is set, besides the ability to export them all

@BenTheElder
Copy link
Member

That's why I'm proposing to add the ability to make it print them (eg systemd logs) when maximum verbosity is set, besides the ability to export them all

I don't see why this makes more sense than export logs, it will be really hard to pick through all of these being dumped from the same process and I've never seen another cluster tool do this.

@BenTheElder
Copy link
Member

I think this is just going to cause users to paste useless log snippets and ignore the instructions to use kind export logs for detailed logs when things break.

What if we automatically did ~kind export logs on kind create cluster failure, with some knobs to opt out and to control where these go?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants