Fix the log duplication issue for --log-file. #65
Conversation
k8s-ci-robot
added
cncf-cla: yes
|
/assign @dims @tallclair |
yuwenma
force-pushed
the
dedup-log
branch
2 times, most recently
from
674f196
to
fc63c40
Compare
|
/cc @adrianlzt @DirectXMan12 (Thanks Adrián for filing the issue) 😃 |
|
/assign @munnerz |
klog.go
Outdated
| // no longer tracks logs separately due to their severity level, but rather | ||
| // only write to the info logfile which later on will be flushed to the | ||
| // dedicated log_file. | ||
| func (l *loggingT) writeToDifferentSeverity(s severity, data []byte) { |
There was a problem hiding this comment.
I recommend factoring this a bit differently:
Make the function:
func (l *logging.T) writeLogData(s severity, data byte) {
if l.shouldLogToStderr(s) { ... }
if l.logFile != "" {
... // handle single-file mode
} else {
... // handle multi-file mode
}
}There was a problem hiding this comment.
Sure. but it will have many duplicate code. Is that okay?
There was a problem hiding this comment.
I don't think it will have too much. I'm worried that the current approach of defaulting to the info log is too error-prone, and folding in the log-to-stderr functionality will actually deduplicate code (and fix a bug) in the fatal log path.
klog.go
Outdated
| // dedicated log_file. | ||
| func (l *loggingT) writeLogData(s severity, data []byte) { | ||
| if l.logFile != "" { // single-file mode | ||
| if l.file[infoLog] == nil { |
There was a problem hiding this comment.
Can we use a dedicated buffer for this, rather than repurposing infoLog?
There was a problem hiding this comment.
Agree we shouldn't repurpose infolog. But I also have some concerns to use a dedicated buffer:
I want to patch a quick/safe fix to the log duplicate problem and catch the 1.15 release. Defining a new buffer results in much more complicated changes and it should have longer testing time before imported to kubernetes/kubernetes.
Updated the code with a dedicated buffer for single mode. Let me know if that's the reasonable approach and is safe to go in. If not, I'd prefer a quick fix that touches the least code.
|
@tallclair happy with this? |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: tallclair, yuwenma The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
What this PR does / why we need it:
We noticed that when a service/app is using the --log-file flag, there could be a severe performance regression due to the duplicated logs generated in log severity cascading (e.g. Error level Log is also tracked and stored in Warning and Info level log). This not only generates duplicate logs in the destination log-file, but also slow down the log-file flush. This PR is specifically adjusting the klog behavior when --log-file is used, and not aimed at providing a generic solution about the log severity cascading or other klog improvements.
For more context, see kubernetes/kubernetes#77904
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #53
Special notes for your reviewer:
Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.
Release note: