Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix for entry data field race condition #1229

Merged
merged 1 commit into from Feb 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
66 changes: 39 additions & 27 deletions entry.go
Expand Up @@ -78,8 +78,20 @@ func NewEntry(logger *Logger) *Entry {
}
}

func (entry *Entry) Dup() *Entry {
data := make(Fields, len(entry.Data))
for k, v := range entry.Data {
data[k] = v
}
return &Entry{Logger: entry.Logger, Data: data, Time: entry.Time, Context: entry.Context, err: entry.err}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not copy caller into new entry? There are some scenarios where I want to pass the caller by myself, for example, many libraries accept a logger interface and use it to output internal log(such as SQL exec log for Gorm), so many users wrap logrus to the needed interface, but the caller got by logrus is inaccuracy, and I get the caller and pass it into Entry, and this change breaks my wrapped logger for these libraries.

Can I send a PR for copying caller in Entry.Dup util? What do you think of it? Thanks! @dgsb @epelc

}

// Returns the bytes representation of this entry from the formatter.
func (entry *Entry) Bytes() ([]byte, error) {
return entry.bytes_nolock()
}

func (entry *Entry) bytes_nolock() ([]byte, error) {
return entry.Logger.Formatter.Format(entry)
}

Expand Down Expand Up @@ -212,68 +224,68 @@ func (entry Entry) HasCaller() (has bool) {

// This function is not declared with a pointer value because otherwise
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment should be updated since it's a pointer method now.

Is it safe to use via pointer?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is safe, the comment should indeed be updated.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanted to point out since I was checking back through this that it's safe because of the entry.Dup() call in the new version.

Hooks are now safe to modify entry because any log call on entry duplicates the entire entry.

// race conditions will occur when using multiple goroutines
func (entry Entry) log(level Level, msg string) {
func (entry *Entry) log(level Level, msg string) {
var buffer *bytes.Buffer

// Default to now, but allow users to override if they want.
//
// We don't have to worry about polluting future calls to Entry#log()
// with this assignment because this function is declared with a
// non-pointer receiver.
if entry.Time.IsZero() {
entry.Time = time.Now()
newEntry := entry.Dup()

if newEntry.Time.IsZero() {
newEntry.Time = time.Now()
}

entry.Level = level
entry.Message = msg
entry.Logger.mu.Lock()
if entry.Logger.ReportCaller {
entry.Caller = getCaller()
newEntry.Level = level
newEntry.Message = msg

newEntry.Logger.mu.Lock()
reportCaller := newEntry.Logger.ReportCaller
newEntry.Logger.mu.Unlock()

if reportCaller {
newEntry.Caller = getCaller()
}
entry.Logger.mu.Unlock()

entry.fireHooks()
newEntry.fireHooks()

buffer = getBuffer()
defer func() {
entry.Buffer = nil
newEntry.Buffer = nil
putBuffer(buffer)
}()
buffer.Reset()
entry.Buffer = buffer
newEntry.Buffer = buffer

entry.write()
newEntry.write()

entry.Buffer = nil
newEntry.Buffer = nil

// To avoid Entry#log() returning a value that only would make sense for
// panic() to use in Entry#Panic(), we avoid the allocation by checking
// directly here.
if level <= PanicLevel {
panic(&entry)
panic(newEntry)
}
}

func (entry *Entry) fireHooks() {
entry.Logger.mu.Lock()
defer entry.Logger.mu.Unlock()
err := entry.Logger.Hooks.Fire(entry.Level, entry)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to fire hook: %v\n", err)
}
}

func (entry *Entry) write() {
entry.Logger.mu.Lock()
defer entry.Logger.mu.Unlock()
serialized, err := entry.Logger.Formatter.Format(entry)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to obtain reader, %v\n", err)
return
}
if _, err = entry.Logger.Out.Write(serialized); err != nil {
fmt.Fprintf(os.Stderr, "Failed to write to log, %v\n", err)
}
func() {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This extra function wrapper isn't required here since the write does nothing else after writing. ie the lock doesn't require it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it's a leftover from temporary step, I will remove it

entry.Logger.mu.Lock()
defer entry.Logger.mu.Unlock()
if _, err := entry.Logger.Out.Write(serialized); err != nil {
fmt.Fprintf(os.Stderr, "Failed to write to log, %v\n", err)
}
}()
}

func (entry *Entry) Log(level Level, args ...interface{}) {
Expand Down
2 changes: 1 addition & 1 deletion entry_test.go
Expand Up @@ -232,7 +232,7 @@ func TestEntryWithIncorrectField(t *testing.T) {

fn := func() {}

e := Entry{}
e := Entry{Logger: New()}
eWithFunc := e.WithFields(Fields{"func": fn})
eWithFuncPtr := e.WithFields(Fields{"funcPtr": &fn})

Expand Down
2 changes: 1 addition & 1 deletion logger.go
Expand Up @@ -12,7 +12,7 @@ import (
// LogFunction For big messages, it can be more efficient to pass a function
// and only call it if the log level is actually enables rather than
// generating the log message and then checking if the level is enabled
type LogFunction func()[]interface{}
type LogFunction func() []interface{}

type Logger struct {
// The logs are `io.Copy`'d to this in a mutex. It's common to set this to a
Expand Down
2 changes: 1 addition & 1 deletion logger_test.go
Expand Up @@ -25,7 +25,7 @@ func TestFieldValueError(t *testing.T) {
t.Error("unexpected error", err)
}
_, ok := data[FieldKeyLogrusError]
require.True(t, ok)
require.True(t, ok, `cannot found expected "logrus_error" field: %v`, data)
}

func TestNoFieldValueError(t *testing.T) {
Expand Down
39 changes: 36 additions & 3 deletions logrus_test.go
Expand Up @@ -588,15 +588,48 @@ func TestLoggingRaceWithHooksOnEntry(t *testing.T) {
logger.AddHook(hook)
entry := logger.WithField("context", "clue")

var wg sync.WaitGroup
var (
wg sync.WaitGroup
mtx sync.Mutex
start bool
)

cond := sync.NewCond(&mtx)

wg.Add(100)

for i := 0; i < 100; i++ {
for i := 0; i < 50; i++ {
go func() {
cond.L.Lock()
for !start {
cond.Wait()
}
cond.L.Unlock()
for j := 0; j < 100; j++ {
entry.Info("info")
}
wg.Done()
}()
}

for i := 0; i < 50; i++ {
go func() {
entry.Info("info")
cond.L.Lock()
for !start {
cond.Wait()
}
cond.L.Unlock()
for j := 0; j < 100; j++ {
entry.WithField("another field", "with some data").Info("info")
}
wg.Done()
}()
}

cond.L.Lock()
start = true
cond.L.Unlock()
cond.Broadcast()
wg.Wait()
}

Expand Down