Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to limit max log's size after escaping #660

Open
joeyave opened this issue Apr 1, 2024 · 0 comments
Open

Add an option to limit max log's size after escaping #660

joeyave opened this issue Apr 1, 2024 · 0 comments

Comments

@joeyave
Copy link

joeyave commented Apr 1, 2024

I use Zerolog for logging some JSON strings with RawJSON function. Sometimes that JSON can be very large. The problem is that logging system has certain limitation for length of the log's line and I can't control that. Max length is about 16KB. I really need to log at least part of that JSON (with some other metadata) so I've tried to trim it to 16KB and use Bytes function.

var LogSizeLimit = 15000

len(jsonBytes) > LogSizeLimit {
    jsonBytesCopy := make([]byte, len(jsonBytes))
    copy(jsonBytesCopy, jsonBytes)
    event = event.Bytes("body", jsonBytesCopy[:util.LogSizeLimit])
}

The problem here is that Bytes function escapes some special characters (double quotes etc). As result that string becomes bigger and exceeds my limit of 16KB.

I've tried to lower LogSizeLimit to let's say 15KB. It helps but not for all cases. There should be more elegant solution.

Also I thought about escaping that trimmed JSON string myself and counting how may new symbols added. Than I can trim that string even more. But that solution sounds somehow complicated and bug-prone.

len(jsonBytes) > LogSizeLimit {
    jsonBytesCopy := make([]byte, len(jsonBytes))
    copy(jsonBytesCopy, jsonBytes)
        
    unescaped := jsonBytesCopy[:util.LogSizeLimit]
    diff := len(util.EscapeForZerolog(unescaped)) - util.LogSizeLimit
        
    event = event.Bytes("body", unescaped[:util.LogSizeLimit-diff])
}

Maybe I can disable escaping in Zerolog? Or should I strictly limit my log's line size to half of the logging system's limit: 16KB / 2 = 7.5KB? Maybe I could limit log's size on Zerolog's side?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant