New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[API Proposal]: rate limited logging #4915
Comments
Tagging subscribers to this area: @dotnet/area-extensions-logging Issue DetailsBackground and motivationWhen developing a web service, logs are usually added for unhappy path, for example, a database query failed because the remote server was unreachable. The code would look like this: try
{
QueryDatabase();
}
catch (Exception e)
{
Logger.LogError(e, "Error querying the database");
} One subtle issue with that code that could have very nasty consequences is that if the database is down, all queries will induce an error log. That would create a huge spikes of logs that could overload the logs collector and make it drop logs that were crucial to understand the root cause of the issue. Because this issue is easy to miss, I think would be great to integrate a token bucket rate limiter in the LoggerMessageAttribute. dotnet/runtime#82465 seems to be related. API Proposalnamespace Microsoft.Extensions.Logging;
public class LoggerMessageAttribute
{
/// <summary>
/// Maximum number of logs to restore each replenishment.
/// </summary>
public int LogsPerPeriod { get; set; }
/// <summary>
/// Period between replenishments in milliseconds.
/// </summary>
public int ReplenishmentPeriod { get; set; }
} If API Usagetry
{
QueryDatabase();
}
catch (DatabaseException e)
{
// A maximum of 5 logs will be sent by second.
LogDatabaseError(e, e.HostName);
}
[LoggerMessage(
EventId = 0,
Level = LogLevel.Error,
Message = "Error querying the database {hostName}"),
LogsPerPeriod = 5,
ReplenishmentPeriod = 1000]
public static partial void LogDatabaseError(ILogger logger, Exception e, string hostName); Alternative DesignsNo response RisksNo response
|
We are planning to augment the logging system with something that achieves the same goal, but in more flexible framework. In particular, we'll have config-time control over the sampling rate, there will be global vs. per-request sampling, and a few more bells and whistles. I'm hoping to have a proposal published on this front in the coming weeks. |
Thanks @geeknoid. I'll transfer this issue to the extensions repo for tracking. |
Background and motivation
When developing a web service, logs are usually added for unhappy path, for example, a database query failed because the remote server was unreachable. The code would look like this:
One subtle issue with that code that could have very nasty consequences is that if the database is down, all queries will induce an error log. That would create a huge spikes of logs that could overload the logs collector and make it drop logs that were crucial to understand the root cause of the issue.
Because this issue is easy to miss, I think would be great to integrate a token bucket rate limiter in the LoggerMessageAttribute.
dotnet/runtime#82465 seems to be related.
API Proposal
If
LogsPerPeriod
orReplenishmentPeriod
are equal or less than 0 they should be both ignored.API Usage
Alternative Designs
No response
Risks
No response
The text was updated successfully, but these errors were encountered: