Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add special BuildMessageEventArgs types for task inputs, outputs, add item and remove item #6007

Closed
KirillOsenkov opened this issue Jan 5, 2021 · 4 comments
Assignees
Milestone

Comments

@KirillOsenkov
Copy link
Member

Currently more than half of all logging is by BuildMessageEventArgs with large message text.

They're categorized into four main categories:

  1. Task Parameter:
  2. Output Item(s):
  3. Added Item(s):
  4. Removed Item(s):

Messages typically reach up to 5MB in size for a single string.

Here's a sampling:
https://gist.github.com/KirillOsenkov/952b0798a760450cc2ae3b5aed3778c3

More stats and details here:
https://github.com/KirillOsenkov/MSBuildStructuredLog/wiki/BinLog-Stats

It would be better if we introduced a custom message type derived from BuildMessageEventArgs, and stored the items on the message in a structured way, without collapsing them into a single giant string. This way loggers would still get the same behavior, but loggers that are aware (such as BinaryLogger) would efficiently serialize the items without concatenating them into a single string unnecessarily.

I expect this to have huge benefits for logging throughput, memory pressure as well as binlog size and binlog loading speed.

@KirillOsenkov KirillOsenkov added performance needs-triage Have yet to determine what bucket this goes in. labels Jan 5, 2021
@KirillOsenkov KirillOsenkov self-assigned this Jan 5, 2021
@KirillOsenkov KirillOsenkov added Area: Logging Performance-Scenario-Build This issue affects build performance. labels Jan 5, 2021
@KirillOsenkov
Copy link
Member Author

See here for the list of types currently deriving from BuildMessageEventArgs:
https://source.dot.net/#Microsoft.Build.Framework/BuildMessageEventArgs.cs,9c87bb1f74820496,references

@KirillOsenkov
Copy link
Member Author

Another thing to pay attention to is we shouldn't be writing the Message field if we can avoid it:
image

@KirillOsenkov
Copy link
Member Author

Start from ItemGroupLoggingHelper

@KirillOsenkov
Copy link
Member Author

Fixed by #6155

@ladipro ladipro added this to the MSBuild 16.10 milestone Dec 9, 2021
@AR-May AR-May added the triaged label Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants