Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider limiting task-input logging character max #4907

Closed
rainersigwald opened this issue Nov 11, 2019 · 4 comments · Fixed by #5210
Closed

Consider limiting task-input logging character max #4907

rainersigwald opened this issue Nov 11, 2019 · 4 comments · Fixed by #5210
Assignees
Labels
Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. performance triaged

Comments

@rainersigwald
Copy link
Member

rainersigwald commented Nov 11, 2019

An internal partner reported (see AB#1019268) out-of-memory errors in VS when using their large solution. @merlynoppenheim debugged and discovered that LogTaskInputs was enabled (by Project System Tools turning on binary logging for design-time builds), and a huge amount of memory was attributable to ItemGroupLoggingHelper.GetParameterText building up strings to log.

In the project in question, there was a single task that tried to log 9966 items, each with ~60 metadata values, some of which were long (I caught one at 172,800 characters and one at 51,860).

We should consider having an on-by-default cap on how much gets logged. It's not useless to log all of this input: you can imagine pulling it out of the log and doing something useful with it. But there's surely a point of diminishing returns between logging one input (yes! Definitely!) and logging a megabyte of text (is this actually useful?).

We spitballed a limit like "the first 10k characters of the item inputs as text".

@rainersigwald rainersigwald added Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. performance labels Nov 11, 2019
@rainersigwald
Copy link
Member Author

Related to #2200, but a different approach: instead of avoiding creating LOH strings, just discard (part of) them.

@rainersigwald
Copy link
Member Author

This might be a duplicate of #2168, though.

@omnigoat
Copy link

omnigoat commented Nov 26, 2019

Hi there. I'm pretty sure we're hitting this whilst running a custom-build step of ours. We have an intermittent failure of a custom-build step, with many inputs. The length of the Inputs field is over 190k characters long. See the bug I filed here: https://developercommunity.visualstudio.com/content/problem/804881/visual-studio-2019-cant-run-custombuildstep-with-l.html?childToView=825578#comment-825578

After further debugging, two different callstacks surface (reproduced below), both in regards to logging. Turning MSBuild project build output verbosity to Quiet works around this issue. This is not a suitable workaround.

Is there a way we can rewrite this as a task or something to avoid this? It's a severe blocker. Thanks!

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Text.StringBuilder.ToString()
   at Microsoft.Build.Shared.ReuseableStringBuilder.ToString()
   at Microsoft.Build.BackEnd.ItemGroupLoggingHelper.GetParameterText(String prefix, String parameterName, IList parameterValue)
   at Microsoft.Build.BackEnd.ItemGroupIntrinsicTask.ExecuteRemove(ProjectItemGroupTaskItemInstance child, ItemBucket bucket)
   at Microsoft.Build.BackEnd.ItemGroupIntrinsicTask.ExecuteTask(Lookup lookup)
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.Split(String[] separator, Int32 count, StringSplitOptions options)
   at Microsoft.Build.BackEnd.Logging.BaseConsoleLogger.IndentString(String s, Int32 indent)
   at Microsoft.Build.BackEnd.Logging.BaseConsoleLogger.WriteLinePretty(Int32 indentLevel, String formattedString)
   at Microsoft.Build.BackEnd.Logging.SerialConsoleLogger.MessageHandler(Object sender, BuildMessageEventArgs e)
   at Microsoft.VisualStudio.ProjectSystem.VS.Implementation.Build.VsOutputWindowFormatter.AnyEventRaisedHandler(Object sender, BuildEventArgs args)
   at Microsoft.Build.Utilities.MuxLogger.SubmissionRecord.RaiseAnyEvent(Object sender, BuildEventArgs buildEvent)
   at Microsoft.Build.BackEnd.Logging.EventSourceSink.RaiseAnyEvent(Object sender, BuildEventArgs buildEvent)

@KirillOsenkov
Copy link
Member

Also improved by #6155

@AR-May AR-May added the triaged label Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Debuggability Issues impacting the diagnosability of builds, including logging and clearer error messages. performance triaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants