Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSBuild logging fails in large builds (when configurations are cached) #6436

Closed
rainersigwald opened this issue May 10, 2021 · 1 comment
Closed

Comments

@rainersigwald
Copy link
Member

Reported by @Tratcher in dotnet/aspnetcore#32482 (comment).

MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: We shouldn't be accessing the ProjectInstance when the configuration is cached.
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Exception innerException, Object[] args)
   at Microsoft.Build.BackEnd.BuildRequestConfiguration.get_Project()
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogProjectStarted(BuildRequest request, BuildRequestConfiguration configuration)
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogRequestHandledFromCache(BuildRequest request, BuildRequestConfiguration configuration, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.LogRequestHandledFromCache(BuildRequest request, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.HandleRequestBlockedByNewRequests(SchedulableRequest parentRequest, BuildRequestBlocker blocker, List`1 responses)
   at Microsoft.Build.BackEnd.Scheduler.ReportRequestBlocked(Int32 nodeId, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.HandleNewRequest(Int32 node, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.ProcessPacket(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.<>c__DisplayClass76_0.<Microsoft.Build.BackEnd.INodePacketHandler.PacketReceived>b__0()
   at Microsoft.Build.Execution.BuildManager.ProcessWorkQueue(Action action)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Build.Execution.BuildManager.EndBuild()

This happens only when the main MSBuild node is under memory pressure, which is how it evaded detection.

The new access here was introduced in 16.9.0 with #5997.

@rainersigwald
Copy link
Member Author

Fortunately (for us to fix this) there's a secret environment variable MSBUILDDEBUGFORCECACHING=1 that pushes MSBuild down this codepath even when it's not under memory pressure, so this is trivially reproducable: set that, then build any multi-project build.

rainersigwald added a commit to rainersigwald/aspnetcore that referenced this issue May 10, 2021
rainersigwald added a commit to rainersigwald/msbuild that referenced this issue May 10, 2021
Fixes dotnet#6436 which causes this crash in cases where MSBuild's result
caching is turned on (generally this is for large builds in 32-bit
MSBuild).

```
MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: We shouldn't be accessing the ProjectInstance when the configuration is cached.
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Exception innerException, Object[] args)
   at Microsoft.Build.BackEnd.BuildRequestConfiguration.get_Project()
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogProjectStarted(BuildRequest request, BuildRequestConfiguration configuration)
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogRequestHandledFromCache(BuildRequest request, BuildRequestConfiguration configuration, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.LogRequestHandledFromCache(BuildRequest request, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.HandleRequestBlockedByNewRequests(SchedulableRequest parentRequest, BuildRequestBlocker blocker, List`1 responses)
   at Microsoft.Build.BackEnd.Scheduler.ReportRequestBlocked(Int32 nodeId, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.HandleNewRequest(Int32 node, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.ProcessPacket(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.<>c__DisplayClass76_0.<Microsoft.Build.BackEnd.INodePacketHandler.PacketReceived>b__0()
   at Microsoft.Build.Execution.BuildManager.ProcessWorkQueue(Action action)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Build.Execution.BuildManager.EndBuild()
```
rainersigwald added a commit to rainersigwald/msbuild that referenced this issue May 10, 2021
Fixes dotnet#6436 which causes this crash in cases where MSBuild's result
caching is turned on (generally this is for large builds in 32-bit
MSBuild).

```
MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: We shouldn't be accessing the ProjectInstance when the configuration is cached.
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Exception innerException, Object[] args)
   at Microsoft.Build.BackEnd.BuildRequestConfiguration.get_Project()
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogProjectStarted(BuildRequest request, BuildRequestConfiguration configuration)
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogRequestHandledFromCache(BuildRequest request, BuildRequestConfiguration configuration, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.LogRequestHandledFromCache(BuildRequest request, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.HandleRequestBlockedByNewRequests(SchedulableRequest parentRequest, BuildRequestBlocker blocker, List`1 responses)
   at Microsoft.Build.BackEnd.Scheduler.ReportRequestBlocked(Int32 nodeId, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.HandleNewRequest(Int32 node, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.ProcessPacket(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.<>c__DisplayClass76_0.<Microsoft.Build.BackEnd.INodePacketHandler.PacketReceived>b__0()
   at Microsoft.Build.Execution.BuildManager.ProcessWorkQueue(Action action)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Build.Execution.BuildManager.EndBuild()
```
rainersigwald added a commit to rainersigwald/msbuild that referenced this issue May 10, 2021
Fixes dotnet#6436 which causes this crash in cases where MSBuild's result
caching is turned on (generally this is for large builds in 32-bit
MSBuild).

```
MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: We shouldn't be accessing the ProjectInstance when the configuration is cached.
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Exception innerException, Object[] args)
   at Microsoft.Build.BackEnd.BuildRequestConfiguration.get_Project()
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogProjectStarted(BuildRequest request, BuildRequestConfiguration configuration)
   at Microsoft.Build.BackEnd.Logging.NodeLoggingContext.LogRequestHandledFromCache(BuildRequest request, BuildRequestConfiguration configuration, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.LogRequestHandledFromCache(BuildRequest request, BuildResult result)
   at Microsoft.Build.BackEnd.Scheduler.HandleRequestBlockedByNewRequests(SchedulableRequest parentRequest, BuildRequestBlocker blocker, List`1 responses)
   at Microsoft.Build.BackEnd.Scheduler.ReportRequestBlocked(Int32 nodeId, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.HandleNewRequest(Int32 node, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.ProcessPacket(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.<>c__DisplayClass76_0.<Microsoft.Build.BackEnd.INodePacketHandler.PacketReceived>b__0()
   at Microsoft.Build.Execution.BuildManager.ProcessWorkQueue(Action action)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Build.Execution.BuildManager.EndBuild()
```
JaynieBai pushed a commit that referenced this issue Feb 8, 2023
)

Bumps Microsoft.CodeAnalysis.BannedApiAnalyzers from 3.3.3 to 3.3.4.

Release notes
Sourced from Microsoft.CodeAnalysis.BannedApiAnalyzers's releases.

v3.3.4
Release build of Roslyn-analyzers based on Microsoft.CodeAnalysis 3.3.1 NuGet packages. Works with VS 2019 16.9 or later.

Contains important functionality and performance bug fixes on top of v3.3.3 release
Commits
22ea642 Merge pull request #6436 from ViktorHofer/patch-1
9018071 Fix buildtransitive vs buildTransitive difference
afa5665 [main] Update dependencies from dotnet/arcade (#6423)
afee469 Merge pull request #6427 from mavasani/CodeAnalysisTreatWarningsAsErrors_Glob...
b858999 Merge pull request #6430 from dotnet/locfiles/be3abf9f-1f22-469b-b26d-7648587...
0cbc3c5 Localized file check-in by OneLocBuild Task: Build definition ID 830: Build I...
113dadc Merge pull request #6429 from dotnet/revert-6364-net7-tests
356147f Revert "Move tests to target .NET 7"
1d7244a Update documentation for CodeAnalysisTreatWarningsAsErrors implementation
6c028d3 Re-implement CodeAnalysisTreatWarningsAsErrors with globalconfig files
Additional commits viewable in compare view
@AR-May AR-May added the triaged label Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants