How to output MSBuild binary logs when using roslyn 4.9.0 or later

[filzrev]

Background

I'm using MSBuildWorkspace to load solution/project files.
And using MSBuild binary file log output. (*.binlog) with the following code.

using var workspace = MSBuildWorkspace.Create();
var msbuildLogger = new BinaryLogger { Parameters = "msbuild.binlog" };
var project = await workspace.OpenProjectAsync("Sample.csproj", msbuildLogger);
var compilation = await project.GetCompilationAsync();

It works if you're using roslyn packages version 4.8.0.
But when upgrading to 4.9.0-3.final (latest pre-release version)
It doesn't work anymore.

This behavior changes are introduced at #70469.
And there are following comments.

TODO: decide what to do with this unused ILogger, since we can't reliabily use it if we're sending builds out of proc

The one major caveat here is we did have some entrypoints for MSBuildWorkspace that accept an ILogger;
currently these will be broken.
I'm still reaching out to folks to decide what to do here.
We can simply obsolete them, or try to get them to work in a few scenarios.
(For example, I could detect if the logger you passed is a well known built in logger we can just recreate that on the OOP side).

So it seems to be intentional behavior changes.

Question
Is there another way to output binlog when using roslyn 4.9.0 or later.
or pass an additional MSBuild parameter (-bl) for a build that is running out of process.

[jasonmalinowski]

Ah yes, I didn't finalize that, did I. 😦

The challenge now is with that rewrite MSBuildWorkspace now does the builds out-of-process. This solves a huge number of problems that people have MSBuildWorkspace but it means I can't easily pass around the ILogger to the other process anymore. And I can of course have each process write out a binlog (we actually have support for that already) but you won't always have a single binlog. I've reached out to the MSBuild team to see if they have something ready-made for doing cross-process logging and merging, but if not we'll have to figure out a better plan. I'm not sure how tricky redoing that ourselves will be, so we might have to do some compromises here.

As a shorter-term solution, if you're using OpenProjectAsync() then I can easily see you're passing in a BinaryLogger, and make this work. The challenge is if you're doing OpenSolutionAsync, since it's possible we might end up sending the underlying build operations to different processes, which would necessitate writing out separate binlogs for now. So @filzrev are you using solutions too or just single projects? Because I can easily make the latter work at least.

Tagging @JoeRobich too since this came up in dotnet/format#2094: are you always opening a single project too? So I won't run into the multiple-buildhost problem either?

[JoeRobich]

@jasonmalinowski dotnet-format will load either a solution or project depending on where/how it is invoked.

[filzrev]

I'm using MSBuild binlog for diagnostics purpose.
It's required when it failed to generate API metadata using docfx

So both Solution and Project support needed if possible.
There is no particular problem if the binlog is written per process.

[jasonmalinowski]

@filzrev So if I were to write additional files like msbuild-1.binlog or something that'd be OK? And your process can just consume all of the binlogs in the directory?

[filzrev]

@jasonmalinowski]
Yes, in my usecase it's acceptable that binlog is separately outputted per MSBuild process.

Additionaly, It's helpful if it can restrict MSBuild process count by parameter or environment variables.
Something like msbuild CLI's -maxcpucount parameter

[jasonmalinowski]

@filzrev We aren't launching processes for parallelization here but correctness. We'll launch two processes if there's a mixed set of projects that we need to analyze (i.e. a mix of classic csproj and SDK-style projects), otherwise we'll just launch one.