Fail in application running on MessagepackCompiler.RunAsync on Windows (works on Linux & macOS)

[SneWs]

Bug description

Starting with dotnet 8 and message pack stand alone generator it crashes when generating on Windows.
This is the case for latest (2.5.140) as well as prev versions, and specifically for Windows. The same code and generation flow works as expected on macOS and Linux respectively

Repro steps

Code to reproduce the behavior have been added to gist https://gist.github.com/SneWs/5c9baa2945f398c7b4661a450876e0da

And I invoke the command to generate as follows:

dotnet mpc -i MsgPackBug.csproj -n wild.Network -o GeneratedMessagePackResolvers.cs

And the tool is defined in .config/dotnet-tools.json as:

{
  "version": 1,
  "isRoot": true,
  "tools": {
    "messagepack.generator": {
      "version": "2.5.140",
      "commands": [
        "mpc"
      ]
    }
  }
}

Expected behavior

That it generates the code just as it's done before and as it still does on macOS and Linux

Actual behavior

What happened instead of what you expected.
I get a crash/exception thrown and the program stops running. The output is:

dotnet mpc -i MsgPackBug.csproj -n wild.Network -o GeneratedMessagePackResolvers.cs
Project Compilation Start:MsgPackBug
failed to get metadata of System.Threading.Tasks.Task`1
failed to get metadata of System.Threading.Tasks.Task
failed to get metadata of System.Runtime.Serialization.IgnoreDataMemberAttribute
Project Compilation Complete:00:00:00.2279938
Method Collect Start
Fail in application running on MessagepackCompiler.RunAsync
System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Collections.Immutable.ImmutableArray`1.get_Item(Int32 index)
   at MessagePackCompiler.CodeAnalysis.TypeCollector.GetObjectInfo(INamedTypeSymbol type) in D:\a\1\s\src\MessagePack.GeneratorCore\CodeAnalysis\TypeCollector.cs:line 616
   at MessagePackCompiler.CodeAnalysis.TypeCollector.CollectObject(INamedTypeSymbol type) in D:\a\1\s\src\MessagePack.GeneratorCore\CodeAnalysis\TypeCollector.cs:line 600
   at MessagePackCompiler.CodeAnalysis.TypeCollector.CollectCore(ITypeSymbol typeSymbol) in D:\a\1\s\src\MessagePack.GeneratorCore\CodeAnalysis\TypeCollector.cs:line 379
   at MessagePackCompiler.CodeAnalysis.TypeCollector.Collect() in D:\a\1\s\src\MessagePack.GeneratorCore\CodeAnalysis\TypeCollector.cs:line 305
   at MessagePackCompiler.CodeGenerator.GenerateFileAsync(Compilation compilation, String output, String resolverName, String namespace, Boolean useMapMode, String multipleIfDirectiveOutputSymbols, String[] externalIgnoreTypeNames) in D:\a\1\s\src\MessagePack.GeneratorCore\CodeGenerator.cs:line 67
   at MessagePack.Generator.MessagepackCompiler.RunAsync(String input, String output, String conditionalSymbol, String resolverName, String namespace, Boolean useMapMode, String multipleIfDirectiveOutputSymbols, String[] externalIgnoreTypeNames) in D:\a\1\s\src\MessagePack.Generator\MessagepackCompiler.cs:line 66
   at ConsoleAppFramework.WithFilterInvoker.RunCore(ConsoleAppContext _)
   at ConsoleAppFramework.WithFilterInvoker.InvokeAsync()
   at ConsoleAppFramework.ConsoleAppEngine.RunCore(Type type, MethodInfo methodInfo, Object instance, String[] args, Int32 argsOffset)
   at ConsoleAppFramework.ConsoleAppEngine.RunCore(Type type, MethodInfo methodInfo, Object instance, String[] args, Int32 argsOffset)

Version used:

• dotnet SDKs available:

dotnet --list-sdks
6.0.202 [C:\Program Files\dotnet\sdk]
6.0.301 [C:\Program Files\dotnet\sdk]
6.0.418 [C:\Program Files\dotnet\sdk]
7.0.405 [C:\Program Files\dotnet\sdk]
8.0.101 [C:\Program Files\dotnet\sdk]

• dotnet runtimes available:

dotnet --list-runtimes
Microsoft.AspNetCore.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.22 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.22 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.24 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Additional context

I can't make out why this only fails on Windows 11 and works on macOS and Linux.

[SneWs]

I'm no windows developer, but fired up a windows box and did some digging, seems that just changing <TargetFrameworks>net6.0;net7.0;net8.0</TargetFrameworks> eg adding net8.0 and rebuilding MessagePack.Generator locally and running with --framework net8.0 makes the error go away. Still puzzled over how this works when I run it on Linux and macOS, the main difference there is that I'm in complete control over the environment and ONLY have dotnet 8 SDK installed

So I guess the real solution to this problem in general is to just update MessagePack-CSharp to dotnet 8 and it should not be a problem. And seems that #1725 is on it's way? But perhaps it might be a good? or maybe a really bad? Idea to just update MPC to support net8.0 as it seems only bumping that tool solves at least the issue reported here?

In the meantime, my workaround was to just clone the repo, from master branch build it and then you can either invoke your mpc commands using dotnet run as below, or take the binary and put it in your $PATH somewhere and call your local build instead of the NuGet provided one for now.

dotnet run --project "C:\Dev\Repos\MessagePack-CSharp\src\MessagePack.Generator\MessagePack.Generator.csproj" -i <YOUR_PROJ_FILE> -n <YOUR_NAMESPACE> -o <YOUR_OUTPUT_PATH> --framework net8.0

[AArnott]

Thanks for the report and sharing your workaround. I don't know why merely targeting .NET 8 would fix this, but it seems like a bug is present and a .NET 8 target may just be masking it. The repro may be useful.

That said, we're not investing in mpc any more. The next major version will have a roslyn source generator. It's based on a lot of the same code, but a lot is different too, so I don't know whether this bug will repro with it or not.

[SneWs]

Alright, good to know.

Would it possible to do a small patch release with just the workaround similar to what I did enabling others that might be upgrading to dotnet8 and happens to be on Windows?

I could do the work needed and submit a PR if it would be an acceptable solution. For my specific use case we can always build and carry our own mpc for the windows folks.

I haven't found any bugs from the generated code we have so far and the code we generate is used quite a lot and using dotnet8 server side and il2cpp client side.

[AArnott]

Thank you for offering the PR. However I hesitate to take a PR that makes a problem go away without understanding why it works.

[SneWs]

Yeah, that's quite understandable... I've been looking at this for a while now as well trying to understand what's going on. It seems that it might be due to a bug in dotnet itself, on later versions of dotnet 8 this is a non-issue. It just won't repro in that case when on dotnet 8.0.200