Tracking issue for remaining AssemblyBuilder.Save work in .NET 9

[buyaa-n]

See #62956 for work done in .NET 8 for adding AssemblyBuilder.Save(string/stream) support. Remaining work include:

• Add new ILGenerator implementation
  • Initial ILGenerator implementation, save minimal/simple IL.
  • Calculate the maxstack appropriately
  • Add support for emitting a call to method, constructor, field, type references
  • Add local variables support
  • Add label (branching) support
  • Add Try/Catch/Finally/Filter/Fault block support
  • Add support for remaining Emit APIs (EmitCall, EmitCalli, Emit Signatures)
• Add support for remaining members
  • Add ConstructorBuilderImpl and save it to file/stream (this should include Emit(OpCode opcode, ConstructorInfo con) implementation)
  • Add PropertyBuilderImpl and save it to file/stream
  • Add EventBuildterImpl and save it to file/stream
  • Add implementation for remaining unimplemented APIs in AssemblyBuilder/ModuleBuilder (CreateGlobalFuntions, DefineGlobalMethod etc)
  • Add implementation for remaining unimplemented APIs for a MethodBuilder/TypeBuilder/FieldBuilder (like SignatureCallingConvention, *RequiredCustomModifiers, *OptionalCustomModifiers)
• Add approved public APIs: Add public APIs for persisted AssemblyBulder #97177
• Entry point support
  • Get API shape approved and implemented
• Add PDBs support
  • Prototype possible solutions to make the design decisions and propose API shape, get proposal approved
  • Implementation and testing

[ghost]

Tagging subscribers to this area: @dotnet/area-system-reflection-emit
See info in area-owners.md if you want to be subscribed.

Issue Details

---

See #62956 for what have done for adding AssemblyBuilder.Save(string/stream) support in .NET 8.
Remaining work include:

• Add ILGenerator implementation
  • Initial ILGenerator implementation, save minimal/simple IL.
  • Calculate maxstack for supported IL.Emit overloads
  • Add support for emitting method, field, constructor call
  • Add local variables support
  • Add support for ControlFlowBuilder branching/scoping
  • Add label (loop) support
  • Add TryCatch block support
• Add support for remaining members
  • Add ConstructorBuilderImpl and save it to file/stream (this needs minimal IL support)
  • Add PropertyBuilderImpl and save it to file/stream (this also needs minimal IL support)
  • Add EventBuildterImpl and save it to file/stream
  • Add implementation for remaining unimplemented APIs for a MethodBuilder/TypeBuilder/FieldBuilder (like SignatureCallingConvention, *RequiredCustomModifiers, *OptionalCustomModifiers)
• Add an API proposal for creating PDBs and implement it
• Entry point support

Author:	buyaa-n
Assignees:	-
Labels:	area-System.Reflection.Emit
Milestone:	-

[masonwheeler]

An important part of Reflection.Emit that I don't see listed here was PDB generation. Is this on the menu anywhere?

[buyaa-n]

Look the last bullet of the list.

[masonwheeler]

Oops! Not sure how I missed that. Good to see it got included.

[buyaa-n]

We are making a good progress, planning to finish the main functionality (excluding entry point and PDB) and make related APIs public in preview 1.

Now it would be great to test the implementation with a real-life scenario. I would really appreciate if you could:

• Prepare a simple app with your use case and share with me (in this thread or email me)
• Provide a sample assembly similar what your app generates, we can use for testing.
• Dogfood the daily build and try out the new AssemblyBuilder implementation yourself and give feedback. (The APIs are still internal so need to use reflection for now as mentioned in the sample)

Thank you in advance!

[masonwheeler]

Here's my real-life scenario: https://github.com/boo-lang/boo

This is not a "simple app" but rather a thorough, comprehensive test of the system: a compiler that uses System.Reflection.Emit as its code generator, with thousands of test cases in its automated test suite, including some scenarios that are supported by refemit but not found in the C# language.

Last I checked, all the tests pass under .NET Framework. So seeing how many pass under this project is a good benchmark for how correct your implementation is.

[buyaa-n]

The feature is now available in https://dotnet.microsoft.com/en-us/download/dotnet/9.0, please give it a try and give us a feedback. We also adding bunch of fixes: #97350 and #98393 in preview 2 mostly related to complex generic types/methods

Here is related doc: https://learn.microsoft.com/dotnet/fundamentals/runtime-libraries/system-reflection-emit-assemblybuilder#persistable-dynamic-assemblies-in-net-core

[GGG-KILLER]

Is there an ETA for SetEntryPoint?

I need that otherwise my use case is still not covered (an optimizing compiler that compiles to CIL).

[buyaa-n]

Is there an ETA for SetEntryPoint?

We decided to add completely different API for setting SetEntryPoint and other options for generated assembly. See more details from the API proposal. This will be added as soon as the API approved (within preview 3).

[leppie]

TypeBuilderInstantiation needs to be implemented.

It currently fails when doing the following (very simple and common scenario):

• Define new type A (simple poco)
• Use existing generic type B<T> to create B<A>
• When trying to reflect on B<A> (ie to get constructor) NotSupported is thrown

Will add code soon.

Edit: IronScheme/IronScheme#160

[buyaa-n]

Thank you @leppie that should have been fixed in preview 2, I can double check when you share the code.

[leppie]

Thank you @leppie that should have been fixed in preview 2, I can double check when you share the code.

Sorrry, was not aware there was a second preview already. But keep them coming!

Edit: I see preview 2 not out yet. You can just grab the code from the PR (IronScheme/IronScheme#160) and follow the steps.

[buyaa-n]

Sorrry, was not aware there was a second preview already. But keep them coming!

Edit: I see preview 2 not out yet. You can just grab the code from the PR (IronScheme/IronScheme#160) and follow the steps.

Sorry @leppie I mistaken, we had similar fixes made in preview 2, but your scenario was not covered, I put up a PR and it will be fixed in preview 3

[buyaa-n]

Is there an ETA for SetEntryPoint?

We decided to add completely different API for setting SetEntryPoint and other options for generated assembly. See more details from the API proposal. This will be added as soon as the API approved (within preview 3).

@GGG-KILLER this change now merged and will be available in preview 3

[GGG-KILLER]

Is there an ETA for SetEntryPoint?

We decided to add completely different API for setting SetEntryPoint and other options for generated assembly. See more details from the API proposal. This will be added as soon as the API approved (within preview 3).

@GGG-KILLER this change now merged and will be available in preview 3

Thanks! In the meantime I ended up vendoring System.Reflection.Emit into my project to be able to do that with the suggested API, so I should be able to migrate to it when .NET 9 p3 comes out

[leppie]

Sorry @leppie I mistaken, we had similar fixes made in preview 2, but your scenario was not covered, I put up a PR and it will be fixed in preview 3

@buyaa-n Still throwing NotSupported.

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilderInstantiation.cs#L165

[jl0pd]

Sorry @leppie I mistaken, we had similar fixes made in preview 2, but your scenario was not covered, I put up a PR and it will be fixed in preview 3

@buyaa-n Still throwing NotSupported.

https://github.com/dotnet/runtime/blob/main/src/libraries/System.Private.CoreLib/src/System/Reflection/Emit/TypeBuilderInstantiation.cs#L165

I guess that's general issue with reflection.emit that exists longer that netcore does. This may help https://stackoverflow.com/a/40221032/10339675

[leppie]

This may help

It has indeed. At least for the 1 specific blocker I have had.

Update: Only had a few more blockers, but IronScheme can now compile and bootstrap itself with .NET 9! Thanks for the pointer @jl0pd

[buyaa-n]

Last item in the list: PDB support is added in preview 4. Closing the issue as completed, if you see any bug/issue please file a new issue

[masonwheeler]

So when is preview 4 expected to be available?

[buyaa-n]

Around 21-May

[masonwheeler]

One bit of API surface that's still noticeably missing is resources. AssemblyBuilder.DefineUnmanagedResource, ModuleBuilder.DefineManifestResource, and ModuleBuilder.DefineResource are all in use in the compiler I'm trying to port, and none of these are available in the .NET Core version. Are there any plans to implement these? If not, what is the "right way" to set up resources in your PE file in Core?

[jkotas]

The recommended replacement is to pass the resource blobs as managedResources and nativeResources arguments directly to the ManagedPEBuilder constructor.

[masonwheeler]

@jkotas Sure, but where do you get "the resource blobs" from? There doesn't seem to be much information out there on the subject. Googling "c# create resource blobs" gives you a bunch of results about using Azure blob storage, and "c# create resources" gives you results about working with RESX files.

[Xyncgas]

Creating resource blobs in the context of .NET typically refers to embedding resources directly into your assembly as binary data. This could include images, XML files, localization strings, or any other data that you want to include with your application.

[jkotas]

where do you get "the resource blobs" from?

• If you called ModuleBuilder.DefineManifestResource before, the stream is the blob.
• If you called ModuleBuilder.DefineResource before, create ResourceWriter and save it into a memory stream that will be the blob. (ModuleBuilder.DefineResource was a thin wrapper over ResourceWriter and DefineManifestResource.)
• Add entries for all resources using MetadataBuilder.AddManifestResource. All streams have to be concatenated together into one BlobBuilder that you pass into ManagedPEBuilder. The offset within the BlobBuilder is argument for AddManifestResource.

@buyaa-n Could you please look into creating a sample for this?

[masonwheeler]

@jkotas Thanks, that's very helpful. And what about the unmanaged resources? It's not at all clear from the API or documentation how ResourceSectionBuilder is supposed to work. I have a bunch of bytes already that are trivial to write to a BlobBuilder, but what in the world am I supposed to do with those two offset values?!?

[jkotas]

If you have the raw bytes, you can just write them out. Here is how it is done in Roslyn: https://github.com/dotnet/roslyn/blob/7a96b1530744f6637ad6bb6bf5b8dfa33ea15b81/src/Compilers/Core/Portable/PEWriter/PeWriter.cs#L431-L447 .

The two offsets are for cases where the native resources have relocs.

[masonwheeler]

@jkotas OK, thanks!