Reference local projects in Source Generator

[trampster]

How do I reference a local project from a Source Generator project. The project is also used from normal .Net projects so cannot be marked as an Analyzer.

I tried adding a normal project reference:

<ProjectReference Include="../JsonSGen/JsonSGen.csproj" />

But I get the following error when trying to use the Source Generator:

'System.IO.FileNotFoundException: Could not load file or assembly 'JsonSGen, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.

I tried marking it's output type as Analyzer in the project reference OutputItemType as follows:

<ProjectReference Include="../JsonSGen/JsonSGen.csproj" OutputItemType="Analyzer" />

But this had exactly the same problem.

All the projects have:

<LangVersion>preview</LangVersion> 

The anaylizer is:

<TargetFramework>net5.0</TargetFramework> 

and the referenced project is:

<TargetFramework>netstandard2.1</TargetFramework>

[trampster]

From the discussions on gitter both @canton7 and @YairHalberstadt have had this problem.

Both @canton7 and I are currently working around this by linking in the .cs files we need, but this is a far from ideal solution.

[canton7]

To clarify: I went straight for linking in the cs files I needed, and didn't attempt to do what you're doing.

[trampster]

I'm not sure if this restriction applies to package references as well, if so that means you can't use code from a nuget package unless that package was specifically built as an Analyzer

[mavasani]

@trampster are you able to see the source generator dependency in the Analyzers node in solution explorer when you add a NuGet package reference to your source generator NuGet package?

[trampster]

No, I'm on vscode for linux

[mavasani]

I'm not sure if this restriction applies to package references as well, if so that means you can't use code from a nuget package unless that package was specifically built as an Analyzer

That is correct, analyzers and source generator can only dependency that is itself marked as an analyzer. This is required to ensure proper execution in presence of shadow copying. We had considered a special AnalyzerDependecy item type, but that did not give us much compared to just reusing Analyzer item type for dependencies.

[trampster]

"but that did not give us much"

What about the ability to use any of the many useful nuget packages that are not analyzers. Which is nearly all of them. This severely limits the size and complexity of a Source Generators, without this you can't scale them, because you have to reimplement everything yourself instead of using off the shelf nuget packages.

[mavasani]

Tagging @jmarolf @sharwell
Is there a way to add a NuGet package reference to an analyzer project and mark all assemblies coming from the package reference as analyzer dependencies?

[sharwell]

mark all assemblies coming from the package reference as analyzer dependencies?

https://github.com/dotnet/roslyn-sdk/blob/0313c80ed950ac4f4eef11bb2e1c6d1009b328c4/samples/CSharp/SourceGenerators/SourceGeneratorSamples/SourceGeneratorSamples.csproj#L13-L30

[jasonmalinowski]

And @jmarolf might also know if we have a bug tracking a better experience here. Ideally we'd have flag in your analyzer/generator producing project that does all of this magically for you.

[Eli-Black-Work]

I've filed an issue in the MSBuild repo for making an analyzer's NuGet packages be automatically included. See dotnet/msbuild#7640

Is MSBuild the correct repo for this, or would this be better filed in the Roslyn repo?

[sharwell]

You can add a dependency to a source generator in the same solution through three steps:

1. Add a <PackageReference>, making sure to set both GeneratePathProperty="true" and PrivateAssets="all"
2. Add a build target to add <TargetPathWithTargetPlatformMoniker> elements as part of GetTargetPath, and add all the required dependency assemblies inside this target, making sure to set IncludeRuntimeDependency="false"
3. Update <GetTargetPathDependsOn> to ensure the target from the previous step is used

You can see an example of these steps here:
https://github.com/dotnet/roslyn-sdk/blob/0313c80ed950ac4f4eef11bb2e1c6d1009b328c4/samples/CSharp/SourceGenerators/SourceGeneratorSamples/SourceGeneratorSamples.csproj#L13-L30

The anaylizer is:

<TargetFramework>net5.0</TargetFramework> 

and the referenced project is:

<TargetFramework>netstandard2.1</TargetFramework>

Note that the source generator must target netstandard2.0, and the dependencies must target netstandard 2.0 or netstandard1.x. See #47087.

[Eli-Black-Work]

@trungnt2910 Can you elaborate? 🙂

Here's how we use <IsRoslynComponent>true</IsRoslynComponent> in our projects:

ExampleProject.csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">
	...
	<ItemGroup>
		<ProjectReference Include="..\CodeGenerators\ExampleGenerator\ExampleGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
		<ProjectReference Include="..\CodeGenerators\GeneratorCommon\GeneratorCommon.csproj" OutputItemType="Analyzer" />
	</ItemGroup>
</Project>

ExampleGenerator.csproj:

<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
		<TargetFramework>netstandard2.0</TargetFramework>
		<LangVersion>8.0</LangVersion>
		<IsRoslynComponent>true</IsRoslynComponent>
	</PropertyGroup>

	<ItemGroup>
		<ProjectReference Include="..\GeneratorCommon\GeneratorCommon.csproj" />
	</ItemGroup>

</Project>

GeneratorCommon.csproj:

<Project Sdk="Microsoft.NET.Sdk">
	// Nothing special. Note that we don't define <IsRoslynComponent>true</IsRoslynComponent> for this project, since it's a shared library that's consumed by ExampleGenerator
</Project>

[trungnt2910]

What I had was:

Consumer lib:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net$(_BrowserNetVersion)</TargetFramework>
    <Nullable>enable</Nullable>
    <LangVersion>latest</LangVersion>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <GenerateDocumentationFile>true</GenerateDocumentationFile>
    <IsTrimmable>true</IsTrimmable>
    <!-- Ignore those spam warnings to filter out the real problems -->
    <!-- No matching param tag in XML comment; Cref attribute cannot be resolved; Referenced assembly does not have a strong name -->
    <NoWarn>CS1573;CS1574;CS8002</NoWarn>
    <!--  
      Microsoft.DotNet.SharedFramework.Sdk requires our framework assemblies to be signed 
      (to have a "public key token")
    -->
    <SignAssembly>True</SignAssembly>
    <AssemblyOriginatorKeyFile>$(_BrowserRootDirectory)/DotnetBrowser.snk</AssemblyOriginatorKeyFile>
  </PropertyGroup>

  <!-- Unrelated items -->

  <ItemGroup>
    <ProjectReference Include="..\Trungnt2910.Browser.Generators.TypeScript\Trungnt2910.Browser.Generators.TypeScript.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
  </ItemGroup>

  <!-- Unrelated items -->

  <Import Project="..\Trungnt2910.Browser.Generators.TypeScript\Trungnt2910.Browser.Generators.TypeScript.props" />

</Project>

Generator lib:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <Nullable>enable</Nullable>
    <LangVersion>latest</LangVersion>
    <ImplicitUsings>true</ImplicitUsings>
    <DefineConstants>NOGOBIE</DefineConstants>
     <!-- Won't work -->
     <!--<IsRoslynComponent>true</IsRoslynComponent>-->
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.3" PrivateAssets="all" />
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.2.0" PrivateAssets="all" />
    <PackageReference Include="TypeScriptAST" Version="1.0.2" NoWarn="NU1701" GeneratePathProperty="true" PrivateAssets="all" />
  </ItemGroup>

  <ItemGroup>
    <Compile Include="../Trungnt2910.Browser/Generators/JsObjectGenerator.cs" />
  </ItemGroup>

  <ItemGroup>
    <None Include="$(OutputPath)\$(AssemblyName).dll" Pack="true" PackagePath="analyzers/dotnet/cs" Visible="false" />
    <None Include="$(PKGTypeScriptAST)\lib\net452\*.dll" Pack="true" PackagePath="analyzers/dotnet/cs" Visible="false" />
    <None Include="Trungnt2910.Browser.Generators.TypeScript.props" Pack="true" PackagePath="build" Visible="false" />
  </ItemGroup>

  <PropertyGroup>
    <GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
  </PropertyGroup>

  <Target Name="GetDependencyTargetPaths">
    <ItemGroup>
      <TargetPathWithTargetPlatformMoniker Include="$(PKGTypeScriptAST)\lib\net452\TypeScriptAST.dll" IncludeRuntimeDependency="false" />
    </ItemGroup>
  </Target>

</Project>

The major difference between my project and yours is that I used an external reference through a NuGet PacakgeReference while you referenced a local project.

I am aware that I'm doing some bad practice by referencing a legacy net452 library, but removing this library and substituting it with a netstandard2.0 version does not work either.

[trungnt2910]

Carefully reading this discussion's title I can see that it explicitly states "local projects", but I came here through a link from another issue.

[Eli-Black-Work]

@trungnt2910 Ah, yes, I think that if you want to debug the source generator, you probably need to reference it as a local project, rather than a NuGet package (although there might be another way that I'm not aware of 🙂)

[trungnt2910]

A workaround that works fine for me is to do whatever I'm doing for my source generator, and then add another dummy source generator that does not have any external dependencies. When I debug the dummy source generator, breakpoints for the first generator still hit.

[nikolaidk]

I had a situation alike in some regards. I have a source generator with a project reference to a utility lib. The utility lib is used in a VSIX extenstion and in a source generator in webapi project. To make this work i had to reference both the utility project and the source generator project as analyser references in the webapi project, and everything works. Now the source generator project picks up the util project without errors.

<ItemGroup>
   ...
    <ProjectReference Include="..\XXX.Utils\XXX.Utils.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false"/> 
    <ProjectReference Include="..\XXX\XXX.Generator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
...
  </ItemGroup>

[xyven1]

This was the problem for me. Seems quite annoying that a simple project reference to the generator is not sufficient? Certainly confusing when trying to reference it from other CS Projects.

Probably Relevant is this issue

[erwan-joly]

with System.Text.Json 6.0.0

warning CS8785: Generator 'nameofthegenerator' failed to generate source. It will not contribute to the output and compilation errors may occur as a result. Exception was of type 'FileNotFoundException' with message 'Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.'

when running this

JsonDocument.Parse(json);

this never fail when doing dotnet build but visual studio 2022 fail to build...

[ganeshnj]

I have experience similar behavior in Visual Studio, not just FileNotFoundException exception but most of the time Visual Studio fails to generated the C# code and I have to close VS and start again.

Give a try restarting VS. No ideal.

[sven-n]

Is your generator project targetting netstandard2.0?

[ganeshnj]

Yes

FYI https://github.com/aws/aws-lambda-dotnet/blob/feature/annotations/Libraries/src/Amazon.Lambda.Annotations.SourceGenerator/Amazon.Lambda.Annotations.SourceGenerator.csproj#L4

[erwan-joly]

yes and it works fine with System.Text.Json 5 as long as I have

	<Target Name="GetDependencyTargetPaths">
		<ItemGroup>
			<TargetPathWithTargetPlatformMoniker Include="$(PKGSystem_Text_Json)\lib\netstandard2.0\System.Text.Json.dll" IncludeRuntimeDependency="false" />
		</ItemGroup>
	</Target>

[vpenades]

I've just started experimenting with source generators, and as soon as I've began doing interesting stuff, that is, referencing useful packages, everything fell down and I wasted several hours until I found this discussion.

I understand what @mavasani said here, but I agree with @thomaslevesque here.

Developers will expect similar behavior compared to other projects, and not being able to reference projects and packages seamlessly severely waters down the usefulness of source generators. Furthermore, this was probably known by microsoft since the very beginning, and it should have been explained and pointed in all the introductions I've read, to prevent countless developers wasting their time

I was planning to do some stuff with source generators based on the expectations I had... but seeing the state of things, to me it's clearly a no go. Maybe next time.

[ptasev]

I completely agree. I spent many hours trying to reference nuget packages and got nowhere. This is such a painful experience and incredibly frustrating. I was hoping to introduce source generators in our code at work, but I will not be bothering until there is a smoother developer experience.

[ViRuSTriNiTy]

Same here, the manual mangling with numerous options to fix FileNotFoundException warnings is mind boggling. You have to have an in-depth understanding how references and NuGet works to even grasp what is happening here. I think that most devs don't want to get in touch with this framework stuff, it should just work. I know that it is sometimes hard to implement it that way, but here ... oh my. My project files currently contain numerous comments hinting to discussions, workarounds and documentation parts easily overlooked to explain why my files look like this. So please, Microsoft, work on making this more seamless.

[vpenades]

I've found this useful link about writing practical source generators, which also covers the transient project dependencies in full.

I'm still not touching source generators until dotnet provides a simple way of adding the dependencies, but I provide the link for those that might find it useful:

https://turnerj.com/blog/the-pain-points-of-csharp-source-generators

[znakeeye]

The only thing that worked for me was this:

<PropertyGroup>
  <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
</PropertyGroup>

[farid-fereidooni]

I know this thread already has an answer for package references, but for some scenarios, you would still want to add a reference to a local project with ProjectReference.
This is my workaround. Suppose I need to reference a class lib project called "DependencyB"
I've also added the Newtonsoft.json NuGet package so you can compare them.

<ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="12.0.1" PrivateAssets="all" GeneratePathProperty="true" />
    
    <!-- we need to add a reference normally but with PrivateAssets="all" property because it's going to be used only in analysing stage-->
    <ProjectReference Include="..\DependencyB\DependencyB.csproj" PrivateAssets="all" />
    
    <!-- We need to add *.dll here so all dlls would be packed inside NuGet package as our DependencyB.dll is not a NuGet package -->
    <None Include="$(OutputPath)\*.dll" Pack="true" PackagePath="analyzers/dotnet/cs" Visible="false" />
    
    <!-- Package the Newtonsoft.Json dependency alongside the generator assembly -->
    <!-- Newtonsoft.json should be added to our NuGet pacakage with a different approach -->
    <None Include="$(PKGNewtonsoft_Json)\lib\netstandard2.0\*.dll" Pack="true" PackagePath="analyzers/dotnet/cs" Visible="false" />

</ItemGroup>

This is enough if you add your Source Generator as a NuGet package.

But if you are going to reference your Source Gerantor as a project reference we should add these too:

  <PropertyGroup>
      <GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
  </PropertyGroup>

And:

  <Target Name="GetDependencyTargetPaths">
      <ItemGroup>
          <!-- Newtonsoft.json will generate a path property because we did the GeneratePathProperty="true"-->
          <TargetPathWithTargetPlatformMoniker Include="$(PKGNewtonsoft_Json)\lib\netstandard2.0\*.dll" IncludeRuntimeDependency="false" />

          <!-- our local project will not generate path variable so we need to find it from it's own build output-->
          <!-- I used $(OutputPath) variable because the dependecy has the exact TFM and configuration as the generator 
               you might need to add configuration manually -->
          <TargetPathWithTargetPlatformMoniker Include="..\DependencyB\$(OutputPath)\*.dll" IncludeRuntimeDependency="false" />
      </ItemGroup>
  </Target>

After these steps, you don't need to add anything else to the target project except the Source Generator itself

[devantler]

I just want to emphasise that I believe this is the best solution for <ProjectReference ...> usage with SourceGenerator projects today. It allows you to configure everything from the generator project.

Another side note is that, if you reference the SourceGenerator project in e.g. a test project, you might want to add $(ProjectDir)/../DependencyB\$(OutputPath)\*.dll, or otherwise it will reference the TargetPathWithTargetPlatformMoniker incorrectly.

@farid-fereidooni thanks for sharing!

[farid-fereidooni]

@adampaquette @devantler
Thank you for the side notes and kind comments.

[seclerp]

I don't really know why this answer is not considered the best one that covers the initial question completely. Thank you!

[CostaBru]

Thank you, Farid! It helped me to extract a code generation logic to a new proj and create a reference to that one in the original source generator.

[znakeeye]

At first I thought you cited a chapter from Shakespeare 😝

It's been three years... maybe time for Microsoft to provide us with an elegant solution?

[kamyker]

Nothing from this thread worked for me. Interestingly enough only incremental generator failed work. Only way I found to fix it is adding missing dll in consuming project as ANALYZER:

        <Analyzer Include="Path\Microsoft.Extensions.ObjectPool.dll" />
        <Analyzer Include="Path\MySourceGenerator.dll" />

[Jacob273]

May anyone tell me whether there is any pattern that this PKG variable has to be set with ?

$(PKGNewtonsoft_Json)

What value shall be set e.g for following:

<PackageReference Include="Swashbuckle.AspNetCore.Annotations" Version="6.*" />	<PackageReference Include="Swashbuckle.AspNetCore.Annotations" Version="6.*" GeneratePathProperty="true" PrivateAssets="all" />

I've tried with following:

$(PKGSwashbuckle_AspNetCore_Annotations) (complete example below)

  <Target Name="GetDependencyTargetPaths">
      <ItemGroup>
          <TargetPathWithTargetPlatformMoniker Include="$(PKGSwashbuckle_AspNetCore_Annotations)\lib\netstandard2.0\Swashbuckle.AspNetCore.Annotations.dll" IncludeRuntimeDependency="false" />
      </ItemGroup>
  </Target>

does it even matter?

[Eli-Black-Work]

Yes, the naming matters 🙂

Here's a complete example from one of our projects:

<ItemGroup>
	<PackageReference Include="Microsoft.OpenApi" Version="1.2.3" GeneratePathProperty="true" PrivateAssets="all" />
	<PackageReference Include="Microsoft.OpenApi.Readers" Version="1.2.3" GeneratePathProperty="true" PrivateAssets="all" />
	<PackageReference Include="SharpYaml" Version="1.6.6" GeneratePathProperty="true" PrivateAssets="all" />
</ItemGroup>

<!-- See https://github.com/dotnet/roslyn-sdk/blob/0313c80ed950ac4f4eef11bb2e1c6d1009b328c4/samples/CSharp/SourceGenerators/SourceGeneratorSamples/SourceGeneratorSamples.csproj#L13-L30
and https://github.com/dotnet/roslyn/discussions/47517#discussioncomment-64145 -->
<PropertyGroup>
	<GetTargetPathDependsOn>$(GetTargetPathDependsOn);GetDependencyTargetPaths</GetTargetPathDependsOn>
</PropertyGroup>

<Target Name="GetDependencyTargetPaths">
	<ItemGroup>
		<TargetPathWithTargetPlatformMoniker Include="$(PKGMicrosoft_OpenApi)\lib\netstandard2.0\Microsoft.OpenApi.dll" IncludeRuntimeDependency="false" />
		<TargetPathWithTargetPlatformMoniker Include="$(PKGMicrosoft_OpenApi_Readers)\lib\netstandard2.0\Microsoft.OpenApi.Readers.dll" IncludeRuntimeDependency="false" />
		<TargetPathWithTargetPlatformMoniker Include="$(PKGSharpYaml)\lib\netstandard2.0\SharpYaml.dll" IncludeRuntimeDependency="false" />
	</ItemGroup>
</Target>

[MeltyPlayer]

Any hope of getting a proper fix for this? The solutions suggested here haven't worked for me and just seem like workarounds. Ideally, adding dependencies into a source generator would "just work".

[MeltyPlayer]

Well, adding the project as a dependency to resolve at runtime worked for a little bit, but now it doesn't work anymore for some reason. (???)

I'm getting errors that it can't be found, hopefully this is just a minor oversight on my end.

[devantler]

Well, adding the project as a dependency to resolve at runtime worked for a little bit, but now it doesn't work anymore for some reason. (???)

I'm getting errors that it can't be found, hopefully this is just a minor oversight on my end.

It looks like you are on the right track. Maybe you can use my solution for reference to find your issue:

Devantler.DataMesh.DataProduct.Generator/Devantler.DataMesh.DataProduct.Generator.csproj

It works for both local and nuget references and shows my approach for both. I fiddled around with it a lot to find the minimal viable solution, so I believe what I do here is the absolute minimum you need to do to reference dependencies in source generators.

I use $(ProjectDir) to reference local project references, as that solves some issues if you have test projects having a reference to the source generator as well. This might also solve your issue here.

Hope it helps!

[MeltyPlayer]

Thanks, switching to your approach for <TargetPathWithTargetPlatformMoniker> will definitely be a lot more scalable!

<Target Name="GetDependencyTargetPaths">
  <ItemGroup>
    <TargetPathWithTargetPlatformMoniker Include="$(ProjectDir)\..\Asserts\$(OutputPath)\*.dll" IncludeRuntimeDependency="false" />
  </ItemGroup
</Target>

Unfortunately, I haven't been able to find any notable differences between your project and mine, though, other than that yours is in netstandard2.1 while mine is still netstandard2.0. Specifically, I have the following situation in a big solution:

• Common netstandard2.0 "Asserts" library that defines basic assertion methods.
• Source generator that generates methods for reading classes from binary depends on "Asserts".
• Program depends on source generator library to generate code, and also depends on "Asserts" separately.
• The source generator is able to resolve "Asserts" at compile-time, but the program is failing to resolve "Asserts" at runtime: FileNotFoundException: Could not load file or assembly 'Asserts, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.

This had been working for a bit earlier today, but I must have done a clean or rebuild or something where it now just doesn't want to work anymore. I've run countless cleans, rebuilds, and cleared all of the bin/obj directories in my solution but nothing has worked. It consistently throws the above error when trying to run my program.

...in Visual Studio, at least. When I try to run it as a standalone executable, it works just fine.

So it seems like this might be a Visual Studio issue? I'll try recloning my project from GitHub and see if that changes anything.

Edit: Unfortunately it didn't help. I think I'll just revert these changes again and table this for now.

[vpenades]

Something I experienced with the little work I did with generators is that Visual Studio loads the generator into its contexts, and once loaded, it cannot unload it. That means that changes that happen in the code afterwards are not reflected in the behavior of the generator. In fact, the safest way I found to test changes in the generator's code is to exit Visual Studio and reopen it again, which triggers compiling and loading the latest changes into the context.

[aspeckt-112]

Very helpful, and still relevant as of November 2023.

To aid discovery, I searched for "unable to use local project reference in source generator C#" and eventually found my way here.

Thanks for sharing, @MeltyPlayer