Looking for a better way to trace source generator and code refactoring performance

[OronDF343]

Hi,

I am working on an incremental source generator (unfortunately I cannot share the code right now). When using it in a large project in Visual Studio, code refactorings become slow (for example, selecting a block of code and pressing Ctrl+.). Builds and normal edit operations, however, are not affected (and the generated source is updated including IntelliSense within a couple of seconds).

I managed to improve it significantly (~4x), by using the performance profiler to figure out what is being executed and improving my caching accordingly, but I have reached a point where I don't know how to proceed.

In my current version, the delay is 7-9 seconds. ~2% of the CPU usage is inside my source generator, while ~75% of the CPU usage occurs inside the built-in method Microsoft.CodeAnalysis.CodeRefactorings.ExtractMethod.ExtractMethodCodeRefactoringProvider.GetCodeActionsAsync() via two call graphs:

This behavior does not appear to exist with other source generators (unsure, not tested many), so it is probably an issue with mine, but I have no way of knowing what the issue is.

Using GeneratorTracer I can see that my generator is being called ~30 times during the operation, with each call taking 75 ms on average. That adds up to ~2.5 seconds, which only accounts for a third of the delay time. Unfortunately, GeneratorTracer does not give me an indication of which steps of the pipeline are being executed or in what order.

My pipeline has TrackingNames, which I can use when running the compilation in my own process, but I am not seeing any issue there when looking at the pipeline steps that were not cached (although it does take a much longer amount of time to run for whatever reason?). Additionally, I have no way of reproducing what ExtractMethod is trying to do.

Q: Is there a better method for tracing a source generator's pipeline when running inside Visual Studio (since the behavior I am trying to trace is specific to VS)?

[CyrusNajmabadi]

Using GeneratorTracer I can see that my generator is being called ~30 times during the operation

This was addressed in #70466 and #70477.

with each call taking 75 ms on average.

Woof. That's a reallllly slow generator. For context, an IDE feature needs to run in a few ms tops (with 50 being the high water mark where we consider things to be seriously problematic). Having a generator itself take 75ms effectively means that every semantic feature is now over budget.

For context, getting the compilation after an edit (without SGs) should take microseconds.

We're adding docs here: #71059 to help explain how to write a good incremental generator.

I am working on an incremental source generator (unfortunately I cannot share the code right now).

It's going to be hard to help out then. However, here's a few good rules of thumb:

1. Only use ForAttributeWithMetadataName. If you use any other mechanism (including other SyntaxProviders) you will have bad perf.
2. Convert the data flowing in from ForAttributeWithMetadataName into your own record models. Ensure that your record models are properly equatable. So literally write unit tests taht instantiate two different instances with the same data, and ensure that those separate instances are equal. This includes ensuring htat if your models have properties that are collections, that equality works for those as well.
3. Never store Nodes, Trees, Symbols, Operations, SemanticModels, Compilations, or other compiler structures in your models. This will break incrementality.

You can then test generator performance by writing a simple console app that does the following:

1. opens a solutoin/project (see MSBuildWorkspace)
2. gets a document and uses GetSemanticModel for it.
3. forks the document, with simple edits, and gets the semantic model from that forked document.
4. repeating 3 over and over again.

Run this under a profiler and you'll see the cost of just the generator.

We can also help out in discord https://discord.com/channels/143867839282020352/598678594750775301. Though, without the source, it will be hard to tell. Most likely something is going wrong in steps 1-3 above.

[OronDF343]

This was addressed in #70466 and #70477.

Confirmed, in 17.9 Preview 2.1 suggestions appear very quickly. Thanks!

I'll look into your other suggestions when I next have time to work on it.

[sharwell]

Having a generator itself take 75ms effectively means that every semantic feature is now over budget.

Source generators don't (and are not allowed to) run during any semantic operation that has a latency target. We use frozen partial semantics to guarantee that the generator latency has no impact on the semantic feature performance. We still want the generator to be fast, but that's primarily for other reasons.