Eagerly parse all files in the project after project load.

[safesparrow]

Once dotnet/fsharp#14848 is fixed and FCS has a usable parsing cache,
can we add optional eager parsing of all files upon project load?
The end goal is to reach a point where every parsing request except one for the currently edited file is hitting the cache.

• Parsing is quicker than type-checking, so much less potential for putting IDE to a crawl.
• Ideally configurable by end-user, so no drawbacks for users who don't want to use it.
• Solution should take into account cache size and either stop eager parsing if the cache is overfilling or increase it when a large codebase is loaded.
• Parsing requests should be asynchronous and not block the UI in any meaningful way.

Related:
"Too many parsing requests" issue: #492

[auduchinok]

@safesparrow Can it be an FCS feature instead? Since we don't have a direct access to the FCS parse cache, it doesn't seem good to build features on top of this black box like this. The project is also being loaded from scratch after its modifications, like adding/renaming a file or changing the solution configuration, and we would need to schedule/interrupt these parsing activities. What if FCS had an API that we could use to notify it about a new project? It could then start evaluating the parsing parts of the project graph inside using its incremental builders.

[safesparrow]

As I understand, project changes rarely affect parse results, except for things like DEFINEs, a few more things and the contents of the file being parsed.

So I don't expect parse results to be invalidated frequently.

Do you think the evaluation would need to be cancelled because it's no longer valid or because any running requests could interact with other requests?

I expect all requests to finish fairly quickly after solution load and after that only individual files get reparsed as the user changes them one by one.

This would require FCS to provide a parsing cache that's shared in all the endpoints, and uses the minimum set of inputs as a key (ie. parsing inputs instead of full project options).

Once that is done, Rider would simply populate that cache by sending parse requests for all files every time a project is loaded. If parsing options haven't changed, all requests would result in a cache hit (except when the source code changed).

Whether this could be built into FCS:
Sure, an API that performs parsing of all files on project load would make sense too. But that's a slower change and I don't see this a vastly different from the first option.

[auduchinok]

As I understand, project changes rarely affect parse results, except for things like DEFINEs, a few more things and the contents of the file being parsed.

In theory, yes, but the cache uses FSharpParsingOptions inside the keys (unless it has been changed in FCS), and these options change on addition/renaming of the files, which is a change that happens more often. The full options are used because the order of the files is relevant for parsing and the last file in a project is parsed using special rules when the project is a console app.

Do you think the evaluation would need to be cancelled because it's no longer valid or because any running requests could interact with other requests?

I think we'd want this cache to be cleared for a project when the project parsing options become outdated. Which also could probably be done easier inside FCS.