Performance Improvement Plan

[jpogran]

Language Server Version

v0.31.4

Problem Statement

We've been investigating performance for terraform-ls to develop a short term and a long term plan to address the issues that have been reported.

We know most of the issues reported surround the work we do in the background to provide features like intellisense, go to references, hover documentation, etc. When you open up the editor, we do a lot of work in the background to understand the code you're working on. We have an indexing process that finds all the Terraform files and modules in your current working directory, parses them, and builds an understanding of all the interdependencies and references inside. We hold this information inside an in-memory database, which we update as you change your files. If you open a directory with many hundreds of folders and files, this may consume more CPU and memory than we intend.

Proposal

In the short term we're working to add ways to measure and record the performance of terraform-ls in different situations so we can pinpoint the best path forward. We need a way to record the time it takes to perform various tasks in a manner that will give us the information we need while also respecting user privacy and not intruding on usage. We’ve started to instrument opt-in only performance tracing that is currently only used by the developers on the team.

We're also reaching out to users who have reported issues to get a closer look at how they are using VS Code to edit Terraform code. We've noticed that people edit their Terraform code using many different approaches. Some open single files, some open single folders, others open a directory containing many hundreds of modules and Terraform files. No individual approach is wrong or right, but each requires some consideration when developing a language server.

We've implemented several targeted improvements over the last several point releases based on our investigations: (#1332 and #1258 ). We’ve also added a language status indicator in hashicorp/vscode-terraform#1547 that approximates an indicator when the language server is ready for use so you can see that there are things happening in the background.

Most of the work we anticipate doing is optimizing these background processes. We plan to use the performance data we collect to inform places where we can either optimize or do less work entirely #1339. We are also planning to add a configuration setting #1341 so that users can adjust/tune the amount of workers handling these background jobs to suit their individual needs.

This is by no means all we intend to do, but it is a start. We are hoping these set of steps will inform us on the best approach for resolving these issues long term.

This issue will be pinned to the repo and updated with links to work being done to address this as they are planned and taken on.

Shipped

• Stable 0.31.5
  • Improve PreloadEmbeddedSchema performance #1369
  • langserver: Avoid re-processing open files after file saving #1372

Current Investigations

• Avoid re-processing files on textDocument/didOpen if content is the same as on disk #1375

References

• Open performance issues

Help Wanted

• I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[jpogran]

We've just released 0.31.5 which contains two fixes (#1369, #1372) in our continuing efforts to address this issue.

With the fix, we should be back to 0.29.2 levels of CPU usage while maintaining the memory improvements of 0.29.3.

If you have the time, please give it a try and let us know how it works for you. Please open a new issue rather than replying here, as we're using this issue as a tracking issue.