Completely dettach IO from parallel request execution

[vinistock]

Motivation

Closes #378

Our request Handler class has leaked some responsibility over to the queue and server files. Both request definition and IO are now spread and that makes it difficult to guarantee that no IO will occur in the main request path.

This refactor tries to centralize all request running logic into an Executor class while leaving all queue and IO managing to the Handler. The only part that leaks into the Handler is defining which requests must be run sequentially inside the main process (typically client notifications and life cycle requests).

Implementation

Before

• All requests are implemented in server.rb
• Handler and Queue work together to decide how to execute and finalize requests, as well as managing the request queue
• A bit difficult to display send arbitrary notifications to the client. We currently use the on_error configuration block in server.rb to show error message notifications, but we are unable to send more than one notification to the client (e.g.: we wouldn't be able to clear diagnostics for a file and display a dialogue in the same request)

After

• The Executor class knows how to run all requests (except for requests that need access to the main process - either the queue or the thread worker). It is basically a big case statement that matches the feature to a method containing the implementation
• In the Executor, every request returns a response and all notifications (which should not return a response) return Void
• In addition to request responses, we may want to display notifications to the user. For that, requests can set the @notifications variable, which is an array of notifications to be returned to the client. For example, this allows us to return a response, clear diagnostics and show a dialogue to the user all within the same request
• The Handler class manages the queue and decides which requests are run sequentially or in parallel. All IO happens here, which should make parallelism a bit easier. We can run requests in separate processes, get the response back and then write to IO

What should reviewers focus on

• Do you think this new structure is cleaner?
• Is it easier to understand?
• How can we better document it in the source code?
• Do you have a suggestion on how to structure it differently?

Manual tests

Please, take the time to play around with the LSP on this branch to see if you can make it break. I tested on a few projects and things seem to be fine.

[bitwise-aiden]

From someone who is unfamiliar with the history of this project, this change seems to make a lot of sense.

One thing that I do want to ask, is if you'd thought at all about this ticket during the refactor: #443

It feels like they align quite well and at the very least would be worth documenting initial thoughts on how plugins could be added with the new code layout.

Otherwise it's looking good!

[vinistock]

This refactor doesn't take much of the plugin mechanisms into consideration, but I don't think it makes it harder either. The way I'm currently thinking about it, we'll need to allow plugins to define middleware and overrides for requests and those will be loaded at boot.

The plugins shouldn't interact with global store, queues and things like that, they should basically just have access to what the request implementation already has (generally the document and some extra parameters like position).

[bitwise-aiden]

Awesome work, this is really easy to follow now

[st0012]

Great job pulling off this huge rewrite. Anyway we can split the changes into multiple PRs/commits so it'd be easier to review?

[st0012]

LGTM. Just 2 nitpicks 🙂