Description

EQL has an "operational" grammar, which is implemented in a form of Java fluent interfaces.

The objective is to improve the EQL grammar and the AST structure in order to:

• Facilitate evolvability (i.e., introduction of new querying capabilities, which means support for
  new expressions, should be easy and reliable in terms of expression correctness).
• The current AST structure needs to be better suited for processing by the EQL semantic analyser.
  It should also be guaranteed that only valid ASTs would result from EQL expressions. It would be
  beneficial to get an already built AST from a fluent API method chain, instead of a sequence of
  tokens as it is now.

The current approach is manual, where EQL's operational grammar is maintained by extending existing
and introducing new fluent interfaces, which in turn need to be implemented to emit the relevant
tokens. This approach worked well in practice, but each time new querying capability is needed, a
very careful informal analysis is required to make sure that any changes or introduction of new
constructs would not lead to a possibility of allowing the creation of invalid EQL expressions.
Also, there is not easy way to guarantee that the resultant AST would strictly conform to the
amended operational grammar.

One well-established tool that supports development of programming languages is ANTLR.
ANTLR accepts a grammar in a specific format and generates a parser for the language described by the given grammar.

Parsers are usually used with plain text as input, and employing them to parse a fluent API is not a
typical task. Nevertheless, parsers generated by ANTLR can be used for this purpose. The main
difference lies in substituting the output of the lexer, which is also generated by ANTLR and
serves as the entry point of the parsing process. Instead of feeding EQL as plain text to the
generated lexer, we should construct the lexer's output ourselves, which is very similar to what we
have now -- a sequence of tokens that result from the fluent API method call chain.

#2178 lays the groundwork for integration with ANTLR.

Expected outcome

• Introduction of new querying capabilities would only require modification of the established EQL
  grammar, which would result in automatic regeneration of a parser for the language.
• AST structure that is well-aligned with the grammar.
• Embedding into Java with a path for embedding into other languages (e.g., JavaScript).