Inline _identifier and _quoted_identifier
#112
Conversation
| @@ -122,7 +122,7 @@ | |||
| "]" [(1, 4), (1, 5)] | |||
| ) | |||
| ) | |||
| close: "]]" [(1, 5), (1, 5)] | |||
| close: "]]" MISSING [(1, 5), (1, 5)] | |||
There was a problem hiding this comment.
We now show MISSING for anonymous and named nodes that have been interpolated by tree-sitter's error recovery (like they do in their cli output). This requires DavisVaughan/r-tree-sitter#7
| rhs: (identifier MISSING [(3, 0), (3, 0)]) | ||
| ) | ||
|
|
||
| Text | ||
| 1 + |
There was a problem hiding this comment.
All of the other places in this PR where MISSING now shows up would have worked even without the changes to the grammar in this PR, because every other case has to do with missing anonymous nodes.
This is a missing named node, and tree-sitter pretty much always seems to inject identifier here as the implied node type, thus causing the issue where prior to the grammar changes here, this would not have shown up as MISSING.
| word: $ => $._identifier, | ||
| word: $ => $.identifier, |
There was a problem hiding this comment.
Note _identifier -> identifier, which just includes quoted identifiers in the word check now, but I don't think that is going to matter much after reading this again https://tree-sitter.github.io/tree-sitter/creating-parsers#keyword-extraction
| identifier: $ => { | ||
| const _identifier = /[\p{XID_Start}._][\p{XID_Continue}.]*/; | ||
| const _quoted_identifier = /`((?:\\(.|\n))|[^`\\])*`/; | ||
|
|
||
| return token( | ||
| choice( | ||
| _identifier, | ||
| _quoted_identifier | ||
| ) | ||
| ) | ||
| }, |
There was a problem hiding this comment.
I pulled the specifics of this idea from the swift grammar
https://github.com/tree-sitter/tree-sitter-swift/blob/db675450dcc1478ee128c96ecc61c13272431aab/grammar.js#L658-L668
To workaround a tree-sitter bug
DavisVaughan
force-pushed
the
fix/workaround-missing-node-issue
branch
from
5065d2e
to
a607767
Compare
Requires DavisVaughan/r-tree-sitter#7 for its updated print method used in snapshot tests
tree-sitter/tree-sitter#3332 prevents us from being able to use
ts_node_is_missing()in many places because ouridentifiernode has internal "hidden" rules. While we wait for this to be fixed, we can workaround this by inlining those hidden rules, being careful about the fact that we still need a "terminal" rule that can be used forword.I've changed from
_identifiertoidentifierfor use inword, but I don't think included quoted identifiers is going to make much of a difference for whatwordis used for (distinguishing keywords likeifandfunctionfrom identifiers)