Skip to content

Commit

Permalink
fix: Allow CommonTokenStream to reset properly
Browse files Browse the repository at this point in the history
  - There is a flag in CommonTokenStream that indicates whether EOF has
    already been fetched/seen. This was not being reset when a new TokenSource
    was given to an existing CommonTokenStream and there was no Reset() function
    which was unfortunately not exported and not part of the interface for TokenStream.
  - Reset the flag when a new source is provided
  - Export the Reset() function so that a TokenStream can be rewound

Signed-off-by: Jim.Idle <jimi@idle.ws>
  • Loading branch information
jimidle authored and parrt committed Apr 23, 2023
1 parent 3aff09a commit 2406774
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 2 deletions.
5 changes: 4 additions & 1 deletion runtime/Go/antlr/v4/common_token_stream.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,9 @@ func (c *CommonTokenStream) Mark() int {

func (c *CommonTokenStream) Release(_ int) {}

func (c *CommonTokenStream) reset() {
func (c *CommonTokenStream) Reset() {
c.fetchedEOF = false
c.tokens = make([]Token, 0)
c.Seek(0)
}

Expand Down Expand Up @@ -196,6 +198,7 @@ func (c *CommonTokenStream) SetTokenSource(tokenSource TokenSource) {
c.tokenSource = tokenSource
c.tokens = make([]Token, 0)
c.index = -1
c.fetchedEOF = false
}

// NextTokenOnChannel returns the index of the next token on channel given a
Expand Down
1 change: 1 addition & 0 deletions runtime/Go/antlr/v4/token_stream.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ type TokenStream interface {
IntStream

LT(k int) Token
Reset()

Get(index int) Token
GetTokenSource() TokenSource
Expand Down
2 changes: 1 addition & 1 deletion runtime/Go/antlr/v4/tokenstream_rewriter.go
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ func (tsr *TokenStreamRewriter) RollbackDefault(instructionIndex int) {
tsr.Rollback(DefaultProgramName, instructionIndex)
}

// DeleteProgram reset the program so that no instructions exist
// DeleteProgram Reset the program so that no instructions exist
func (tsr *TokenStreamRewriter) DeleteProgram(programName string) {
tsr.Rollback(programName, MinTokenIndex) //TODO: double test on that cause lower bound is not included
}
Expand Down

0 comments on commit 2406774

Please sign in to comment.