Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scanner: Find quoted token from context #327

Closed
wants to merge 5 commits into from

Commits on Nov 24, 2022

  1. scanner: Find quoted token from context

    To resolve an issue with quoted map keys not correctly parsing, this PR
    changes how it progresses after scanning a quote token.
    
    The old behaviour was:
    1. scanner reaches a :
    2. pull the characters out of the current scan context buffer and turn it into the map key token
    3. set the previous indent column to the beginning of the buffer
    So next time it gets to another key in the map, it'll check whether the indent was the same.
    
    When scanning a quote:
    1. Scan from the beginning to the end of the quote sequence
    2. Add a token, send back to the lexer, and reset the scan context
    In this scenario, it reaches the :, but nothing is in the scan context buffer because it just went ahead and added the quote token. The previous indent column doesn't get reset, so when it gets to the next map key it thinks the indent increased, which is invalid yaml.
    
    The new behaviour is essentially to not reset the context after scanning a quote, and when reaching a : check the context tokens, and if it's a quote set the previous indent column to that
    braydonk committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    690fbb5 View commit details
    Browse the repository at this point in the history
  2. scanner: resolve off-by-one scanning single quote

    When scanning an escaped single quote `''` there was an off by one error
    because while the local `idx` variable is increased, the context's `idx`
    is not. This leads to the progress returned being off by one, so the
    scanner gets stuck in the `ctx.next()` loop forever, unable to progress.
    braydonk committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    b84d0bd View commit details
    Browse the repository at this point in the history
  3. scanner: apply column skip in quote scanning

    When we scan escape characters in quoted scalars, the local idx is
    shifted but not the underlying context. This adds a column skip to
    ensure the ctx progresses in sync with the local quote scanning.
    braydonk committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    9de8125 View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2022

  1. scanner: fix buf and obuf sync issue

    `scanQuote` will add to the origin buffer for the purpose of creating
    the quote token, but would never reset it. This caused a problem when
    nothing would reset the buffer, i.e. when a quoted value was followed by
    a newline and the next map key.
    braydonk committed Nov 25, 2022
    Configuration menu
    Copy the full SHA
    8834a3c View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2022

  1. add tests, fixed bad scanner logic

    I had adjusted the `:` scanning logic in a way that didn't cover all
    cases.
    braydonk committed Nov 27, 2022
    Configuration menu
    Copy the full SHA
    d56f06e View commit details
    Browse the repository at this point in the history