Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compute TokenList.value dynamically (v2) #710

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Commits on Mar 28, 2023

  1. Strip comments from a token list before sublists

    Avoid stripping T.Comment tokens contained within an sql.Comment before
    stripping the sql.Comment itself.  Now an sql.Comment token will be
    stripped first along with any contained T.Comment tokens.
    living180 committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    3d0b218 View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2023

  1. Restructure token class hierarchy.

    Rename Token to TokenBase and make it a superclass for TokenList and a
    new Token class.  Move some of the functionality of TokenBase into Token
    and TokenList.  This will make it easier to maintain separate
    functionality for Token versus TokenList.
    living180 committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    7de0ab1 View commit details
    Browse the repository at this point in the history
  2. Make TokenList.value a property not an attribute.

    The fact that a new value was being computed each time
    TokenList.group_tokens() was called caused supra-linear runtime when
    token grouping was enabled.
    
    Address by making TokenList.value a dynamically-computed property rather
    than a static attribute.
    living180 committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    ff4f391 View commit details
    Browse the repository at this point in the history