Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Custom Tokenizer/Renderer extensions #2043

Merged
merged 49 commits into from Jun 15, 2021
Merged

feat: Custom Tokenizer/Renderer extensions #2043

merged 49 commits into from Jun 15, 2021

Commits on May 22, 2021

  1. Initial Commit

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    04fef1b View commit details
    Browse the repository at this point in the history

  2. Paragraph Tokenizer handles custom "start" points for extensions

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    108d4bc View commit details
    Browse the repository at this point in the history

  3. Move "start" logic from Paragraph tokenizer to Lexer

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    ee563e1 View commit details
    Browse the repository at this point in the history

  4. Clear extensions after each "Use Extension" unit test

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    a0798c3 View commit details
    Browse the repository at this point in the history

  5. Unit test: require blank lines for custom block tokens if not using "… 

    …start" property
    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    1308bc3 View commit details
    Browse the repository at this point in the history

  6. Handle inline tokens

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    803bbac View commit details
    Browse the repository at this point in the history

  7. Put extensions in map to prevent looping

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    94befe4 View commit details
    Browse the repository at this point in the history

  8. Remove 'overwrite' keyword / copied runTokenizerExtension between all… 

    … Tokenizers
    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    f4c2b44 View commit details
    Browse the repository at this point in the history

  9. Typo

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    e7da3a0 View commit details
    Browse the repository at this point in the history

  10. Another Typo

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    87ade7e View commit details
    Browse the repository at this point in the history

  11. Run all Tokenizer extensions at top of Lexer loop / remove "before"

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    244e07f View commit details
    Browse the repository at this point in the history

  12. Add unit test showing interacting block and inline extensions

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    b107826 View commit details
    Browse the repository at this point in the history

  13. Remove TokenParsed from Parser.js

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    47538e9 View commit details
    Browse the repository at this point in the history

  14. Better Paragraph interruption

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    31b7fa5 View commit details
    Browse the repository at this point in the history

  15. Lint

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    a03a64f View commit details
    Browse the repository at this point in the history

  16. Put renderers in its own property

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    16fc409 View commit details
    Browse the repository at this point in the history

  17. Unit test for custom renderer that returns false.

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    15be918 View commit details
    Browse the repository at this point in the history

  18. Make 'start' rules into functions.

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    0cb670e View commit details
    Browse the repository at this point in the history

  19. Extensions can use lexer and parser functions

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    6ec699e View commit details
    Browse the repository at this point in the history

  20. add walkableTokens property to extensions 

    function that will return an array of any child tokens that should be reachable by `walkTokens`, in case they are not all included in a `.tokens` property.
    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    cc23bc9 View commit details
    Browse the repository at this point in the history

  21. Add walkableTokens option for extensions.

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    375255e View commit details
    Browse the repository at this point in the history

  22. walkable Tokens requires options passed in

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    b602e71 View commit details
    Browse the repository at this point in the history

  23. Update src/marked.js 

    Co-authored-by: Tony Brix <tony@brix.ninja>
    @calculuschild @UziTech
    calculuschild and UziTech committed May 22, 2021
    Configuration menu
    Copy the full SHA
    9c9c557 View commit details
    Browse the repository at this point in the history

  24. Update src/marked.js 

    Co-authored-by: Tony Brix <tony@brix.ninja>
    @calculuschild @UziTech
    calculuschild and UziTech committed May 22, 2021
    Configuration menu
    Copy the full SHA
    eccaff4 View commit details
    Browse the repository at this point in the history

  25. lint

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    adace8c View commit details
    Browse the repository at this point in the history

  26. Update unit test functions to object method syntax

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    7fafa71 View commit details
    Browse the repository at this point in the history

  27. walktokens uses marked.defaults instead of opts parameter

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    e993545 View commit details
    Browse the repository at this point in the history

  28. Change hasExtensions to a boolean

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    95ceb59 View commit details
    Browse the repository at this point in the history

  29. Don't walk .tokens param if defined in the extension

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    70d3e2b View commit details
    Browse the repository at this point in the history

  30. Update Lexer.js

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    892ef65 View commit details
    Browse the repository at this point in the history

  31. add multiple extension test

    @UziTech @calculuschild
    UziTech authored and calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    fa7aaca View commit details
    Browse the repository at this point in the history

  32. Update src/marked.js 

    Co-authored-by: Tony Brix <tony@brix.ninja>
    @calculuschild @UziTech
    calculuschild and UziTech committed May 22, 2021
    Configuration menu
    Copy the full SHA
    ec85ef3 View commit details
    Browse the repository at this point in the history

  33. Fix multiple extensions overwriting each other

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    8a25d94 View commit details
    Browse the repository at this point in the history

  34. Rebase

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    d8bbb74 View commit details
    Browse the repository at this point in the history

  35. Update src/marked.js 

    Co-authored-by: Tony Brix <tony@brix.ninja>
    @calculuschild @UziTech
    calculuschild and UziTech committed May 22, 2021
    Configuration menu
    Copy the full SHA
    e6b4bbd View commit details
    Browse the repository at this point in the history

  36. Small fixes

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    d9ec74d View commit details
    Browse the repository at this point in the history

  37. Change walkableTokens to childTokens 

    * Also only walk `token.tokens` by default if `childTokens` is not provided, or it is specifically listed in `childTokens`
    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    caab77b View commit details
    Browse the repository at this point in the history

  38. Unit test that shows working "style tag" type syntax

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    ac35030 View commit details
    Browse the repository at this point in the history

  39. Last loaded extension gets priority, but will fall back to next exten… 

    …sion with same name if returns false
    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    eb0e790 View commit details
    Browse the repository at this point in the history

  40. Cleanup

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    ccb9869 View commit details
    Browse the repository at this point in the history

  41. Errors for missing name or invalid 'level'

    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    bbc8685 View commit details
    Browse the repository at this point in the history

  42. override original tokenizer/renderer with same name, but fall back if… 

    … returns false
    @calculuschild
    calculuschild committed May 22, 2021
    Configuration menu
    Copy the full SHA
    ecc47d4 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2021

  1. test fallback if returning false in extension

    @UziTech
    UziTech committed May 31, 2021
    Configuration menu
    Copy the full SHA
    c175894 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2021

  1. Update Docs, first pass

    @calculuschild
    calculuschild committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    697315a View commit details
    Browse the repository at this point in the history

  2. Formatting changes to docs

    @calculuschild
    calculuschild committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    dc828a0 View commit details
    Browse the repository at this point in the history

  3. Update docs/USING_PRO.md 

    Co-authored-by: Tony Brix <tony@brix.ninja>
    @calculuschild @UziTech
    calculuschild and UziTech committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    00a91cc View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2021

  1. Change marked.use to accept multiple parameters instead of an array

    @calculuschild
    calculuschild committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    9eb1aa8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'MarkedExtensions' of https://github.com/calculuschild/m… 

    …arked into MarkedExtensions
    @calculuschild
    calculuschild committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    4c9b38f View commit details
    Browse the repository at this point in the history

  3. update docs

    @calculuschild
    calculuschild committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    0664ddd View commit details
    Browse the repository at this point in the history