private __emitTokens API

[joshgoebel]

Needed for #3619. Plan is to release this with 10.7.

Changes

• Allows grammars to provide their own parsers that output to the private HLJS Emitter API
• A grammar may supply a __emitTokens(code, emitter) function as part of the language data it returns when called

Checklist

• Add unit tests for __emitTokens
• Updated the changelog at CHANGES.md

[joshgoebel]

@wooorm Relevant to you because once this goes public the Emitter API will become part of our public API since custom grammars will need to trust this API to be stable. No rush, but it'll happen eventually I think. I'd love to see what people could do with this.

[joshgoebel]

@allejo When you have a moment.