Register Walktokens Behavior for extensions with different token structure

[calculuschild]

Describe the feature
In walkTokens(), traversing child tokens has to be handled differently for tables and lists, due to not holding a standard tokens object.

https://github.com/markedjs/marked/blob/6f4467ca9762f3a502c400ddc7687a27985f050d/src/Instance.ts#L44C7-L61

I have recently noticed that my extension marked-extended-tables fails to be walked correctly, since it has nearly the same structure as table but a different token type spanTable. I would like some way to register the expected walk path for extensions that do not have the standard tokens object.

Why is this feature necessary?
For compatibility with other extensions that use walkTokens(), because child tokens of marked-extended-table are not properly traversed.

Describe alternatives you've considered
Some kind of "on-the-fly" changing of the spanTable type back and forth between table and spanTable so it can still land in the correct spanTable Renderer, but this also fails due to slight differences in my token (spanTable supports multiple header rows for example).

[UziTech]

You can use childTokens to tell walkTokens which fields of the token should be walked if it is not just tokens

[calculuschild]

LMAO. Did I put that there way back in the original Extensions package at #2043? Wow. Yep, it was there from the start. It's been way too long. .....nevermind.....

[calculuschild]

Ah, wait. There is still the case of multiple rows in the table. I can list rows as a childToken, but rows is an array of arrays, not an array of tokens as walkTokens() expects.

We don't currently handle the case where you pass walkTokens an array of token arrays, right?

[UziTech]

Right. That should be easy to handle just flatten childTokens array.