Unify docs and embedded docs and/or define them alongside the functions

[josdejong]

Continuation from the discussion in #2047 (which became a bit off-topic)

In #2047 two ideas where brought up by @gwhitney:

1. It would be nice to have the embedded docs defined in or alongside the function themselves. Right now, the code and docs of a single function are spread in multiple places
2. It would be neat to have to define the documentation of a function only once. Right now it is defined twice: once in code comments on top of the function, which are used to auto generate the documentation on the website. And once in the embedded docs, which can be used via the expression parser like math.evaluate('help(sin)').

Some practical notes:

• The embedded docs show how to use the expression parser. The expression parser works differently than the low level JavaScript functions itself, like it uses 1-based indexes. See docs: https://mathjs.org/docs/expressions/syntax.html#differences-from-javascript
• The embedded docs are relatively large (in bytes). So we need to be careful not to add this to browser bundles when it is not needed. Right now that is possible because the help function with embedded docs is separated from the source code of the individual functions.

[gwhitney]

Basically the scheme I am thinking about is to extend docgenerator.js so that while it is at it, it writes a JSON file in the embedded docs format with everything it's extracted from the comments, placing it basically where the current embedded docs json files go. So running directly from source without having done the build, very few (if any) symbols would have embedded docs. But in the full bundle, the embedded docs would be there. And the tree-shaking would work just as it does now, because comments are not included in the bundles, and the generated JSON file would be accessed just the way the current hand-written ones are, i.e., it would not included in a bundle exactly as the current help is or is not.

In the generation of the embedded-docs JSON, the math. in the examples would be excised, and indices (where they can be automatically identified) would be increased by one, but I would also add a tag like Parser Examples to override the auto-translated Examples to take care of specific cases in which that auto-translation doesn't capture how the function works in the parser properly.

Does that sound plausible and worthwhile? I feel it would eliminate a significant amount of duplicated information. Let me know and I will take a stab at implementation, if it seems like a good direction to go.

[josdejong]

I agree with you that the best approach would be to use the jsdoc comments as the "single source of thuth", and from there, auto generate the docs for the web site and the embedded docs (using the docgenerator).

In the generation of the embedded-docs JSON, the math. in the examples would be excised, and indices (where they can be automatically identified) would be increased by one, but I would also add a tag like Parser Examples to override the auto-translated Examples to take care of specific cases in which that auto-translation doesn't capture how the function works in the parser properly.

Yes that makes a lot of sense. In like 80% of the cases, the examples for plain js vs expression parser are 1:1 the same except for stripping the math.* prefixes. For the other cases, it's probably easiest to fallback on a manually created section "parser examples" instead of trying to implement automatic conversion.

[gwhitney]

Great, I will give this a stab when I have a chance. Doing only the math. stripping and otherwise relying on handwritten Parser Examples tag will certainly ease the initial implementation and then we can see how many functions need e.g. index shifting.

[josdejong]

💪