You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the signature for the to_string method of the Codec traits returns a mapping of the each node to the character position in the returned string:
The Mapping allows us to associate cursor positions in the generated format (e.g. Markdown) with each node in the document. This in turn enables UX such as showing information related to a node or Ctrl+Enter to execute the node that the cursor is currently on.
The problem with this is that the generated Markdown is not necessarily exactly the same as the source Markdown (e.g. multiple blank lines between paragraphs are ignored). This requires us to update the editor with the generated Markdown (and the mapping associated with it), which in turn causes "jumpiness".
A better alternative would be to generate the mapping from the source by changing the signature of from_str to:
Having the mapping generated from the source Markdown would solve the jumpiness issue in the SourceView editor while still having the benefits of a position-to-node mapping. Perhaps more importantly, it would allow us to have a mapping for Markdown in non-Stencila editors, such as VSCode.
This has not been done yet because the MarkdownCodec is based on pulldown-cmark which does not retain source location of parsed nodes and looks like it is unlikely to in the near future.
Given that we need to make changes to the Markdown decoding anyway, I suggest switching to the markdown crate which has a schema similar to Stencila (in fact, several years ago, the mdast schema was part inspiration for many of the node types in Stencila Schema) and retains a Position for each parsed node, e.g. Paragraph.
Moving from pulldown-cmark to markdown will need a rewrite of the codec_markdown::decode module but the rewritten code is likely to be far simpler given the similarity of mdast schema to Stencila schema. nom-based parsers for extensions should be able to be reused.
The text was updated successfully, but these errors were encountered:
Currently the signature for the
to_string
method of theCodec
traits returns a mapping of the each node to the character position in the returned string:The
Mapping
allows us to associate cursor positions in the generated format (e.g. Markdown) with each node in the document. This in turn enables UX such as showing information related to a node orCtrl+Enter
to execute the node that the cursor is currently on.The problem with this is that the generated Markdown is not necessarily exactly the same as the source Markdown (e.g. multiple blank lines between paragraphs are ignored). This requires us to update the editor with the generated Markdown (and the mapping associated with it), which in turn causes "jumpiness".
A better alternative would be to generate the mapping from the source by changing the signature of
from_str
to:Having the mapping generated from the source Markdown would solve the jumpiness issue in the
SourceView
editor while still having the benefits of a position-to-node mapping. Perhaps more importantly, it would allow us to have a mapping for Markdown in non-Stencila editors, such as VSCode.This has not been done yet because the
MarkdownCodec
is based onpulldown-cmark
which does not retain source location of parsed nodes and looks like it is unlikely to in the near future.Given that we need to make changes to the Markdown decoding anyway, I suggest switching to the
markdown
crate which has a schema similar to Stencila (in fact, several years ago, themdast
schema was part inspiration for many of the node types in Stencila Schema) and retains aPosition
for each parsed node, e.g.Paragraph
.Moving from
pulldown-cmark
tomarkdown
will need a rewrite of thecodec_markdown::decode
module but the rewritten code is likely to be far simpler given the similarity ofmdast
schema to Stencila schema.nom
-based parsers for extensions should be able to be reused.The text was updated successfully, but these errors were encountered: