Convert Documentation from reST to Markdown

[mj023]

As described in #112 I started converting all .rst-files to .md-files. The docs will use MyST-Markdown, because it retains most of the reST functionality while being compatible with ReadTheDocs. The README will use github-flavored-markdown (gfm) as GitHub can't render some of the MyST Syntax.

• Convert all documentation files to MyST
• Convert the Readme.rst and Changes.rst to gfm

[codecov]

Codecov Report

Merging #117 (a45aa7f) into main (0277f2d) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main      #117   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            1         1           
  Lines           55        55           
=========================================
  Hits            55        55           

Flag	Coverage Δ
end_to_end	100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[mj023]

The pre-commit bot tries to use mdformat to correct the Markdown format. Because he does not know any of the extensions of MyST, this leads to him for example escaping a bunch of directives that use colon fencing.
Should I try to correct the Markdown Files and avoid using extensions or should we try to modify mdformat or stop the bot from applying it to the docs?

[hmgaudecker]

Thanks! Modifying the behavior of mdformat seems like the way to go, see e.g.

https://github.com/executablebooks/mdformat-myst/blob/master/.pre-commit-config.yaml

[mj023]

It seems to me, that there is no way to modify mdformat to include MyST extensions, without writing the extensions for mdformat ourselves. See this Issue executablebooks/mdformat-myst/issues/5, where they talk about writing an extension to include colon fences, which sadly did not happen yet.

I guess we should stop the bot from reformatting the docs, at least until mdformat is updated to include extensions?

[hmgaudecker]

Cool, thanks for the investigation!

Do we strictly need those colon fences or are they more a convenience making the automated conversion simpler?

If the former, it might well be worth investigating whether we could help mdformat-myst, in case you'd enjoy that!

[mj023]

We don't really need any extensions right now, they are mostly for convenience or to make MyST look more markdownish. I will just change some of the md-files and remove the extensions that are not supported from the conf.py to prevent confusion for future contributors.

[hmgaudecker]

Suggested change

	myst_enable_extensions = ["dollarmath"]
	myst_enable_extensions = ["amsmath"]

While browsing things earlier today, I noted that dollarmath might be gone from MyST in the future executablebooks/MyST-Parser#505. Makes sense because for parsers, you definitely want to differentiate between start/end sequences delimiting environments. So we'll want to be future-proof here :-)

Reference on AMSmath: https://myst-parser.readthedocs.io/en/latest/syntax/optional.html#syntax-amsmath

[mj023]

We would probably have to remove it completely then and only use the math-directive for equations. Because dollarmath is the only extension that works with mdformat (for legacy reasons, might also be changed in the future).

[hmgaudecker]

Yes, let's remove it then. Thanks!

[timmens]

Guide for translation of other projects

Thanks to @mj023!

Translation of files that are rendered using sphinx

This should include all files in the docs folder.

1. Install rst-to-myst. On windows make sure that click >= 8.1.3.
2. Use the command line interface of rst-to-myst to run a basic conversion:

   $ cd /into/project/root
   $ rst2myst convert docs/**/*.rst
   

   CLI options can be found here.
3. Compare results and fix remaining problems manually.

Known Problems:

• Include blocks are not translated
  • use eval-rst instead of include directive; in eval-rst the included file will be evaluated as rst, even if its md
• Include blocks will still have links to .rst-files; need to be replaced with the new md.-files.
• Heading levels need to be changed according to where a file is included.
• Inline link targets: '_' becomes '-' only in the targets, not the links.
• MdFormat compatibility problems:
  • only recognizes base MyST-Syntax while rst-to-myst sometimes uses colon fences etc. (maybe solvable by using --no-colon-fences Option)

Translation of files that are rendered on GitHub

This should include, for example, the README.md.

1. Install pandoc.
2. Use pandoc's command line interface to translate the specific files:

   $ pandoc file.rst -f rst -t gfm -o file.md

3. Compare results and fix remaining problems manually.

Known Problems:

• GitHub Badges will be displayed stacked on top of each other