Create Markdown corpus readers

[elespike]

Hello,

I've written these markdown corpus readers to handle markdown documents, and figured I'd ask about including them in NLTK.

There are 4 requirements that would be new:

• markdown_it
• mdit_plain
• mdit_py_plugins
• yaml

Hopefully you'll find this useful!

[tomaarsen]

@elespike
#2903 has fixed a broken test case which snuck into this PR. I've merged develop into this PR to resolve this broken test.

[stevenbird]

@elespike – thanks for your contribution. Expanding NLTK functionality brings a maintenance cost, so there needs to be a clear case for adding code. Can you please make the case, e.g. by pointing us to markdown corpora used in NLP work? Thanks.

[elespike]

@stevenbird While I'm not familiar with and haven't found current NLP work using markdown corpora, the inspiration here is to be able to more easily perform NLP on knowledge bases built with markdown, either with:

• traditional text editors
• specialized applications such as https://obsidian.md/ or https://www.inkdrop.app/
• online/self-hosted platforms from which you can read/export markdown, such as https://hackmd.io/ and https://hedgedoc.org/, https://www.notion.so, https://dillinger.io/, or even entire GitHub Wikis.

These readers allow for easy extraction of quotes, code blocks, links, images, and sections, as well as everything that NLTK can do with plain text. =)

edit: forgot to mention https://stackedit.io/, and also Jupyter notebooks!

[stevenbird]

@elespike: thanks for your patience... this does look useful, thanks!