Modernised nltk.org website

[tomaarsen]

Hello!

Pull request overview

• Modernised the nltk.org website by modifying the Sphinx theme to nltk_theme, which is based on @autophagy's Insegel.
• Started using better_apidoc instead of sphinxcontrib.apidoc, as the former allows templates, which proved quite useful.
• Updated RELEASE-HOWTO to reflect the new documentation build flow.
• Moved the team information from contribute.rst into a separate file called team.rst.
• Renamed items on the Table of Contents menu.

Example page on the new website's documentation

The core

First things first, you can check out the new website, as generated with code from this PR, on the GitHub Pages of my nltk.github.com fork: https://tomaarsen.github.io/nltk.github.com/

This website was generated by running the following command on the root directory of nltk:

sphinx-build -E ./web ./build

This command uses Sphinx (version 4.2.0) to look in the web folder for a config and more, while outputting the build in a build folder. This build folder was then copied to the modernised branch of https://github.com/nltk/nltk.github.com. Note that in order to be able to execute this command, a new dependency of nltk_theme must be downloaded:

pip install -U nltk_theme

See https://github.com/tomaarsen/nltk_theme for more information on the newly made theme.

Something to keep in mind is that the website links to the source code at a particular tag in the footer. With other words, if the website is built for e.g. NLTK 3.6.5 (which doesn't exist yet), and there's no tag with that name, then this link will 404. So, when a new website is being built, ensure that there is also a tag for that version.

---

Further changes

Using better_apidoc instead of sphinxcontrib.apidoc

The generated Table of Content trees would have a redundant layer in their tree called "Submodules" or "Subpackages", which only adds extra confusion to the website. Something similar can be seen in this SO: https://stackoverflow.com/questions/29385564/customize-templates-for-sphinx-apidoc.

As a result, I switched to using better_apidoc. I made simple templates for modules and packages where the "Submodules" and "Subpackages" will no longer be included in TOC trees.

---

The remainder of the further changes are smaller, and mostly speak for themselves in the PR overview.

---

Feedback

At this point, I'm most interested in how users perceive the usability of the website, and how easy they consider it to find what they're after in the documentation. If you have any comments on that, I'd love to hear them.
Beyond that, I welcome any feedback on the fonts, page title, structure, layout, colors, content, the footer content, anything.

Future Changes

This PR is by no means a complete overhaul of the website, but we are due one! There are plenty of small tasks left to work on. I've enumerated some below:

• Building the website's documentation generates a whopping 520 warnings. Some sample warnings are:

  WARNING: Unexpected indentation.
  WARNING: Block quote ends without a blank line; unexpected unindent.
  WARNING: Inline literal start-string without end-string.
  WARNING: Field list ends without a blank line; unexpected unindent.
  WARNING: Undefined substitution referenced: "s_1".
  WARNING: Unexpected section title.
  

  In short, most of these can be fixed quite quickly. It's simply a matter of converting the docstrings all over the codebase to exactly what Sphinx expects.
• The HOWTO page (now under "Example Usage") is still pretty much just a blank page. This section of the website can definitely still use some TLC.
• Most of the website pages can be updated or rewritten. For example, in index.html we can definitely list more interesting uses of NLTK, and we should definitely create links from these examples to the concrete documentation of those classes/functions.
• news.rst ought to be updated. Instead of having two spaces at the start of each update, we should have a * . That way, we get a nice bulleted list in news.html.

What now?

Alongside this PR I've also created nltk/nltk.github.com#18, within which I've pushed the website built for NLTK version 3.6.3. I've mostly done this so I can use the GitHub pages from my own fork of nltk.github.com to launch the website here. I will create a PR there too.
However, it seems smart to first merge this and release NLTK 3.6.5. That way, we can use the develop branch to generate the website for the new version 3.6.5, without having to do something complicated like checking out tag 3.6.3, and then copying in the web folder from this PR, and then generating the website.

---

I'd love to hear your thoughts on this.

• Tom Aarsen

[stevenbird]

@tomaarsen: wonderful! I say we go with this and continue to make incremental changes. I like your suggestion to merge this, release 3.6.5, and build the updated docs from here.

[stevenbird]

@dannysepler thanks for your glowing review.