docsdev.md

Development Guidelines for Documentation

The pipeline documentation has been developed using the python package mkdocsand the material theme{:target="_blank"}. It is published as a static website using GitHub pages.

Documentation development server

!!! note Remember to install the development dependencies which include the modules required for the documentation.

This section describes how to set up a development server to live reload your changes to the pipeline documentation.

The main code of the documentation is located in the docs directory. In order to keep the repository, CHANGELOG.md, LICENSE.txt and CODE_OF_CONDUCT.md on the root path, relative soft links have been created under the docs folder:

adminusage
architecture
changelog.md -> ../CHANGELOG.md
code_of_conduct.md -> ../CODE_OF_CONDUCT.md
design
developing
exploringwebsite
faqs.md
gen_doc_stubs.py
gettingstarted
img
index.md
license.md -> ../LICENSE.txt
outputs
reference
theme
using

Start the development server:

(pipeline_env)$ mkdocs serve

Files in the docs directory are then 'watched' such that any changes will cause the mkdocs server to reload the new content.

The structure of the site (see nav section) and the settings are in the mkdocs.yml in the root of the repository.

Plugins and Customisations

mkdocs and the material theme have a lot of customisations and plugins available. The more involved plugins or customisations that are in use for this documentation are detailed in the following sections.

Custom theme directory

The theme directory contains various overrides to the standard material template. This is enabled by the following line in the mkdocs.yml file:

!!! example "mkdocs.yml" yaml theme: custom_dir: docs/theme

Custom javascript and css files are also stored in this directory and are enabled in the following mkdocs.yml file:

!!! example "mkdocs.yml" yaml extra_css: - theme/css/extra.css extra_javascript: - theme/js/extra.js - https://polyfill.io/v3/polyfill.min.js?features=es6 - https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js

The contents of the theme directory are shown below:

docs/theme
├── assets
│   └── stylesheets
│       └── overrides.css
├── css
│   ├── extra.css
│   └── lightgallery.min.css
├── fonts
│   ├── lg.svg
│   ├── lg.ttf
│   └── lg.woff
├── home.html
├── img
│   └── loading.gif
├── js
│   ├── extra.js
│   ├── lg-zoom.js
│   └── lightgallery.min.js
└── main.html

Full documentation on customising the mkdocs-material theme can be found in the documentation{:target="_blank"}.

main.html contains edits to the main.html as described in the documentation linked to above. The other files shown are used for the custom homepage and the Lightgallery.

Custom Homepage

The homepage is overwritten by a custom stylised HTML page. This is achieved using the following files:

docs
├── index.md
├── theme
│   ├── assets
│   │   └── stylesheets
│   │       └── overrides.css
│   ├── home.html
│   └── main.html

The index.md file in the main docs directory should only contain the following.

!!! example "index.md" ```md --- title: VAST Pipeline template: home.html ---

```

Creation and Last Updated Dates

Each page displays a date of creation and also a "last updated" date. This is done by using the plugin mkdocs-git-revision-date-localized-plugin{:target="_blank"}. The following options are used:

!!! example "mkdocs.yml" yaml plugins: - git-revision-date-localized: fallback_to_build_date: true enable_creation_date: true

!!! warning "Dates and Code Reference Pages" The option fallback_to_build_date: true is required for the code reference pages that are auto generated by the mkdocs-gen-files{:target="_blank"} plugin (see the Python Docstrings and Source Code section below). These pages will show the build date rather than the "last updated" date. During the build process the following warning will be seen, which is expected and ok to ignore: console WARNING - [git-revision-date-localized-plugin] Unable to find a git directory and/or git is not installed. Option 'fallback_to_build_date' set to 'true': Falling back to build date

Lightgallery

Images are displayed in the documentation using the lightgallery-markdown{:target="_blank"} plugin. As described in the repository linked to above, certain assets from lightgallery.js{:target="_blank"} are copied over to make the extension work, the files that apply here are:

docs/theme
├── css
│   └── lightgallery.min.css
├── fonts
│   ├── lg.svg
│   ├── lg.ttf
│   └── lg.woff
├── img
│   └── loading.gif
├── js
│   ├── extra.js
│   ├── lg-zoom.js
│   └── lightgallery.min.js
└── main.html

lg-zoom.js is also copied over to activate the zoom feature on the gallery images.

!!! note The javascript to select the lightgallery class objects and run them through the lightgallery function is placed in theme/js/extra.js such that the plugin works with instant navigation{:target="_blank"}.

MathJax

MathJax is enabled as described in the mkdocs-material documentation{:target="_blank"}.

Python Docstrings and Source Code

Python docstrings and source code are rendered using the mkdocstrings{:target="_blank"} plugin.

!!! warning "Docstrings Format" All docstrings must be done using the Google format{target="_blank"}.

The markdown files for the docstrings are automatically generated using mkdocs-gen-files{:target="_blank"} in the file docs/gen_doc_stubs.py. It is activated by the following lines in the mkdocs.yml file:

!!! example "mkdocs.yml" yaml hl_lines="3 4 5" plugins: - search - gen-files: scripts: - docs/gen_doc_stubs.py

The markdown files are programmatically generated, this means that the markdown files don't actually appear, but are virtually part of the site build. The files must still be referenced in the nav: section of mkdocs.yml, for example:

!!! example "mkdocs.yml" yaml - nav: - Code Reference: - vast_pipeline: - image: - main.py: reference/image/main.md - utils.py: reference/image/utils.md

The files are virtually created relative to the doc path that is stated in gen_doc_stubs.py. For example, using the vast_pipeline as the base for the python files and reference as the doc path, the markdown file for the docstrings in vast_pipeline/image/main.py is created at reference/image/main.md.

Please refer to the mkdocstrings documentation {:target="_blank"} for full details of the options available to declare in mkdocs.yml.

Versioning

Versioning is enabled on the documentation by using the mike{:target="_blank"} package. The documentation is published for each release version, along with a development version that is updated with every commit to the default dev branch. Setup of this has been completed on the initial deployment, and the GitHub workflows will automatically take care of the continual deployment. Vist the mike package page linked above for details on general management commands such as how to delete a version.

Deployment to GitHub pages

Automatic deployment to GitHub pages is set up using GitHub actions and workflows. See the workflows ci-docs-dev.yml and ci-docs-release.yml.