DOC: major documentation rewrite

Closes mesonbuild#197, mesonbuild#230, mesonbuild#253, and mesonbuild#264
Sets us up to fix mesonbuild#138, mesonbuild#233, and mesonbuild#224

Signed-off-by: Filipe Laíns <lains@riseup.net>

README.md

@@ -11,29 +11,26 @@ SPDX-License-Identifier: MIT
 [![codecov](https://codecov.io/gh/mesonbuild/meson-python/branch/main/graph/badge.svg?token=xcb2u2YvVk)](https://codecov.io/gh/mesonbuild/meson-python)
 [![Documentation Status](https://readthedocs.org/projects/meson-python/badge/?version=stable)](https://meson-python.readthedocs.io/en/stable/?badge=stable)
 
-Python build backend ([PEP 517](https://www.python.org/dev/peps/pep-0517/)) for Meson projects.
-
-See the [documentation](https://meson-python.readthedocs.io/en/stable/) for more details.
-
-### Status
-
-- Pure Python modules :+1:
-- Native modules
-  - Don't link against anything :+1:
-  - Link against external libraries :+1:
-  - Link against libraries from the Meson project :+1:
-  - Detect the ABI :+1:
-- Scripts (executables in Meson)
-  - Don't link against anything :+1:
-  - Link against external libraries :+1:
-  - Link against libraries from the Meson project :hammer:
-
-#### Platform Support
-
-- Linux :+1:
-- Windows :hammer:
-  - Does not support linking against libraries from the Meson project
-- MacOS :hammer:
-  - Does not support linking against libraries from the Meson project
-- Other UNIX-like :warning:
-  - Most platforms should work, but currently that is not tested or guaranteed
+`meson-python` is a Python build backend built on top of the [Meson][meson]
+build-system. It enables you to use [Meson][meson] for your Python packages.
+
+See our [documentation][docs] for more details.
+
+
+## Support
+
+If you have a general question feel free to [start a discussion][new discussion]
+on Github. If you want to report a bug, request a feature, or propose an improvement, feel
+free to open an issue on our [bugtracker][bugtracker].
+
+## Contributing
+
+If you are interested in contributing, please check out
+[our documentation][contributing docs].
+
+
+[meson]: https://github.com/mesonbuild/meson
+[docs]: https://meson-python.readthedocs.io/en/stable/
+[new discussion]: https://github.com/mesonbuild/meson-python/discussions/new/choose
+[bugtracker]: https://github.com/mesonbuild/meson-python/issues
+[contributing docs]: https://meson-python.readthedocs.io/en/lastest/contributing/getting-started.html

docs/conf.py

@@ -37,7 +37,10 @@
 extensions = [
     'sphinx.ext.autodoc',
     'sphinx.ext.intersphinx',
+    'sphinx.ext.todo',
     'sphinx_autodoc_typehints',
+    'sphinx_copybutton',
+    'sphinx_design',
 ]
 
 intersphinx_mapping = {
@@ -55,6 +58,8 @@
 
 default_role = 'any'
 
+todo_include_todos = True
+
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
@@ -63,6 +68,15 @@
 html_theme = 'furo'
 html_title = f'meson-python {version}'
 
+html_theme_options = {
+    'light_css_variables': {
+        'font-stack': (
+            'system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,'
+            'Helvetica,Arial,sans-serif,Apple Color Emoji,Segoe UI Emoji'
+        ),
+    },
+}
+
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named 'default.css' will overwrite the builtin 'default.css'.

docs/contributing/commit-format.rst

@@ -0,0 +1,18 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _contributing-commit-format:
+
+*************
+Commit format
+*************
+
+
+.. todo::
+
+   - Explain is the same as Numpy and SciPy
+   - List valid prefixes
+   - Speak a bit about message clarity
+   - Speak a bit about splitting commits into logical changes
+   - Mention we require a linear history

docs/contributing/documentation.rst

@@ -0,0 +1,36 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _contributing-documentation:
+
+*************
+Documentation
+*************
+
+Our documentation is built on sphinx_, is written using reStructuredText_, and
+follows the Diátaxis_ framework.
+
+To automatically re-build the documentation and serve it on an web server, you
+can pass the ``serve`` argument to the ``docs`` nox_ task.
+
+
+.. code-block:: console
+
+   $ nox -s docs -- serve
+
+
+When using this argument, the task will watch the documentation source files,
+and every time you edit something, it will automatically re-build the
+documentation and make it available on the provided web server.
+
+
+.. todo::
+
+   Elaborate more with tips on how to write.
+
+
+.. _sphinx: https://www.sphinx-doc.org
+.. _reStructuredText: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
+.. _Diátaxis: https://diataxis.fr/
+.. _nox: https://github.com/wntrblm/nox

docs/contributing/getting-started.rst

@@ -0,0 +1,15 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _contributing-getting-started:
+
+***************
+Getting started
+***************
+
+
+.. todo::
+
+   - Introduction to the development workflow.
+   - ?

docs/contributing/test-suite.rst

@@ -0,0 +1,16 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _contributing-test-suite:
+
+**********
+Test suite
+**********
+
+
+.. todo::
+
+   - Introduce nox
+   - Explain our most relevant pytest fixtures (``package_*``, ``sdist_*``,
+     ``wheel_*``, ``editable_*``, ``venv``, etc.)

docs/explanations/design-old.rst

@@ -0,0 +1,86 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+:orphan:
+
+.. _explanations-design-old:
+
+*******************************
+How does ``meson-python`` work?
+*******************************
+
+
+.. admonition:: Old documentation
+   :class: attention
+
+   You are looking at the old documentation. We are currently in the process of
+   refactoring the whole documentation and this page was part of the old
+   version. While it still contains lots of relevant information, it is not
+   necessarily in the format we ultimately want to present, and is not well
+   integrated with the other parts of the documentation.
+
+
+
+How does it work?
+=================
+
+We implement the Python build system hooks, enabling any standards compliant
+Python tool (pip_, `pypa/build`_, etc.) to build and install the project.
+
+``meson-python`` will build a Python sdist (source distribution) or
+wheel (binary distribution) from Meson_ project.
+
+Source distribution (sdist)
+---------------------------
+
+The source distribution is based on ``meson dist``, so make sure all your files
+are included there. In git projects, Meson_ will not include files that are not
+checked into git, keep that in mind when developing. By default, all files
+under version control will be included in the sdist. In order to exclude files,
+use ``export-ignore`` or ``export-subst`` attributes in ``.gitattributes`` (see
+the ``git-archive`` documentation for details; ``meson dist`` uses
+``git-archive`` under the hood).
+
+Local (uncommitted) changes to files that are under version control will be
+included. This is often needed when applying patches, e.g. for build issues
+found during packaging, to work around test failures, to amend the license for
+vendored components in wheels, etc.
+
+Binary distribution (wheels)
+----------------------------
+
+The binary distribution is built by running Meson_ and introspecting the build
+and trying to map the files to their correct location. Due to some limitations
+and/or bugs in Meson_, we might not be able to map some of the files with
+certainty. In these cases, we will take the safest approach and warn the user.
+In some cases, we might need to resort to heuristics to perform the mapping,
+similarly, the user will be warned. These warnings are not necessarily a reason
+for concern, they are there to help identifying issues. In these cases, we
+recommend that you check if the files in question were indeed mapped to the
+expected place, and open a bug report if they weren't.
+
+If the project itself includes shared libraries that are needed by other files,
+those libraries will be included in the wheel, and native files will have their
+library search path extended to include the directory where the libraries are
+placed.
+
+If the project depends on external shared libraries, those libraries will not
+be included in the wheel. This can be handled in several ways by the package
+developer using ``meson-python``:
+
+1. Finding or creating a suitable Python package that provides those shared
+   libraries and can be added to ``dependencies`` in ``pyproject.toml``.
+2. Vendoring those libraries into the wheel. Currently ``meson-python`` does
+   not provide an OS-agnostic way of doing that; on Linux ``auditwheel`` can be
+   used, and on macOS ``delocate``. Or, the package developer can copy the
+   shared libraries into the source tree and patch ``meson.build`` files to
+   include them.
+3. Documenting that the dependency is assumed to already be installed on the
+   end user's system, and letting the end user install it themselves (e.g.,
+   through their Linux distro package manager).
+
+
+.. _pip: https://github.com/pypa/pip
+.. _pypa/build: https://github.com/pypa/build
+.. _Meson: https://github.com/mesonbuild/meson

docs/explanations/design.rst

@@ -0,0 +1,26 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _explanations-design:
+
+*******************************
+How does ``meson-python`` work?
+*******************************
+
+
+.. todo::
+
+   - Explain how ``meson-python`` maps files to the wheel
+
+     - Link to :ref:`tutorials-executable` in the relevant section
+
+   - Explain why the heuristics are necessary and how they work
+   - Explain why we don't support ``install_data``
+
+     - Link to the relevant :ref:`reference-limitations` section
+
+   - ?
+
+
+You can find the old documentation :ref:`here <explanations-design-old>`.

docs/explanations/editable-installs.rst

@@ -0,0 +1,19 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _explanations-editable-installs:
+
+******************************
+How do editable installs work?
+******************************
+
+
+.. todo::
+
+   - Explain how the custom finder works
+   - Explain what is the purpose of the ``_MESONPY_EDITABLE_SKIP`` environment
+     variable
+
+     - Warn that the approach described is purely informational, and should
+       not be relied on

docs/explanations/internal-dependencies.rst

@@ -0,0 +1,17 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _explanations-internal-dependencies:
+
+***************************************
+How do we handle internal dependencies?
+***************************************
+
+
+.. todo::
+
+   - Explain why bundle internal dependencies
+   - Explain how we bundle internal dependencies
+
+     - Explain how we patch the ``RPATH``

docs/how-to-guides/add-dependencies.rst

@@ -0,0 +1,17 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _how-to-guides-add-dependencies:
+
+*******************
+Adding dependencies
+*******************
+
+
+.. todo::
+
+   - Mention how to add dependencies
+   - Mention how to add optional dependencies
+   - Mention how to add build dependencies
+   - Give a quick introduction to the Python dependency specification format

docs/how-to-guides/build-directory.rst

@@ -0,0 +1,18 @@
+.. SPDX-FileCopyrightText: 2023 The meson-python developers
+..
+.. SPDX-License-Identifier: MIT
+
+.. _how-to-guides-build-directory:
+
+*******************************
+Use a permanent build directory
+*******************************
+
+
+.. todo::
+
+   - Explain why you want to use the ``builddir`` option
+   - Explain how ``builddir`` works
+   - Warn that user-configured build directories are not "use at your own risk"
+   - Warn that ``meson-python`` version mismatches in the build directory can
+     cause trouble