Merge branch 'master' into python-3-10

.github/workflows/CI.yml

@@ -28,7 +28,8 @@ jobs:
       - name: Install package
         run: |
           python -m pip install --upgrade pip
-          pip install .[tests]
+          pip install -r requirements/tests
+          pip install .
       - name: Run test suite
         run: pytest
 
@@ -44,7 +45,8 @@ jobs:
       - name: Install package
         run: |
           python -m pip install --upgrade pip
-          pip install .[dev]
+          pip install -r requirements/dev
+          pip install .
       - name: Run tox environments
         run: tox
       - name: Build documentation

.readthedocs.yaml

@@ -14,6 +14,4 @@ python:
     - requirements: docs/requirements.txt
     - method: pip
       path: .
-      extra_requirements:
-        - docs
   system_packages: false

contributing.rst

@@ -24,9 +24,11 @@ Using a virtual environment of your choice for the installation is recommended.
 
     $ git clone https://github.com/felix-hilden/sphinx-codeautolink.git
     $ cd sphinx-codeautolink
-    $ pip install -e .[dev]
+    $ pip install -e .
+    $ pip install -r requirements/dev
 
-The last command installs all the necessary extra dependencies for development.
+The last command installs all the necessary tools for development
+as well as all optional dependencies.
 
 If you forked, consider adding the upstream repository as a remote to easily
 update your main branch with the latest upstream changes.

docs/requirements.txt

@@ -1,4 +1,6 @@
+# Manually pin Sphinx + docs requirements
 sphinx==4.2.0
 sphinx-rtd-theme==1.0.0
 docutils==0.16
 matplotlib==3.4.3
+ipython==7.30.1

docs/src/about.rst

@@ -47,14 +47,28 @@ Caveats
   on the expected failures, see our `test suite on GitHub <https://github.com
   /felix-hilden/sphinx-codeautolink>`_. Please report any unexpected failures!
 
-Clean Sphinx build
-------------------
+Sphinx semantics
+----------------
+Clean build
+***********
 For correct partial builds, code reference information is saved to a file
 which is updated when parsing new or outdated files.
 It shouldn't become outdated, but a clean build can be achieved with
 `sphinx-build -E <https://www.sphinx-doc.org/en/master/man/sphinx-build.html
 #cmdoption-sphinx-build-E>`_ or by deleting the build directory.
 
+Sphinx cache
+************
+A function specified in :confval:`codeautolink_custom_blocks` prevents Sphinx
+from caching documentation results. Consider using an importable instead.
+For more information, see the discussion in :issue:`76`.
+
+Parallel build and custom parsers
+*********************************
+Locally defined custom block parsers in :confval:`codeautolink_custom_blocks`
+cannot be passed to Pickle, which prevents parallel Sphinx builds.
+Please consider using an importable function instead.
+
 Copying code blocks
 -------------------
 If you feel like code links make copying code a bit more difficult,
@@ -72,15 +86,15 @@ a bug, which you can report on `GitHub
 If third-party code blocks are in use, matching may fail because of
 inconsistent or unrecognised CSS classes. The class related to the block lexer
 name is automatically added to the list of CSS classes that are searched when
-matching code examples as ``highlight-{lexer} notranslate``.
+matching code examples as ``highlight-{lexer}``.
 If the class has another value, :confval:`codeautolink_search_css_classes`
 can be used to extend the search. To find out which classes should be added,
 build your documentation, locate the code example and use the class of the
 outermost ``div`` tag. For example:
 
 .. code:: python
 
-   codeautolink_search_css_classes = ["highlight-default notranslate"]
+   codeautolink_search_css_classes = ["highlight-default"]
 
 Secondly, matching can fail on a specific line or range of lines.
 This is often a bug, but the known expected failure cases are presented here:

docs/src/conf.py

@@ -41,6 +41,7 @@
 }
 autodoc_typehints = "description"
 intersphinx_mapping = {
+    "python": ('https://docs.python.org/3/', None),
     "numpy": ("https://numpy.org/doc/stable/", None),
     "matplotlib": ("https://matplotlib.org/stable/", None),
 }

docs/src/examples.rst

@@ -158,11 +158,18 @@ An object type "class" seems to work for other types as well.
 
 Intersphinx integration
 -----------------------
-When writing documentation that references other libraries, `intersphinx
+When writing code examples that use builtins or other libraries, `intersphinx
 <https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html>`_
-is a great extension to use. It enables links to documentation on other sites.
-sphinx-codeautolink integrates this functionality seamlessly,
-linking objects as long as the correct ``intersphinx_mapping`` is defined.
+can be used to enable links to documentation on other Sphinx-generated sites.
+Intersphinx is integrated seamlessly, linking objects as long as the correct
+``intersphinx_mapping`` is defined.
+
+.. code:: python
+
+   if __debug__:
+       print(...)
+   else:
+       raise RuntimeError(f"Could not debug!")
 
 .. code:: python
 
@@ -195,13 +202,14 @@ is provided as a ready-made transformer.
 
 .. code:: python
 
-   from sphinx_codeautolink import clean_pycon
-
    extensions = [
        ...,
        "sphinx.ext.doctest",
    ]
-   codeautolink_custom_blocks = {"python3": None, "pycon3": clean_pycon}
+   codeautolink_custom_blocks = {
+       "python3": None,
+       "pycon3": "sphinx_codeautolink.clean_pycon",
+   }
 
 ``doctest`` and ``testcode`` blocks now work as expected.
 However, any test setup and teardown code is not taken into account.
@@ -211,6 +219,33 @@ However, any test setup and teardown code is not taken into account.
    >>> import lib
    >>> lib.Knight()
 
+IPython code blocks
+-------------------
+Code blocks using the language setting ``ipython3`` are supported by default.
+The function :func:`~sphinx_codeautolink.clean_ipython` is used to handle
+IPython-specific syntax like so-called `magic functions`_.
+
+.. _magic functions: https://ipython.readthedocs.io/en/stable/
+   interactive/tutorial.html#magic-functions
+
+.. code:: ipython3
+
+   %reset
+   import lib
+   lib.Knight().taunt()
+
+This is useful for integrating Jupyter notebooks, which is possible with
+separate Sphinx extensions like nbsphinx_ or MyST-NB_.
+IPython processing is enabled if the ``ipython`` library is installed.
+It is also included in the ``ipython`` extra of sphinx-codeautolink.
+
+.. code:: sh
+
+   pip install sphinx-codeautolink[ipython]
+
+.. _nbsphinx: https://nbsphinx.readthedocs.io/
+.. _MyST-NB: https://myst-nb.readthedocs.io/
+
 Third-party code blocks
 -----------------------
 Third-party code blocks that use the basic Pygments lexers for Python

docs/src/index.rst

@@ -2,11 +2,10 @@ sphinx-codeautolink
 ===================
 |pypi| |license|
 
-Automatic links from Python code examples to reference documentation
-at the flick of a switch!
-sphinx-codeautolink analyses the code in your documentation
-and inserts links to definitions that you use.
-Click any names in the code example below.
+sphinx-codeautolink makes code examples clickable by inserting links
+from individual code elements to the corresponding reference documentation.
+We aim for a minimal setup assuming your examples are already valid Python.
+Click any names in the example below for a demonstration:
 
 .. code:: python
 

docs/src/reference.rst

@@ -31,11 +31,12 @@ Configuration
 
 .. confval:: codeautolink_custom_blocks
 
-   Type: ``Dict[str, None | Callable[[str], Tuple[str, str]]]``.
+   Type: ``Dict[str, None | str | Callable[[str], Tuple[str, str]]]``.
    Register custom parsers for lexers of unknown types of code blocks.
    They are registered as a dict mapping a block lexer name to a function
    possibly cleaning up the block content to valid Python syntax.
    If none is specified, no transformations are applied.
+   A string is interpreted as an importable transformer function.
    The transformer must return two strings: the code appearing in documentation
    (often just the original source) and the cleaned Python source code.
    The transformer must preserve line numbers for correct matching.
@@ -110,9 +111,23 @@ CSS class
 ---------
 The CSS class used in all code block links is ``sphinx-codeautolink-a``.
 
-Cleanup function
-----------------
-The function below is usable for cleaning ``pycon`` code blocks.
-It is intended to be used with :confval:`codeautolink_custom_blocks`.
+Cleanup functions
+-----------------
+The functions below are usable for cleaning
+``pycon`` and ``ipython`` code blocks.
+They are intended to be used with :confval:`codeautolink_custom_blocks`.
 
 .. autofunction:: sphinx_codeautolink.clean_pycon
+
+.. autofunction:: sphinx_codeautolink.clean_ipython
+
+Warning types
+-------------
+Sphinx logging machinery is used to issue warnings during documentation builds.
+All warning subtypes below are in the ``codeautolink.*`` namespace
+and can be ignored with configuring ``suppress_warnings``.
+
+- ``user_error``: issued when a directive is used incorrectly
+- ``parsing_error``: issued when parsing a code block fails
+- ``match_block``: issued when a block cannot be matched
+- ``match_name``: issued when a code snippet cannot be matched

docs/src/release_notes.rst

@@ -11,7 +11,19 @@ sphinx-codeautolink adheres to
 Unreleased
 ----------
 - Support Python 3.10 (:issue:`33`)
+- Use Sphinx logging instead of raising exceptions (:issue:`86`)
+- Link builtins if visible to intersphinx (:issue:`87`)
+- Use Sphinx logging instead of the builtin ``warnings`` to warn (:issue:`89`)
+
+0.8.0 (2021-12-16)
+------------------
 - Correctly test for optional types in annotations (:issue:`72`)
+- Don't check for ``notranslate`` CSS class, allowing for additional classes
+  (:issue:`75`)
+- Allow to specify block parsers as importable references (:issue:`76`)
+- Allow parallel builds (:issue:`77`)
+- Automatic support for ``ipython3`` code blocks (:issue:`79`)
+- Correctly produce links for ``py`` code blocks (:issue:`81`)
 
 0.7.0 (2021-11-28)
 ------------------

readme.rst

@@ -2,10 +2,9 @@ sphinx-codeautolink
 ===================
 |pypi| |license| |readthedocs| |build|
 
-Automatic links from Python code examples to reference documentation
-at the flick of a switch!
-sphinx-codeautolink analyses the code in your documentation
-and inserts links to definitions that you use.
+sphinx-codeautolink makes code examples clickable by inserting links
+from individual code elements to the corresponding reference documentation.
+We aim for a minimal setup assuming your examples are already valid Python.
 
 For a live demo, see our online documentation on
 `Read The Docs <https://sphinx-codeautolink.rtfd.org>`_.

readme_pypi.rst

@@ -2,10 +2,9 @@ sphinx-codeautolink
 ===================
 |pyversions| |downloads| |license| |readthedocs|
 
-Automatic links from Python code examples to reference documentation
-at the flick of a switch!
-sphinx-codeautolink analyses the code in your documentation
-and inserts links to definitions that you use.
+sphinx-codeautolink makes code examples clickable by inserting links
+from individual code elements to the corresponding reference documentation.
+We aim for a minimal setup assuming your examples are already valid Python.
 
 For a live demo, see our online documentation on
 `Read The Docs <https://sphinx-codeautolink.rtfd.org>`_.

requirements/dev

@@ -0,0 +1,15 @@
+-r extras
+-r docs
+-r tests
+
+# Tests / checks tooling
+tox
+doc8>=0.9
+flake8
+flake8-bugbear
+pydocstyle[toml]>=6.1
+pygments
+
+# Build / publishing
+build
+twine

requirements/docs

@@ -0,0 +1,4 @@
+-r extras
+
+sphinx-rtd-theme
+matplotlib

requirements/extras

@@ -0,0 +1 @@
+ipython

requirements/tests

@@ -0,0 +1,4 @@
+-r extras
+
+pytest>=6
+coverage[toml]>=5

setup.py

@@ -6,30 +6,6 @@
 version_file = root / "src" / "sphinx_codeautolink" / "VERSION"
 readme_file = root / "readme_pypi.rst"
 
-extras_require = {
-    "build": [
-        "build",
-        "twine",
-    ],
-    "docs": [
-        "sphinx-rtd-theme",
-        "matplotlib"
-    ],
-    "tests": [
-        "pytest>=6",
-        "coverage[toml]>=5",
-    ],
-    "checks": [
-        "tox",
-        "doc8>=0.9",
-        "flake8",
-        "flake8-bugbear",
-        "pydocstyle[toml]>=6.1",
-        "pygments",
-    ],
-}
-extras_require["dev"] = sum(extras_require.values(), [])
-
 setuptools.setup(
     name="sphinx-codeautolink",
     version=version_file.read_text().strip(),
@@ -62,7 +38,10 @@
         'beautifulsoup4',
         'dataclasses;python_version<"3.7"',
     ],
-    extras_require=extras_require,
+    # Keep extras in sync with requirements manually
+    extras_require={
+        "ipython": ["ipython"],
+    },
 
     classifiers=[
         'Development Status :: 4 - Beta',

src/sphinx_codeautolink/VERSION

@@ -1 +1 @@
-0.7.0
+0.8.0

src/sphinx_codeautolink/__init__.py

@@ -4,7 +4,7 @@
 
 from sphinx.application import Sphinx
 from .extension import backref, directive, SphinxCodeAutoLink
-from .extension.block import clean_pycon  # NOQA
+from .extension.block import clean_pycon, clean_ipython  # NOQA
 
 _version_file = _Path(_os.path.realpath(__file__)).parent / "VERSION"
 __version__ = _version_file.read_text().strip()
@@ -29,6 +29,9 @@ def setup(app: Sphinx):
     app.connect('builder-inited', state.build_inited)
     app.connect('autodoc-process-docstring', state.autodoc_process_docstring)
     app.connect('doctree-read', state.parse_blocks)
+    app.connect('env-merge-info', state.merge_environments)
+    app.connect('env-purge-doc', state.purge_doc_from_environment)
+    app.connect('env-updated', state.create_references)
     app.connect('doctree-resolved', state.generate_backref_tables)
     app.connect('build-finished', state.apply_links)
 
@@ -38,4 +41,4 @@ def setup(app: Sphinx):
     app.add_node(
         backref.SummaryNode, html=(backref.visit_summary, backref.depart_summary)
     )
-    return {'version': __version__}
+    return {'version': __version__, 'parallel_read_safe': True}