TypeError in sphinx logging building CPython docs with Sphinx 3.0

[tirkarthi]

Describe the bug

TypeError in sphinx logging building CPython docs with Sphinx 3.0

To Reproduce
Steps to reproduce the behavior:

$ git clone --depth 1 https://github.com/python/cpython/
$ cd cpython/Doc
$ Edit Makefile to ensure it's not pinned to 2.2 at https://github.com/python/cpython/blob/38aefc585f60a77d66f4fbe5a37594a488b53474/Doc/Makefile#L146
$ make venv
$ make html

Expected behavior

No internal error

Error log

# Sphinx version: 3.0.0
# Python version: 3.8.0 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 2.11.1
# Last messages:
#   reading sources... [ 98%] whatsnew/3.3
#   reading sources... [ 98%] whatsnew/3.4
#   reading sources... [ 98%] whatsnew/3.5
#   reading sources... [ 98%] whatsnew/3.6
#   reading sources... [ 99%] whatsnew/3.7
#   reading sources... [ 99%] whatsnew/3.8
#   reading sources... [ 99%] whatsnew/3.9
#   reading sources... [ 99%] whatsnew/changelog
#   reading sources... [100%] whatsnew/index
#   
# Loaded extensions:
#   sphinx.ext.mathjax (3.0.0) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (1.0.3) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.4) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /root/cpython/Doc/venv/lib/python3.8/site-packages/alabaster/__init__.py
#   sphinx.ext.coverage (3.0.0) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/ext/coverage.py
#   sphinx.ext.doctest (3.0.0) from /root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/ext/doctest.py
#   pyspecific (1.0) from /root/cpython/Doc/tools/extensions/pyspecific.py
#   c_annotations (1.0) from /root/cpython/Doc/tools/extensions/c_annotations.py
#   escape4chm (1.0) from /root/cpython/Doc/tools/extensions/escape4chm.py
#   python_docs_theme (unknown version) from /root/cpython/Doc/venv/lib/python3.8/site-packages/python_docs_theme/__init__.py
Traceback (most recent call last):
  File "/root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/application.py", line 348, in build
    self.builder.build_update()
  File "/root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 297, in build_update
    self.build(to_build,
  File "/root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 311, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/util/logging.py", line 213, in pending_warnings
    memhandler.flushTo(logger)
  File "/root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/util/logging.py", line 178, in flushTo
    logger.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1587, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 1649, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 946, in handle
    rv = self.filter(record)
  File "/usr/lib/python3.8/logging/__init__.py", line 807, in filter
    result = f.filter(record)
  File "/root/cpython/Doc/venv/lib/python3.8/site-packages/sphinx/util/logging.py", line 415, in filter
    raise SphinxWarning(location + ":" + message)
TypeError: can only concatenate str (not "DefinitionError") to str

Environment info

• OS: Linux
• Python version: 3.8
• Sphinx version: 3.0
• Sphinx extensions: listed in the error log

Additional context

Cpython issue : https://bugs.python.org/issue40204

[jakobandersen]

I'm not sure of the specific problem, but I used cpython as a testing ground for the revision of the C domain. I have opened python/cpython#19397 with the changes I ended up with.

[StrikerRUS]

We are facing the same error while building docs for out C API via breathe package.

# Sphinx version: 3.0.0
# Python version: 3.7.4 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 2.10.3
# Last messages:
#   making output directory...
#   done
#   building [mo]: targets for 0 po files that are out of date
#   building [html]: targets for 2 source files that are out of date
#   updating environment:
#   [new config]
#   2 added, 0 changed, 0 removed
#   reading sources... [ 50%] C-API
#   reading sources... [100%] index
#   
# Loaded extensions:
#   sphinx.ext.mathjax (3.0.0) from e:\miniconda3\lib\site-packages\sphinx\ext\mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from e:\miniconda3\lib\site-packages\sphinxcontrib\applehelp\__init__.py
#   sphinxcontrib.devhelp (1.0.2) from e:\miniconda3\lib\site-packages\sphinxcontrib\devhelp\__init__.py
#   sphinxcontrib.htmlhelp (1.0.3) from e:\miniconda3\lib\site-packages\sphinxcontrib\htmlhelp\__init__.py
#   sphinxcontrib.serializinghtml (1.1.4) from e:\miniconda3\lib\site-packages\sphinxcontrib\serializinghtml\__init__.py
#   sphinxcontrib.qthelp (1.0.3) from e:\miniconda3\lib\site-packages\sphinxcontrib\qthelp\__init__.py
#   alabaster (0.7.12) from e:\miniconda3\lib\site-packages\alabaster\__init__.py
#   breathe (4.15.0) from e:\miniconda3\lib\site-packages\breathe\__init__.py
Traceback (most recent call last):
  File "e:\miniconda3\lib\site-packages\sphinx\cmd\build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "e:\miniconda3\lib\site-packages\sphinx\application.py", line 348, in build
    self.builder.build_update()
  File "e:\miniconda3\lib\site-packages\sphinx\builders\__init__.py", line 299, in build_update
    len(to_build))
  File "e:\miniconda3\lib\site-packages\sphinx\builders\__init__.py", line 311, in build
    updated_docnames = set(self.read())
  File "e:\miniconda3\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "e:\miniconda3\lib\site-packages\sphinx\util\logging.py", line 213, in pending_warnings
    memhandler.flushTo(logger)
  File "e:\miniconda3\lib\site-packages\sphinx\util\logging.py", line 178, in flushTo
    logger.handle(record)
  File "e:\miniconda3\lib\logging\__init__.py", line 1524, in handle
    self.callHandlers(record)
  File "e:\miniconda3\lib\logging\__init__.py", line 1586, in callHandlers
    hdlr.handle(record)
  File "e:\miniconda3\lib\logging\__init__.py", line 890, in handle
    rv = self.filter(record)
  File "e:\miniconda3\lib\logging\__init__.py", line 751, in filter
    result = f.filter(record)
  File "e:\miniconda3\lib\site-packages\sphinx\util\logging.py", line 415, in filter
    raise SphinxWarning(location + ":" + message)
TypeError: can only concatenate str (not "DefinitionError") to str

To reproduce:

pip install -r requirements.txt
make html

requirements.txt

sphinx
breathe

conf.py

import os
import sys
import sphinx

from subprocess import PIPE, Popen

CURR_PATH = os.path.abspath(os.path.dirname(__file__))

# -- General configuration ------------------------------------------------

# The master toctree document.
master_doc = 'index'

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

# -- Configuration for C API docs generation ------------------------------

extensions = [
    'breathe',
]
breathe_projects = {
    "LightGBM": os.path.join(CURR_PATH, 'doxyoutput', 'xml')
}
breathe_default_project = "LightGBM"
breathe_domain_by_extension = {
    "h": "c",
}
breathe_show_define_initializer = True


def generate_doxygen_xml(app):
    """Generate XML documentation for C API by Doxygen.

    Parameters
    ----------
    app : object
        The application object representing the Sphinx process.
    """
    doxygen_args = [
        "INPUT={}".format(os.path.join(CURR_PATH, 'c_api.h')),
        "OUTPUT_DIRECTORY={}".format(os.path.join(CURR_PATH, 'doxyoutput')),
        "GENERATE_HTML=NO",
        "GENERATE_LATEX=NO",
        "GENERATE_XML=YES",
        "XML_OUTPUT=xml",
        "XML_PROGRAMLISTING=YES",
        r'ALIASES="rst=\verbatim embed:rst:leading-asterisk"',
        r'ALIASES+="endrst=\endverbatim"',
        "ENABLE_PREPROCESSING=YES",
        "MACRO_EXPANSION=YES",
        "EXPAND_ONLY_PREDEF=NO",
        "SKIP_FUNCTION_MACROS=NO",
        "SORT_BRIEF_DOCS=YES",
        "WARN_AS_ERROR=YES",
    ]
    doxygen_input = '\n'.join(doxygen_args)
    is_py3 = sys.version[0] == "3"
    if is_py3:
        doxygen_input = bytes(doxygen_input, "utf-8")
    if not os.path.exists(os.path.join(CURR_PATH, 'doxyoutput')):
        os.makedirs(os.path.join(CURR_PATH, 'doxyoutput'))
    try:
        # Warning! The following code can cause buffer overflows on RTD.
        # Consider suppressing output completely if RTD project silently fails.
        # Refer to https://github.com/svenevs/exhale
        # /blob/fe7644829057af622e467bb529db6c03a830da99/exhale/deploy.py#L99-L111
        process = Popen(["doxygen", "-"],
                        stdin=PIPE, stdout=PIPE, stderr=PIPE)
        stdout, stderr = process.communicate(doxygen_input)
        output = '\n'.join([i.decode('utf-8') if is_py3 else i
                            for i in (stdout, stderr) if i is not None])
        if process.returncode != 0:
            raise RuntimeError(output)
        else:
            print(output)
    except BaseException as e:
        raise Exception("An error has occurred while executing Doxygen\n" + str(e))


def setup(app):
    """Add new elements at Sphinx initialization time.

    Parameters
    ----------
    app : object
        The application object representing the Sphinx process.
    """
    app.connect("builder-inited", generate_doxygen_xml)

index.rst

.. toctree::
   :caption: Contents:

   C API <C-API>

C-API.rst

C API
=====

.. doxygenfile:: c_api.h

c_api.h

#ifndef LIGHTGBM_C_API_H_
#define LIGHTGBM_C_API_H_


/*!
 * \brief Free space for booster.
 * \param handle Handle of booster to be freed
 * \return 0 when succeed, -1 when failure happens
 */
LIGHTGBM_C_EXPORT int LGBM_BoosterFree(int handle);

#endif  // LIGHTGBM_C_API_H_

Removing LIGHTGBM_C_EXPORT from c_api.h makes build succeed.

[jakobandersen]

Using the instructions of @tirkarthi I can reproduce it, but only with the venv, not when I run sphinx-build directly. Similarly, the example of @StrikerRUS does not reproduce the problem on my end.
I have made #7437 which I believe fixes the problem. Are you able to confirm it?

(Btw. it would be nice if the Python logging module would have type annotations. I believe this bug then would have been caught before release :-)).