Added documentation page about extension modules (#3368)

changelog.d/3368.doc.rst

@@ -0,0 +1 @@
+Added documentation page about extension modules -- by :user:`mkoeppe`

docs/userguide/ext_modules.rst

@@ -0,0 +1,116 @@
+==========================
+Building Extension Modules
+==========================
+
+Setuptools can build C/C++ extension modules.  The keyword argument
+``ext_modules`` of ``setup()`` should be a list of instances of the
+:class:`setuptools.Extension` class.
+
+
+For example, let's consider a simple project with only one extension module::
+
+    <project_folder>
+    ├── pyproject.toml
+    └── foo.c
+
+and all project metadata configuration in the ``pyproject.toml`` file:
+
+.. code-block:: toml
+
+   # pyproject.toml
+   [build-system]
+   requires = ["setuptools"]
+   build-backend = "setuptools.build_meta"
+
+   [project]
+   name = "mylib-foo"  # as it would appear on PyPI
+   version = "0.42"
+
+To instruct setuptools to compile the ``foo.c`` file into the extension module
+``mylib.foo``, we need to add a ``setup.py`` file similar to the following:
+
+.. code-block:: python
+
+   from setuptools import Extension, setup
+
+   setup(
+       ext_modules=[
+           Extension(
+               name="mylib.foo",  # as it would be imported
+                                  # may include packages/namespaces separated by `.`
+
+               sources=["foo.c"], # all sources are compiled into a single binary file
+           ),
+       ]
+   )
+
+.. seealso::
+    You can find more information on the `Python docs about C/C++ extensions`_.
+    Alternatively, you might also be interested in learn about `Cython`_.
+
+    If you plan to distribute a package that uses extensions across multiple
+    platforms, :pypi:`cibuildwheel` can also be helpful.
+
+
+Compiler and linker options
+===========================
+
+The command ``build_ext`` builds C/C++ extension modules.  It creates
+a command line for running the compiler and linker by combining
+compiler and linker options from various sources:
+
+.. Reference: `test_customize_compiler` in distutils/tests/test_sysconfig.py
+
+* the ``sysconfig`` variables ``CC``, ``CXX``, ``CCSHARED``,
+  ``LDSHARED``, and ``CFLAGS``,
+* the environment variables ``CC``, ``CPP``,
+  ``CXX``, ``LDSHARED`` and ``LDFLAGS``,
+  ``CFLAGS``, ``CPPFLAGS``, ``LDFLAGS``,
+* the ``Extension`` attributes ``include_dirs``,
+  ``library_dirs``, ``extra_compile_args``, ``extra_link_args``,
+  ``runtime_library_dirs``.
+
+.. Ignoring AR, ARFLAGS, RANLIB here because they are used by the (obsolete?) build_clib, not build_ext.
+
+The resulting command line is then processed by the compiler and linker.
+According to the GCC manual sections on `directory options`_ and
+`environment variables`_, the C/C++ compiler searches for files named in
+``#include <file>`` directives in the following order:
+
+* first, in directories given by ``-I`` options (in left-to-right order),
+* then, in directories given by the environment variable ``CPATH`` (in left-to-right order),
+* then, in directories given by ``-isystem`` options (in left-to-right order),
+* then, in directories given by the environment variable ``C_INCLUDE_PATH`` (for C) and ``CPLUS_INCLUDE_PATH`` (for C++),
+* then, in standard system directories,
+* finally, in directories given by ``-idirafter`` options (in left-to-right order).
+
+The linker searches for libraries in the following order:
+
+* first, in directories given by ``-L`` options (in left-to-right order),
+* then, in directories given by the environment variable ``LIBRARY_PATH`` (in left-to-right order).
+
+.. important::
+   All files used to compile your extension need to be available on the system
+   when building the package, so please make sure to include some documentation
+   on how developers interested in building your package from source
+   can obtain operating system level dependencies
+   (e.g. compilers and external binary libraries/artifacts).
+
+   You will also need to make sure that all auxiliary files that are contained
+   inside your :term:`project` (e.g. C headers authored by you or your team)
+   are configured to be included in your :term:`sdist <Source Distribution (or "sdist")>`.
+   Please have a look on our section on :ref:`Controlling files in the distribution`.
+
+
+----
+
+API Reference
+-------------
+
+.. autoclass:: setuptools.Extension
+
+
+.. _Python docs about C/C++ extensions: https://docs.python.org/3/extending/extending.html
+.. _Cython: https://cython.readthedocs.io/en/stable/index.html
+.. _directory options: https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html
+.. _environment variables: https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html>

docs/userguide/index.rst

@@ -28,6 +28,7 @@ Contents
     package_discovery
     entry_point
     dependency_management
+    ext_modules
     datafiles
     development_mode
     distribution

setuptools/extension.py

@@ -28,7 +28,92 @@ def _have_cython():
 
 
 class Extension(_Extension):
-    """Extension that uses '.c' files in place of '.pyx' files"""
+    """
+    Describes a single extension module.
+
+    This means that all source files will be compiled into a single binary file
+    ``<module path>.<suffix>`` (with ``<module path>`` derived from ``name`` and
+    ``<suffix>`` defined by one of the values in
+    ``importlib.machinery.EXTENSION_SUFFIXES``).
+
+    In the case ``.pyx`` files are passed as ``sources and`` ``Cython`` is **not**
+    installed in the build environment, ``setuptools`` may also try to look for the
+    equivalent ``.cpp`` or ``.c`` files.
+
+    :arg str name:
+      the full name of the extension, including any packages -- ie.
+      *not* a filename or pathname, but Python dotted name
+
+    :arg list[str] sources:
+      list of source filenames, relative to the distribution root
+      (where the setup script lives), in Unix form (slash-separated)
+      for portability.  Source files may be C, C++, SWIG (.i),
+      platform-specific resource files, or whatever else is recognized
+      by the "build_ext" command as source for a Python extension.
+
+    :keyword list[str] include_dirs:
+      list of directories to search for C/C++ header files (in Unix
+      form for portability)
+
+    :keyword list[tuple[str, str|None]] define_macros:
+      list of macros to define; each macro is defined using a 2-tuple:
+      the first item corresponding to the name of the macro and the second
+      item either a string with its value or None to
+      define it without a particular value (equivalent of "#define
+      FOO" in source or -DFOO on Unix C compiler command line)
+
+    :keyword list[str] undef_macros:
+      list of macros to undefine explicitly
+
+    :keyword list[str] library_dirs:
+      list of directories to search for C/C++ libraries at link time
+
+    :keyword list[str] libraries:
+      list of library names (not filenames or paths) to link against
+
+    :keyword list[str] runtime_library_dirs:
+      list of directories to search for C/C++ libraries at run time
+      (for shared extensions, this is when the extension is loaded)
+
+    :keyword list[str] extra_objects:
+      list of extra files to link with (eg. object files not implied
+      by 'sources', static library that must be explicitly specified,
+      binary resource files, etc.)
+
+    :keyword list[str] extra_compile_args:
+      any extra platform- and compiler-specific information to use
+      when compiling the source files in 'sources'.  For platforms and
+      compilers where "command line" makes sense, this is typically a
+      list of command-line arguments, but for other platforms it could
+      be anything.
+
+    :keyword list[str] extra_link_args:
+      any extra platform- and compiler-specific information to use
+      when linking object files together to create the extension (or
+      to create a new static Python interpreter).  Similar
+      interpretation as for 'extra_compile_args'.
+
+    :keyword list[str] export_symbols:
+      list of symbols to be exported from a shared extension.  Not
+      used on all platforms, and not generally necessary for Python
+      extensions, which typically export exactly one symbol: "init" +
+      extension_name.
+
+    :keyword list[str] swig_opts:
+      any extra options to pass to SWIG if a source file has the .i
+      extension.
+
+    :keyword list[str] depends:
+      list of files that the extension depends on
+
+    :keyword str language:
+      extension language (i.e. "c", "c++", "objc"). Will be detected
+      from the source extensions if not provided.
+
+    :keyword bool optional:
+      specifies that a build failure in the extension should not abort the
+      build process, but simply not install the failing extension.
+    """
 
     def __init__(self, name, sources, *args, **kw):
         # The *args is needed for compatibility as calls may use positional