Python packaging has come a long way.
The traditional setuptools
way of packaging Python modules
uses a setup()
function within the setup.py
script. Commands such as
python setup.py bdist
or python setup.py bdist_wheel
generate a
distribution bundle and python setup.py install
installs the distribution.
This interface makes it difficult to choose other packaging tools without an
overhaul. Because setup.py
scripts allow for arbitrary execution, it
is difficult to provide a reliable user experience across environments
and history.
PEP 517 therefore came to the rescue and specified a new standard to package and distribute Python modules. Under PEP 517:
A
pyproject.toml
file is used to specify what program to use for generating the distribution.Then, two functions provided by the program,
build_wheel(directory: str)
andbuild_sdist(directory: str)
create the distribution bundle at the specifieddirectory
. The program is free to use its own configuration script or extend the.toml
file.Lastly,
pip install *.whl
orpip install *.tar.gz
does the actual installation. If*.whl
is available,pip
will go ahead and copy the files intosite-packages
directory. If not,pip
will look atpyproject.toml
and decide what program to use to 'build from source' (the default issetuptools
).
With this standard, switching between packaging tools is a lot easier. build_meta
implements setuptools
's build system support.
Start with a package that you want to distribute. You will need your source
files, a pyproject.toml
file and a setup.cfg
file:
~/meowpkg/ pyproject.toml setup.cfg meowpkg/ __init__.py module.py
The pyproject.toml file is required to specify the build system (i.e. what is being used to package your scripts and install from source). To use it with setuptools, the content would be:
[build-system] requires = ["setuptools"] build-backend = "setuptools.build_meta"
The setuptools
package implements the build_sdist
command and the wheel
package implements the build_wheel
command; the latter is a dependency of the former
exposed via PEP 517 hooks.
Use setuptools
' :ref:`declarative config <declarative config>` to
specify the package information:
[metadata] name = meowpkg version = 0.0.1 description = a package that meows [options] packages = find:
Now generate the distribution. To build the package, use PyPA build:
$ pip install -q build $ python -m build
And now it's done! The .whl
file and .tar.gz
can then be distributed
and installed:
dist/ meowpkg-0.0.1.whl meowpkg-0.0.1.tar.gz $ pip install dist/meowpkg-0.0.1.whl
or:
$ pip install dist/meowpkg-0.0.1.tar.gz
With the changes introduced by PEP 517 and PEP 518, the
setup_requires
configuration field was made deprecated in setup.cfg
and
setup.py
, in favour of directly listing build dependencies in the
requires
field of the build-system
table of pyproject.toml
.
This approach has a series of advantages and gives package managers and
installers the ability to inspect in advance the build requirements and
perform a series of optimisations.
However some package authors might still need to dynamically inspect the final
users machine before deciding these requirements. One way of doing that, as
specified by PEP 517, is to "tweak" setuptools.build_meta
by using a
:pep:`in-tree backend <517#in-tree-build-backends>`.
Tip
Before implementing a in-tree backend, have a look on :pep:`PEP 508 <508#environment-markers>`. Most of the times, dependencies with environment markers are enough to differentiate operating systems and platforms.
If you add the following configuration to your pyproject.toml
:
[build-system]
requires = ["setuptools", "wheel"]
build-backend = "backend"
backend-path = ["_custom_build"]
then you should be able to implement a thin wrapper around build_meta
in
the _custom_build/backend.py
file, as shown in the following example:
from setuptools import build_meta as _orig
prepare_metadata_for_build_wheel = _orig.prepare_metadata_for_build_wheel
build_wheel = _orig.build_wheel
build_sdist = _orig.build_sdist
def get_requires_for_build_wheel(self, config_settings=None):
return _orig.get_requires_for_build_wheel(config_settings) + [...]
def get_requires_for_build_sdist(self, config_settings=None):
return _orig.get_requires_for_build_sdist(config_settings) + [...]
Note that you can override any of the functions specified in :pep:`PEP 517 <517#build-backend-interface>`, not only the ones responsible for gathering requirements.
Important
Make sure your backend script is included in the :doc:`source distribution </userguide/distribution>`, otherwise the build will fail. This can be done by using a SCM/VCS plugin (like :pypi:`setuptools-scm` and :pypi:`setuptools-svn`), or by correctly setting up :ref:`MANIFEST.in <manifest>`.
If this is the first time you are using a customised backend, please have a
look on the generated .tar.gz
and .whl
.
On POSIX systems that can be done with tar -tf dist/*.tar.gz
and unzip -l dist/*.whl
.
On Windows systems you can rename the .whl
to .zip
to be able to
inspect it on the file explorer, and use the same tar
command in a
command prompt (alternativelly there are GUI programs like 7-zip that
handle .tar.gz
).
In general the backend script should be present in the .tar.gz
(so the
project can be build from the source) but not in the .whl
(otherwise the
backend script would end up being distributed alongside your package).
See ":doc:`/userguide/package_discovery`" for more details about package
files.