Support pyproject.toml-style configuration (PEP 621) - Round 2 #2970

This will facilitate the implementation of other configuration formats (such as pyproject.toml as initially defined by PEP 621)

We can split the process of interpreting configuration files into 2 steps: 1. The parsing the file contents from strings to value objects that can be understand by Python (for example a string with a comma separated list of keywords into an actual Python list of strings). 2. The expansion (or post-processing) of these values according to the semantics ``setuptools`` assign to them (for example a configuration field with the ``file:`` directive should be expanded from a list of file paths to a single string with the contents of those files concatenated) The idea of this change is to extract the functions responsible for (2.) into a new module, so they can be reused between different config file formats.

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

This eventually will allow reading project metadata directly from `pyproject.toml`

In order to minimise dependencies, `validate-pyproject` has the ability to "dump" only the code necessary to run the validations to a given directory. This special strategy is used instead of the default `pip install -t`. The idea of using JSONSchema for validation was suggested in pypa#2671, and the rationale for that approach is further discussed in https://github.com/abravalheri/validate-pyproject/blob/main/docs/faq.rst Using a library such as `validate-pyproject` has the advantage of incentive sing reuse and collaboration with other projects. Currently `validate-pyproject` ships a JSONSchema for the proposed use of `pyproject.toml` as means of configuration for setuptools. In the future, if there is interest, setuptools could also ship its own schema and just use the shared infrastructure of `validate-pyproject` (by advertising the schemas via entry-points).

Enforcing local imports is no longer needed.

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

This is the first step towards making setuptools understand `pyproject.toml` as a configuration file. The implementation deliberately allows splitting the act of loading the configuration from a file in 2 stages: the reading of the file itself and the expansion of directives (and other derived information).

The user might specify dynamic `entry-points` via a `file:` directive (a similar feature for `setup.cfg` is documented in [declarative config]). The changes introduced here add the ability to expand them when reading the configuration from `pyproject.toml`. [declarative config]: https://setuptools.pypa.io/en/latest/userguide/declarative_config.html

There is frequent an opinion in the community that `include_package_data=True` is a better default (and a quality of life improvement). Since we are migrating to a new configuration file, this change can be implemented in a backward compatible way (to avoid breaking existing packages): - Config from `setup.cfg` defaults to `include_package_data=False` - Config from `pyproject.toml` defaults to `include_package_data=True` This also takes advantage that `ini2toml` (the provided library for automatic conversion between `setup.cfg` and `pyproject.toml`) will backfill `include_package_data=False` when the field is missing.

Since the Distrubition and DistributionMetadata classes are modeled after (an old version of) core metadata, it is necessary to add a translation layer between them and the configuration read from pyproject.toml

The apply_configuration is implemented in a way that it is consistent for both pyproject.toml and setup.cfg

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

According to PEP 621, the backend should fill-in the content-type if the `readme` field is passed as a string. The value is derived from the extension of the file (an error should be raised when that is not possible). Previously all READMEs were wrongly assumed rst. This error was reported in: https://discuss.python.org/t/help-testing-experimental-features-in-setuptools/13821/4

Since now setuptools supports 2 types of files for configuration (`setup.cfg` and `pyproject.toml`), it is very trick to provide a single `read_configuration` function that will provide compatible outputs for both formats. Instead the `config.{setupcfg,pyprojecttoml}` modules have their own `read_configuration` functions that differ between themselves in terms of arguments and format of the return value. Therefore the users should be importing directly the specific submodule and calling the read function from there. The `config.setupcfg` submodule is advertised as "provisional" in the deprecation note because the main proposal debated in the setuptools issue tracker reached some level of consensus around deprecating `setup.cfg` files.

… as suggested in code review

This is a regression test for a problem identified in: pypa#2970 (comment)

These tests were initially motivated by a discussion in: pybind/pybind11#3711 (comment)

…ns (pypa#3065) We can split the process of interpreting configuration files into 2 steps: 1. The parsing the file contents from strings to value objects that can be understand by Python (for example a string with a comma separated list of keywords into an actual Python list of strings). 2. The expansion (or post-processing) of these values according to the semantics setuptools assign to them (for example a configuration field with the file: directive should be expanded from a list of file paths to a single string with the contents of those files concatenated) The idea of this change is to extract the functions responsible for (2.) into a new module, so they can be reused between different config file formats.

Some extra dependencies can be used to support ``pyproject.toml`` configuration. This change just adds them as vendored packages.

This change targets adding a standardised project configuration format as initially proposed by PEP 621 (with the possibility of modification/evolution according to follow-up PEPs). For the configuration fields not covered by the standards, the approach proposed in [ini2toml] is adopted. This operation relies on vendorised external libraries `tomli` and `validate-pyproject`. [ini2toml]: https://ini2toml.readthedocs.io/en/latest/setuptools_pep621.html

This change builds on top of the recently added ability to parse and apply configurations from TOML files to distribution objects by tapping into the existing mechanism for handling configuration and making these TOML files to be read by default. A series of deprecation warnings and tests is also added.

Type hints can help to increase confidence in the code and prevent unexpected bugs. To typecheck the subpackage the following command can be used: ``` mypy setuptools/config --tb --show-error-codes --show-error-context --pretty ```

The adoption of configuration via pyprojec.toml is a perfect opportunity to consider namespaces by default (PEP 420) when automatically finding projects. In projects using `setup.cfg` or `setup.py` nothing will be changed for backward compatibility, but for projects including metadata in `pyproject.toml` (as initially proposed in PEP 621), namespaces will be considered by default.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support pyproject.toml-style configuration (PEP 621) - Round 2 #2970

Support pyproject.toml-style configuration (PEP 621) - Round 2 #2970

Commits on Mar 5, 2022