[Doubt] Distribution.get_name() always "UNKNOWN" in extensions when using pyproject.toml (in setuptools.finalize_distribution_options entry-point)

[hansingt]

setuptools version

setuptools==69.1.1

Python version

Python 3.11

OS

Fedoa Workstation 39

Additional environment information

No response

Description

I've a custom setuptools extension, which tries to do some things, based on the name of the Distribution. It has worked before, but now, we are migrating the keywords from the setup.py to the pyproject.toml and this resulted in the Distribution.get_name() always being UNKNOWN in the Options finalizer (setuptools.finalize_distribution_options entry-point).

Expected behavior

I'd have expected the distribution options to be set when running the option finalizers, regardless of whether the options are set using the pyproject.toml or the setup.py.

How to Reproduce

1. Clone the example from https://github.com/hansingt/setuptools-dist-name-unknown
2. python -m venv venv
3. ./venv/bin/pip install -e .
4. ./venv/bin/python setup.py --name

Output

DIST NAME: UNKNOWN
test

PS: The behaviour is demonstrated using a direct call to setup.py, but it behaves the same, when building the distribution, either using setup.py bdist_wheel,pip wheel, or python -m build. The only difference is, that you don't see the output when running these commands.

[abravalheri]

Hi @hansingt, there is probably a misalignment of expectations around setuptools.finalize_distribution_options.

This hook runs as the last step of the creation of the Distribution object. In a simplified way, this is more or less what happens:

build_meta (build API hooks)
 ├── Distribution.__init__()
 │       ├── "initialize options" (to default values or values given via `setup.py`)
 │       ├── entry-point hooks: `distutils.setup_keywords`
 │       └── Distribytion.finalize_options()
 │                 └── entry-point hooks: setuptools.finalize_distribution_options
 ├── Distribution.parse_config_files()
 │      ├── parses `setup.cfg`
 │      └── parses `pyproject.toml`
 ├── Distribution.parse_command_line()
 └── Distribution.run_command()

If you want to have a hook that is able to access the distribution object after the config files are parsed, that would probably be a feature request.

[hansingt]

Hi @abravalheri,

Thanks for the reply. What I wanted to say, is that I've observed a difference in the behavior, when specifying the distribution options in setup.py and when specifying them in pyproject.toml.

If I specify the name directly in the setup.py everything is fine. If I specify it in pyproject.toml, it is not set yet. This is a difference in behavior, which (at least) I had not expected.

Especially, as the name says, that the hook is to finalize the options, thus, I would have expected all (defined) options to be already set. But I might have misunderstood this.

If it is not possible to include parsing the configuration files in the distribution initialization, I'd need to work around this issue, somehow.

[abravalheri]

I see...

Yes, the name is a bit confusing. Sorry for that.

The entry-point hook runs inside the __init__() function.

The parse_config_files() function is called at a later stage. The reason why setup.py files work is because the setup() function is a kind of wrapper around the full pipeline (it runs all the functions that I mentioned above).

[hansingt]

From technical point of view, I understand the reason for the current behavior.

But from the user's point of view, this is inconsistent or even unexpected and therefore, IMHO, should be either decided (and documented) or changed.

Out of curiosity, is there anything against changing the order of the calls, to parse the configuration files before calling the finalizers?

I think about changing the call structure to something similar to this:

build_meta (build API hooks)
 ├── Distribution.__init__()
 │       ├── "initialize options" (to default values or values given via `setup.py`)
 │       ├── entry-point hooks: `distutils.setup_keywords`
 |       ├── Distribution.parse_config_files()
 │       |      ├── parses `setup.cfg`
 │       |      └── parses `pyproject.toml`
 │       └── Distribytion.finalize_options()
 │              └── entry-point hooks: setuptools.finalize_distribution_options
 ├── Distribution.parse_command_line()
 └── Distribution.run_command()

[abravalheri]

But from the user's point of view, this is inconsistent or even unexpected and therefore, IMHO, should be either decided (and documented) or changed.

For backward compatibility, I believe that the best approach is documentation instead of change (otherwise we risk breaking existing plugins). Would you like to provide a PR for that?

Out of curiosity, is there anything against changing the order of the calls, to parse the configuration files before calling the finalizers?

Calling Distribution.parse_config_files inside of Distribution.__init__ probably would break things
(I haven't tried, maybe not... But this aspect is defined in distutils codebase, so the change would require cross-repository updates).

But even if it does not break the setuptools implementation itself, there is always Hyrum's law. In a project the size of setuptools, the likelihood of breaking 3rd-party code that indirectly relies on this behaviour is very high.

[hansingt]

I see. I agree, that the chance to break things is very high. I've played around with this, to try to find a suitable workaround, but this is way more difficult than I thought, because of many internal dependencies in the execution order of the code.

Would you like to provide a PR for that?

I can try to do so.