Poetry 2.0 removals and other breaking changes

[radoering]

This issue tracks possible removals and other breaking changes we want to include if the next release (or the one after that) will be a major release. (That does not mean that the changes listed here cannot be included in a minor release after a decent deprecation period.)

Please add a comment if you have concerns or would like to include another pending breaking change that is not on the list.

Background & Motivation

As support for PEP 621 is slowly taking shape (see #9135 to track the current state) and this will be a big, significant change (even if the old way will still be supported), this might be the time for a major version bump. Even if we try to avoid obvious breaking changes, just not ignoring the project section anymore will be a subtle breaking change for anyone who is already using this section (for whatever reason). Further, since supporting PEP 621 requires massive changes in many areas of the code, the risk of unintentional breaking changes is higher than usual.

Long story short: If we do a major version bump anyway, we should make use of it and remove things / add breaking changes we were holding back so far.

Candidates

Disclaimer: decision is not final and might be subject to change

change	decision	details
remove dependency on poetry-plugin-export	❓	#5980, although #5740 would be nice
remove deprecated source types secondary and default	❓	results in significantly less complicated code and less complicated documentation
remove support of scripts with extras	❓	already done in python-poetry/poetry-core#708
fail if project cannot be installed	❓	#9333
make lock --no-update the default	❓	#9327
remove deprecated command line options	❓
...	...	...

[dimbleby]

maybe an opportunity to do something about poetry shell - see eg discussion in #4216

[dimbleby]

and also to take a view on poetry self #7872

[johnthagen]

A humble 🙏 that if poetry-plugin-export is removed by default, it's accompanied by something like

• How do I make part of and auto-enable plugins in my project #5729
• poc: isolated project plugins & tools #5740

This would then be much less disruptive for users of tools such as nox-poetry.

[Evan-Zhao]

(Sorry if this is not the right place to add an Issue on this version.)
I've been trying this new version out, and it's been working great overall!

One problem I noticed is whenever Poetry tries to add a [tool.poetry] section to a PEP 621-compliant pyproject.toml file, it erases everything after the point where it inserts the new section.

For example, I'm working on the following pyproject.toml here.
When I run poetry add --group plot matplotlib from the cmdline, Poetry will fall back to adding a [tool.poetry] section:

[tool.poetry.group.plot.dependencies]
matplotlib = "^3.8.3"

which makes sense because dep group is not a thing in 621.
However, this is added after [project.optional-dependencies], and Poetry also remove everything below that in this process -- the [tool.isort] and [tool.black] sections are lost.

[radoering]

Poetry also remove everything below that in this process -- the [tool.isort] and [tool.black] sections are lost.

Thanks for reporting this. I fixed it and added a test.

(Sorry if this is not the right place to add an Issue on this version.)

Since it's an open PR that is not yet merged, you can report issues directly in #9135.

[PabloAlexis611]

I think it'd be nice to also get rid of the deprecated --dev flag for the poetry add command since Poetry support for groups is already in place

[radoering]

Yes, we should take a look at all of the deprecated command line options.

[rmorshea]

Is there any chance that some of these changes could be enabled via feature flags?

I'm particularly interested in #3248 but this could be generally useful for easing the community into most of the listed breaking changes. Perhaps something like:

[tool.poetry.future]
lock-no-update-by-default = true

Of course doing this would mean maintaining two different code paths, but that's something the maintainers would have to weigh against the benefits of early feedback and (hopefully) a smoother transition.

[Secrus]

Is there any chance that some of these changes could be enabled via feature flags?

I'm particularly interested in #3248 but this could be generally useful for easing the community into most of the listed breaking changes. Perhaps something like:

[tool.poetry.future]
lock-no-update-by-default = true

Of course doing this would mean maintaining two different code paths, but that's something the maintainers would have to weigh against the benefits of early feedback and (hopefully) a smoother transition.

No. The reason for a major version bump is for us to have an opportunity to cleanup the code. Maintaining multiple code paths goes against that goal.

[trim21]

suggest to make install --sync/--remove-untracked default

[aureliojargas]

Looking at the changelog, we've had already some breaking changes in the configuration format:

• 1.0.0: Specifying dependencies with allows-prereleases in the pyproject.toml file is deprecated for consistency with the add command. Use allow-prereleases instead.
• 1.5.0: Introduce priority key for sources and deprecate flags default and secondary
• 1.8.0: Deprecate source priority default.

Here we're discussing some additional breaking changes on the configuration.

Would it make sense having some metadata (say, in [tool.poetry]) informing which specific Poetry version(s) the current configuration is intended for? Or having some versioned configuration schema? Otherwise it can be a guess work finding the Poetry version that works with a given configuration file.

[radoering]

suggest to make install --sync/--remove-untracked default

Although I mostly use --sync myself, I am not sure about it. (Waiting for other opinions.) I suppose, the current default could be named --keep-untracked if we decided to change the default.

Would it make sense having some metadata (say, in [tool.poetry]) informing which specific Poetry version(s) the current configuration is intended for?

There is #3316. I think we are open to adding a version constraint so the user can define compatible poetry versions.

Or having some versioned configuration schema?

I think that could cause more confusion than it helps respectively increases the maintainance burden. In the end, pyproject.toml is a file that is manually edited by users so users are responsible for specifying the correct version. If I remember correctly, there are some discussions on DPO why there is no version in pyproject.toml in general.

Otherwise it can be a guess work finding the Poetry version that works with a given configuration file.

At the moment, your best chance is the first line of poetry.lock.

[johnthagen]

suggest to make install --sync/--remove-untracked default

I don't have a strong opinion, but I do myself use --sync all the time and recommend to all of my teams they do the same. I think setting it as a default is probably less surprising (e.g., someone comes back to a project months later and has old things installed in the venv, the default poetry install doesn't remove them and get them to a clean slate).

I would lean towards making it the default because it feels more "pure" (e.g., by default Poetry will always make sure that you get the same environment as others on the same platform).

The counter would be if some users install "extra" things into their venv often and are frustrated that now Poetry uninstalls them. I'd call this an anti-pattern and they should be putting these things in groups to make things sound.

Overall, I would support making --sync the default.