Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update package metadata #338

Merged
merged 5 commits into from Jun 20, 2022
Merged

Update package metadata #338

merged 5 commits into from Jun 20, 2022

Conversation

ofek
Copy link
Contributor

@ofek ofek commented Mar 27, 2022

Background

Hello there! The Python packaging ecosystem has standardized on the interface for build backends (PEP 517/PEP 660) and the format for metadata declaration (PEP 621/PEP 631). As a result, the execution of setup.py files is now heavily discouraged.

So, I'm spending my free time updating important projects so that they are modernized and set an example for others 馃槃

Summary of changes

This implements PEP 621, obviating the need for setup.py, setup.cfg, and MANIFEST.in. The build backend hatchling (of which I am a maintainer) is quite stable and actively developed, the only reason the version is 0.x is because 1.0.0 will drop support for Python 2. It's also on the major distribution channels such as Debian, Fedora, Arch Linux, conda-forge, Nixpkgs, etc.

Other projects have done this such as pipx, tox, all Datadog Agent integrations, etc.

In comparison to setuptools, Hatchling has reproducible builds, supports the modern PEP 660 editable installation standard, has a more intuitive plugin system, and the codebase is an order of magnitude smaller thus making it more auditable and less prone to bugs (setuptools is based on a deprecated standard library module that it now must vendor).

Notes

  • Similar to Add support for Python 3.7 and 3.8, drop EOL 3.4聽#228, this also drops support for EOL Python 3.x versions since hatchling and other build backends like setuptools have already done so. This will not break anything because pip only looks for version candidates that satisfy the requires-python field. Here are stats for the curious: https://pypistats.org/packages/colorama
  • The source distributions on PyPI are erroneously shipping a build artifact *.egg-info from pip install -e .; this is now fixed:
    c

@ofek ofek force-pushed the modernize-metadata branch 2 times, most recently from 476f164 to 1918721 Compare March 27, 2022 20:39
@ofek
Copy link
Contributor Author

ofek commented May 8, 2022

@tartley
Copy link
Owner

tartley commented Jun 14, 2022

Hi. Thanks for the contribution! We're talking about making a release, and this seems like a laudable change. I'd like to get it in if I can. @ofek Presumably your opinion is that it is currently good to merge as-is? I'll check it out...

Makefile Outdated Show resolved Hide resolved
@tartley
Copy link
Owner

tartley commented Jun 14, 2022

I see there are failures on 2.7 on Windows. I'll try to run a build on master as a sanity check.

@ofek
Copy link
Contributor Author

ofek commented Jun 14, 2022

Fixed, this looks good to me now!

@ofek
Copy link
Contributor Author

ofek commented Jun 14, 2022

When you drop py2 just remove the pin

@tartley
Copy link
Owner

tartley commented Jun 14, 2022

We're scrambling to find a little time to make a release before going on vacation. In order to keep things simple, I'm going to postpone merging this right now, but will merge it and make another release after my vacation, around July 5th.

Thanks heaps for the contribution, and all your work leading up to this point! :-)

@tartley tartley added the 0.4.6 Get merged for 0.4.6. release label Jun 14, 2022
@ofek
Copy link
Contributor Author

ofek commented Jun 14, 2022

Happy to help! Have fun 馃檪

@wiggin15
Copy link
Collaborator

Hi @ofek . Can you explain the benefits of using hatchling vs. using python -m build once we have pyproject.toml?
Maybe these are just two different build backends and build are using a name that is too generic - but my knowledge about these recent developments is not up-to-date so this is a bit confusing.

@ofek
Copy link
Contributor Author

ofek commented Jun 15, 2022

@wiggin15 Hey! See https://peps.python.org/pep-0517/

build is a build frontend like pip. So python -m build and pip wheel . both invoke the configured build backend which is what builds the sdist & wheel. Hatchling is a build backend like setuptools.

Some benefits:

  • easier to configure
  • reproducible builds
  • plugin ecosystem
  • small, modern codebase which makes it easier to audit

@wiggin15
Copy link
Collaborator

Thanks. I think I got it.

Makefile Outdated
@@ -51,15 +50,14 @@ test: ## Run tests
# build packages

build: ## Build a release (sdist and wheel)
$(python) -m pip install --upgrade setuptools wheel
$(python) setup.py sdist bdist_wheel
$(python) -m pip install --upgrade build
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think maybe "build" should be in requirements-dev.txt (which is installed during make bootstrap - see the release process in README-hacking.md) instead of being installed here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think not since it's unrelated to testing; I think twine should also not be in there

Copy link
Collaborator

@wiggin15 wiggin15 Jun 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requirements-dev.txt is not just for testing - we install these packages when running make bootstrap before running make build/make release as part of the build/release process.

OTOH now I see that we ran pip install for setuptools and wheel before this change...

Anyway, this is not blocking a merge.

@wiggin15
Copy link
Collaborator

I'm ok with merging this now that we released v0.4.5. @ofek can you please rebase the PR to resolve the latest conflicts? Thanks.

@ofek
Copy link
Contributor Author

ofek commented Jun 19, 2022

Done!

Makefile Outdated
.PHONY: build

test-release: build ## Test a built release
./test-release
.PHONY: test-release

release: ## Upload a built release
$(twine) upload dist/colorama-$(version)*
$(twine) upload dist/colorama-*{.whl,.tar.gz}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We encountered an issue in our last release where the globs using {} didn't work in the Makefile. We had to remove them.

@@ -1,5 +1,6 @@
[tox]
envlist = py27, py35, py36, py37, py38, py39, py310, pypy, pypy3
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are removing py35 and py36 from the testing matrix we should reflect that in the README (we list tested versions under Installation)

@wiggin15
Copy link
Collaborator

Sorry for piling on more comments each time, and thanks for working with us :)
I feel like we're almost there!

@ofek
Copy link
Contributor Author

ofek commented Jun 19, 2022

Addressed!

@wiggin15 wiggin15 merged commit cd653d7 into tartley:master Jun 20, 2022
@wiggin15
Copy link
Collaborator

Thank you very much @ofek . This brings colorama to the modern age of Python packaging 馃コ

@ofek ofek deleted the modernize-metadata branch June 20, 2022 06:05
@tartley
Copy link
Owner

tartley commented Oct 19, 2022

We have a release candidate on PyPI containing this PR. Thank you for your contributions! https://pypi.org/project/colorama/0.4.6rc1/

If nobody spots any problems with it, I'll push the final 0.4.6 tomorrow.

@ofek
Copy link
Contributor Author

ofek commented Oct 20, 2022

Awesome!

SomberNight added a commit to SomberNight/electrum that referenced this pull request Nov 4, 2022
or more like four:
```
$ pipdeptree
hatchling==1.11.1
  - editables [required: >=0.3, installed: 0.3]
  - packaging [required: >=21.3, installed: 21.3]
    - pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
  - pathspec [required: >=0.10.1, installed: 0.10.1]
  - pluggy [required: >=1.0.0, installed: 1.0.0]
  - tomli [required: >=1.2.2, installed: 2.0.1]
```

Let's monitor how the situation evolves, and whether other packages start requiring hatchling,
but for now I am not going to add four new packages into the trusted base set...
Pinning colorama to an older version for now.

related:
tartley/colorama#338
tartley/colorama#349
SomberNight added a commit to SomberNight/electrum that referenced this pull request Nov 4, 2022
or more like four:
```
$ pipdeptree
hatchling==1.11.1
  - editables [required: >=0.3, installed: 0.3]
  - packaging [required: >=21.3, installed: 21.3]
    - pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
  - pathspec [required: >=0.10.1, installed: 0.10.1]
  - pluggy [required: >=1.0.0, installed: 1.0.0]
  - tomli [required: >=1.2.2, installed: 2.0.1]
```

Let's monitor how the situation evolves, and whether other packages start requiring hatchling,
but for now I am not going to add four new packages into the trusted base set...
Pinning colorama to an older version for now.

related:
tartley/colorama#338
tartley/colorama#349
SomberNight added a commit to SomberNight/electrum that referenced this pull request Nov 4, 2022
or more like four:
```
$ pipdeptree
hatchling==1.11.1
  - editables [required: >=0.3, installed: 0.3]
  - packaging [required: >=21.3, installed: 21.3]
    - pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
  - pathspec [required: >=0.10.1, installed: 0.10.1]
  - pluggy [required: >=1.0.0, installed: 1.0.0]
  - tomli [required: >=1.2.2, installed: 2.0.1]
```

Let's monitor how the situation evolves, and whether other packages start requiring hatchling,
but for now I am not going to add four new packages into the trusted base set...
Pinning colorama to an older version for now.

related:
tartley/colorama#338
tartley/colorama#349
SomberNight added a commit to SomberNight/electrum that referenced this pull request Nov 4, 2022
or more like four:
```
$ pipdeptree
hatchling==1.11.1
  - editables [required: >=0.3, installed: 0.3]
  - packaging [required: >=21.3, installed: 21.3]
    - pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
  - pathspec [required: >=0.10.1, installed: 0.10.1]
  - pluggy [required: >=1.0.0, installed: 1.0.0]
  - tomli [required: >=1.2.2, installed: 2.0.1]
```

Let's monitor how the situation evolves, and whether other packages start requiring hatchling,
but for now I am not going to add four new packages into the trusted base set...
Pinning colorama to an older version for now.

related:
tartley/colorama#338
tartley/colorama#349
SomberNight added a commit to spesmilo/electrum that referenced this pull request Nov 5, 2022
or more like four:
```
$ pipdeptree
hatchling==1.11.1
  - editables [required: >=0.3, installed: 0.3]
  - packaging [required: >=21.3, installed: 21.3]
    - pyparsing [required: >=2.0.2,!=3.0.5, installed: 3.0.9]
  - pathspec [required: >=0.10.1, installed: 0.10.1]
  - pluggy [required: >=1.0.0, installed: 1.0.0]
  - tomli [required: >=1.2.2, installed: 2.0.1]
```

Let's monitor how the situation evolves, and whether other packages start requiring hatchling,
but for now I am not going to add four new packages into the trusted base set...
Pinning colorama to an older version for now.

related:
tartley/colorama#338
tartley/colorama#349
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 9, 2022
0.4.6 Current release
* tartley/colorama#139 Add alternative to 'init()',
  called 'just_fix_windows_console'. This fixes many longstanding problems
  with 'init', such as working incorrectly on modern Windows terminals, and
  wonkiness when init gets called multiple times. The intention is that it
  just makes all Windows terminals treat ANSI the same way as other terminals
  do. Many thanks the njsmith for fixing our messes.
* tartley/colorama#352 Support Windows 10's ANSI/VT
  console. This didn't exist when Colorama was created, and avoiding us
  causing havok there is long overdue. Thanks to segeviner for the initial
  approach, and to njsmith for getting it merged.
* tartley/colorama#338 Internal overhaul of package
  metadata declaration, which abolishes our use of the now heavily
  discouraged setuptools (and hence setup.py, setup.cfg and MANIFEST.in), in
  favor of hatchling (and hence pyproject.toml), generously contributed by
  ofek (author of hatchling). This includes dropping support Python3.5 and
  3.6, which are EOL, and were already dropped from setuptools, so this
  should not affect our users.
* tartley/colorama#353 Attention to detail award to
  LqdBcnAtWork for a spelling fix in demo06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.4.6 Get merged for 0.4.6. release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants