Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update dev requirements #688

Merged
merged 8 commits into from Dec 9, 2022
Merged

Update dev requirements #688

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
2d24b56
Update dev requirements
willkg Dec 8, 2022
bf2f2ce
Fix lint workflow
willkg Dec 8, 2022
4d6eae5
Redo linting and testing in CI
willkg Dec 8, 2022
2e5e575
Rework testing and CI to be tox-based
willkg Dec 8, 2022
79fef51
Add Makefile to MANIFEST.in
willkg Dec 8, 2022
1c322ec
Remove scripts/run_tests.sh since we don't use it anymore
willkg Dec 8, 2022
cc1ce87
Fix CI for pypy
willkg Dec 8, 2022
92ecbf2
Update cache action to v3
willkg Dec 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
40 changes: 0 additions & 40 deletions .github/workflows/lint.yml
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions .github/workflows/test.yml
View file
Open in desktop
Expand Up @@ -40,6 +40,6 @@ jobs:
python -m pip install -U pip setuptools>=18.5
python -m pip install -r requirements-dev.txt

- name: Tests
- name: Run tests, linting, and everything else
shell: bash
run: ./scripts/run_tests.sh
run: tox
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching it to be tox-based where tox-gh-action will run the various tox environments depending on the Python version. Now we're running all the tox environments. Previously, we were only running the basic ones and skipping the tinycss2 ones and the build-no-lang ones in CI.

13 changes: 8 additions & 5 deletions MANIFEST.in
View file
Open in desktop
@@ -1,19 +1,22 @@
include CHANGES
include CONTRIBUTORS
include CONTRIBUTING.rst
include CODE_OF_CONDUCT.md
include SECURITY.md
include tox.ini
include CONTRIBUTING.rst
include CONTRIBUTORS
include LICENSE
include Makefile
include README.rst
include requirements-dev.txt
include requirements-flake8.txt
include SECURITY.md
include tox.ini

include docs/conf.py
include docs/Makefile
include docs/requirements.txt

include scripts/*

recursive-include bleach *.py *.json *.rst *.sh *.txt INSTALLER METADATA RECORD WHEEL LICENSE REQUESTED *.SHA256SUM
recursive-include bleach *.py *.rst *.sh *.txt INSTALLER METADATA RECORD WHEEL LICENSE REQUESTED *.SHA256SUM
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were no json files in the bleach/ directory, so I removed this bit.

recursive-include docs *.rst
recursive-include tests *.py *.test
recursive-include tests_website *.html *.py *.rst
33 changes: 33 additions & 0 deletions Makefile
View file
Open in desktop
@@ -0,0 +1,33 @@
DEFAULT_GOAL := help
PROJECT=bleach

.PHONY: help
help:
@echo "Available rules:"
@fgrep -h "##" Makefile | fgrep -v fgrep | sed 's/\(.*\):.*##/\1: /'

.PHONY: test
test: ## Run tests
tox

.PHONY: docs
docs: ## Build docs
tox -e py39-docs

.PHONY: lint
lint: ## Lint files
black --target-version=py37 --exclude=_vendor setup.py bleach/ tests/ tests_website/
tox -e py39-lint
tox -e py39-format-check

.PHONY: vendorverify
vendorverify: ## Verify vendored files
tox -e py39-vendorverify

.PHONY: clean
clean: ## Clean build artifacts
rm -rf build dist ${PROJECT}.egg-info .tox .pytest_cache
rm -rf docs/_build/*
rm -rf .eggs
find . -name __pycache__ | xargs rm -rf
find . -name '*.pyc' | xargs rm -rf
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs talk about a Makefile, but there wasn't one in the repo. This adds one.

19 changes: 9 additions & 10 deletions docs/dev.rst
View file
Open in desktop
Expand Up @@ -42,10 +42,10 @@ Run::

$ tox

That'll run Bleach tests in all the supported Python environments. Note
that you need the necessary Python binaries for them all to be tested.
That'll run Bleach tests in all the supported Python environments. Note that
you need the necessary Python binaries for them all to be tested.

Tests are run as github actions for test and pull request events.
Tests are run as GitHub actions for test and pull request events.


Release process
Expand All @@ -67,7 +67,7 @@ Release process

5. Verify correctness.

1. Run tests with tox::
1. Run linting, tests, and everything else with tox::

$ tox

Expand All @@ -85,8 +85,7 @@ Release process
exit with ``/tmp/vendor-test exists. Please remove.`` and the exit
code should be zero)::

$ ./scripts/run_tests.sh vendorverify
$ ./scripts/run_tests.sh vendorverify
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why this has to be run twice. I think it was an error.

$ make vendorverify

5. Run any additional tests to verify everything else works

Expand All @@ -102,21 +101,21 @@ Release process

9. Generate distribution files::

$ python setup.py sdist bdist_wheel
$ python -m build
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to use the build module now. https://pypi.org/project/build/


10. Sanity check the release contents and sizes::

$ ls -lh dist/* # file sizes should be similar
$ tar tvzf dist/bleach-${VERSION}.tar.gz
$ unzip -v dist/bleach-${VERSION}-py2.py3-none-any.whl

11. Upload them to PyPI::
11. Using a PyPI API token, upload dist files to PyPI::

$ twine upload dist/*
$ twine upload -r [REPO] dist/*

12. Push the new tag::

$ git push --tags official main
$ git push --tags [REMOTE] main

That will push the release to PyPI.

Expand Down
20 changes: 10 additions & 10 deletions requirements-dev.txt
View file
Open in desktop
@@ -1,12 +1,12 @@
-e .

black==22.3.0; implementation_name == 'cpython'
build==0.8.0
flake8==4.0.1
mypy==0.961; implementation_name =='cpython'
pip-tools==6.6.2
pytest==7.1.2
Sphinx==4.3.2
tox==3.25.0
twine==4.0.1
wheel==0.37.1
black==22.10.0; implementation_name == 'cpython'
build==0.9.0
mypy==0.991; implementation_name =='cpython'
pip-tools==6.11.0
pytest==7.2.0
Sphinx==5.3.0
tox==3.27.1
tox-gh-actions==2.11.0
twine==4.0.2
wheel==0.38.4
2 changes: 2 additions & 0 deletions requirements-flake8.txt
View file
Open in desktop
@@ -0,0 +1,2 @@
# Requirements for running flake8
flake8==6.0.0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flake8 has dependency conflicts with other packages, so I moved it into its own environment.

41 changes: 0 additions & 41 deletions scripts/run_tests.sh
View file
Open in desktop

This file was deleted.

47 changes: 28 additions & 19 deletions tox.ini
View file
Open in desktop
Expand Up @@ -5,10 +5,19 @@ envlist =
py{37,38,39,310,311,py3}
py{37,38,39,310,311,py3}-tinycss2
py{37,38,39,310,311}-build-no-lang
docs
format-check
lint
vendorverify
py39-docs
py39-format-check
py39-lint
py39-vendorverify
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the py39 encodes which Python version to run it in.


[gh-actions]
python =
3.7: py37
3.8: py38
3.9: py39
3.10: py310
3.11: py311
py3: pypy-3.8
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this isn't working correctly.

Run actions/setup-python@v4
Successfully set up PyPy 7.3.9
 with Python (3.8.13)

It should be installing PyPy 3.8 not PyPy 7.3.9 using Python 3.8.13. Either that or I'm misreading the output.


[testenv]
deps = -rrequirements-dev.txt
Expand Down Expand Up @@ -82,32 +91,32 @@ setenv =
commands =
python setup.py build

[testenv:lint]
basepython = python3.9
changedir = scripts
deps = -rrequirements-dev.txt
[testenv:py39-lint]
changedir = {toxinidir}
deps = -rrequirements-flake8.txt
platform = linux
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding platform = linux here means this will only run when the platform is linux. This prevents it from running on windows and macos--we don't need to run these utility environments on those environments.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably also means that devs who are using Windows and macOS can't run these environments. I'm the only dev at the moment and I'm using Linux this week, so I'm going to ignore this for now, but I'll write it up as an issue.

Copy link
Member Author

@willkg willkg Dec 8, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[grumble] I should fix it now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For things like vendorverify and docs, we want to run them in CI only on Linux, but when not in CI on whatever platform. I can't figure out a way to do that that doesn't involve writing a bash script or something else that's platform-dependent, so I'm going to skip it for now.

commands =
./run_tests.sh lint
flake8 setup.py tests/ bleach/ tests_website/

[testenv:vendorverify]
basepython = python3.9
changedir = scripts
[testenv:py39-vendorverify]
changedir = {toxinidir}
deps = -rrequirements-dev.txt
platform = linux
commands =
./run_tests.sh vendorverify
{toxinidir}/scripts/vendor_verify.sh

[testenv:format-check]
basepython = python3.9
changedir = scripts
[testenv:py39-format-check]
changedir = {toxinidir}
deps = -rrequirements-dev.txt
platform = linux
commands =
./run_tests.sh format-check
black --target-version=py37 --check --diff --exclude=_vendor setup.py bleach/ tests/ tests_website/

[testenv:docs]
basepython = python3.9
[testenv:py39-docs]
changedir = docs
deps = -rrequirements-dev.txt
extras = css
platform = linux
commands =
sphinx-build -b html -d {envtmpdir}/doctrees . {envtmpdir}/html
sphinx-build -b doctest -d {envtmpdir}/doctrees . {envtmpdir}/doctest