Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.8 as minimum #16954

Merged
merged 12 commits into from
Nov 29, 2023
Merged

Python 3.8 as minimum #16954

merged 12 commits into from
Nov 29, 2023

Conversation

mr-c
Copy link
Contributor

@mr-c mr-c commented Nov 1, 2023

Walrus

Fixes: #16394

Major or minor packages bumps now that Python 3.8 is the minimum

Lint

Dev

Typecheck

If any of the above release have desired features, we should document their new minimum versions (which will also speed up future poetry runs).

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@nsoranzo
Copy link
Member

nsoranzo commented Nov 1, 2023

Bleach is deprecated since 2023-01-23 What to do?

It looks like that https://github.com/messense/nh3 can be used, but it's not urgent, I think.

@mr-c

This comment was marked as resolved.

@nsoranzo

This comment was marked as resolved.

@nsoranzo nsoranzo force-pushed the python3.8 branch 2 times, most recently from f312f63 to c7b8162 Compare November 24, 2023 14:08
@mr-c mr-c force-pushed the python3.8 branch 2 times, most recently from e00f08d to b18c47f Compare November 24, 2023 16:18
mr-c and others added 12 commits November 26, 2023 18:34
Fixes: galaxyproject#16394

Also upgrade code to Python 3.8 syntax with:

```
ack --type=python -f | grep -v '^lib/galaxy/schema/bco/\|^lib/galaxy/schema/drs/\|lib/tool_shed_client/schema/trs.*.py\|^tools/\|^.venv/\|^.tox/' | grep -v '^lib/galaxy/files/sources/\|^lib/galaxy/job_metrics/\|^lib/galaxy/objectstore/\|^lib/galaxy/tool_util/\|^lib/galaxy/util/' | xargs pyupgrade --py38-plus
```
Fixed by running:

```
ack --type=python -f | grep -v '^lib/galaxy/schema/bco/\|^lib/galaxy/schema/drs/\|^tools/\|^.venv/\|^.tox/' | grep -v '^lib/galaxy/files/sources/\|^lib/galaxy/job_metrics/\|^lib/galaxy/objectstore/\|^lib/galaxy/tool_util/\|^lib/galaxy/util/' | xargs auto-walrus
make format
```

Note that the directories for the packages listed in
`packages/packages_for_pulsar_by_dep_dag.txt`
were explictly excluded.
"Unpack is already enabled by default" in version 1.7.0
twill 3.2 replaced `requests` with `httpx`, and `httpx` has a default
timeout of 5 s on a request, while `requests` didn't enforce a timeout by
default.

Fix errors like the following in ToolShed tests, where resetting
repository metadata takes longer than 5 s:

```
self = <tool_shed.test.functional.test_0000_basic_repository_features.TestBasicRepositoryFeatures object at 0x7f28f99197c0>

    def test_0090_verify_repository_metadata(self):
        """Verify that resetting the metadata does not change it."""
        repository = self._get_repository_by_name_and_owner(repository_name, common.test_user_1_name)
>       self.verify_unchanged_repository_metadata(repository)

lib/tool_shed/test/functional/test_0000_basic_repository_features.py:264:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
lib/tool_shed/test/base/twilltestcase.py:2073: in verify_unchanged_repository_metadata
    self.reset_repository_metadata(repository)
lib/tool_shed/test/base/twilltestcase.py:1774: in reset_repository_metadata
    self.visit_url("/repository/reset_all_metadata", params=params)
lib/tool_shed/test/base/twilltestcase.py:845: in visit_url
    return self._browser.visit_url(url, allowed_codes=allowed_codes)
lib/tool_shed/test/base/twillbrowser.py:35: in visit_url
    return visit_url(url, allowed_codes=allowed_codes)
lib/tool_shed/test/base/twillbrowser.py:20: in visit_url
    new_url = tc.go(url)
.venv/lib/python3.8/site-packages/twill/commands.py:120: in go
    browser.go(url)
.venv/lib/python3.8/site-packages/twill/browser.py:192: in go
    self._journey("open", try_url)
.venv/lib/python3.8/site-packages/twill/browser.py:677: in _journey
    result = self._client.get(url)
.venv/lib/python3.8/site-packages/httpx/_client.py:1041: in get
    return self.request(
.venv/lib/python3.8/site-packages/httpx/_client.py:814: in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
.venv/lib/python3.8/site-packages/httpx/_client.py:901: in send
    response = self._send_handling_auth(
.venv/lib/python3.8/site-packages/httpx/_client.py:929: in _send_handling_auth
    response = self._send_handling_redirects(
.venv/lib/python3.8/site-packages/httpx/_client.py:966: in _send_handling_redirects
    response = self._send_single_request(request)
.venv/lib/python3.8/site-packages/httpx/_client.py:1002: in _send_single_request
    response = transport.handle_request(request)
.venv/lib/python3.8/site-packages/httpx/_transports/default.py:228: in handle_request
    resp = self._pool.handle_request(req)
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/contextlib.py:131: in __exit__
    self.gen.throw(type, value, traceback)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    @contextlib.contextmanager
    def map_httpcore_exceptions() -> typing.Iterator[None]:
        try:
            yield
        except Exception as exc:
            mapped_exc = None

            for from_exc, to_exc in HTTPCORE_EXC_MAP.items():
                if not isinstance(exc, from_exc):
                    continue
                # We want to map to the most specific exception we can find.
                # Eg if `exc` is an `httpcore.ReadTimeout`, we want to map to
                # `httpx.ReadTimeout`, not just `httpx.TimeoutException`.
                if mapped_exc is None or issubclass(to_exc, mapped_exc):
                    mapped_exc = to_exc

            if mapped_exc is None:  # pragma: no cover
                raise

            message = str(exc)
>           raise mapped_exc(message) from exc
E           httpx.ReadTimeout: timed out

.venv/lib/python3.8/site-packages/httpx/_transports/default.py:83: ReadTimeout
```
with `make update-requirements`
@dannon dannon merged commit 426e24f into galaxyproject:dev Nov 29, 2023
52 checks passed
@mr-c mr-c deleted the python3.8 branch November 29, 2023 07:00
@nsoranzo nsoranzo added highlight/admin Included in admin/dev release notes kind/refactoring cleanup or refactoring of existing code, no functional changes labels Nov 29, 2023
@nsoranzo
Copy link
Member

nsoranzo commented Nov 29, 2023

Thanks a lot @mr-c !

@mr-c
Copy link
Contributor Author

mr-c commented Nov 29, 2023

Thank you @nsoranzo and @dannon !

@galaxyproject galaxyproject deleted a comment from github-actions bot Nov 29, 2023
nsoranzo added a commit to nsoranzo/bioblend that referenced this pull request Dec 4, 2023
@mvdbeek mvdbeek modified the milestones: 23.2, 24.0 Dec 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Drop support for Python 3.7
4 participants