Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Port latest changes from master to v5-dev (#5942)
* Fix serialization and deserialization of nested exception classes (#5717) * Fix #5597: chain priority (#5759) * adding `worker_process_shutdown` to __all__ (#5762) * Fix typo (#5769) * Reformat code. * Simplify commands to looking for celery worker processes (#5778) * update doc- celery supports storage list. (#5776) * Update introduction.rst * Update introduction.rst * Fail xfailed tests if the failure is unexpected. * Added integration coverage for link_error (#5373) * Added coverage for link_error. * Use pytest-rerunfailed plugin instead of rolling our own custom implementation. * Added link_error with retries. This currently fails. * Remove unused import. * Fix import on Python 2.7. * retries in link_error do not hang the worker anymore. * Run error callbacks eagerly when the task itself is run eagerly. Fixes #4899. * Adjust unit tests accordingly. * Grammar in documentation (#5780) * Grammar in documentation * Address review. * pypy 7.2 matrix (#5790) * removed extra slashes in CELERY_BROKER_URL (#5792) The Celery broker URL in settings.py had 2 slashes in the end which are not required and can be misleading. so I changed :- CELERY_BROKER_URL = 'amqp://guest:guest@localhost//' to CELERY_BROKER_URL = 'amqp://guest:guest@localhost' * Fix #5772 task_default_exchange & task_default_exchange_type not work (#5773) * Fix #5772 task_default_exchange & task_default_exchange_type not work * Add unit test: test_setting_default_exchange * Move default_exchange test to standalone class * Run integration suite with memcached results backend. (#5739) * Fix hanging forever when fetching results from a group(chain(group)) canvas. (#5744) PR #5739 uncovered multiple problems with the cache backend. This PR should resolve one of them. PR #5638 fixed the same test case for our async results backends that support native join. However, it did not fix the test case for sync results backends that support native join. * Fix regression in PR #5681. (#5753) See comment in the diff for details. * Grammatical fix to CONTRIBUTING.rst doc (#5794) * Fix #5734 Celery does not consider authMechanism on mongodb backend URLs (#5795) * Fix #5734 Celery does not consider authMechanism on mongodb backend URLs * Add unit test: test_get_connection_with_authmechanism * Add unit test: test_get_connection_with_authmechanism_no_username * Fix errors in Python 2.7 Remove "," after "**" operator * Revert "Revert "Revert "Added handle of SIGTERM in BaseTask in celery/task.py to prevent kill the task" (#5577)" (#5586)" (#5797) This reverts commit f79894e. * Add Python 3.8 Support (#5785) * Added Python 3.8 to the build matrix. * Ensure a supported tblib version is installed for Python 3.8 and above. In addition, modernize the relevant tests. * Workaround patching problem in test. * py 3.8 in clasifier * ubuntu bionic (#5799) * ubuntu bionic * fast finish * sync bumversion with pypi release * Dev.req (#5803) * update docker config * undo hardpin * devr req install from github master * update docker config (#5801) * update docker config * make dockerfile to install from github master dev branch by default * update download link * Isort. * Grammatical & punctuation fixes for CONTRIBUTING.rst document (#5804) * update dockerfile * switched to ubuntu bionic * update docker * keep it empty until we reconfigure it again with autopep8 * Fixed Dockerfile (#5809) * Update document CONTRIBUTING.rst & fix Dockerfile typo (#5813) * Added an issue template for minor releases. * reference gocelery Go Client/Server for Celery (#5815) * Add enterprise language (#5818) * Fix/correct minor doc typos (#5825) * Correct a small typo * Correct bad contributing documentation links * Preserve the task priority in case of a retry (#5820) * Preserve the task priority in case of a retry * Created test case for retried tasks with priority * Implement an integration test for retried tasks with priorities * bump kombu * basic changelog for celery 4.4.0rc4 * bump celery 4.4.0rc4 * events bootstep disabled if no events (#5807) * events bootstep disabled if no events * Added unit tests. * update bug report template * fixing ascii art to look nicer (#5831) * Only rerun flaky tests when failures can be intermediary. * Rename Changelog to Changelog.rst * The test_nested_group_chain test can run without native_join support. (#5838) * Run integration tests with Cassandra (#5834) * Run integration tests with Cassandra. * Configure cassandra result backend * Pre-create keyspace and table * Fix deprecation warning. * Fix path to cqlsh. * Increase connection timeout. * Wait until the cluster is available. * SQS - Reject on failure (#5843) * reject on failure * add documentation * test fix * test fix * test fix * Add a concurrency model with ThreadPoolExecutor (#5099) * Add a concurrency model with ThreadPoolExecutor * thread model test for pypy * Chain primitive's code example fix in canvas documentation (Regression PR#4444) (#5845) * Changed multi-line string (#5846) This string wasn't rendering properly and was printing the python statement too. Although the change isn't as pretty code-wise, it gets rid of an annoyance for the user. * Add auto expiry for DynamoDB backend (#5805) * Add auto expiry for DynamoDB backend This adds auto-expire support for the DynamoDB backend, via the DynamoDB Time to Live feature. * Require boto3>=1.9.178 for DynamoDB TTL support boto3 version 1.9.178 requires botocore>=1.12.178. botocore version 1.12.178 introduces support for the DynamoDB UpdateTimeToLive call. The UpdateTimeToLive call is used by the DynamoDB backend to enable TTL support on a newly created table. * Separate TTL handling from table creation Handle TTL enabling/disabling separately from the table get-or-create function. Improve handling of cases where the TTL is already set to the desired state. DynamoDB only allows a single TTL update action within a fairly long time window, so some problematic cases (changing the TTL attribute, enabling/disabling TTL when it was recently modified) will raise exceptions that have to be dealt with. * Handle older boto3 versions If the boto3 TTL methods are not found, log an informative error. If the user wants to enable TTL, raise an exception; if TTL should be disabled, simply return. * Improve logging - Handle exceptions by logging the error and re-raising - Log (level debug) when the desired TTL state is already in place * Add and use _has_ttl() convenience method Additional changes: - Handle exceptions when calling boto3's describe_time_to_live() - Fix test cases for missing TTL methods * Update ttl_seconds documentation * Log invalid TTL; catch and raise ValueError * Separate method _get_table_ttl_description * Separate ttl method validation function * Clarify tri-state TTL value * Improve test coverage * Fix minor typo in comment * Mark test as xfail when using the cache backend. (#5851) * [Fix #5436] Store extending result in all backends (#5661) * [Fix #5436] Store extending result in all backends * Fix sqlalchemy * More fixu * Fixing tests * removing not necessary import * Removing debug code * Removing debug code * Add tests for get_result_meta in base and database * Revert "Add auto expiry for DynamoDB backend (#5805)" (#5855) This reverts commit f7f5bcf. * Revert "Mark test as xfail when using the cache backend. (#5851)" (#5854) This reverts commit 1b303c2. * docs: Document Redis commands used by celery (#5853) * remove cache back end integrtion test. (#5856) * Fix a race condition when publishing a very large chord header (#5850) * Added a test case which artificially introduces a delay to group.save(). * Fix race condition by delaying the task only after saving the group. * update tox * Remove duplicate boto dependency. (#5858) * Revert "remove cache back end integrtion test. (#5856)" (#5859) This reverts commit e0ac7a1. * Revert "Revert "Add auto expiry for DynamoDB backend (#5805)" (#5855)" (#5857) This reverts commit 4ddc605. * Revert "update tox" This reverts commit 49427f5. * Fix the test_simple_chord_with_a_delay_in_group_save test. * Revert "Revert "Skip unsupported canvas when using the cache backend"" (#5860) * Revert "Revert "Mark test as xfail when using the cache backend. (#5851)" (#5854)" This reverts commit fc101c6. * Make the xfail condition stricter. * Fix the xfail condition. * Linters should use Python 3.8. * Move pypy unit tests to the correct stage. * Temporarily allow PyPy to fail since it is unavailable in Travis. * Remove unused variables. * Fix unused imports. * Fix pydocstyle errors in dynamodb. * Fix pydocstyle errors in redis backend. * bump kombu to 4.6.7 * celery 4.4.0rc5 changelog * celery 4.4.0rc5 * rm redundant code (#5864) * isort. * Document the threads task pool in the CLI. * Removed the paragraph about using librabbitmq. Refer to #5872 (#5873) * Task class definitions can have retry attributes (#5869) * autoretry_for * retry_kwargs * retry_backoff * retry_backoff_max * retry_jitter can now be defined as cls attributes. All of these can be overriden from the @task decorator #4684 * whatsnew in Celery 4.4 as per projects standard (#5817) * 4.4 whatsnew * update * update * Move old whatsnew to history. * Remove old news & fix markers. * Added a section notifying Python 3.4 has been dropped. * Added a note about ElasticSearch basic auth. * Added a note about being able to replace eagerly run tasks. * Update index. * Address comment. * Described boto3 version updates. * Fix heading. * More news. * Thread pool. * Add Django and Config changes * Bump version 4.4.0 * upate readme * Update docs regarding Redis Message Priorities (#5874) * Update docs regarding Redis Message Priorities * fixup! Update docs regarding Redis Message Priorities * Update 4.4.0 docs (#5875) * Update 4.4 release changelog * Update whatsnew-4.4 * Update tasks docs * Fix recent tasks doc file update (#5879) * Include renamed Changelog.rst in source releases. (#5880) Changelog.rst was renamed from Changelog in fd023ec but MANIFEST.in was not updated to include the new name. This fixes the file name so Changelog.rst will show up in future source releases again. * Reorganised project_urls and classifiers. (#5884) * Use safequote in SQS Getting Started doc (#5885) * Have appveyor build relevant versions of Python. (#5887) * Have appveyor build relevant and buildable versions of Python. * Appveyor is missing CI requirements to build. * Pin pycurl to version that will build with appveyor (because wheels files exist) * Restrict python 2.7 64 bit version of python-dateutil for parse. * Use is_alive instead of isAlive for Python 3.9 compatibility. (#5898) * Very minor tweak to commen to improve docs (#5900) As discussed here: https://stackoverflow.com/questions/58816271/celery-task-asyncresult-takes-task-id-but-is-documented-to-get-asyncresult-inst this comment seems to flow to a very confusing and misleading piece of documentation here: https://docs.celeryproject.org/en/latest/reference/celery.app.task.html#celery.app.task.Task.AsyncResult * Support configuring schema of a PostgreSQL database (#5910) * Support configuring schema of a PostgreSQL database * Add unit test * Remove blank line * Fix raise issue to make exception message more friendly (#5912) Signed-off-by: Chenyang Yan <memory.yancy@gmail.com> * Add progress for retry connections (#5915) This will show current retry progress so it will clear confusion about how many retries will be tried for connecting to broker. Closes #4556 * chg: change xrange to range (#5926) * update docs for json serializer and add note for int keys serialization (#5932) * fix indentation for note block in calling.rst (#5933) * Added links to other issue trackers. (#5939) * Add labels automatically for issues. (#5938) * Run pyupgrade. Co-authored-by: Michal Čihař <michal@cihar.com> Co-authored-by: ptitpoulpe <ptitpoulpe@ptitpoulpe.fr> Co-authored-by: Didi Bar-Zev <didi@hiredscore.com> Co-authored-by: Santos Solorzano <santosjavier22@gmail.com> Co-authored-by: manlix <manlix@yandex.ru> Co-authored-by: Jimmy <54828848+sckhg1367@users.noreply.github.com> Co-authored-by: Борис Верховский <boris.verk@gmail.com> Co-authored-by: Asif Saif Uddin <auvipy@gmail.com> Co-authored-by: Jainal Gosaliya <jainal09gosalia@gmail.com> Co-authored-by: gsfish <caoyu97@hotmail.com> Co-authored-by: Dipankar Achinta <di.punk.car19@gmail.com> Co-authored-by: Pengjie Song (宋鹏捷) <spengjie@sina.com> Co-authored-by: Chris Griffin <chris-griffin@users.noreply.github.com> Co-authored-by: Muhammad Hewedy <mhewedy@gmail.com> Co-authored-by: Blaine Bublitz <blaine.bublitz@gmail.com> Co-authored-by: Tamu <tamsanh@gmail.com> Co-authored-by: Erik Tews <erik@datenzone.de> Co-authored-by: abhinav nilaratna <anilaratna2@bloomberg.net> Co-authored-by: Wyatt Paul <wpaul@hearsaycorp.com> Co-authored-by: gal cohen <gal.nevis@gmail.com> Co-authored-by: as <alfred@huji.fr> Co-authored-by: Param Kapur <paramkapur2002@gmail.com> Co-authored-by: Sven Ulland <sven.ulland@gmail.com> Co-authored-by: Safwan Rahman <safwan.rahman15@gmail.com> Co-authored-by: Aissaoui Anouar <tobia@crossbone.cc> Co-authored-by: Neal Wang <qdzzyb2015@gmail.com> Co-authored-by: Alireza Amouzadeh <alireza@amouzadeh.net> Co-authored-by: Marcos Moyano <marcos@anue.biz> Co-authored-by: Stepan Henek <stepan+github@henek.name> Co-authored-by: Andrew Sklyarov <AndrewPix@users.noreply.github.com> Co-authored-by: Michael Fladischer <michael@fladi.at> Co-authored-by: Dejan Lekic <dejan.lekic@gmail.com> Co-authored-by: Yannick Schuchmann <yannick.schuchmann@googlemail.com> Co-authored-by: Matt Davis <matteius@gmail.com> Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com> Co-authored-by: Bernd Wechner <bernd-wechner@users.noreply.github.com> Co-authored-by: Sören Oldag <soeren_oldag@freenet.de> Co-authored-by: uddmorningsun <memory.yancy@gmail.com> Co-authored-by: Amar Fadil <34912365+marfgold1@users.noreply.github.com> Co-authored-by: woodenrobot <woodenrobot1993@gmail.com> Co-authored-by: Sardorbek Imomaliev <sardorbek.imomaliev@gmail.com>
- Loading branch information
42 people
committed
Feb 26, 2020
1 parent
c2f55f0
commit 7001102