Celery K8s

Run Celery Tasks as Kubernetes Jobs

Free software: GNU Affero General Public License v3+

Getting Started

Installation

Using Poetry (preferred)

poetry add celery-k8s

Using pip & PyPI.org

pip install celery-k8s

Using pip & GitHub

pip install git+https://github.com/the-wondersmith/celery-k8s.git

Using pip & A Local Copy Of The Repo

git clone https://github.com/the-wondersmith/celery-k8s.git
cd celery-k8s
pip install -e "$(pwd)"

Configure Celery

• Import celery_k8s in the same module where your Celery "app" is defined
• Ensure that celery_k8s is imported before any other Celery code is actually called (thus ensuring that the K8sTransport class is able to properly register itself with kombu)

"""My super awesome Celery app."""

# ...
from celery import Celery

# add the following import
import celery_k8s  # noqa: F401

# ensure the import is called *before*
# your Celery app is defined

app = Celery(
    "my-super-awesome-celery-app",
    # use 'k8s://' as your broker
    # url, everything after the
    # double slashes is ignored
    broker="k8s://celery-namespace",
)

Developing / Testing / Contributing

NOTE: Our preferred packaging and dependency manager is Poetry. Installation instructions can be found here.

Developing

Clone the repo and install the dependencies

$ git clone https://github.com/the-wondersmith/celery-k8s.git \
  && cd celery-k8s \
  && poetry install --sync

Optionally, if you do not have or prefer not to use Poetry, celery-k8s is fully PEP-517 compliant and can be installed directly by any PEP-517-compliant package manager.

$ cd celery-k8s \
  && pip install -e "$(pwd)"

TODO: Coming Soon™

Testing

To run the test suite:

$ poetry run pytest tests/

Contributing

TODO: Coming Soon™