Snakemake Orchestrator API w/ Kubernetes

Setup

This project uses Pixi to manage packages and the environment.

# linux / mac os
curl -fsSL https://pixi.sh/install.sh | bash

To install the dependencies, run the following command:

pixi install

What this does is create a virtual environment and install the dependencies listed in the pyproject.toml file. By starting command line commands with pixi run, you can run the commands in the specific environment.

pixi run python -m orcestrator.core.main
# > Hello World!

This ensures that any other packages or dependencies can be synchronized across different environments, systems, and developers.

A feature of pixi is that it can be used to run common tasks. For example, the above command has been implemented as a pixi task in the pyproject.toml file.

To run the task, use the following command:

pixi run hello
# > ✨ Pixi task (default): python -m src.core.main
# > Hello, World!

TODO:: as development progresses, remove the hello_world function and add a more meaningful function to show the functionality of the project and PIXI in general.

Pre-commit

This project has pre-commit hooks enabled to ensure that the code is formatted correctly and that the tests pass before committing. To install the pre-commit hooks, run the following command:

pixi run pre-commit install

Then run:

pixi run pre-commit run --all-files

This will run the pre-commit hooks on all files in the repository. If there are any issues, they will be displayed in the terminal, and most often, the file will be automatically formatted.

From now on, anytime you commit, the pre-commit hooks will run automatically. If there are any issues, the commit will be rejected until the issues are resolved. Most often, you can just run git commit again and the issues will be fixed.

Pipeline Specification

This project is aims to orchestrate the deployment and execution of Snakemake workflows on Kubernetes clusters. As such, it makes some assumptions about the structure of the workflow and the environment in which it is executed.

All pipelines must follow the Pipeline Standards (WORK IN PROGRESS)

Semantic Versioning

For a detailed explanation of the release process, please refer to the GitFlow Process and Release Cycle.

This project uses Python Semantic Release for automatic versioning and changelog generation.

It is implemented via GitHub Actions and the configuration can be found in the pyproject.toml file.