Developer Guide

Before getting started, check out the #bentoml-contributors channel in the BentoML community slack.

If you are interested in contributing to existing issues and feature requets, check out the good-first-issue and help-wanted issues list.

If you are interested in proposing a new feature, make sure to create a new feature request ticket here and share your proposal in the #bentoml-contributors slack channel for feedback.

Start Developing

with the Command Line

Make sure to have Git, pip, and Python3.7+ installed.

Optionally, make sure to have GNU Make available on your system if you aren't using a UNIX-based system for a better developer experience. If you don't want to use make then please refer to the Makefile for specific commands on a given make target.
Fork the BentoML project on GitHub.
Clone the source code from your fork of BentoML's GitHub repository:
```
git clone git@github.com:username/BentoML.git && cd BentoML
```
Add the BentoML upstream remote to your local BentoML clone:
```
git remote add upstream git@github.com:bentoml/BentoML.git
```

Configure git to pull from the upstream remote:

git switch main # ensure you're on the main branch
git branch --set-upstream-to=upstream/main

Install BentoML with pip in editable mode:
```
pip install -e .
```
This installs BentoML in an editable state. The changes you make will automatically be reflected without reinstalling BentoML.

Install the BentoML development requirements:

pip install -r ./requirements/dev-requirements.txt

Test the BentoML installation either with bash:
```
bentoml --version
```
or in a Python session:
```
import bentoml
print(bentoml.__version__)
```

with VS Code

Confirm that you have the following installed:
- Python3.7+
- VS Code with the Python and Pylance extensions
Fork the BentoML project on GitHub.
Clone the GitHub repository:
1. Open the command palette with Ctrl+Shift+P and type in 'clone'.
2. Select 'Git: Clone(Recursive)'.
3. Clone BentoML.
Add an BentoML upstream remote:
1. Open the command palette and enter 'add remote'.
2. Select 'Git: Add Remote'.
3. Press enter to select 'Add remote' from GitHub.
4. Enter https://github.com/bentoml/BentoML.git to select the BentoML repository.
5. Name your remote 'upstream'.
Pull from the BentoML upstream remote to your main branch:
1. Open the command palette and enter 'checkout'.
2. Select 'Git: Checkout to...'
3. Choose 'main' to switch to the main branch.
4. Open the command palette again and enter 'pull from'.
5. Click on 'Git: Pull from...'
6. Select 'upstream'.
Open a new terminal by clicking the Terminal dropdown at the top of the window, followed by the 'New Terminal' option. Next, add a virtual environment with this command:
```
python -m venv .venv
```
Click yes if a popup suggests switching to the virtual environment. Otherwise, go through these steps:
1. Open any python file in the directory.
2. Select the interpreter selector on the blue status bar at the bottom of the editor.
3. Switch to the path that includes .venv from the dropdown at the top.
Update your PowerShell execution policies. Win+x followed by the 'a' key opens the admin Windows PowerShell. Enter the following command to allow the virtual environment activation script to run:
```
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
```

Making Changes

using the Command Line

Make sure you're on the main branch.
```
git switch main
```
Use the git pull command to retrieve content from the BentoML Github repository.
```
git pull
```
Create a new branch and switch to it.
```
git switch -c my-new-branch-name
```
Make your changes!
Use the git add command to save the state of files you have changed.
```
git add <names of the files you have changed>
```
Commit your changes.
```
git commit
```
Push all changes to your fork on GitHub.
```
git push
```

using VS Code

Switch to the main branch:
1. Open the command palette with Ctrl+Shift+P.
2. Search for 'Git: Checkout to...'
3. Select 'main'.
Pull from the upstream remote:
1. Open the command palette.
2. Enter and select 'Git: Pull...'
3. Select 'upstream'.
Create and change to a new branch:
1. Type in 'Git: Create Branch...' in the command palette.
2. Enter a branch name.
Make your changes!
Stage all your changes:
1. Enter and select 'Git: Stage All Changes...' in the command palette.
Commit your changes:
1. Open the command palette and enter 'Git: Commit'.
Push your changes:
1. Enter and select 'Git: Push...' in the command palette.

Run BentoML with verbose/debug logging

To view internal debug loggings for development, set the BENTOML_DEBUG environment variable to TRUE:

export BENTOML_DEBUG=TRUE

And/or use the --verbose option when running bentoml CLI command, e.g.:

bentoml get IrisClassifier --verbose

Style check, auto-formatting, type-checking

formatter: black, isort, buf

linter: pylint, buf

type checker: pyright

We are using buf for formatting and linting of our proto files. Configuration can be found here. Currently, we are running buf with docker, hence we kindly ask our developers to have docker available. Docker installation can be found here.

Run linter/format script:

make format

make lint

Run type checker:

make type

Editing proto files

The proto files for the BentoML gRPC service are located under bentoml/grpc. The generated python files are not checked in the git repository, and are instead generated via this script. If you edit the proto files, make sure to run ./scripts/generate_grpc_stubs.sh to regenerate the proto stubs.

Deploy with your changes

Test test out your changes in an actual BentoML model deployment, you can create a new Bento with your custom BentoML source repo:

Install custom BentoML in editable mode. e.g.:
- git clone your bentoml fork
- pip install -e PATH_TO_THE_FORK
Set env var export BENTOML_BUNDLE_LOCAL_BUILD=True and export SETUPTOOLS_USE_DISTUTILS=stdlib
- make sure you have the latest setuptools installed: pip install -U setuptools
Build a new Bento with bentoml build in your project directory
The new Bento will include a wheel file built from the BentoML source, and bentoml containerize will install it to override the default BentoML installation in base image

Distribute a custom BentoML release for your team

If you want other team members to easily use your custom BentoML distribution, you may publish your branch to your fork of BentoML, and have your users install it this way:

pip install git+https://github.com/{YOUR_GITHUB_USERNAME}/bentoml@{YOUR_REVISION}

And in your BentoML projects' bentofile.yaml, force the Bento to install this distribution, e.g.:

service: "service:svc"
description: "file: ./README.md"
include:
  - "*.py"
python:
  packages:
    - pandas
    - git+https://github.com/{YOUR_GITHUB_USERNAME}/bentoml@{YOUR_REVISION}
docker:
  system_packages:
    - git

Testing

Make sure to install all test dependencies:

pip install -r requirements/tests-requirements.txt

Unit tests

You can run unit tests in two ways:

Run all unit tests directly with pytest:

# GIT_ROOT=$(git rev-parse --show-toplevel)
pytest tests/unit --cov=bentoml --cov-config="$GIT_ROOT"/pyproject.toml

Run all unit tests via ./scripts/ci/run_tests.sh:

./scripts/ci/run_tests.sh unit

# Or on UNIX-based system
make tests-unit

Integration tests

Run given tests after defining a target under scripts/ci/config.yml with run_tests.sh:

# example: run Keras TF1 integration tests
./scripts/ci/run_tests.sh keras_tf1

E2E tests

# example: run e2e tests to check for http general features
./scripts/ci/run_tests.sh http_server

Running the whole suite

To run the whole test suite, minus frameworks integration, you can use:

make tests-suite

Adding new test suite

If you are adding new ML framework support, it is recommended that you also add a separate test suite in our CI. Currently we are using GitHub Actions to manage our CI/CD workflow.

We recommend using nektos/act to run and test Actions locally.

The following tests script run_tests.sh can be used to run tests locally.

./scripts/ci/run_tests.sh -h
Running unit/integration tests with pytest and generate coverage reports. Make sure that given testcases is defined under ./scripts/ci/config.yml.

Usage:
  ./scripts/ci/run_tests.sh [-h|--help] [-v|--verbose] <target> <pytest_additional_arguments>

Flags:
  -h, --help            show this message
  -v, --verbose         set verbose scripts


If `pytest_additional_arguments` is given, the additional arguments will be passed to all of the tests run by the tests script.

Example:
  $ ./scripts/ci/run_tests.sh pytorch --run-gpus-tests --capture=tee-sys

All tests are then defined under config.yml where each field follows the following format:

<target>: &tmpl
  root_test_dir: "tests/integration/frameworks"
  is_dir: false
  override_name_or_path:
  dependencies: []
  external_scripts:
  type_tests: "integration"

By default, each of our frameworks tests files with the format: test_<frameworks>_impl.py. If is_dir set to true we will try to match the given <target> under root_test_dir to run tests from.

Keys	Type	Defintions
`root_test_dir`	`<str>`	root directory to run a given tests
`is_dir`	`<bool>`	whether `target` is a directory instead of a file
`override_name_or_path`	`<str>`	optional way to override a tests file name if doesn't match our convention
`dependencies`	`<List[str]>`	define additional dependencies required to run the tests, accepts `requirements.txt` format
`external_scripts`	`<str>`	optional shell scripts that can be run on top of `./scripts/ci/run_tests.sh` for given testsuite
`type_tests`	`<Literal["e2e","unit","integration"]>`	define type of tests for given `target`

When type_tests is set to e2e, ./scripts/ci/run_tests.sh will change current directory into the given root_test_dir, and will run the testsuite from there.

The reason why we encourage developers to use the scripts in CI is that under the hood when we use pytest, we will create a custom report for the given tests. This report can then be used as carryforward flags on codecov for consistent reporting.

Example:

# e2e tests
http:
  root_test_dir: "tests/e2e/bento_server_http"
  is_dir: true
  type_tests: "e2e"
  dependencies:
    - "Pillow"

# framework
pytorch_lightning:
  <<: *tmpl
  dependencies:
    - "pytorch-lightning"
    - "-f https://download.pytorch.org/whl/torch_stable.html"
    - "torch==1.9.0+cpu"
    - "torchvision==0.10.0+cpu"

Refer to config.yml for more examples.

Python tools ecosystem

Currently, BentoML is PEP518 compatible. We define package configuration via [pyproject.toml][https://github.com/bentoml/bentoml/blob/main/pyproject.toml].

Benchmark

BentoML has moved its benchmark to bentoml/benchmark.

Optional: git hooks

BentoML also provides git hooks that developers can install with:

make hooks

Creating Pull Requests on GitHub

Push changes to your fork and follow this article on how to create a pull request on github. Name your pull request with one of the following prefixes, e.g. "feat: add support for PyTorch". This is based on the Conventional Commits specification

feat: (new feature for the user, not a new feature for build script)
fix: (bug fix for the user, not a fix to a build script)
docs: (changes to the documentation)
style: (formatting, missing semicolons, etc; no production code change)
refactor: (refactoring production code, eg. renaming a variable)
perf: (code changes that improve performance)
test: (adding missing tests, refactoring tests; no production code change)
chore: (updating grunt tasks etc; no production code change)
build: (changes that affect the build system or external dependencies)
ci: (changes to configuration files and scripts)
revert: (reverts a previous commit)

Once your pull request is created, an automated test run will be triggered on your branch and the BentoML authors will be notified to review your code changes. Once tests are passed and a reviewer has signed off, we will merge your pull request.

Documentations

Refer to BentoML Documentation Guide for how to build and write docs.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEVELOPMENT.md

DEVELOPMENT.md

Developer Guide

Start Developing

with the Command Line

with VS Code

Making Changes

using the Command Line

using VS Code

Run BentoML with verbose/debug logging

Style check, auto-formatting, type-checking

Editing proto files

Deploy with your changes

Distribute a custom BentoML release for your team

Testing

Unit tests

Integration tests

E2E tests

Running the whole suite

Adding new test suite

Python tools ecosystem

Benchmark

Optional: git hooks

Creating Pull Requests on GitHub

Documentations

Files

DEVELOPMENT.md

Latest commit

History

DEVELOPMENT.md

File metadata and controls

Developer Guide

Start Developing

with the Command Line

with VS Code

Making Changes

using the Command Line

using VS Code

Run BentoML with verbose/debug logging

Style check, auto-formatting, type-checking

Editing proto files

Deploy with your changes

Distribute a custom BentoML release for your team

Testing

Unit tests

Integration tests

E2E tests

Running the whole suite

Adding new test suite

Python tools ecosystem

Benchmark

Optional: git hooks

Creating Pull Requests on GitHub

Documentations