Skip to content

snakepacker/python

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A simple way to build a Python project

This repository provides and demonstrates a way to pack python package into a compact Docker image, based on modern Ubuntu Jammy operation system.

Awailable images

This project is available both in the official docker repository and also on the Github Container Registry (ghcr.io).

ghcr.io docker.io
ghcr.io/snakepacker/python/all snakepacker/python:all
ghcr.io/snakepacker/python/all-pillow snakepacker/python:all-pillow
ghcr.io/snakepacker/python/3.13 snakepacker/python:3.13
ghcr.io/snakepacker/python/3.13-pillow snakepacker/python:3.13-pillow
ghcr.io/snakepacker/python/3.12 snakepacker/python:3.12
ghcr.io/snakepacker/python/3.12-pillow snakepacker/python:3.12-pillow
ghcr.io/snakepacker/python/3.11 snakepacker/python:3.11
ghcr.io/snakepacker/python/3.11-pillow snakepacker/python:3.11-pillow
ghcr.io/snakepacker/python/3.10 snakepacker/python:3.10
ghcr.io/snakepacker/python/3.10-pillow snakepacker/python:3.10-pillow
ghcr.io/snakepacker/python/3.9 snakepacker/python:3.9
ghcr.io/snakepacker/python/3.9-pillow snakepacker/python:3.9-pillow
ghcr.io/snakepacker/python/3.8 snakepacker/python:3.8
ghcr.io/snakepacker/python/3.8-pillow snakepacker/python:3.8-pillow
ghcr.io/snakepacker/python/pylama snakepacker/python:pylama
ghcr.io/snakepacker/python/pylava snakepacker/python:pylava
ghcr.io/snakepacker/python/ipython snakepacker/python:ipython
ghcr.io/snakepacker/python/certbot snakepacker/python:certbot
ghcr.io/snakepacker/python/black snakepacker/python:black
ghcr.io/snakepacker/python/gray snakepacker/python:gray
ghcr.io/snakepacker/python/ruff snakepacker/python:ruff
ghcr.io/snakepacker/python/jupyterlab snakepacker/python:jupyterlab
ghcr.io/snakepacker/python/base snakepacker/python:base

Image descriptions

The images according to their purpose and features:

Tag Info Purpose Features
all build stage all available python versions, libpython headers and compiler
all-pillow build stage all available python versions, libpython headers, graphics libraries headers and compiler
3.13 target stage pure python 3.13
3.12 target stage pure python 3.12
3.11 target stage pure python 3.11
3.10 target stage pure python 3.10
3.9 target stage pure python 3.9
3.8 target stage pure python 3.8
3.13-pillow target stage pure python 3.13 with graphics libraries binaries
3.12-pillow target stage pure python 3.12 with graphics libraries binaries
3.11-pillow target stage pure python 3.11 with graphics libraries binaries
3.10-pillow target stage pure python 3.10 with graphics libraries binaries
3.9-pillow target stage pure python 3.9 with graphics libraries binaries
3.8-pillow target stage pure python 3.8 with graphics libraries binaries
pylama ready to use pylama application image (useful for CI)
pylava ready to use pylava application image (useful for CI)
ipython ready to use ipython application image
certbot ready to use certbot application image
black ready to use black application image (useful for CI)
gray ready to use gray application image (useful for CI)
ruff ready to use ruff linter image (useful for CI)
jupyterlab ready to use jupyterlab image
base common layers

Concept

The main idea of this method is to build a virtualenv for your package using heavy full-powered image (e.g. ghcr.io/snakepacker/python:all, that contains all necessary headers, libraries, compiler, etc.), and then copy it into thin base image with suitable Python version.

Reasons

Why so complex? You could just COPY directory with your python project into Docker container, and for the first point of view this seems to be reasonable.

But just copying directory with python project cause several problems:

  • Generated on different operating system .pyc files can be put into Docker image accidentally. Thus, python would try to rewrite .pyc with correct ones each time when Docker image would be started. If you would run Docker image in read-only mode - your application would break.

  • Large possibility that you would also pack garbage files: pytest and tox cache, developer's virtualenv and other files, that just increate the size of the resulting image.

  • No explicit entrypoint. It is not obvious what commands end user is able to run (we hope you've implemented -h or --help arguments).

  • By default, tox interprets your package as python module, e.g. it tries to run pip install . when preparing environment.

Yes, of course, you can solve all of those problems using hacks, specific settings, .dockeridnore file, and other tricks. But it would be non-intuitive and non-obvious for your users.

So, we recommend to spend a little more time and pack your package carefully, so your users would run it with pleasure.

Example

For example, you may build the jupyter notebook. Just create a Dockerfile with the following content:

#################################################################
####################### BUILD STAGE #############################
#################################################################
# This image contains:
# 1. All the Python versions
# 2. required python headers
# 3. C compiler and developer tools
FROM ghcr.io/snakepacker/python:all as builder

# Create virtualenv on python 3.10
# Target folder should be the same on the build stage and on the target stage
RUN python3.10 -m venv /usr/share/python3/app

# Install target package
RUN /usr/share/python3/app/bin/pip install -U pip 'ipython[notebook]'

# Will be find required system libraries and their packages
RUN find-libdeps /usr/share/python3/app > /usr/share/python3/app/pkgdeps.txt

#################################################################
####################### TARGET STAGE ############################
#################################################################
# Use the image version used on the build stage
FROM ghcr.io/snakepacker/python:3.10

# Copy virtualenv to the target image
COPY --from=builder /usr/share/python3/app /usr/share/python3/app

# Install the required library packages
RUN xargs -ra /usr/share/python3/app/pkgdeps.txt apt-install

# Create a symlink to the target binary (just for convenience)
RUN ln -snf /usr/share/python3/app/bin/ipython /usr/bin/

CMD ["ipython"]

And just build this:

docker build -t ipython .

Useful tools

All images contain ready to use and simple wrappers for easy image building.

apt-install

Pretty simple bash script. The main purpose is removing the apt cache and temporary files after installation when you want to install something through apt-get install.

Otherwise, you have to write something like this

apt-get update && \
apt-get install -y tcpdump && \
rm -fr /var/lib/apt/lists /var/lib/cache/* /var/log/*

It might be replaced like this:

apt-install tcpdump

wait-for-port

Python script which waits for availability one or multiple TCP ports. It's very useful for tests and with docker-compose.

wait-for-port --period=0.5 --timeout=600 postgres:5432 pgbouncer:6432 && python myscript.py

Or shorter (values from previous example are defaults):

wait-for-port postgres:5432 pgbouncer:6432 && python myscript.py

This script will be trying to make connections to passed endpoints until timeout would be reached or endpoints stay connectable.

find-libdeps

A shell script which find binary *.so files and resolve required system package for install library dependencies.

Save required packages

find-libdeps /usr/share/python3/app > /usr/share/python3/app/pkgdeps.txt

Install saved packages

xargs -ra /usr/share/python3/app/pkgdeps.txt apt-install