Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move migrations to init container #356

Open
sergei-maertens opened this issue Jun 22, 2021 · 2 comments
Open

Move migrations to init container #356

sergei-maertens opened this issue Jun 22, 2021 · 2 comments
Assignees

Comments

@sergei-maertens
Copy link
Contributor

Since we're deploying on Kubernetes, we're making use of liveness and readiness probes. If these probes fail too often, the container is deemed unhealthy and is restarted.

Sometimes we have long-running data migrations, which are executed as part of the regular migrate step in the docker command script. These can run so long that the liveness probes fail so often, resulting in a container being restarted. If this happens over and over again, it can mean that the new version/container is never coming up.

One way to deal with this, would be to use the same container image in an init container which only runs the migrations. These containers don't have the "time restriction" that the "main" containers have.

No code changes are particularly needed for this, except for maybe a script/entrypoint similar to bin/docker_start.sh which stops at running the migrations, so we can adapt the infrastructure to that.

@sergei-maertens sergei-maertens self-assigned this Jun 22, 2021
@damm89
Copy link
Collaborator

damm89 commented Sep 27, 2022

@jordi-t heb jij enig idee hoe relevant dit nog is?

@jordi-t
Copy link
Collaborator

jordi-t commented Oct 15, 2022

@damm89 It's indeed good practice to run the migrations from an init-container. This is currently not the case, which in practice means that you'll bump into the issue described by Sergei when your migrations + application start-up run longer than 55 seconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants