Docker Workflow for Development

Instructions for setting up the development environment

NOTE: Development Docker composition is not suitable for Production use.

This is a Development Docker composition of Specify 7

It has nice features like hot reloading and debugging support

However, it is not suited for production use due to memory leaks and security issues

Though, you can use it for evaluation of Specify 7

Members of Specify Collections Consortium get access to production-ready Docker compositions. Please contact us at support@specifysoftware.org if you would like to get access

This tutorial assumes basic knowledge of Docker and Docker Compose.

You can read these tutorials on Docker and Docker Compose to get started.

Installation

Install Docker and Docker compose:

On macOS (assuming brew is installed):
```
brew install --cask docker
```
On Linux:
```
sudo apt-get install \
  apt-transport-https \
  ca-certificates \
  curl \
  gnupg \
  lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker
sudo systemctl enable docker.service
sudo systemctl enable containerd.service
curl -L https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh | sh
docker plugin install --grant-all-permissions vieux/sshfs
```
On Windows:

a. Install and run Windows Subsystem For Linux (WSL) b. On your Linux Subsystem, add the following to /etc/wsl.conf (If the file does not exist, create it)
```
[boot]
systemd=true
```
c. Repeat the steps from the previous On Linux section d. To run Specify, port 80 will need to be open on your computer. By default windows already has a service listening to port 80. (You can check which services are listening on which ports with netstat -aon | findstr <port>) As the Internet Information Services feature listens on port 80, there are two options
- Disable Windows Internet Information Services
  - In the Start Menu, search for "features"
  - In the Windows Features window, find and uncheck "Internet Information Services"
  - Click OK
- Change the port of the default home page
  - With Internet Information Services enabled, search for Internet Information Services Manager in the start menu
  - Under the Actions toolbar (located on the right), click on View Sites
  - Click on "Default Web Site". Under the Actions toolbar, click on "Bindings..." under Edit Site
  - Change the port of http to an unused port on your computer (If you need a port, 3200 is available by default)

Checkout the repository (assuming git is installed)

git clone https://github.com/specify/specify7/
cd specify7

Put a MySQL dump of the database you want to use into the ./seed-database folder (name of the file does not matter)
Build the images and run the containers:
```
docker compose up --build
# or, if you are on a legacy version:
docker-compose up --build
```
On subsequent runs, you don't have to provide the "--build" argument, unless the list of front-end or back-end dependencies has changed.
Make the code changes you want. Both the backend and the frontend would be updated in real-time.

If you want to receive audible notifications when the rebuild process is finished, here are the instructions for setting that up

Afterward, you can stop the containers by pressing Ctrl+C or typing docker compose down in a separate terminal window.

Changing the schema version

Open the ./docker-compose.yml file

Find the following lines:

specify6:                                                             
  image: specifyconsortium/specify6-service:6.8.01

Replace "6.8.01" with the desired version. Full list of available versions
Rebuild the containers:
```
docker compose up --build
```

Changing the branch

Checkout the branch you want:
```
git switch production
```
Full list of branches
Get the latest changes from GitHub:
```
git pull
```
If the branch you are switcing from and the new one have the same dependencies, you don't have to stop the containers and rebuild them. Simply run docker compose up if containers are not yet running.

If there were some dependency changes, or you are not sure, run docker compose up --build to rebuild the containers.

Changing the database

Put the database dump into ./seed-database. More information
SSH into the MariaDB container. You can do so by pressing the >_ button next to the container name in Docker Dashboard, or by executing the following command in the terminal:
```
# Assumming your container is named "specify7-mariadb-1":
docker exec -it specify7-mariadb-1 env TERM=xterm-256color /bin/sh
```

Run the following commands in order:

# Navigate into the mounted volume in the container that holds the database dump
cd doc*
# List the files in that directory. If everything is correct, you should see the name of your database dump file
ls
# Connect to the database
mysql -uroot -proot

# Delete current database from the MariaDB server. This will delete all data. If you need to preserver data, make a database dump using `mysqldump` first
DROP DATABASE `specify`;
# Create a blank database
CREATE DATABASE `specify`;
# Select the database
USE `specify`;
# Import the database from the dump. Assuming your dump file is called "dump.sql":
SOURCE "./dump.sql";
# Disconnect from MySQL
EXIT;

# Disconnect from the container
exit;

Run database migrations. This step is needed only once for a database before it is used with Specify 7 for the first time or after a Specify 7 update.
```
docker exec -it specify7-specify7-1 ve/bin/python manage.py migrate
```

After this, refresh the "Specify 7" window in your browser and log in again if prompted.

A note of caution:

Make sure there is at most one file with the .sql extension in your seed-database directory at any given point. If you need to store multiple databases there just in case, than rename the files to not end with .sql (i.e, change extension to be .sql_)

This is because every time docker starts up, it will execute all of those SQL files. If you have more than one, they will conflict and may lead to unexpected circumstances

Connecting to database from outside the container

Open ./docker-compose.yml
Find the section that begins with mariadb:

Add a new line:

mysql:
   # Add these two lines line:
   ports:
   - '127.0.0.1:3306:3306'

Rebuild the containers:
```
docker compose up --build
```

This would make database accessible at localhost with port 3306.

Running the server on a remote machine

A common use case for using Docker is having the containers run on a dedicated machine. This frees up resources on your laptop for other tasks.

This can be atchived in Docker by using code syncing and Docker contexts.

Configuring Docker Contexts

Docker Contexts allows to manage remote Docker servers.

Configuring Code Sync

We need to update the remove server in real time as soon as any code changes are made. mirror is designed just for that task.

Instructions for setting up Code Sync with Docker

The commands I used:

# Add this to your .zshrc file:
alias sync_client='docker run --rm --init -it -u $(id -u):$(id -g) -v $(pwd):/data \
  quay.io/stephenh/mirror client \
  --local-root /data \
  --remote-root /data \
  --host 192.168.50.165 \
  --include "./seed-database" --exclude "./idea" --exclude "nohup.out"'
alias sync_server='docker run \
  --rm --init -it -u $(id -u):$(id -g) -v $(pwd):/data -p 49172:49172 \
  quay.io/stephenh/mirror server'

Then just run sync_server on the server and sync_client on the client machine.

Overwriting default preferences

The easiest way currently to override Specify settings that aren't connected to an environment variable in a Docker container is:

Copy the local_specify_settings.py out of the container docker-compose exec specify7 cat /opt/specify7/settings/local_specify_settings.py > override_specify_settings.py
Make the desired changes in override_specify_settings.py
Mount the modified settings file on top of the file in the container by adding - "./override_specify_settings.py:/opt/specify7/settings/local_specify_settings.py" to the volumes section of specify7 service.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly