Add Docker Compose Support

[jalessio]

This PR is primarily intended as a way of concretely demonstrating potential improvements / responding to reviewers in PR #2272 by @fazlerabbi37.

Notable changes

• Reorganized the directory structure based on experience using this structure successfully on several other projects.
• Use a postgis Docker image instead of postgres since we definitely need PostGIS.
• Combine RUN commands to reduce the number of layers in Docker images.
• Don't compile Ruby or install gems in the db image. We just need a single header file in order to build libpgosm.so.
• Turns out the docker_postgres.sh script which installs the OSM-specific Postgres extensions wasn't actually being run because of a missing slash. Fixed that _(and renamed the file to include openstreetmap in the name to make its purpose clear).
• Switched from ruby:2.5-slim to ruby:2.5 since we're installing so much stuff anyway (ok, fine - it's mostly because it has curl installed by default which saves some bootstrapping trouble 😛). I'm happy to be talked out of this one.

TODO

• Review and update DOCKER.md. I haven't touched this other than moving it to the top-level directory.
• I wonder if just using an Ubuntu image for web would be easier since we'd get sane Ruby and NodeJS defaults? Additionally, we could just copy/paste the current instructions from INSTALL.md and what is used for Vagrant instead of inventing a separate set of installation instructions.
• Get feedback on the addition of a Makefile at the top level for running docker-compose commands (doesn't seem like that great of an idea).
• Deal with Postgres persistence. Instructions for this included at https://hub.docker.com/_/postgres
• "create dedicated user & group to run the server, don't just use root." per @mmd-osm in Add docker support #2272 (comment)
• Get tests running
• Get tests PASSING locally
• Clean up commit history

[simonpoole]

* Use a `postgis` Docker image instead of `postgres` since we definitely need PostGIS.

What's the rationale for that? Given that the rails-port definitely doesn't require PostGIS.

[mmd-osm]

With #2383 in place, most of docker/postgres/Dockerfile and docker/postgres/openstreetmap-postgres-init.sh will no longer be needed.

[jalessio]

* Use a `postgis` Docker image instead of `postgres` since we definitely need PostGIS.

What's the rationale for that? Given that the rails-port definitely doesn't require PostGIS.

@simonpoole During the course of working on this yesterday I somehow managed to convince myself that PostGIS was in fact needed... but you're right, it's not!

I reverted back to Postgres in 3e5b8ce.

[jalessio]

With #2383 in place, most of docker/postgres/Dockerfile and docker/postgres/openstreetmap-postgres-init.sh will no longer be needed.

@mmd-osm PR #2383 just merged by @tomhughes so I removed the additional PostgreSQL functions from the db Docker image. After rebase from master the changes are in commit b0d9d03.

[zerebubuth]

Hi! Looks great so far. I was able to get a couple of containers up using your instructions. Please let me know if there's anything I can do to help 😄

[jalessio]

Hi! Looks great so far. I was able to get a couple of containers up using your instructions. Please let me know if there's anything I can do to help 😄

@zerebubuth Any thoughts on how to handle persistence of Postgres data? There is some background information in the "Where to Store Data" section here: https://hub.docker.com/_/postgres

I personally prefer the "Let Docker manage the storage" approach but I have found that people find that confusing since their data vanishes when you run docker-compose down. For me, that's a feature but it seems to be surprising to people and can lead to unexpected data destruction which nobody likes.

Because of that I'm inclined to do with the "Create a data directory on the host system (outside the container)" option. I find it weird to have a PGDATA directory on my host system that is only used in a container, but I suspect I'm in the minority.

Thoughts?

btw, right now it's using the "Let Docker manage the storage" approach

[Firefishy]

Use a docker volume (managed via docker-compose.yml) for postgres persistance. In "production" one would not use postgres in the same way. A volume is not automatically removed on a docker-compose down

[Firefishy]

Quick example of docker-compose.yml volume with postgres: https://gist.github.com/Firefishy/c6ba34c1eab11d5c3426137011d4869a

[mmd-osm]

I wonder why imagemagick is still in the list. GPX preview image generation was moved to gd2-ffij earlier this year, and I'm not aware of other parts of the website depending on imagemagick.

[jalessio]

The imagemagick requirements are still listed in the "Minimum Requirements" section of INSTALL.md so if they are dropped here they should probably be dropped there as well.

https://github.com/openstreetmap/openstreetmap-website/blob/master/INSTALL.md#minimum-requirements

[mmd-osm]

Right, it seems the INSTALL.md file also needs a bit of cleanup.

[jalessio]

@mmd-osm is there anything in the test suite which would verify whether this dependency is needed? If not, any suggestions on a suitable test I could add?

[mmd-osm]

I thought we got rid of imagemagick, but it seems that 6c20244 introduced mini_magick, which depends on imagemagick. This is only used to resize avatar images and might be a bit heavyweight for the task at hand. Maybe @tomhughes knows more.

[tomhughes]

Well I don't believe there is any other supported approach. It only uses it via the command line though so there is no need for the development package.

[mmd-osm]

This one should also be reviewed, not sure it is still needed.

[jalessio]

Use a docker volume (managed via docker-compose.yml) for postgres persistance. In "production" one would not use postgres in the same way. A volume is not automatically removed on a docker-compose down

I switched over to use a Docker volume for Postgres in 8d84170

[jalessio]

I'm closing this PR out and replacing it with PR #2409, which has a significantly cleaner commit history. For outstanding issues in the comments of this PR I propose we move them to Issues and address them in follow-on PRs if/when #2409 merges.

[tomhughes]

It doesn't matter but if you're not aware you don't have to open a new PR just because you've rewritten the branch history - you can just force push and the existing PR will get the new clean history.