Update Dockerfile configuration #195
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This helps ensure that when a Docker image is built the expacted source image is used regardless of what repository is configured as the default on the host system. It also makes our Dockerfiles more seamlessly convertible to using the GitHub Container Registry or any other Open Container Initiative (OCI) compatible registry.
Use the full tag that includes the Alpine Linux version to ensure the pulled image is always the same.
Use the full path for source container images
Instead of downloading the source archive, extracting it, and then installing it with pip we instead just let pip directly install the package.
Since we are now installing cisagov/skeleton-python-library directly with pip we no longer need these OS packages.
Now that we are not overwriting the internal Python package file the text we look for must match what is output by default. The Docker Compose secret configuration is left in place to continue to serve as an example and to be leveraged for a future update to cisagov/skeleton-python-library that can provide similar functionality to what was removed in this project.
…hon-library_directly Install cisagov/skeleton-python-library directly with `pip`
We should not blindly upgrade all pre-installed packages. This can create inconsistent build results due to changes in installed versions.
Pin the versions of the pip, setuptools, and wheel packages that are installed.
We can move this instruction to the end of the Dockerfile now that we are no longer working with files in the Docker container when building.
…eatable Pin Python package versions and improve build repeatability
Instead of relying on `pip3` being on the PATH we instead call the module through the Python executable. This ensures that the `pip` being used is in the same environment as the `python3` being used.
Using a virtual environment is a Python best practice. We also consolidate all of the Python dependency installation steps into a single RUN instruction. This ensures that Python setup is cached in one layer and mirrors the logical organization of this being a single step.
Since we cannot use long options on Alpine Linux we should explain what the short options we are using do. I also changed the order of options so that they are in alphabetical order. Co-authored-by: Shane Frasier <jeremy.frasier@gwe.cisa.dhs.gov>
Use a Python virtual environment
This configuration includes a Pipfile configuration file and the generated Pipfile.lock file that pins to specific versions for the Python dependencies for this project. This will help us ensure repeatable builds. The pipenv package is added as a developmental requirement to support these files.
Now that we have a pipenv configuration we will use it to install the Python dependencies for the image. The `build` workflow is updated to no longer pass the VERSION build argument in line with this change.
Switch to using a multi-stage build in the Dockerfile. This reduces image size since pipenv and its dependencices are not needed in the final image. It also ensures that the system Python environment is unmodified.
Install the core Python packages (pip, setuptools, and wheel) into the system Python environment before installing pipenv. This keeps things consistent with our usual approach to Python environments.
The comment references a command that is no longer being run. Co-authored-by: Shane Frasier <jeremy.frasier@gwe.cisa.dhs.gov>
Co-authored-by: dav3r <david.redmin@trio.dhs.gov>
…tion Install Python dependencies with `pipenv`
Change the tags used in the table to match the version of the project. Previously "1.2.3" was used as an example version but there is no reason not to use the real version of the image.
Update the README
Update the Dockerfile and testing to accommodate changes in the new version.
Update dependencies
mcdonnnj
added
breaking change
mcdonnnj
requested review from
dav3r,
felddy,
jasonodoom and
jsf9k
as code owners
jsf9k
approved these changes
Mar 6, 2024
dav3r
reviewed
Mar 7, 2024
dav3r
reviewed
Mar 7, 2024
dav3r
approved these changes
Mar 7, 2024
Co-authored-by: dav3r <david.redmin@trio.dhs.gov>
jsf9k
approved these changes
Mar 14, 2024
felddy
approved these changes
May 28, 2024
| # because the ln binary in Alpine Linux does not support long flags. The -f instructs | ||
| # ln to remove the existing file and the -s instructs ln to create a symbolic link. | ||
| ### | ||
| COPY --from=compile-stage --chown=${CISA_USER}:${CISA_GROUP} ${VIRTUAL_ENV} ${VIRTUAL_ENV} |
There was a problem hiding this comment.
TIL you can chown during a COPY in a Dockerfile. Bravo.
| # cache. This results in a smaller final image, at the cost of | ||
| # slightly longer install times. | ||
| ### | ||
| RUN python3 -m pip install --no-cache-dir --upgrade \ |
There was a problem hiding this comment.
Since this is the compile-stage of the build, do we need to optimize for layer count. i.e., does it still make sense to concatenate each command with && versus having discreet RUNs.
I ask because I think you will get a more localized error statement when something fails when the command is decomposed. There may be some other gains to be had as well with caching, and parallelization.
| && pip3 install --no-cache-dir --requirement requirements.txt \ | ||
| && ln -snf /run/secrets/quote.txt src/example/data/secret.txt \ | ||
| && rm sourcecode.tgz | ||
| # Note that we symlink the Python binary in the venv to the system-wide Python so that |
There was a problem hiding this comment.
Should this comment be moved down to after the COPY but before the RUN ln ... line?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗣 Description
This pull request represents a rework of this project's Docker configuration. It includes the following pull requests:
pip#188pipenv#191💭 Motivation and context
We have been striving to use configurations that are better at producing repeatable builds in downstream projects. This represents a backporting of a lot of that work to this skeleton so that all of our Docker projects are similarly configured.
🧪 Testing
Automated tests pass.
✅ Pre-approval checklist
to reflect the changes in this PR.
✅ Post-merge checklist