Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: hooks for RUN instructions (use cases: reproducible builds, cross-compilation, malware detection, ...) #4576

Open
AkihiroSuda opened this issue Jan 23, 2024 · 4 comments · May be fixed by #4669

Comments

@AkihiroSuda
Copy link
Member

AkihiroSuda commented Jan 23, 2024

I'd like to propose a hooking mechanism for RUN instructions of Dockerfile.

e.g.,

buildctl build \
  --frontend dockerfile.v0 \
  --opt hook="$(cat hook.json)"

with hook.json as follows:

{
  "RUN": {
    "entrypoint": ["/dev/.dfhook/entrypoint"],
    "mounts": [
       {"from": "example.com/hook", "target": "/dev/.dfhook"},
       {"type": "secret", "source": "something", "target": "/etc/something"}
    ]
  }
}

This will let the frontend treat RUN foo as:

RUN \
  --mount=from=example.com/hook,target=/dev/.dfhook \
  --mount=type=secret,source=something,target=/etc/something \
  /dev/.dfhook/entrypoint foo

docker history will still show this as RUN foo.

Note

The proposed json schema may change.
See the PR for the latest status:

Use cases

Reproducible builds

A hook can be used for wrapping apt-get command to use snapshot.debian.org for reproducing package versions without modifying the Dockerfile.

The /dev/.dfhook/entrypoint script can be like this:

#!/bin/bash
set -eu -o pipefail

: "${SOURCE_DATE_EPOCH:=$(stat --format=%Y /etc/apt/sources.list.d/debian.sources)}"
snapshot="$(printf "%(%Y%m%dT%H%M%SZ)T\n" "${SOURCE_DATE_EPOCH}")"
. /etc/os-release

# Rewrite /etc/apt to use snapshot.debian.org
cp -a /etc/apt /etc/apt.bak
rm -f /etc/apt/sources.list.d/debian.sources
cat <<EOF >>/etc/apt/sources.list
deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/${snapshot} ${VERSION_CODENAME} main
deb [check-valid-until=no] http://snapshot.debian.org/archive/debian-security/${snapshot} ${VERSION_CODENAME}-security main
deb [check-valid-until=no] http://snapshot.debian.org/archive/debian/${snapshot} ${VERSION_CODENAME}-updates main
EOF

# Run the command
set +e
"$@"
status=$?
set -e

# Restore /etc/apt
rm -rf /etc/apt
mv /etc/apt.bak /etc/apt

exit $status

A hook may also push/pull dpkg blobs to an OCI registry (or whatever) for efficient caching.

Cross-compilation

xx-apt, etc. (https://github.com/tonistiigi/xx) can be reimplemented as a hook.

Malware detection

A hook may use seccomp, etc. to hook the syscalls and detect malicious actions, etc.

Enterprise networking

Enterprise networks often require installing a MITM proxy cert.
This can be easily automated with a hook.

FAQs

  • Q. Why not just modify Dockerfile?
    • A. Because it affects the history object in OCI Image Config and decreases reproducibility
@AkihiroSuda
Copy link
Member Author

@tonistiigi @thaJeztah SGTY?

@thaJeztah
Copy link
Member

thaJeztah commented Feb 19, 2024

Would these hooks run inside the container that's run as part of the RUN, or is this all running on the host? If it's inside the container, would this be similar to a custom SHELL (to set a custom entry point for RUN steps)?

@AkihiroSuda AkihiroSuda linked a pull request Feb 19, 2024 that will close this issue
@AkihiroSuda
Copy link
Member Author

@AkihiroSuda
Copy link
Member Author

Would these hooks run inside the container that's run as part of the RUN, or is this all running on the host?

Inside.

If it's inside the container, would this be similar to a custom SHELL (to set a custom entry point for RUN steps)?

Yes, somewhat similar.

(I noticed I accidentally edited @thaJeztah's original comment 😅 . Now reverted. )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants