Add support for Podman daemon (#87)

* add dial with podman ssh key Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * got docker client talking to podman Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add podman support via docker client Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * got docker client talking to podman Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix broken git rebase Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * simpler ssh client creation Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * even simpler ssh client creation Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add integration test for podman Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add podman integration test helpers Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add env vars for custom execution CONTAINER_HOST or PODMAN_HOST to define address to talk to podman daemon CONTAINER_SSHKEY defines path for ssh key used to talk to podman daemon Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * [wip] DI the docker client for DaemonProvider Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * simpler provider/client creation Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add support to local rootless connections feedback changes Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix unix client host address Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix golangCI linter errors Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * remove wrong error log Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * [wip] get podman running with systemd Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * get 502 from env Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * drop a couple commands Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * [wip] getting podman going Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * feedback changes Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * get remote uid from remote machine Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix linter errors Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix mime type test Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * explain podman socket http address Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * run tmate if integration tests fail Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * log when it can't talk to unix socket and std err of command Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * setup podman daemon before integration test Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * rename tmate step for clarity Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add timeout to tmate session tmate will start if integration tests fail, which blocks the workflow from finishing until they hit the default of 6h timeout. A tmate timeout of 40min is a happy medium, where someone can ssh into the action and debug it, but if they don't do it the workflow won't hang on for 6h. add tmate to validation workflow validation workflow also runs integration tests (not sure why, since basic checks does that). Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * remove old basic-checks gh actions Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * using inputs to trigger tmate Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fancy ssh Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix default timeout Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * ping podman if docker is not available Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * enable podman's client secure flag This flag is meant to control whether the ssh handshake validates the server's host key against the local known keys in .ssh/known_hosts, which is important when talking to remote Podman servers. Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * read podman engine address from config file added unit test for toml reading Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * clean up comments and code for podman address Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * get all podman ssh and unix socket addr from config file Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * remove env vars Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * use XDG_RUNTIME_DIR to find podman's socket address Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * go mod tidy Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * run tmate before integration tests Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * remove broken unit test Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * set before integration tests Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * redesign to unit test better Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * expand integration tests Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix github workflow config Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix github workflow config and lint issues Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add qemu setup action Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix validation yaml typo Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * less typos Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * remove podman VM setup Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * expanded unit podman unit tests added integration tests for unix socket and ssh Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * concept for testing podman overssh within a container Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * improve unit tests for podman client Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * podman - inte test without QEMU tries to ssh into localhost to validate ssh client Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix authorized_keys path Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add username to integration test Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add ssh default port Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * make ssh dir if not there Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * use default unix address no other is found Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * clean up integration tests Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add setup service Signed-off-by: Alex Goodman <alex.goodman@anchore.com> * wip testing podman on docker Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * move from RSA to ed25519 keys Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix auth code Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix file permissions Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * fix file permissions via setup.sh Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * move ssh fixtures to its own .gitignore Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * add tests for config reading & cleanup Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> * clean up Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com> Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
anchore · Jan 28, 2022 · c30d664 · c30d664
1 parent c91cf94
commit c30d664
Show file tree

Hide file tree

Showing 34 changed files with 1,381 additions and 139 deletions.
diff --git a/.github/workflows/validations.yaml b/.github/workflows/validations.yaml
@@ -1,6 +1,15 @@
 name: "Validations"
 on:
   workflow_dispatch:
+    inputs:
+      run-tmate:
+        description: "Start a tmate session"
+        required: false
+        default: ""
+      tmate-duration:
+        description: "Tmate session duration"
+        required: false
+        default: 5
   push:
   pull_request:
 
@@ -96,6 +105,26 @@ jobs:
 
       - uses: actions/checkout@v2
 
+      - name: Enable systemd for podman socket activation
+        run: |
+          set -x
+          loginctl enable-linger $(whoami)
+          
+          # we need to make certain that the bus instance is up and ready before systemctl attempts to use it.
+          # if we can figure a non-racey way to do this, that would be nice.
+          sleep 5
+          export XDG_RUNTIME_DIR=/run/user/$UID
+          mkdir -p $HOME/.config/systemd/user
+          systemctl --user enable --now podman.socket
+          sudo systemctl start podman.socket
+
+      - name: Setup tmate session
+        if: ${{ github.event.inputs.run-tmate == 'true' }}
+        uses: mxschmitt/action-tmate@v3
+        timeout-minutes: ${{ fromJSON(github.event.inputs.tmate-duration) }}
+        with:
+          limit-access-to-actor: true
+
       - name: Restore tool cache
         id: tool-cache
         uses: actions/cache@v2.1.3
@@ -127,7 +156,7 @@ jobs:
         with:
           path: ${{ github.workspace }}/test/integration/test-fixtures/cache
           key: ${{ runner.os }}-integration-test-cache-${{ hashFiles('test/integration/test-fixtures/cache.fingerprint') }}
-
+          
       - name: Run integration tests
         run: make integration
 

diff --git a/.gitignore b/.gitignore
@@ -5,10 +5,10 @@
 *.log
 .images
 .tmp/
+*.DS_Store
 coverage.txt
 **/test-fixtures/cache/
 
-
 # Binaries for programs and plugins
 *.exe
 *.exe~
@@ -20,4 +20,4 @@ coverage.txt
 *.test
 
 # Output of the go coverage tool, specifically when used with LiteIDE
-*.out
+*.out
diff --git a/client.go b/client.go
@@ -4,7 +4,9 @@ import (
 	"fmt"
 
 	"github.com/anchore/stereoscope/internal/bus"
+	dockerClient "github.com/anchore/stereoscope/internal/docker"
 	"github.com/anchore/stereoscope/internal/log"
+	"github.com/anchore/stereoscope/internal/podman"
 	"github.com/anchore/stereoscope/pkg/file"
 	"github.com/anchore/stereoscope/pkg/image"
 	"github.com/anchore/stereoscope/pkg/image/docker"
@@ -25,7 +27,17 @@ func GetImageFromSource(imgStr string, source image.Source, registryOptions *ima
 		// note: the imgStr is the path on disk to the tar file
 		provider = docker.NewProviderFromTarball(imgStr, &tempDirGenerator, nil, nil)
 	case image.DockerDaemonSource:
-		provider = docker.NewProviderFromDaemon(imgStr, &tempDirGenerator)
+		c, err := dockerClient.GetClient()
+		if err != nil {
+			return nil, err
+		}
+		provider = docker.NewProviderFromDaemon(imgStr, &tempDirGenerator, c)
+	case image.PodmanDaemonSource:
+		c, err := podman.GetClient()
+		if err != nil {
+			return nil, err
+		}
+		provider = docker.NewProviderFromDaemon(imgStr, &tempDirGenerator, c)
 	case image.OciDirectorySource:
 		provider = oci.NewProviderFromPath(imgStr, &tempDirGenerator)
 	case image.OciTarballSource:
@@ -49,8 +61,8 @@ func GetImageFromSource(imgStr string, source image.Source, registryOptions *ima
 	return img, nil
 }
 
-// GetImage parses the user provided image string and provides an image object; note: the source where the image should
-// be referenced from is automatically inferred.
+// GetImage parses the user provided image string and provides an image object;
+// note: the source where the image should be referenced from is automatically inferred.
 func GetImage(userStr string, registryOptions *image.RegistryOptions) (*image.Image, error) {
 	source, imgStr, err := image.DetectSource(userStr)
 	if err != nil {

diff --git a/examples/basic.go b/examples/basic.go
@@ -2,18 +2,23 @@ package main
 
 import (
 	"fmt"
-	"github.com/anchore/stereoscope/pkg/filetree/filenode"
 	"io/ioutil"
 	"os"
 
 	"github.com/anchore/stereoscope"
 	"github.com/anchore/stereoscope/pkg/file"
+	"github.com/anchore/stereoscope/pkg/filetree/filenode"
+	"github.com/sirupsen/logrus"
 )
 
 func main() {
 	// note: we are writing out temp files which should be cleaned up after you're done with the image object
 	defer stereoscope.Cleanup()
 
+	lctx := logrus.New()
+	lctx.Level = logrus.DebugLevel
+	stereoscope.SetLogger(lctx)
+
 	/////////////////////////////////////////////////////////////////
 	// pass a path to an Docker save tar, docker image, or OCI directory/archive as an argument:
 	//    ./path/to.tar
@@ -24,8 +29,12 @@ func main() {
 		panic(err)
 	}
 
-	////////////////////////////////////////////////////////////////
-	// Show the filetree for each layer
+	for _, layer := range image.Layers {
+		fmt.Printf("layer: %s\n", layer.Metadata.Digest)
+	}
+
+	//////////////////////////////////////////////////////////////////
+	//// Show the filetree for each layer
 	for idx, layer := range image.Layers {
 		fmt.Printf("Walking layer: %d", idx)
 		err = layer.Tree.Walk(func(path file.Path, f filenode.FileNode) error {
@@ -38,8 +47,8 @@ func main() {
 		}
 	}
 
-	////////////////////////////////////////////////////////////////
-	// Show the squashed filetree for each layer
+	//////////////////////////////////////////////////////////////////
+	//// Show the squashed filetree for each layer
 	for idx, layer := range image.Layers {
 		fmt.Printf("Walking squashed layer: %d", idx)
 		err = layer.SquashedTree.Walk(func(path file.Path, f filenode.FileNode) error {
@@ -52,8 +61,8 @@ func main() {
 		}
 	}
 
-	////////////////////////////////////////////////////////////////
-	// Show the final squashed tree
+	//////////////////////////////////////////////////////////////////
+	//// Show the final squashed tree
 	fmt.Printf("Walking squashed image (same as the last layer squashed tree)")
 	err = image.SquashedTree().Walk(func(path file.Path, f filenode.FileNode) error {
 		fmt.Println("   ", path)
@@ -63,8 +72,8 @@ func main() {
 		panic(err)
 	}
 
-	////////////////////////////////////////////////////////////////
-	// Fetch file contents from the (squashed) image
+	//////////////////////////////////////////////////////////////////
+	//// Fetch file contents from the (squashed) image
 	filePath := file.Path("/etc/group")
 	contentReader, err := image.FileContentsFromSquash(filePath)
 	if err != nil {

diff --git a/go.mod b/go.mod
@@ -5,21 +5,22 @@ go 1.16
 require (
 	github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
 	github.com/bmatcuk/doublestar/v4 v4.0.2
-	github.com/containerd/containerd v1.5.9 // indirect
-	github.com/docker/cli v20.10.10+incompatible
-	github.com/docker/docker v20.10.11+incompatible
-	github.com/gabriel-vasile/mimetype v1.3.0
-	github.com/go-test/deep v1.0.7
+	github.com/docker/cli v20.10.12+incompatible
+	github.com/docker/docker v20.10.12+incompatible
+	github.com/gabriel-vasile/mimetype v1.4.0
+	github.com/go-test/deep v1.0.8
 	github.com/google/go-containerregistry v0.7.0
 	github.com/hashicorp/go-multierror v1.1.0
 	github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381
 	github.com/mitchellh/go-homedir v1.1.0
+	github.com/pelletier/go-toml v1.9.3
 	github.com/pkg/errors v0.9.1
 	github.com/scylladb/go-set v1.0.2
 	github.com/sergi/go-diff v1.1.0
+	github.com/sirupsen/logrus v1.8.1
 	github.com/spf13/afero v1.6.0
 	github.com/stretchr/testify v1.7.0
 	github.com/wagoodman/go-partybus v0.0.0-20200526224238-eb215533f07d
 	github.com/wagoodman/go-progress v0.0.0-20200621122631-1a2120f0695a
-	golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2 // indirect
+	golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2
 )