timeout is not working as expected

[sourabhgupta385]

Description

I am using trivy 0.26.0 to scan docker images and I am using timeout of 10m in the Jenkins pipelines. Below is my command to do scan:

trivy --cache-dir /common-lab/trivy image --skip-update  --timeout 10m0s --format template --template @/usr/local/share/trivy/templates/html.tpl <docker_image>

What did you expect to happen?

In case of taking more than 10m to do scanning, it should timed out at 10 min threshold and report an error.

What happened instead?

Scanning ran for about 20 min and then reported error. Sometimes it is running for more time also.

Output of run:

[2022-06-08T07:47:00.857Z] + trivy --cache-dir /common-lab/trivy image --skip-update  --timeout 10m0s --format template --template @/usr/local/share/trivy/templates/html.tpl <docker_image>

[2022-06-08T08:07:11.275Z] 2022-06-08T08:07:11.211Z	WARN	Increase --timeout value

[2022-06-08T08:07:11.280Z] 2022-06-08T08:07:11.212Z	FATAL	scan error: image scan failed: failed analysis: analyze error: timeout: context deadline exceeded

[DmitriyLewen]

Hello @sourabhgupta385
Thanks for your report!

Can you update Trivy and retry the scan?
If you still have a problem - can you send your image to investigate this problem?

Regards, Dmitriy

[elchenberg]

I am wondering if the timeout only considers the scan time. Could it be that it took ~10 minutes to pull the image and then another 10 minutes to scan?

I am asking because I have a different problem with the timeout when I cannot connect to the Trivy server in ClientServer mode (I used the wrong port intentionally in the following command). I set the timeout to 30s but the command takes more than 2 minutes before it times out:

+ trivy --timeout=30s image --debug --security-checks=vuln --server=http://trivy.trivy.svc:443 alpine@sha256:1304f174557314a7ed9eddb4eab12fed12cb0cd9809e4c28f29af86979a3c870
  2022-09-02T09:21:38.600Z      DEBUG   Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
  2022-09-02T09:21:38.601Z      INFO    Vulnerability scanning is enabled
  2022-09-02T09:21:38.601Z      DEBUG   Vulnerability type:  [os library]
  2022-09-02T09:21:41.096Z      DEBUG   Image ID: sha256:9c6f0724472873bb50a2ae67a9e7adcb57673a183cea8b06eb778dca859181b5
  2022-09-02T09:21:41.096Z      DEBUG   Diff IDs: [sha256:994393dc58e7931862558d06e46aa2bb17487044f670f310dffe1d24e4d1eec7]
  2022-09-02T09:21:41.096Z      DEBUG   Base Layers: [sha256:994393dc58e7931862558d06e46aa2bb17487044f670f310dffe1d24e4d1eec7]
  2022-09-02T09:23:50.468Z      FATAL   image scan error:
      github.com/aquasecurity/trivy/pkg/commands/artifact.Run
          /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:367
    - scan error:
      github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact
          /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:227
    - image scan failed:
      github.com/aquasecurity/trivy/pkg/commands/artifact.scan
          /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:531
    - failed analysis:
      github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact
          /home/runner/work/trivy/trivy/pkg/scanner/scan.go:127
    - unable to get missing layers:
      github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.Inspect
          /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:103
    - unable to fetch missing layers:
      github.com/aquasecurity/trivy/pkg/cache.RemoteCache.MissingBlobs
          /home/runner/work/trivy/trivy/pkg/cache/remote.go:61
    - twirp error internal: failed to do request: Post "http://trivy.trivy.svc:443/twirp/trivy.cache.v1.Cache/MissingBlobs": dial tcp 10.96.22.175:443: connect: connection timed out

[DmitriyLewen]

Hello @elchenberg
Thanks for your report!

Currently, Trivy has a total time to pull the image and scan it.

About timeout for connecting to Trivy server:
At the moment Trivy can't stop when trying to connect to Trivy server.
If i remember correctly - the default client timeout is 2 minutes.
But timeout includes connection time.
e.g.:
You have timeout = 3min.
Connection to Trivy server took 1 minute.
Then (after successful connection) Trivy will be stopped after 2 minute of scanning.

Regards, Dmitriy

[arxeiss]

I see the same with v0.33.0 and v0.34.0 because it never finished as reported here #3212.

But I executed command below and it was running 15 minutes before I killed it.

trivy config --format table --exit-code 1 --severity HIGH,CRITICAL --skip-dirs ./vendor --ignorefile ./.trivyignore --debug --timeout 1m .