Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(report): Apply ignore policies from a directory #6338

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/docs/configuration/filtering.md
Expand Up @@ -422,7 +422,9 @@ Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
Rego is part of the popular [Open Policy Agent (OPA)](https://www.openpolicyagent.org) CNCF project.
For advanced filtering, Trivy allows you to use Rego language to filter vulnerabilities.

Use the `--ignore-policy` flag which takes a path to a Rego file that defines the filtering policy.
Use the `--ignore-policy` flag which takes a path to a Rego file that defines the filtering policy. The flag can also take
a directory path containing Rego policy files (each `.rego` file found in the directory is applied for the filtering).

The Rego package name must be `trivy` and it must include a "rule" named `ignore` which determines if each individual scan result should be excluded (ignore=true) or not (ignore=false).
The `input` for the evaluation is each [DetectedVulnerability](https://github.com/aquasecurity/trivy/blob/00f2059e5d7bc2ca2e3e8b1562bdfede1ed570e3/pkg/types/vulnerability.go#L9) and [DetectedMisconfiguration](https://github.com/aquasecurity/trivy/blob/00f2059e5d7bc2ca2e3e8b1562bdfede1ed570e3/pkg/types/misconfiguration.go#L6).

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/references/configuration/cli/trivy_aws.md
Expand Up @@ -81,7 +81,7 @@ trivy aws [flags]
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for aws
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignorefile string specify .trivyignore file (default ".trivyignore")
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--list-all-pkgs enabling the option will output all packages regardless of vulnerability
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/references/configuration/cli/trivy_config.md
Expand Up @@ -25,7 +25,7 @@ trivy config [flags] DIR
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for config
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignorefile string specify .trivyignore file (default ".trivyignore")
--include-non-failures include successes and exceptions, available with '--scanners misconfig'
--k8s-version string specify k8s version to validate outdated api by it (example: 1.21.0)
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/references/configuration/cli/trivy_convert.md
Expand Up @@ -24,7 +24,7 @@ trivy convert [flags] RESULT_JSON
--exit-on-eol int exit with the specified code when the OS reaches end of service/life
-f, --format string format (table,json,template,sarif,cyclonedx,spdx,spdx-json,github,cosign-vuln) (default "table")
-h, --help help for convert
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignorefile string specify .trivyignore file (default ".trivyignore")
--list-all-pkgs enabling the option will output all packages regardless of vulnerability
-o, --output string output file name
Expand Down
Expand Up @@ -40,7 +40,7 @@ trivy filesystem [flags] PATH
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for filesystem
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignore-status strings comma-separated list of vulnerability status to ignore (unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life)
--ignore-unfixed display only fixed vulnerabilities
--ignored-licenses strings specify a list of license to ignore
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/references/configuration/cli/trivy_image.md
Expand Up @@ -56,7 +56,7 @@ trivy image [flags] IMAGE_NAME
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for image
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignore-status strings comma-separated list of vulnerability status to ignore (unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life)
--ignore-unfixed display only fixed vulnerabilities
--ignored-licenses strings specify a list of license to ignore
Expand Down
Expand Up @@ -51,7 +51,7 @@ trivy kubernetes [flags] { cluster | all | specific resources like kubectl. eg:
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for kubernetes
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignore-status strings comma-separated list of vulnerability status to ignore (unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life)
--ignore-unfixed display only fixed vulnerabilities
--ignorefile string specify .trivyignore file (default ".trivyignore")
Expand Down
Expand Up @@ -40,7 +40,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for repository
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignore-status strings comma-separated list of vulnerability status to ignore (unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life)
--ignore-unfixed display only fixed vulnerabilities
--ignored-licenses strings specify a list of license to ignore
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/references/configuration/cli/trivy_rootfs.md
Expand Up @@ -43,7 +43,7 @@ trivy rootfs [flags] ROOTDIR
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for rootfs
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignore-status strings comma-separated list of vulnerability status to ignore (unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life)
--ignore-unfixed display only fixed vulnerabilities
--ignored-licenses strings specify a list of license to ignore
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/references/configuration/cli/trivy_sbom.md
Expand Up @@ -33,7 +33,7 @@ trivy sbom [flags] SBOM_PATH
--file-patterns strings specify config file patterns
-f, --format string format (table,json,template,sarif,cyclonedx,spdx,spdx-json,github,cosign-vuln) (default "table")
-h, --help help for sbom
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignore-status strings comma-separated list of vulnerability status to ignore (unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life)
--ignore-unfixed display only fixed vulnerabilities
--ignored-licenses strings specify a list of license to ignore
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/references/configuration/cli/trivy_vm.md
Expand Up @@ -40,7 +40,7 @@ trivy vm [flags] VM_IMAGE
--helm-set-string strings specify Helm string values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
--helm-values strings specify paths to override the Helm values.yaml files
-h, --help help for vm
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignore-policy string specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability
--ignore-status strings comma-separated list of vulnerability status to ignore (unknown,not_affected,affected,fixed,under_investigation,will_not_fix,fix_deferred,end_of_life)
--ignore-unfixed display only fixed vulnerabilities
--ignorefile string specify .trivyignore file (default ".trivyignore")
Expand Down
2 changes: 1 addition & 1 deletion pkg/flag/report_flags.go
Expand Up @@ -65,7 +65,7 @@ var (
IgnorePolicyFlag = Flag[string]{
Name: "ignore-policy",
ConfigName: "ignore-policy",
Usage: "specify the Rego file path to evaluate each vulnerability",
Usage: "specify the Rego file path (or dir path with Rego files) to evaluate each vulnerability",
}
ExitCodeFlag = Flag[int]{
Name: "exit-code",
Expand Down
51 changes: 49 additions & 2 deletions pkg/result/filter.go
Expand Up @@ -3,17 +3,20 @@ package result
import (
"context"
"fmt"
"io/fs"
"os"
"path/filepath"
"sort"

"github.com/open-policy-agent/opa/bundle"
"github.com/open-policy-agent/opa/rego"
"github.com/samber/lo"
"golang.org/x/exp/maps"
"golang.org/x/exp/slices"
"golang.org/x/xerrors"

dbTypes "github.com/aquasecurity/trivy-db/pkg/types"
"github.com/aquasecurity/trivy/pkg/log"
"github.com/aquasecurity/trivy/pkg/sbom/core"
sbomio "github.com/aquasecurity/trivy/pkg/sbom/io"
"github.com/aquasecurity/trivy/pkg/types"
Expand Down Expand Up @@ -69,8 +72,16 @@ func FilterResult(ctx context.Context, result *types.Result, ignoreConf IgnoreCo
filterLicenses(result, severities, opt.IgnoreLicenses, ignoreConf)

if opt.PolicyFile != "" {
if err := applyPolicy(ctx, result, opt.PolicyFile); err != nil {
return xerrors.Errorf("failed to apply the policy: %w", err)
// Get ignore policy files from the input path (either file or files in dir)
policyFiles, err := findPolicyFiles(opt.PolicyFile)
if err != nil {
return err
}

for _, policyFile := range policyFiles {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, loading all Rego files together and evaluating the policies collectively is more intuitive for Rego users than loading and repeatedly evaluating individual Rego files. The current implementation may not work correctly if variable definitions are spread across multiple files.

To address this, using the rego.Load() function to load all Rego files from a directory recursively would be more appropriate. This ensures that all files are considered together, allowing for proper resolution of variables, rules, and dependencies.

However, since I've been away from OPA recently, I would greatly appreciate insights from @simar7 to ensure the correctness and effectiveness of this approach.

Copy link
Member

@simar7 simar7 Apr 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 I'd also like to mention that running .PrepareForEval() (and the eventual evaluation itself) is an expensive operation. Doing it recursively on a directory that is user defined to load all files that match the Rego extension, whether relevant or not, can be costly.

However if we go down the route of loading all rego files via rego.Load() as @knqyf263 mentioned, I'm not sure if we can have multiple checks that can contain the same default as the compiler will error out. Since Rego checks often contain defaults that result in "fail-close" type of checks, this would be a common occurrence (e.g. multiple checks having default allow=false).

Maybe a safer route is to allow this flag to have values that can be a list of filenames rather than a directory? This would limit the scope of rego files that get loaded and evaluated.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, we need to understand the use case precisely. @dstrelbytskyi Could you elaborate on it?

if err := applyPolicy(ctx, result, policyFile); err != nil {
return xerrors.Errorf("failed to apply ignore policy %s: %w", policyFile, err)
}
}
}
sort.Sort(types.BySeverity(result.Vulnerabilities))
Expand Down Expand Up @@ -239,6 +250,42 @@ func summarize(status types.MisconfStatus, summary *types.MisconfSummary) {
}
}

func findPolicyFiles(policiesPath string) ([]string, error) {
fi, err := os.Stat(policiesPath)
if err != nil {
return nil, xerrors.Errorf("failed to analyze ignore policy path %q: %w", policiesPath, err)
}

// The ignore policy option is a file
if !fi.IsDir() {
return []string{
policiesPath,
}, nil
}

// If the ignore policy option is a dir find rego files in it
var files []string
if err = filepath.WalkDir(policiesPath, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.Type().IsRegular() || filepath.Ext(path) != bundle.RegoExt {
return nil
}

files = append(files, path)
return nil
}); err != nil {
return nil, xerrors.Errorf("failed to find policy files in %q: %w", policiesPath, err)
}

if len(files) == 0 {
log.Logger.Warnf("No ignore policies found in %q", policiesPath)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
log.Logger.Warnf("No ignore policies found in %q", policiesPath)
log.Warn("No ignore policies found", log.String("dir", pliciesPath))

}

return files, nil
}

func applyPolicy(ctx context.Context, result *types.Result, policyFile string) error {
policy, err := os.ReadFile(policyFile)
if err != nil {
Expand Down
65 changes: 65 additions & 0 deletions pkg/result/filter_test.go
Expand Up @@ -713,6 +713,71 @@ func TestFilter(t *testing.T) {
},
},
},
{
name: "ignore policy directory",
args: args{
report: types.Report{
Results: types.Results{
{
Vulnerabilities: []types.DetectedVulnerability{
vuln1,
vuln3, // ignored by policy
vuln4, // ignored by policy
},
Misconfigurations: []types.DetectedMisconfiguration{
misconf1,
misconf3, // ignored by policy
},
},
},
},
severities: []dbTypes.Severity{
dbTypes.SeverityLow,
dbTypes.SeverityHigh,
},
policyFile: "./testdata/ignore-dir",
},
want: types.Report{
Results: types.Results{
{
Vulnerabilities: []types.DetectedVulnerability{
vuln1,
},
MisconfSummary: &types.MisconfSummary{
Successes: 0,
Failures: 1,
Exceptions: 1,
},
Misconfigurations: []types.DetectedMisconfiguration{
misconf1,
},
ModifiedFindings: []types.ModifiedFinding{
{
Type: types.FindingTypeMisconfiguration,
Status: types.FindingStatusIgnored,
Statement: "Filtered by Rego",
Source: "testdata/ignore-dir/ignore-misconf.rego",
Finding: misconf3,
},
{
Type: types.FindingTypeVulnerability,
Status: types.FindingStatusIgnored,
Statement: "Filtered by Rego",
Source: "testdata/ignore-dir/ignore-vuln.rego",
Finding: vuln3,
},
{
Type: types.FindingTypeVulnerability,
Status: types.FindingStatusIgnored,
Statement: "Filtered by Rego",
Source: "testdata/ignore-dir/ignore-vuln.rego",
Finding: vuln4,
},
},
},
},
},
},
{
name: "happy path with duplicates, one with empty fixed version",
args: args{
Expand Down
9 changes: 9 additions & 0 deletions pkg/result/testdata/ignore-dir/ignore-misconf.rego
@@ -0,0 +1,9 @@
package trivy

import data.lib.trivy

default ignore=false

ignore {
input.AVDID != "AVD-ID100"
}
9 changes: 9 additions & 0 deletions pkg/result/testdata/ignore-dir/ignore-vuln.rego
@@ -0,0 +1,9 @@
package trivy

import data.lib.trivy

default ignore=false

ignore {
input.VulnerabilityID != "CVE-2019-0001"
}