feat: yaml support for iac wasm bundles [CFG-1271]

[aron]

• Follows CONTRIBUTING rules

What does this PR do?

This PR bumps @open-policy-agent/opa-wasm to 1.6.0 to include support for the yaml.unmarshal() function in Rego and support for Node 10. This allows the snyk iac command to support additional policies that unmarshall YAML content within Terraform configuration.

Where should the reviewer start?

package.json is the primary change here but the implementation of yaml.unmarshal() is here: open-policy-agent/npm-opa-wasm#100

This also removes the use of the mock-fs package in the file-scanner.spec.ts test as the opa-wasm package will use console.error to write out a warning to stderr when loading the wasm fixture and this somehow causes mock-fs to throw the following error bringing the test suite down:

ENOENT: no such file or directory, lstat '/Users/Aron/Code/snyk/node_modules/@jest/console/node_modules'

Rather than debug this weird behavior we've decided to just mock out the function that loads the fixtures directly in the test suite.

How should this be manually tested?

rules.zip

Download the above zip file containing a custom rules bundle and unzip:

With the snyk-iac-rules tool build a custom bundle:

% snyk-iac-rules build

Then test the fixture with the latest snyk cli:

% snyk-dev iac test --rules=bundle.tar.gz rules/HELLO/fixtures/sg.tf

You should see 15 issues found vs. 14 when run with current snyk.

What are the relevant tickets?

CFG-1271

[ipapast]

I validated it locally by generating a new bundle and running with the new version: ✅

Edit: This fails when being run with node 10

[aron]

I tracked this down to an issue in the npm-opa-wasm package. Looks like something changed in the WASM API between Node 10 and 12. So when we look at the part of the opa-wasm package that checks the ABI version:

https://github.com/open-policy-agent/npm-opa-wasm/blob/f1be838de45e3310e7c37bcebbfa387ac17c4db3/src/opa.js#L195-L212

In Node 12+ we see the wasm.instance.exports.opa_wasm_abi_version is an object with a version property that returns the value 1. In Node 10 the value is just a number 1 so when it tries to access abiVersionGlobal.value it get's undefined.

If we patch the code to remove the .value the tests pass fine on Node 10 :/ so it looks like we'll need to open another PR on the opa-wasm project.

[ipapast]

For whoever picks it up next - the last 2 (or even 3) commits can be reverted.

Aron has actually had the PR merged with the fix already open-policy-agent/npm-opa-wasm#108, so when this is a new release in that repo, revert these commits and include the new version in the package.json here.

[aron]

This has now been updated to use the latest opa-wasm release with Node 10 support. All tests are now passing but I wasn't able to remove the mock-fs workaround. Our fixture still doesn't have the correct opa_wasm_abi_version and I'm not sure what is needed to get it working.

@ipapast would you mind re-reviewing on behalf of @snyk/group-infrastructure-as-code?

[ipapast]

This is tested by generating a new bundle with the snyk-iac-rules tool and then running a scan with the CLI using node versions (16, 14, 12, 10).

> ./snyk-iac-rules build rules
Generated bundle: bundle.tar.gz

The following brings up 15 issues as expected:

> snyk-dev iac test --rules=../bundle.tar.gz rules/HELLO/fixtures/sg.tf

Attaching screenshots for the buggy version of node10 as well.