Skip to content

Commit

Permalink
Fix integration tests to not group error messages (#241)
Browse files Browse the repository at this point in the history
This PR disables the error messages when running integration tests of
NilAway (against different analyzer drivers) such that the output is
more accurate for comparisons.

As part of this change, we fixed one want string in testdata where one
false positive was silently ignored due to grouping. It is now properly
marked as FP and a TODO is added to remove it once cross-package
contract support is fixed in NilAway.
  • Loading branch information
yuxincs committed May 13, 2024
1 parent f3f320c commit 3062237
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 2 deletions.
4 changes: 3 additions & 1 deletion testdata/integration/contracts/downstream/downstream.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,7 @@ func GiveUpstreamNil() {
func GiveUpstreamNonnil() {
a := 1
r := upstream.NonnilToNonnil(&a)
print(*r) // Safe!
// TODO: FP: this should be safe due to the contract. We should remove this once contract
// support is fixed in NilAway.
print(*r) //want "result 0 of `NonnilToNonnil\(\)` dereferenced"
}
7 changes: 6 additions & 1 deletion tools/cmd/integration-test/standalone.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,12 @@ func (d *StandaloneDriver) Run(dir string) (map[Position]string, error) {
}

// Run the NilAway binary on the integration test project, with redirects to an internal buffer.
cmd := exec.Command(filepath.Join("..", "..", "bin", "nilaway"), "-json", "-pretty-print=false", "./...")
cmd := exec.Command(filepath.Join("..", "..", "bin", "nilaway"),
"-json", "-pretty-print=false",
// Disable group error messages to make the output accurate for comparisons.
"-group-error-messages=false",
"./...",
)
cmd.Dir = dir
out, err := cmd.CombinedOutput()
if err != nil {
Expand Down

0 comments on commit 3062237

Please sign in to comment.