Reduce build matrix

[bai]

Short-term workaround to address build flakiness.

[dnwe]

@bai i think we'd still want/need both the push and pr type of build as doesn't that alter that code under test? Head of the branch vs merged into master state?

/pr builds automerged state (as if you merged the PR and ran the tests on that, note: it won't run if the PR can't be automerged).
/push builds for the current state of the branch you pushed to (as if you ran the tests on your local copy you just pushed),