Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add display to the engine tests #16050

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add display to the engine tests #16050

wants to merge 4 commits into from

Conversation

tgummerer
Copy link
Collaborator

@tgummerer tgummerer commented Apr 24, 2024

We want to add more test coverage to the display code. The best way to do that is to add it to the engine tests, that already cover most of the pulumi functionality.

It's probably not really possible to review all of the output, but at least it gives us a baseline, which we can work with.

There's a couple of tests that are flaky for reasons I don't quite understand yet. I marked them as to skip and we can look at them later. I'd rather get in the baseline tests sooner, rather than spending a bunch of time looking at that. The output differences also seem very minor, so not super concerning.

The biggest remaining issue is that this doesn't interact well with the Chdir we're doing in the engine. We could either pass the CWD through, or just try to get rid of that Chdir. So this should only be merged after #15607.

I've tried to split this into a few commits, separating out adding the testdata, so it's hopefully a little easier to review, even though the PR is still quite large.

One other thing to note is that we're comparing that the output has all the same lines, and not that it is exactly the same. Because of how the engine is implemented, there's a bunch of race conditions otherwise, that would make us have to skip a bunch of tests, just because e.g. resource A is sometimes deleted before resource B and sometimes it's the other way around.

The biggest downside of that is that running with PULUMI_ACCEPT will produce a diff even when there are no changes. Hopefully we won't have to run that way too often though, so it might not be a huge issue?

@pulumi-bot
Copy link
Contributor

pulumi-bot commented Apr 24, 2024

Changelog

[uncommitted] (2024-04-29)

@tgummerer tgummerer force-pushed the tg/display-tests branch 2 times, most recently from b35a02b to 5fc7c92 Compare April 25, 2024 11:37
@tgummerer tgummerer added the impact/no-changelog-required This issue doesn't require a CHANGELOG update label Apr 25, 2024
@tgummerer tgummerer force-pushed the tg/display-tests branch 2 times, most recently from b629e87 to e856c07 Compare April 29, 2024 15:13
@tgummerer tgummerer marked this pull request as ready for review April 30, 2024 08:24
@tgummerer tgummerer requested a review from a team as a code owner April 30, 2024 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact/no-changelog-required This issue doesn't require a CHANGELOG update
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants