Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support record/replay for supportability and testing scenarios #12127

Open
lukehoban opened this issue Feb 9, 2023 · 0 comments
Open

Support record/replay for supportability and testing scenarios #12127

lukehoban opened this issue Feb 9, 2023 · 0 comments
Labels
area/testing kind/enhancement Improvements or new features

Comments

@lukehoban
Copy link
Member

Although users interface with Pulumi through a program written in a particular programming language, the behaviour of that program ultimately boils down to a sequence of RPC calls made to the Pulumi engine. For many use cases, it is useful to be able to record, share, and reply these RPC calls:

  1. For supportability scenarios, such as debugging a user problem without requiring them to share source code.
  2. For testing scenarios, such as being able to lock in, in a language agnostic way, the expected results of a program or component.

For testing, we have a few types of tests that are used frequently in Pulumi today:

  1. Integration tests, which run a program written in a supported Pulumi programming language against a real cloud provider
  2. Unit tests, which run within the program as part of the language's built-in unit testing capability, using mocks instead of a real cloud provider.

There are cases though where we would like to test in a more language-agnostic way, and decouple testing between the language/component layer (which generates a sequence of RPC calls) and the engine (which handles those calls).

It would be nice if we had a language-agnostic framework to run Pulumi programs against a mock monitor RPC server, where
we would could supply the canned responses to particular RPC requests through config.

As a bonus, it would be nice if we could capture the RPC request/response when running a Pulumi program normally to use
that as config to run subsequent tests with Pulumi programs written in another language.

Some explorations of these ideas were done in #11710 and #11085.

@mikhailshilkov mikhailshilkov added kind/enhancement Improvements or new features area/testing labels Feb 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/testing kind/enhancement Improvements or new features
Projects
None yet
Development

No branches or pull requests

2 participants