Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add debug GRPC interceptors that print conversations in JSON #11085

Merged
merged 1 commit into from Nov 8, 2022

Conversation

t0yv0
Copy link
Member

@t0yv0 t0yv0 commented Oct 19, 2022

Description

Fixes #10232

PULUMI_DEBUG_GRPC=./debug.json pulumi preview now works.

The debug APIs are probably unstable so I moved them out to their own repo rather than hosting under sdk/.

Debug interceptors are no-op unless the process initializes them in main. Currently only pulumi process initializes them if the env var is set. This way we observe conversations only once, as a server for ResourceMonitor etc, and as a client for plugin conversations (language and resource).

I can now see provider conversations logged which was of interest to me.

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • I have run make changelog and committed the changelog/pending/<file> documenting my change
  • Yes, there are changes in this PR that warrants bumping the Pulumi Service API version

@pulumi-bot
Copy link
Contributor

pulumi-bot commented Oct 19, 2022

Changelog

[uncommitted] (2022-11-08)

Features

  • [cli] Enables debug tracing of Pulumi gRPC internals: PULUMI_DEBUG_GRPC=$PWD/grpc.json pulumi up
    #11085

@t0yv0 t0yv0 requested a review from a team October 19, 2022 14:53
Copy link
Member

@AaronFriel AaronFriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested a couple changes, mostly around moving the control for when the interceptor is active and chained into this repo, and having it not be an unconditional part of the gRPC stack.

pkg/engine/plugin_host.go Outdated Show resolved Hide resolved
pkg/cmd/pulumi/main.go Outdated Show resolved Hide resolved
pkg/cmd/pulumi/main.go Outdated Show resolved Hide resolved
@t0yv0 t0yv0 force-pushed the t0yv0/grpc-trace branch 2 times, most recently from 84af240 to 77c7894 Compare November 2, 2022 15:41
@t0yv0
Copy link
Member Author

t0yv0 commented Nov 2, 2022

@Frassle @AaronFriel I've cleaned this up a bit:

  • lib is inlined in pkg/
  • lib is stateless
  • Serve -> ServeWithOptions so that only ResourceMonitor service receives debug tracing options
  • Avoiding static state is hard in this codebase - how do we extend engine state? I snuck in a sync.Mutex into plugin.Context for this; it works as a compromise
  • Snuck in client interceptor setup into ProjectInfoContext constructor for plugin.Context

The changes should be to my knowledge fully bw-compat, only affecting things when PULUMI_DEBUG_GRPC is set.

Copy link
Member

@AaronFriel AaronFriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good!

@t0yv0
Copy link
Member Author

t0yv0 commented Nov 8, 2022

bors merge

bors bot added a commit that referenced this pull request Nov 8, 2022
11085: Add debug GRPC interceptors that print conversations in JSON r=t0yv0 a=t0yv0

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes #10232

`PULUMI_DEBUG_GRPC=./debug.json pulumi preview` now works.

The debug APIs are probably unstable so I moved them out to their own repo rather than hosting under sdk/.

Debug interceptors are no-op unless the process initializes them in main. Currently only `pulumi` process initializes them if the env var is set. This way we observe conversations only once, as a server for ResourceMonitor etc, and as a client for plugin conversations (language and resource).

I can now see provider conversations logged which was of interest to me.


## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Anton Tayanovskyy <anton.tayanovskyy@gmail.com>
Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
@bors
Copy link
Contributor

bors bot commented Nov 8, 2022

Build failed:

@t0yv0
Copy link
Member Author

t0yv0 commented Nov 8, 2022

bors merge

@bors
Copy link
Contributor

bors bot commented Nov 8, 2022

Build succeeded:

@bors bors bot merged commit 362523f into master Nov 8, 2022
@pulumi-bot pulumi-bot deleted the t0yv0/grpc-trace branch November 8, 2022 19:33
abhinav pushed a commit to pulumi/pulumi-dotnet that referenced this pull request Jan 11, 2023
11085: Add debug GRPC interceptors that print conversations in JSON r=t0yv0 a=t0yv0

<!--- 
Thanks so much for your contribution! If this is your first time contributing, please ensure that you have read the [CONTRIBUTING](https://github.com/pulumi/pulumi/blob/master/CONTRIBUTING.md) documentation.
-->

# Description

<!--- Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. -->

Fixes pulumi/pulumi#10232

`PULUMI_DEBUG_GRPC=./debug.json pulumi preview` now works.

The debug APIs are probably unstable so I moved them out to their own repo rather than hosting under sdk/.

Debug interceptors are no-op unless the process initializes them in main. Currently only `pulumi` process initializes them if the env var is set. This way we observe conversations only once, as a server for ResourceMonitor etc, and as a client for plugin conversations (language and resource).

I can now see provider conversations logged which was of interest to me.


## Checklist

<!--- Please provide details if the checkbox below is to be left unchecked. -->
- [ ] I have added tests that prove my fix is effective or that my feature works
<!--- 
User-facing changes require a CHANGELOG entry.
-->
- [ ] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change
<!--
If the change(s) in this PR is a modification of an existing call to the Pulumi Service,
then the service should honor older versions of the CLI where this change would not exist.
You must then bump the API version in /pkg/backend/httpstate/client/api.go, as well as add
it to the service.
-->
- [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version
  <!-- `@Pulumi` employees: If yes, you must submit corresponding changes in the service repo. -->


Co-authored-by: Anton Tayanovskyy <anton.tayanovskyy@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a facility to easily debug gRPC conversations with jq .
4 participants