helper/resource: Potential Goroutine Leak with go-plugin #1063
Assignees
Labels
bug
Something isn't working
subsystem/tests
Issues and feature requests related to the testing framework.
Milestone
Comments
bflad
added
bug
bflad
added a commit
that referenced
this issue
Nov 4, 2022
Reference: #1063 Reference: hashicorp/go-plugin#220 This includes the following small bug fixes: - Clarifies/fixes the trace logging during `terraform show` commands - Improves performance slightly for `terraform show` raw plan usage (terraform-exec already captures and returns stdout for raw plans) - Prevents a goroutine leak specific to the detached stop context in `schema.GRPCProvider` Previously, leak detection on testing would contain an entry per Terraform command such as: ``` [Goroutine 276 in state select, with github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.mergeStop on top of the stack: goroutine 276 [select]: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.mergeStop({0x103a5f298?, 0x14000390c00?}, 0x1400033e5f0, 0x140004bb560) /Users/bflad/src/github.com/hashicorp/terraform-plugin-sdk/helper/schema/grpc_provider.go:46 +0x64 created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).StopContext /Users/bflad/src/github.com/hashicorp/terraform-plugin-sdk/helper/schema/grpc_provider.go:63 +0x178 ] ``` Leaked goroutines such as: ``` [Goroutine 53 in state select, with github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv on top of the stack: goroutine 53 [select]: github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0x29?) /Users/bflad/go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.4/grpc_broker.go:121 +0x58 github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0x140003ca190) /Users/bflad/go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.4/grpc_broker.go:411 +0x40 created by github.com/hashicorp/go-plugin.(*GRPCServer).Init /Users/bflad/go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.4/grpc_server.go:85 +0x424 ] ``` Will likely require a fix upstream in go-plugin.
bflad
added a commit
that referenced
this issue
Nov 7, 2022
Reference: #1063 Reference: hashicorp/go-plugin#220 This includes the following small bug fixes: - Clarifies/fixes the trace logging during `terraform show` commands - Improves performance slightly for `terraform show` raw plan usage (terraform-exec already captures and returns stdout for raw plans) - Prevents a goroutine leak specific to the detached stop context in `schema.GRPCProvider` Previously, leak detection on testing would contain an entry per Terraform command such as: ``` [Goroutine 276 in state select, with github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.mergeStop on top of the stack: goroutine 276 [select]: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.mergeStop({0x103a5f298?, 0x14000390c00?}, 0x1400033e5f0, 0x140004bb560) /Users/bflad/src/github.com/hashicorp/terraform-plugin-sdk/helper/schema/grpc_provider.go:46 +0x64 created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).StopContext /Users/bflad/src/github.com/hashicorp/terraform-plugin-sdk/helper/schema/grpc_provider.go:63 +0x178 ] ``` Leaked goroutines such as: ``` [Goroutine 53 in state select, with github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv on top of the stack: goroutine 53 [select]: github.com/hashicorp/go-plugin.(*gRPCBrokerServer).Recv(0x29?) /Users/bflad/go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.4/grpc_broker.go:121 +0x58 github.com/hashicorp/go-plugin.(*GRPCBroker).Run(0x140003ca190) /Users/bflad/go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.4/grpc_broker.go:411 +0x40 created by github.com/hashicorp/go-plugin.(*GRPCServer).Init /Users/bflad/go/pkg/mod/github.com/hashicorp/go-plugin@v1.4.4/grpc_server.go:85 +0x424 ] ``` Will likely require a fix upstream in go-plugin.
|
#1091 which will be part of the next release covers the sdk-based provider global StopContext leak. hashicorp/go-plugin#220 (or something similar) would cover the gRPC broker leak. |
|
#1095 should cover the gRPC broker leak. |
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
SDK version
Relevant provider source code
Refer to the terraform-provider-random acceptance testing.
Debug Output
Go timeout panic stacktrace
Expected Behavior
Running
go test -count=1 -parallel=1 -timeout=10m -v ./internal/providerwould yield a relatively small number of go-plugin-related Goroutines because the testing should be serialized and each test should properly close them.Actual Behavior
Refer to Gist above, there's a lot of this:
Steps to Reproduce
TF_ACC=1 go test -count=1 -parallel=1 -timeout=10m -v ./internal/provider(and have it time out so it raises the stacktrace)The text was updated successfully, but these errors were encountered: