[Bug]: Goroutine Leaks in NewDockerCompose #2008
Labels
bug
An issue with the library
Comments
|
Thanks @mhogara for raising this issue. Would you be interested in contributing a fix? |
|
👍🏻 If I have the time, I'll start work on a fix @mdelapenya - will update this issue if I make any progress or have any questions. |
|
hey @mhogara I'm able to reproduce this issue adding the goleak dependency to a test in the compose module (not in the LocalDockerCompose, which is deprecated and will eventually be removed): func TestDockerComposeAPIWithVolume(t *testing.T) {
defer goleak.VerifyNone(t)
path := RenderComposeWithVolume(t)
compose, err := NewDockerCompose(path)
require.NoError(t, err, "NewDockerCompose()")
t.Cleanup(func() {
require.NoError(t, compose.Down(context.Background(), RemoveOrphans(true), RemoveImagesLocal), "compose.Down()")
})
ctx, cancel := context.WithCancel(context.Background())
t.Cleanup(cancel)
err = compose.Up(ctx, Wait(true))
require.NoError(t, err, "compose.Up()")
}with the result you described: From that report, I can only say that the leak comes from another package, not from testcontainers-go. I've even run all the tests in the func TestMain(m *testing.M) {
defer goleak.VerifyTestMain(m)
}the goleak report for the entire execution shows that no testcontainers-go code is in use: |
|
@mhogara if you agree, we should close this one as my guess is that it's not caused by us. Wdyt? |
|
Hi @mdelapenya I'd prefer to keep this issue open as long as the go leak is still occuring in the code though and we have no solid solution; this leak does require a workaround/change in my code, so to have it closed would make it harder for me to track when/how long I need this workaround to be present. I would also be interested in which package exactly this leak is coming from 🤔 - I'm not sure how much investigation that would take. If we could pinpoint the package that causes this, we should make an issue for them to track it and further investigate. I was also thinking over your hypothesis about an HTTP connection causing the leak and found this SO post: https://stackoverflow.com/questions/75026700/goroutine-leaks-when-trying-to-readallresponse-body-where-response-is-retu. That feels promising, I'm not sure if you've seen it. After thinking it through, this could stop the leak if it coming from the polling for the wait strategy(assuming there's an internal client?). I'd be happy to look into this possible solution, but it might take a while for me to have enough time to get to it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Testcontainers version
0.26.0
Using the latest Testcontainers version?
Yes
Host OS
Linux
Host arch
x86_64
Go version
1.21.3
Docker version
Docker info
What happened?
This is somewhat of a follow up to #2007 (possibly same issue, but I'll let the team here reorganize as needed). I did some searching in the current issues and could not find anything on go leaks.
Problem at Hand
Running a simple test that takes up a
NewDockerComposefromtestcontainers-go/modules/composeleaks 2 goroutines (as caught bygoleak.VerifyNone(t). There are 2 leaks with a wait strategy; without a wait strategy, 4 go routines are leaking. The container or docker compose file does not affect anything.Interestingly enough, these problems do not happen if you use the deprecated
NewLocalDockerCompose.I'll include reproduction code below and under "Relevant log output", I'll put the two goroutine leaks you get when you have a wait strategy.
What I've Observed/Debugging
It seems to be like 2 of the goroutines come from the creation of the Docker compose and another 2 come from some sort of waiting that happens after. From debugging, it seems like some of those leaks could come from calls to
NewClientHTTPSorNewDockerClientWithOpts.Let me know if I can share anything else to help.
Relevant log output
--- FAIL: TestIntegrationSuite (1.80s) /glnxa64/golang/src/internal/registrystore/redis/store_test.go:50: found unexpected goroutines: [Goroutine 52 in state IO wait, with internal/poll.runtime_pollWait on top of the stack: goroutine 52 [IO wait]: internal/poll.runtime_pollWait(0x7f17fe4dede8, 0x72) /glnxa64/golang/src/runtime/netpoll.go:343 +0x85 internal/poll.(*pollDesc).wait(0xc000714b00?, 0xc00023b000?, 0x0) /glnxa64/golang/src/internal/poll/fd_poll_runtime.go:84 +0x27 internal/poll.(*pollDesc).waitRead(...) /glnxa64/golang/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc000714b00, {0xc00023b000, 0x1000, 0x1000}) /glnxa64/golang/src/internal/poll/fd_unix.go:164 +0x27a net.(*netFD).Read(0xc000714b00, {0xc00023b000?, 0xc000549ec8?, 0xc000518960?}) /glnxa64/golang/src/net/fd_posix.go:55 +0x25 net.(*conn).Read(0xc0000906c8, {0xc00023b000?, 0x0?, 0x0?}) /glnxa64/golang/src/net/net.go:179 +0x45 net/http.(*persistConn).Read(0xc0006517a0, {0xc00023b000?, 0x451a20?, 0xc000549ec8?}) /glnxa64/golang/src/net/http/transport.go:1954 +0x4a bufio.(*Reader).fill(0xc0000b6e40) /glnxa64/golang/src/bufio/bufio.go:113 +0x103 bufio.(*Reader).Peek(0xc0000b6e40, 0x1) /glnxa64/golang/src/bufio/bufio.go:151 +0x53 net/http.(*persistConn).readLoop(0xc0006517a0) /glnxa64/golang/src/net/http/transport.go:2118 +0x1b9 created by net/http.(*Transport).dialConn in goroutine 51 /glnxa64/golang/src/net/http/transport.go:1776 +0x169f Goroutine 53 in state select, with net/http.(*persistConn).writeLoop on top of the stack: goroutine 53 [select]: net/http.(*persistConn).writeLoop(0xc0006517a0) /glnxa64/golang/src/net/http/transport.go:2421 +0xe5 created by net/http.(*Transport).dialConn in goroutine 51 /glnxa64/golang/src/net/http/transport.go:1777 +0x16f1 ] FAILAdditional information
Here's some code that causes the 2 goroutine leaks(docker compose file/wait strategy can be anything, I'm using a Redis container):
To see the case where there are 4 leaking goroutines, you just need to comment out the "WaitForService" line.
Here's the code that has no goroutine leaks:
The text was updated successfully, but these errors were encountered: