[Bug]: Sporadically fails to create reaper in GitHub Actions

[abemedia]

Testcontainers version

v0.27.0

Using the latest Testcontainers version?

Yes

Host OS

Ubuntu 22.04.3 LTS

Host arch

amd64

Go version

1.20

Docker version

GitHub Actions 🤷🏼‍♂️

Docker info

GitHub Actions 🤷🏼‍♂️

What happened?

This does not seem to happen when running locally but occasionally my tests fail in GitHub Actions with the error creating reaper failed.

Relevant log output

2024/01/28 17:56:02 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 24.0.7
  API Version: 1.43
  Operating System: Ubuntu 22.04.3 LTS
  Total Memory: 6921 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: 1db7667978fa78cee3873c39e54eaa6e9e4e8bd9b8a5025ceb0b2b7c20578725
  Test ProcessID: c0f2e63c-5ad4-4ae2-a60d-de0250ff089f
2024/01/28 17:56:02 🔥 Reaper obtained from Docker for this test session 24e370b447d41033419fea8aefd579d45338a16882557d81a9ee8e0df38cfaad
2024/01/28 17:56:02 port not found: creating reaper failed: failed to create container

Additional information

No response

[Bablzz]

Hi @abemedia ! I hope you are doing well! can you share link to the job? I check actions in this repo - repear works fine - for example - https://github.com/testcontainers/testcontainers-go/actions/runs/7730045562/job/21075148438#step:10:41

[kostyay]

Did you manage to solve this issue? I'm facing an issue with reaper failing with the following error.. it only happens on GHA:

2024/02/13 14:47:24 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 25.0.3
  API Version: 1.43
  Operating System: Alpine Linux v3.19 (containerized)
  Total Memory: 31291 MB
  Resolved Docker Host: unix:///run/docker.sock
  Resolved Docker Socket Path: /run/docker.sock
  Test SessionID: c00c76c83c7180c78797a94d2146b5216f91200d033f9eb87efd620b53b27115
  Test ProcessID: e60189dd-ddfc-4b4c-98ab-1[70](xxx)99ea7a937
2024/02/13 14:47:24 🐳 Creating container for image testcontainers/ryuk:0.6.0
    internal_handler_test.go:283: failed to start container: look up reaper container returned nil although creation failed due to name conflict: creating reaper failed: failed to create container

[pablocalvo-zh]

Did you manage to solve this issue? I'm facing an issue with reaper failing with the following error.. it only happens on GHA:

2024/02/13 14:47:24 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 25.0.3
  API Version: 1.43
  Operating System: Alpine Linux v3.19 (containerized)
  Total Memory: 31291 MB
  Resolved Docker Host: unix:///run/docker.sock
  Resolved Docker Socket Path: /run/docker.sock
  Test SessionID: c00c76c83c7180c78797a94d2146b5216f91200d033f9eb87efd620b53b27115
  Test ProcessID: e60189dd-ddfc-4b4c-98ab-1[70](xxx)99ea7a937
2024/02/13 14:47:24 🐳 Creating container for image testcontainers/ryuk:0.6.0
    internal_handler_test.go:283: failed to start container: look up reaper container returned nil although creation failed due to name conflict: creating reaper failed: failed to create container

I am getting the same error apparently.

[pablocalvo-zh]

@kostyay I was able to bypass this error by cleaning up containers on the tests and then using TESTCONTAINERS_RYUK_DISABLED: true. FYI I realize this happens mostly when there are multiple tests spawning containers.

https://java.testcontainers.org/features/configuration/

[webstradev]

I am experiencing this exact same issue in gitlab runners when running a few different jobs that are all using testcontainers.
4/5 of my jobs always pass and one job (which runs tests in parallel) always fails when the other 4 are running but passes when I retry it (and it is running by itself).

[abdulkk49]

I am getting a similar error in Github Actions:

Server Version: 25.0.2
  API Version: 1.44
  Operating System: Ubuntu 20.04.6 LTS
  Total Memory: 7751 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
2024/03/19 13:49:59 🐳 Creating container for image testcontainers/ryuk:0.6.0
        	Error:      	Received unexpected error:
        	            	look up reaper container returned nil although creation failed due to name conflict: creating reaper failed: failed to create container

[shepherdjerred]

I've seen this quite frequently in Jenkins when running a large number of tests. Disabling the reaper with TESTCONTAINERS_RYUK_DISABLED fixed the issue for me.

If you're in an ephemeral environment enabling TESTCONTAINERS_RYUK_DISABLED should be a fine workaround without any negative consequences.

[tuxxi]

Adding a data point here: we never hit this until upgrading testcontainers-go from 0.19 to 0.28.
Soon after, starting hitting this issue fairly regularly (I would estimate about 1 out of 4 jobs fail due to either

look up reaper container returned nil although creation failed due to name conflict

or

connecting to reaper failed: failed to create container