Allow tests to be run inside a docker container #141

wngr · 2019-12-13T20:46:31Z

This adds the ip_address method to Docker trait in order to to query the ip address of the spawned container. This is derived from a docker inspect call.

This also introduces a new usage pattern when used from the host system:
Rather than relying on a port mapping to the host, the presumably known target port can be directly queried by using the ip address of the container in the internal docker network.
Let me know whether this actually breaks any workflows I didn't think of.
Note: This relies on the fact, that the default docker bridge network is configured for all involved containers.

I looked into shiplift to create a new Client impl, but refrained from doing so, as the current API is completely sync, and it felt rather heavy pulling in shiplift w/ a runtime. However, I'm very much interested in a follow-up discussion, where this crate could offer an additional async API, or a way forward with the sync API.
Futhermore, with the approach presented in this PR, the orchestrating container needs to have the docker cli available. When adding more and more features, it might make sense eventually to use shiplift to interface with the docker daemon, and by this also get rid of this requirement..

Closes #129
Related #139

I also bumped the version in .travis.yml #140.

This adds the `ip_address` method to `Docker` trait in order to to query the ip address of the spawned container. This is derived from a `docker inspect` call. This also introduces a new usage pattern when used from the host system: Rather than relying on a port mapping to the host, the target port can be directly queried by using the ip address of the container in the internal docker network. Note: This relies on the fact, that the default docker bridge network is configured for all involved containers. testcontainers#129

testcontainers#140

wngr · 2019-12-13T20:54:49Z

src/clients/cli.rs

+            ports
+        } else {
+            ports
+        }


Open question, whether this API should be changed in any way.

wngr · 2019-12-13T20:55:03Z

src/clients/cli.rs

+            && image.volumes().into_iter().collect::<Vec<_>>().len() > 0
+        {
+            // TODO: Panic here or just log and continue?
+            panic!("Running inside Docker: Unable to map volumes.");


How should this be handled?

This is very restrictive. It is still possible to map volumes if they mapped correctly

bsideup · 2019-12-15T09:06:43Z

src/clients/cli.rs

+    /// default docker bridge network.
+    fn ip_address(&self, id: &str) -> String {
+        let info = self.get_container_info(id);
+        info.network_settings.ip_address


if container is connected to a network that is different from the tests' one, you will not be able to use this IP.

This is why in testcontainers-java we always connect to the gateway's IP

bsideup · 2019-12-15T09:09:20Z

Rather than relying on a port mapping to the host, the presumably known target port can be directly queried by using the ip address of the container in the internal docker network.
Let me know whether this actually breaks any workflows I didn't think of.

From testcontainers-java experience I can say that this breaks in many environments. This is why in the previous PR ( #139 ) we did it differently (but the way it works in every case)

bsideup · 2019-12-15T09:11:49Z

Also, I am a bit disappointed that you decided not not listen to the feedback based on testcontainers-java and will let you discover all the fun yourself :)

thomaseizinger · 2019-12-30T03:50:48Z

From testcontainers-java experience I can say that this breaks in many environments. This is why in the previous PR ( #139 ) we did it differently (but the way it works in every case)

@bsideup are you only talking about not using the port mapping or also determining the ip address via docker inspect?

thomaseizinger · 2019-12-30T04:55:46Z

Also, I am a bit disappointed that you decided not not listen to the feedback based on testcontainers-java and will let you discover all the fun yourself :)

Just to clarify, I am very open to allow testcontainers-rs to become as mature as testcontainers-java! My main concern in #139 was purely implementation-wise.

If the approach in #139 is superior, maybe we can take the best of both worlds and implement #139 with a design similar to this PR?

I am unfortunately not too familiar with the internals of docker, hence I don't exactly know the constraints we are facing here. Would it be too much to ask to create a structured write-up of what our constraints are for implementing this?

thomaseizinger · 2019-12-30T04:58:12Z

I looked into shiplift to create a new Client impl, but refrained from doing so, as the current API is completely sync, and it felt rather heavy pulling in shiplift w/ a runtime. However, I'm very much interested in a follow-up discussion, where this crate could offer an additional async API, or a way forward with the sync API.

@wngr That is a good point! Now that async/await is maturing in Rust and we have things like #[tokio::test] and #[async_std::test], we can at some point migrate the API to be async which then should allow us to fairly easily implement a client on top of shiplift. There is also https://github.com/fussybeaver/bollard btw.

wngr · 2020-01-15T13:46:03Z

Sorry for being passive on this PR, vacation days took its toll on me :-)..

@bsideup

Also, I am a bit disappointed that you decided not not listen to the feedback based on testcontainers-java and will let you discover all the fun yourself :)

I'm honestly confused now, whether #139 and the PR presented here are actually about the same thing. The scenario I want to realize with the proposed approach is to be able to run the actual testing code inside a docker container itself. Thus, I have to get the container's IP address. You rightfully noted, that this assumes that both containers are on the same network (TODO!).
I'd greatly appreciate if you could elaborate on what you mean by ..

This is why in testcontainers-java we always connect to the gateway's IP
.. and how the scenario I described above can be done in a more robust way.

rkudryashov · 2020-07-21T11:02:43Z

I have just encountered an issue that I think is related to this thread:

---- test_sign_in stdout ----
thread 'test_sign_in' panicked at 'Failed to execute docker command: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/testcontainers-0.9.1/src/clients/cli.rs:102:21
Panic in Arbiter thread.

It would be great to make this working! Are there any news on the PR?

thomaseizinger · 2020-07-27T23:58:01Z

It would be great to make this working! Are there any news on the PR?

Nothing other than what you can see on this thread unfortunately :)

From my perspective, I think efforts stalled mainly due to confusion of what usecases we actually want to support 😬

Like I said above, it would be very helpful if we could have a write-up of which different usecases we want to solve and what the traps around this are. @bsideup mentioned that they discovered several of those in testcontainers-java :)

mergify · 2020-09-30T02:53:24Z

This repository is using GitFlow. New features should be merged into dev instead of master. Please open a new pull request that targets dev instead.

wngr added 2 commits December 13, 2019 21:36

Bump stable rust version on travis to 1.36.0, respectively 1.39.0.

c55024b

testcontainers#140

wngr commented Dec 13, 2019

View reviewed changes

bsideup reviewed Dec 15, 2019

View reviewed changes

wngr mentioned this pull request Jan 15, 2020

Provide an async API #142

Closed

thomaseizinger mentioned this pull request Sep 14, 2020

add possibility to start container in host network #201

Closed

mergify bot closed this Sep 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow tests to be run inside a docker container #141

Allow tests to be run inside a docker container #141

wngr commented Dec 13, 2019

wngr Dec 13, 2019

wngr Dec 13, 2019

bsideup Dec 15, 2019

bsideup Dec 15, 2019

bsideup commented Dec 15, 2019

bsideup commented Dec 15, 2019

thomaseizinger commented Dec 30, 2019

thomaseizinger commented Dec 30, 2019

thomaseizinger commented Dec 30, 2019

wngr commented Jan 15, 2020

rkudryashov commented Jul 21, 2020

thomaseizinger commented Jul 27, 2020

mergify bot commented Sep 30, 2020

Allow tests to be run inside a docker container #141

Allow tests to be run inside a docker container #141

Conversation

wngr commented Dec 13, 2019

wngr Dec 13, 2019

Choose a reason for hiding this comment

wngr Dec 13, 2019

Choose a reason for hiding this comment

bsideup Dec 15, 2019

Choose a reason for hiding this comment

bsideup Dec 15, 2019

Choose a reason for hiding this comment

bsideup commented Dec 15, 2019

bsideup commented Dec 15, 2019

thomaseizinger commented Dec 30, 2019

thomaseizinger commented Dec 30, 2019

thomaseizinger commented Dec 30, 2019

wngr commented Jan 15, 2020

rkudryashov commented Jul 21, 2020

thomaseizinger commented Jul 27, 2020

mergify bot commented Sep 30, 2020