Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple compose files #43

Open
l3s2d opened this issue Nov 4, 2019 · 3 comments · May be fixed by #64
Open

Multiple compose files #43

l3s2d opened this issue Nov 4, 2019 · 3 comments · May be fixed by #64
Labels
enhancement New feature or request

Comments

@l3s2d
Copy link

l3s2d commented Nov 4, 2019

I've looked around for examples but couldn't find any. I have a directory structure like this:

└── tests
    ├── test1
    │   ├── docker-compose.yml
    │   └── test_1.py
    └── test2
        ├── docker-compose.yml
        ├── test_2.py
        └── test_3.py

I'm looking for a way to call pytest from the project root (the directory above tests, and have pytest recursively run each test. Each test should be run against the compose file in the same directory as itself. Is there a way to specify with pytest.ini or conftest.py? Alternatively, is it possible to specify in the test_*.py which compose file to use?

@todofixthis todofixthis added the enhancement New feature or request label Nov 4, 2019
@RmStorm
Copy link
Contributor

RmStorm commented Feb 11, 2020

First off: sorry for the very slow reply!

I have to think a little bit about this. At first glance my guess is that it would be quite difficult to implement. Currently there is a 'top level' fixture in this plugin called docker_project. This fixture manages the docker-compose project and verifies that there are not already containers running beforehand and stuff. This fixture is session scoped to allow spinning up the containers once for all the tests. To use multiple docker compose files this fixture has to be downgraded to at least the module level. This would mean that the containers also have to be spun up and down per module. To prevent this, the project fixture would have to come in multiple 'scope flavours'. This would allow some people to use a session scoped project fixture and run all the tests against a single set of running containers. People in your situation could instantiate a project fixture per module or class and scope the containers to the same scope.

While that sounds like a fairly straightforward change, the docker_project fixture gets its configuration from command line options. To implement this in a nice way there would have to be some way to configure it as a fixture from conftest instead or fixture parameters instead.

I think it's possible and it can be made to be fairly practical. I do however think it will be hard to add this in a backwards incompatible way. The current plugin design is just quite unfit for it.

If you make a fork and get something to work I would be happy to look at a PR!

@RmStorm
Copy link
Contributor

RmStorm commented Sep 10, 2020

Hi @l3s2d, in #54 @mathieu-lemay added the ability to use multiple docker compose files as described here. Does this fit your usecase? I think it's quite different from using a different compose file in every directory but maybe it helps and we can close this issue?

Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Sep 11, 2020
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict that keeps all docker_project instances involved in the pytest session. At the same time, docker_project session scoped fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
@Toilal
Copy link

Toilal commented Sep 11, 2020

@RmStorm Please check my PR, it's implemented and now far easier to use/configure the plugin.

Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Sep 11, 2020
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict that keeps all docker_project instances involved in the pytest session. At the same time, docker_project session scoped fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Sep 11, 2020
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Sep 11, 2020
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Feb 10, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Feb 10, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Feb 10, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Feb 10, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Feb 10, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Feb 10, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Feb 10, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Toilal added a commit to Toilal/pytest-docker-compose that referenced this issue Oct 14, 2021
…ns inside test directory structure.

docker-compose configuration is now loaded from 3 directories:
 - Directory containing the actual test
 - "docker-compose" directory at the same level of the directory containing the actual test
 - Current working directory.

Effective directory is the first one in this list where docker-compose can run properly. Effective directory can be forced with --docker-compose-directory option.

All docker_project objects are now registered in all_docker_projects registry, a simple dict available as a session scoped fixture that keeps all docker_project instances involved in the pytest session. At the same time, docker_project fixture has been dropped. This change makes possible to have different docker_project instances for the run session, while still allowing to reuse the docker_project instance when the same configuration is involved.

ContainerGetter has been improved to wait for container to be in running state before returning. To retrieve a stopped service, user can use set wait_running argument to False. This changes makes tests more stable. (In my use case, network_info was often empty because it was retrieved when a container was still starting or restarting)

Some broken tests were fixed too.

I also tried to update CircleCI configuration to make it test with Python 3.7/3.8.

Close pytest-docker-compose#43
Close pytest-docker-compose#63

BREAKING CHANGE: --docker-compose option doesn't support "," separator anymore and should now be used many times to include many configuration files.
BREAKING CHANGE: docker_project fixture has been dropped, use all_docker_projects instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
4 participants