Relative group paths broke when combined with included test suites

[simonberger]

Given a simplified project structure with a root level codeception.yml which includes others with multiple suites and defined groups.

I made a reproducer for this: https://github.com/simonberger/codeception-groups-problem It contains a readme with the following steps:

Demonstraiting a break of including codeception configurations + using groups in included suites in codeception patch release 4.1.24.
Codeception 5 behaves same as ^4.1.24

1. run `composer update`
2. note running multiple variants on groups and suites are working with codeception 4.1.23
  - `vendor/bin/codecept run`
  - `vendor/bin/codecept run -g unit1`
  - `vendor/bin/codecept run -c src/Core`
  - `vendor/bin/codecept run -c src/Core -g unit1`
3. change `codeception/codeception` constraint in composer.json to `^5` or `4.1.24` (same result) and execute `composer update`
4. run any of the above codeception commands. All are failing.

To give some more details. The problem occurs in a structure like this:

• codeception.yml
• src
  • Core
    • codeception.yml
    • tests
      • api
      • api.suite.yml

api.suite.yml

...

groups:
    api: [./tests/api]

The group is handled with the base path of the first codeception.yml file because of this code change which broke in in version 4.1.24 as also mentioned here.

I think this was a regular use case which made sense or wasn't it? I also did not find a way to work around it without breaking at least any way tests can be executed or rebuilding the whole big project structure.
I could work on a patch for codeception but for this I need to understand why the code change has been done. Beside the break of our use case I can not come up with a scenario this fixed.

[simonberger]

This is still an issue, but I doubt something can easily and will be fixed, so I will close here.

We resolved it on our side after updating our code to PHP 8 using codeception 5 with sharding.
While sharding is no 1:1 replacement for groups having advantages and disadvantages, it suits our needs of parallel execution.