Codeception/Robot - Groups - Php 8.1 - tests data wrong path

[v4ssi404]

Hello, I am doing an upgrade from php 7.3 to php 8.1 and i am also updating my composer dependencies.
The framework i am using is Yii2 but i don't think that is something framework related, probably dependencies breaking changes, but i found the causes but no the solutions.

After the upgrade the command vendor/bin/codecept run -f seems fail with my backend suite that use Groups.
And also Parallel Test (with codeception/robo-paracept) are going crazy.

With Php 7.3 everything is working fine.

I have my main codeception.yml in my root folder

include:
    - common
    - backend
    - console
paths:
    log: console/runtime/logs
settings:
    memory_limit: 2560M

And in backend folder i have my codeception.yml

namespace: backend\tests
paths:
    tests: tests
    output: tests/_output
    data: tests/_data
    support: tests/_support
    envs: tests/_envs
actor_suffix: Tester
bootstrap: _bootstrap.php
     ...other configs....
groups:
    paracept_*: tests/_data/paracept_*

On my 8.1 upgraded branch if i run vendor/bin/codecept run -f i get this error:

[backend\tests]: tests from /var/www/html/backend

In Finder.php line 592:                                                             
  The "/var/www/html/tests/_data" directory does not exist.  

He's basically not getting data inside backend/tests folder, but he's searching into the root.
Weird behavior that seems related to the groups config option because if remove

groups:
    paracept_*: tests/_data/paracept_*

or if replace it with backend/tests/_data/paracept_* my backend tests run without problem like common/console are already doing because they don't use groups.

If i also run vendor/bin/robo parallel:run i get:

 GroupManager: File or directory /var/www/html/**backend/backend**/tests/api/TestCrudApiCest.php set in paracept_1 group does not exist

This because the path of the tests are now written adding "backend" into paracept_1:

On my 7.3 branch: tests/api/TestCrudApiCest.php:testCreatingRecordWithGoodRequest
On my 8.1 branch: backend/tests/api/TestCrudApiCest.php:testCreatingRecordWithGoodRequest

Can you maybe pointing me on the right direction?
I have red Codeception and Robo docs but i didn't found breaking changes.

Thank you.

Composer - Working Version - Php 7.3

• Codeception version: was Codeception/base
• PHP Version: 7.3
• Operating System: Linux (VM)
• Installation type: Composer

    "require": {
        "php": "^7.3",
        "ext-intl": "*",
        "ext-json": "*",
        "ext-pdo": "*",
        "codeception/robo-paracept": "^0.4.2",
        "firebase/php-jwt": "^5.0",
        "guzzlehttp/guzzle": "^6.3",
        "ifsnop/mysqldump-php": "^2.5",
        "imagine/imagine": "^1.2",
        "kartik-v/yii2-mpdf": "1.0.2",
        "mdmsoft/yii2-admin": "^2.8",
        "paypal/paypal-checkout-sdk": "^1.0",
        "slam/mysql-php": "^0.2.0",
        "slam/php-validatore-fattura-elettronica": "v1.2-patch20190809.1",
        "slam/yii2-js-cookie": "^1.0",
        "webmozart/assert": "^1.3",
        "yii2tech/authlog": "^1.0",
        "yiisoft/yii2": "^2.0",
        "yiisoft/yii2-httpclient": "^2.0",
        "yiisoft/yii2-swiftmailer": "^2.1"
    },
    "require-dev": {
        "codeception/base": "^3.1",
        "codeception/verify": "^1.0",
        "flow/jsonpath": "^0.4.0",
        "friendsofphp/php-cs-fixer": "2.16.7",
        "grifart/phpstan-oneline": "^0.4.1",
        "phpstan/phpstan": "^0.12.1",
        "phpstan/phpstan-phpunit": "^0.12.2",
        "phpstan/phpstan-strict-rules": "^0.12.9",
        "phpstan/phpstan-webmozart-assert": "^0.12.13",
        "slam/php-debug-r": "^1.5",
        "slam/phpstan-extensions": "^5.0.2",
        "thecodingmachine/phpstan-strict-rules": "^0.12.1",
        "yiisoft/yii2-debug": "^2.0"
    },

Composer - Almost Working Version - Php 8.1

• Codeception version: 4.1.31
• PHP Version: 8.1
• Operating System: Linux (VM)
• Installation type: Composer

    "require": {
        "php": "^8.1",
        "ext-intl": "*",
        "ext-json": "*",
        "ext-pdo": "*",
        "firebase/php-jwt": "^5.0",
        "guzzlehttp/guzzle": "^6.3",
        "ifsnop/mysqldump-php": "^2.5",
        "imagine/imagine": "^1.2",
        "kartik-v/yii2-mpdf": "1.0.6",
        "mdmsoft/yii2-admin": "^3.2",
        "slam/mysql-php": "^0.2.0",
        "webmozart/assert": "^1.3",
        "yiisoft/yii2": "^2.0",
        "yiisoft/yii2-httpclient": "^2.0",
        "yiisoft/yii2-symfonymailer": "^2.0"
    },
    "require-dev": {
        "codeception/codeception": "^4.1",
        "codeception/module-asserts": "^1.3",
        "codeception/module-db": "^1.1",
        "codeception/module-filesystem": "2.0",
        "codeception/module-rest": "^1.3",
        "codeception/module-yii2": "^1.1",
        "codeception/robo-paracept": "^2.0",
        "codeception/verify": "^2.1.1",
        "flow/jsonpath": "^0.4.0",
        "friendsofphp/php-cs-fixer": "^3.7.0",
        "grifart/phpstan-oneline": "^0.4.2",
        "phpstan/phpstan": "^1.4.10",
        "phpstan/phpstan-phpunit": "^1.0.0",
        "phpstan/phpstan-strict-rules": "^1.1.0",
        "phpstan/phpstan-webmozart-assert": "^1.1.2",
        "slam/php-debug-r": "^v1.7.0",
        "slam/phpstan-extensions": "^v6.0.0",
        "thecodingmachine/phpstan-strict-rules": "^v1.0.0",
        "yiisoft/yii2-debug": "^2.1"
    },

[v4ssi404]

I tried using a sed command into the RoboFile to clean the wrong path or to add an asbolute one.
Robo is now working thanks to sed if path are all absolute:

/var/www/html/backend/tests/api/CompanyCrudApiCest.php:testCreatingRecordWithGoodRequest
/var/www/html/backend/tests/unit/CompanyInfoTest.php:testCompanyFullInfo

But vendor/bin/codecept run -f still fail and now if i change it like i described adding backend/ to my codeception.yml
Robo fail because find backend/backend.

If can be useful i guess through my debug that the problem is the way that path of tests are handled differently when are Cest and Unit.
The first one use backend/tests/api/CompanyCrudApiCest.php:testCreatingRecordWithGoodRequest
The second one use /var/www/html/backend/tests/unit/CompanyInfoTest.php:testCompanyFullInfo

Or on config part, the fact that Codeception run do not automatically add the suite directory when running group tests instead robo-paracept does it.

[Naktibalda]

I guess that it was broken by #6292 in 4.1.24, try downgrading to 4.1.23 and see if it helps

[v4ssi404]

I guess that it was broken by #6292 in 4.1.24, try downgrading to 4.1.23 and see if it helps

Thank you @Naktibalda , you saved me! 😄

It helped, now i changed my sed to let only relative path:
tests/api/CompanyCrudApiCest.php:testCreatingRecordWithGoodRequest and both tests type are working.

[DavertMik]

Sorry, @sviluppo looks like I accidentally broke your setup, however, this allowed tests to run on included group.

[simonberger]

@DavertMik sorry to mention you here, I can also create a new issue but I think that might come to a quicker solution.

Can you say how this change helps to run tests on included groups? This was what we did prior to this release and this changed line breaks it https://github.com/Codeception/Codeception/pull/6292/files#diff-a7930bc6948cf3fb3765c3221caebc0ee98bfb5a93adcd078c4586e74f5032edL113

We have a codeception.yml on root level, including other modules which again have suites.

Let's say simplified

• codeception.yml
• src
  • Core
    • codeception.yml
    • tests
      • api
      • api.suite.yml

api.suite.yml

...

groups:
    api: [./tests/api]

Because of the mentioned line this resolves to /tests/api and this path does not exist running to the exception in this class. Instead src/Core/tests/api need to be used.
Is there an additional configuration to be used? And what has been fixed by this change that couldn't be done before.