Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestDox result printer does not handle repeated test execution correctly #4632

Closed
ricfio opened this issue Mar 28, 2021 · 2 comments
Closed
Assignees
Labels
feature/test-runner CLI test runner feature/testdox The TextDox printer/formatter type/bug Something is broken

Comments

@ricfio
Copy link

ricfio commented Mar 28, 2021

Q A
PHPUnit version 9.5.4
PHP version 8.0.1
Installation Method Composer

Summary

PHP Warnings (Undefined array key) occurs (in TestDoxPrinter.php and CliTestDoxPrinter.php) when use --repeat parameter with --testdox parameter.
No problem when use --repeat parameter without --testdox parameter.
(detected with several tests of the phpunit tutorial).

Current behavior

vscode ➜ /workspaces/phpunit-tutorial/www (develop ✗) $ ./vendor/bin/phpunit tests/PHPUnitRepeatTest.php --repeat 3 --testdox
PHPUnit 9.5.4 by Sebastian Bergmann and contributors.

PHPUnit Repeat
 ✘ Basic
   ┐
   ├ Failed asserting that false is true.
   │
   ╵ /workspaces/phpunit-tutorial/www/tests/PHPUnitRepeatTest.php:9
   ┴

 ✘ Basic
   ┐
   ├ Failed asserting that false is true.
   │
   ╵ /workspaces/phpunit-tutorial/www/tests/PHPUnitRepeatTest.php:9
   ┴

Warning: Undefined array key 1 in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/TestDoxPrinter.php on line 243

Warning: Undefined array key "testName" in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 187

Warning: Undefined array key "className" in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 188


Warning: Undefined array key "className" in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 193
PHPUnit Repeat
 ✘ Basic
   ┐
   ├ Failed asserting that false is true.
   │
   ╵ /workspaces/phpunit-tutorial/www/tests/PHPUnitRepeatTest.php:9
   ┴

Time: 00:00.711, Memory: 6.00 MB


FAILURES!
Tests: 3, Assertions: 3, Failures: 3.

How to reproduce

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class PHPUnitRepeatTest extends TestCase 
{
    public function testBasic()
    {
        $this->assertTrue(false);
    }
}

./vendor/bin/phpunit tests/PHPUnitRepeatTest.php --repeat 3

Expected behavior

No PHP Warnings (Undefined array key) showed (see following):

Warning: Undefined array key 1 in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/TestDoxPrinter.php on line 243

Warning: Undefined array key "testName" in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 187

Warning: Undefined array key "className" in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 188


Warning: Undefined array key "className" in /workspaces/phpunit-tutorial/www/vendor/phpunit/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 193
@ricfio ricfio added the type/bug Something is broken label Mar 28, 2021
@sebastianbergmann sebastianbergmann self-assigned this Mar 28, 2021
@sebastianbergmann sebastianbergmann added feature/test-runner CLI test runner feature/testdox The TextDox printer/formatter labels Mar 28, 2021
@sebastianbergmann
Copy link
Owner

Issue4632Test.php

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class Issue4632Test extends TestCase 
{
    public function testOne(): void
    {
        $this->assertTrue(false);
    }
}
$ phpunit Issue4632Test.php --testdox --repeat 1 
PHPUnit 9.5.4 by Sebastian Bergmann and contributors.

Issue4632
 ✘ One
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/Issue4632Test.php:8
   │

Time: 00:00.022, Memory: 4.00 MB


FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
$ phpunit Issue4632Test.php --testdox --repeat 2
PHPUnit 9.5.4 by Sebastian Bergmann and contributors.

Issue4632
 ✘ One
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/Issue4632Test.php:8
   │

 ✘ One
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/Issue4632Test.php:8
   │

Time: 00:00.021, Memory: 4.00 MB


FAILURES!
Tests: 2, Assertions: 2, Failures: 2.
$ phpunit Issue4632Test.php --testdox --repeat 3
PHPUnit 9.5.4 by Sebastian Bergmann and contributors.

Issue4632
 ✘ One
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/Issue4632Test.php:8
   │

 ✘ One
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/Issue4632Test.php:8
   │

Warning: Undefined array key 1 in /usr/local/src/phpunit/src/Util/TestDox/TestDoxPrinter.php on line 243

Warning: Undefined array key "testName" in /usr/local/src/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 187

Warning: Undefined array key "className" in /usr/local/src/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 188


Warning: Undefined array key "className" in /usr/local/src/phpunit/src/Util/TestDox/CliTestDoxPrinter.php on line 193
Issue4632
 ✘ One
   │
   │ Failed asserting that false is true.
   │
   │ /home/sb/Issue4632Test.php:8
   │

Time: 00:00.022, Memory: 4.00 MB


FAILURES!
Tests: 3, Assertions: 3, Failures: 3.

@sebastianbergmann sebastianbergmann changed the title PHP Warnings (Undefined array key) occurs when use --repeat parameter with --testdox parameter TestDox result printer does not handle repeated test execution correctly Mar 28, 2021
@sebastianbergmann
Copy link
Owner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/test-runner CLI test runner feature/testdox The TextDox printer/formatter type/bug Something is broken
Projects
None yet
Development

No branches or pull requests

2 participants