Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependent test of skipped test has status -1 #3379

Closed
OndraM opened this issue Nov 1, 2018 · 0 comments
Closed

Dependent test of skipped test has status -1 #3379

OndraM opened this issue Nov 1, 2018 · 0 comments
Labels
type/bug Something is broken

Comments

@OndraM
Copy link
Sponsor Contributor

OndraM commented Nov 1, 2018

Q A
PHPUnit version 7.4.3
PHP version 7.2.11
Installation Method Composer

Hi,
we have a test method which is now being skipped using markTestSkipped(), and another test method which depends on this (now skipped) test:

<?php declare(strict_types=1);

namespace My;

class SkippedTest extends \PHPUnit\Framework\TestCase
{
    public function testFirst(): void
    {
        $this->markTestSkipped('skipped');
    }

    /**
     * @depends testFirst
     */
    public function testSecond(): void
    {
        $this->assertTrue(true);
    }
}

We utilize custom TestListeners, especially addSkippedTest and endTest methods.

<?php declare(strict_types=1);

namespace My;

use PHPUnit\Framework\Test;
use PHPUnit\Framework\TestCase;

class MyTestStatusListener implements TestListener
{
    public function addSkippedTest(Test $test, \Throwable $t, float $time): void
    {
        if ($test instanceof TestCase) {
            echo "Skipped test " . $test->getName() . ", status: "  . $test->getStatus() .  "\n";
        }
    }

    public function endTest(Test $test, float $time): void
    {
        if ($test instanceof TestCase) {
            echo "End test " . $test->getName() . ", status: "  . $test->getStatus() .  "\n";
        }
    }

    // ...
}

In this scenario, expected behavior is this output:

Skipped test testFirst, status: 1
End test testFirst, status: 1
Skipped test testSecond, status: 1
End test testSecond, status: 1

However the actual output is:

Skipped test testFirst, status: 1
End test testFirst, status: 1
Skipped test testSecond, status: -1
End test testSecond, status: -1

Ie. even though the addSkippedTest() method gets called, the test status property (getStatus()) of this actual TestCase is not 1 (BaseTestRunner::STATUS_SKIPPED), but -1 (BaseTestRunner::STATUS_UNKNOWN), which seems unexpected and inconsistent.

OndraM added a commit to lmc-eu/steward that referenced this issue Nov 1, 2018
@sebastianbergmann sebastianbergmann added the type/bug Something is broken label Nov 2, 2018
OndraM added a commit to lmc-eu/steward that referenced this issue Nov 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something is broken
Projects
None yet
Development

No branches or pull requests

2 participants