Cannot mark tests as skipped or incomplete in beforeClass hooks

[jderusse]

Q	A
PHPUnit version	9.5.2
PHP version	8.0
Installation Method	Composer / PHAR

Summary

When calling self::markTestSkipped in a setUpBeforeClass method, all tests inside the file are flagged Skipped without the error "Test skipped because of an error in hook method"

When looking at https://github.com/sebastianbergmann/phpunit/blob/master/src/Framework/TestSuite.php#L438-L448
It looks like we can fix this by throwing a SkippedTestSuiteError instead. But this class is @internal.

Current behavior

1) FooTest::testMethod
Test skipped because of an error in hook method

How to reproduce

class FooTest extends TestCase {
    public static function setUpBeforeClass(): void
    {
        self::markTestSkipped('My Custom message');
    }
}

Expected behavior

1) FooTest::testMethod
My Custom message

Suggestion

Add a public static markTestSuiteSkipped method to the API

[sebastianbergmann]

Thank you for your report.

Please provide a minimal, self-contained, reproducing test case that shows the problem you are reporting.

Without such a minimal, self-contained, reproducing test case I will not be able to investigate this issue.

[jderusse]

Please find here a reproducer: https://github.com/jderusse/phpunit-4689

git clone https://github.com/jderusse/phpunit-4689
cd phpunit-4689
composer i
./vendor/bin/phpunit tests -v

output

1) FooTest::testAdd
My Custom message

/data/oss/debug/phpunit/tests/FooTest.php:9

2) FooTest::testAdd2
Test skipped because of an error in hook method

[nicolas-grekas]

FYI on Symfony, we merged symfony/symfony#41384 meanwhile.
This might break in the future as SkippedTestSuiteError is @internal, but that's the best option we have at the moment.

[sebastianbergmann]

Superseded by #5164

[sebastianbergmann]

I apologize for dropping the ball on this. I split this issue into two new issues, #5164 for PHPUnit 9.6 and #5165 for PHPUnit 10. #5164 is about to be closed, will look into #5165 next.