Exit with non-zero code if no file/directory to test was found

[mvorisek]

(opening as a feature request, even it looks like a bug to me)

This is a feature request to exit phpunit with non-zero status code if no file to test was found.

Currently, phpunit exits with zero status code which is dangerous, as tests passes even if data directory with tests is completely missing.

Example config (phpunit passes even if tests directory does not exist):

<phpunit bootstrap="vendor/autoload.php" colors="true">
    <testsuites>
        <testsuite name="tests">
            <directory>tests</directory>
        </testsuite>
    </testsuites>
</phpunit>
``

[sebastianbergmann]

Duplicate of #4493.

[mvorisek]

@sebastianbergmann can you please advise why I am getting the following error:

PHPUnit 9.5-dev by Sebastian Bergmann and contributors.
Runtime:       PHP 8.0.0
Configuration: prefix-phpunit.xml.dist
Call to undefined method Maha\PhpCiTestProject\Tests\ExampleTest::getAnnotations()
ERROR: Job failed: exit code 1

when I use dev phpunit downloaded by:

composer require 'phpunit/phpunit:dev-master#60890a4 as 9.5.0-alpha0' --ansi --prefer-dist --prefer-stable --no-progress --classmap-authoritative

I think this error must be in phpunit, tested with this minimalistic configuration and one simple test:

<phpunit bootstrap="vendor/autoload.php" colors="true">
    <testsuites>
        <testsuite name="tests">
            <directory>tests</directory>
        </testsuite>
    </testsuites>
    <listeners>
        <listener class="JohnKary\PHPUnit\Listener\SpeedTrapListener" />
    </listeners>
</phpunit>

<?php

declare(strict_types=1);

namespace Maha\PhpCiTestProject\Tests;

use Maha\PhpCiTestProject\Example;

class ExampleTest extends \PHPUnit\Framework\TestCase
{
    public function testBasic(): void
    {
        $this->assertTrue(1 === 1);
    }
}

thanks in advance.

PS: with 09ba0b8 commit it works, with 60890a4 (used above) it does not even CI here is green

[sebastianbergmann]

I would not call a setup that includes a PHPUnit extension (<listener class="JohnKary\PHPUnit\Listener\SpeedTrapListener" />) minimal.

[sebastianbergmann]

PHPUnit 9.4 has a TestCase::getAnnotations() method that is annotated with @internal This method is not covered by the backward compatibility promise for PHPUnit. This method no longer exists in what will be PHPUnit 9.5. @johnkary's SpeedTrapListener class uses this method when it should not.

[mvorisek]

thanks!

[johnkary]

Thank you @sebastianbergmann for explaining the fix. johnkary/phpunit-speedtrap listener has been updated to address this issue.

Thank you for your continued efforts on PHPUnit and within the PHP community :)