-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Closed
Labels
feature/eventsIssues related to PHPUnit's event systemIssues related to PHPUnit's event systemfeature/test-runnerCLI test runnerCLI test runnertype/bugSomething is brokenSomething is brokenversion/10Something affects PHPUnit 10Something affects PHPUnit 10
Description
Q | A |
---|---|
PHPUnit version | 10.0.9 |
PHP version | 8.2.3 |
Installation Method | Composer |
Summary
A PHPUnit\Event\Test\PreparationStartedSubscriber
throws an Exception.
Current behavior
The test that has had preparation started is silently discarded. There are no indications that the test failed.
How to reproduce
- Create an Extension.
- Register a PreparationStartedSubscriber and throw an Exception.
- Ensure the Extension is set to be loaded in your PHPUnit Configuration.
- Execute PHPUnit and see that no tests are run.
I have also reproduced this in an isolated repo: https://github.com/cspray/phpunit10-bugs under the PreparationStartedException
directory.
Expected behavior
I would expect a test that has an exception thrown in a PreparationStartedSubscriber
to be properly marked as a failure/error with information on the Exception that was thrown.
Composer Info Output
myclabs/deep-copy 1.11.0 Create deep copies (clones) of your objects
nikic/php-parser v4.15.3 A PHP parser written in PHP
phar-io/manifest 2.0.3 Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version 3.2.1 Library for handling version information and constraints
phpunit/php-code-coverage 10.0.0 Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator 4.0.1 FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker 4.0.0 Invoke callables with a timeout
phpunit/php-text-template 3.0.0 Simple template engine.
phpunit/php-timer 6.0.0 Utility class for timing
phpunit/phpunit 10.0.9 The PHP Unit Testing framework.
roave/security-advisories dev-latest 9206ad5 Prevents installation of composer packages with known security vulnerabilities: no API, simply require it
sebastian/cli-parser 2.0.0 Library for parsing CLI options
sebastian/code-unit 2.0.0 Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 3.0.0 Looks up which function or method a line of code belongs to
sebastian/comparator 5.0.0 Provides the functionality to compare PHP values for equality
sebastian/complexity 3.0.0 Library for calculating the complexity of PHP code units
sebastian/diff 5.0.0 Diff implementation
sebastian/environment 6.0.0 Provides functionality to handle HHVM/PHP environments
sebastian/exporter 5.0.0 Provides the functionality to export PHP variables for visualization
sebastian/global-state 6.0.0 Snapshotting of global state
sebastian/lines-of-code 2.0.0 Library for counting the lines of code in PHP source code
sebastian/object-enumerator 5.0.0 Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector 3.0.0 Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context 5.0.0 Provides functionality to recursively process PHP variables
sebastian/type 4.0.0 Collection of value objects that represent the types of the PHP type system
sebastian/version 4.0.1 Library that helps with managing the version number of Git-hosted PHP projects
theseer/tokenizer 1.2.1 A small library for converting tokenized PHP source code into XML and potentially other formats
Metadata
Metadata
Assignees
Labels
feature/eventsIssues related to PHPUnit's event systemIssues related to PHPUnit's event systemfeature/test-runnerCLI test runnerCLI test runnertype/bugSomething is brokenSomething is brokenversion/10Something affects PHPUnit 10Something affects PHPUnit 10
Projects
Milestone
Relationships
Development
Select code repository
Activity
sebastianbergmann commentedon Feb 19, 2023
An exception thrown in a subscriber must not have any effect on how tests are run.
sebastianbergmann commentedon Feb 19, 2023
Here is what I get:
phpunit.xml
src/autoload.php
src/Extension.php
src/TestPreparedSubscriber.php
tests/Issue5219Test.php
You wrote
And this is exactly what I observe:
However, as I already wrote in #5219 (comment), this is wrong. The exception in the subscriber should not have any effect on how tests are run and/or how their outcome is evaluated.
sebastianbergmann commentedon Feb 19, 2023
I think this is what we are missing:
This would ignore all exceptions thrown in subscribers.
Closes #5219
sebastianbergmann commentedon Feb 19, 2023
I just verified that the patch shown in #5219 (comment) has the desired effect for your use case, too.
[-]Exception in PreparationStartedSubscriber silently discards test[/-][+]Exceptions thrown in event subscribers affect how tests are run and/or how their outcome is evaluated[/+]sebastianbergmann commentedon Feb 19, 2023
@cspray Thank you for reporting this issue!
cspray commentedon Feb 19, 2023
@sebastianbergmann I noticed in the example you replicated, the event is the
PHPUnit\Event\Test\Prepared
event. The event I originally submitted the issue about was thePHPUnit\Event\Test\PreprationStarted
event. I copied your example to my local machine, updated theTestPreparedSubscriber
to implement thePreparationStartedSubscriber
and I see the same error I reported.The changes I made to your
TestPreparedSubscriber
sebastianbergmann commentedon Feb 19, 2023
I am sorry, but I do not understand what you are trying to tell me.
As of 74ce5b6, all exceptions thrown in third-party subscribers are ignored (as they should be). For this, it does not matter which event a subscriber subscribes to.