fix compatibility with PHPUnit error expectations

[calvinalkan]

I'm surprised that nobody noticed this before, but PHPUnits expectNotice() expectWarning(), expectError() methods dont seem to work at all.

This is because the ErrorHandler transforms all errors into exceptions instead of conditionally looking at the error code (like the PHPUnit error handler).

I created tests that prove this.

For deprecations, it's the same issue and I created a failing test. I don't now how to fix that tho because I don't understand how deprecations are currently handled here.

[calvinalkan]

Tests are failing because of tests/unit/Codeception/Subscriber/ErrorHandlerTest.php::testShowsLocationOfWarning which is a test that IMO is only proving that things don't work with the PHPUnit API.

But maybe that's desired..?

[Naktibalda]

There methods are quite recent, they there introduced in PHPUnit 8.4,

[Naktibalda]

To get this change into Codeception 4, it must be compatible with PHPUnit 5.7.
Classes you used here were renamed in PHPUnit 6.0, so this code would cause fatal error.
Please add some test that runs on all versions of PHP.
Then add class aliases to https://github.com/Codeception/phpunit-wrapper/blob/6.0/src/shim.php

[calvinalkan]

@Naktibalda What about the deprecation stuff?

Codeception should support expectDeprecation aswell.

But I don't understand what is done there currently. Looks like something symfony-related.

[Naktibalda]

Codeception doesn't have to support any of this.

Deprecations are counted and summary is printed below test results:

[Naktibalda]

There is no equivalent of Notice class in PHPUnit 5.7.

I think that it would be the best to wrap this section with

if (version_compare(\PHPUnit\Runner\Version::id(), '8.4.0', '>=')) {
   ...
}

[calvinalkan]

Codeception doesn't have to support any of this.

Deprecations are counted and summary is printed below test results:

@Naktibalda But doesn't this just show the deprecations that occurred while running the test?

What I'm talking about (and what PHPUnit supports in the TestCase class) is making assertions about deprecations.

class Foo {

   public function foo(): {
     trigger_error('foo is deprecated', E_USER_DEPRECATED);
   }

}

class FooTest extends Codeception\Unit {

  public function test_that_foo_triggers_deprecation() {
  
     $this->expectDeprecation('foo is deprecated');
     
     (new Foo())->foo();
     
  }


}

[Naktibalda]

Since 8.5.21 PHPUnit doesn't convert deprecations to exceptions by default, it requires setting configuration option.

https://github.com/sebastianbergmann/phpunit/blob/8.5/ChangeLog-8.5.md#8521---2021-09-25

You could implement convert_deprecations_to_exceptions option too.

• See how error_level option is handled in handle.
• Add new option to $defaultSuiteSettings https://github.com/Codeception/Codeception/blob/4.2/src/Codeception/Configuration.php#L116-L137 (the default value should be false too).
• Document at https://github.com/Codeception/codeception.github.com/blob/master/docs/reference/Configuration.md#settings

[Naktibalda]

testNotice

[Naktibalda]

There is no equivalent of Notice class in PHPUnit 5.7.

I think that it would be the best to wrap this section with

if (version_compare(\PHPUnit\Runner\Version::id(), '8.4.0', '>=')) {
   ...
}

[Naktibalda]

And change this condition to

if (version_compare(\PHPUnit\Runner\Version::id(), '8.4.0', '<')) {

[Naktibalda]

I implemented all proposed changes myself in #6469

[calvinalkan]

@Naktibalda Great. Thanks a lot.