Failed assert() should show a backtrace

[joachim-n]

Q	A
PHPUnit version	9.5.16
PHP version	7.4.2
Installation Method	Composer

Summary

If a test fails because an assert() fails, the test output only shows the assert message. There is no backtrace, unlike when a test fails because of a thrown exception.

Current behavior

How to reproduce

Expected behavior

[Csardelacal]

I have the exact same issue when testing my code. It drives me insane. I have assert.exception = On, but PHPUnit ignores the fact that it's an exception with a trace and instead treats it like a $this->assertX() failure condition.

I assume that it has to do with the fact that PHPUnit catches AssertionError and treats it like a failure instead of an error. Which, IMO, is incorrect behavior.

Usually a $this->assertX() is called within the test itself. Which means it's trivial to find the reason why the assertion failed in the first place. A message like Failed assert that 'xyz' contains 'abc' is easy to find in the code.

In the event of having a sanity check buried deep within the code, you get an output like assert($collection->contains($item)) in /long/path/to/file.php in line 123 but the context of the call is lost. Considering that AssertionError objects have a trace, and it deviates from the Test, it should be printed.

PHPUnit version 9.5.21
PHP version 8.1
Installation Method Composer

Current behavior

There was 1 failure:

1) tests\SomethingTest::testSomethinf
assert(array_key_exists($key, $this->original)) in /home/.../src/storage/file.php:142

How to reproduce

Any assert() call causes it

Expected behavior

There was 1 failure:

1) tests\SomethingTest::testSomethinf
AssertionError: assert(array_key_exists($key, $this->original)) in /home/.../src/storage/file.php:142

/home/.../src/file1.php:122
/home/.../src/file2.php:44
/home/.../src/file3.php:19
/home/.../src/tests/test.php:189

[joachim-n]

The fix doesn't seem to be working.

I am on PHPUnit 9.5.28, and in the TestResult class I see this which matches the fix:

            $e = new AssertionFailedError(
                sprintf(
                    '%s in %s:%s',
                    $e->getMessage(),
                    $frame['file'] ?? $e->getFile(),
                    $frame['line'] ?? $e->getLine()
                ),
                0,
                $e
            );

However, my output for a failed assertion is:

There was 1 failure:

1) Drupal\Tests\computed_field\Kernel\ComputedFieldKernelTest::testComputedFields
Cannot load the "entity_test_bundle" entity with NULL ID. in /Users/joachim/Sites/drupal-contrib/web/core/lib/Drupal/Core/Entity/EntityStorageBase.php:295

/Users/joachim/Sites/drupal-contrib/vendor/phpunit/phpunit/src/Framework/TestResult.php:757

Another simpler example to reproduce:

  public function testFailAssert() {
    $this->tryassert();
  }

  function tryassert() {
    $this->tryassert2();
  }

  function tryassert2() {
    $this->tryassert3();
  }

  function tryassert3() {
    assert(0);
  }

I get:

There was 1 failure:

1) Drupal\Tests\computed_field\Kernel\ComputedFieldKernelTest::testFailAssert
assert(0) in /Users/joachim/Sites/drupal-contrib/web/modules/custom/computed_field/tests/src/Kernel/ComputedFieldKernelTest.php:114

/Users/joachim/Sites/drupal-contrib/vendor/phpunit/phpunit/src/Framework/TestResult.php:757

[sebastianbergmann]

Please understand that I cannot debug examples that depend on third-party software such as Drupal. I will gladly reopen this issue and investigate further if you can provide a minimal, self-contained, reproducing example.

[joachim-n]

Understood - try the code snippet with the testFailAssert() method.

[joachim-n]

Scratch that - I get a backtrace on a simpler project with that code:

Caused by
AssertionError: assert(0) in /Users/joachim/Sites/_sandbox/drupal-code-builder/Test/Unit/TestArrgh.php:25
Stack trace:
#0 /Users/joachim/Sites/_sandbox/drupal-code-builder/Test/Unit/TestArrgh.php(25): assert(0, 'assert(0)')
#1 /Users/joachim/Sites/_sandbox/drupal-code-builder/Test/Unit/TestArrgh.php(21): DrupalCodeBuilder\Test\Unit\TestArrgh->tryassert3()
#2 /Users/joachim/Sites/_sandbox/drupal-code-builder/Test/Unit/TestArrgh.php(17): DrupalCodeBuilder\Test\Unit\TestArrgh->tryassert2()

[joachim-n]

Drupal runs its kernel tests in a separate process - could that be the problem?

[joachim-n]

Here's a test case that shows the problem:

<?php

namespace DrupalCodeBuilder\Test\Unit;

// see https://github.com/sebastianbergmann/phpunit/issues/4913
use PHPUnit\Framework\TestCase;

 class TestAssertionBacktrace extends TestCase {

  /**
   * @runInSeparateProcess
   */
  public function testFailAssert() {
    $this->tryassert();
  }

  function tryassert() {
    $this->tryassert2();
  }

  function tryassert2() {
    $this->tryassert3();
  }

  function tryassert3() {
    assert(0);
  }

 }