Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TeamCity logger logs warnings as test failures #4591

Closed
tlamy opened this issue Jan 25, 2021 · 1 comment
Closed

TeamCity logger logs warnings as test failures #4591

tlamy opened this issue Jan 25, 2021 · 1 comment
Labels
feature/logging Issues related to logging test results type/bug Something is broken

Comments

@tlamy
Copy link

tlamy commented Jan 25, 2021

Q A
PHPUnit version 9.5.1
PHP version 7.4.14
Installation Method Composer

Summary

(See https://youtrack.jetbrains.com/issue/WI-57958)
While working on a project making heavy use of at() in the test suite I noticed (again) that soft deprecations cause tests to be marked as failed, although the exit code is 0.

I've traced this to \PHPUnit\Util\Log\TeamCity::addWarning(), which logs a testFailed event.
A quick search did not reveal a list of possible event names, so I put a comment in https://youtrack.jetbrains.com/issue/WI-57958 to get help from Jetbrains.

Current behavior

Test cases with warnings are logged with testFailed event type

How to reproduce

Run a test with at() matcher or assertRegExp() or other soft-deprecated features using the --teamcity command line option:

tlamy@Thomass-iMac-Pro messaging % ./vendor/bin/phpunit tests/unittests --teamcity
PHPUnit 9.5.1 by Sebastian Bergmann and contributors.


##teamcity[testCount count='252' flowId='25286']

##teamcity[testSuiteStarted name='/Users/tlamy/git/XXX/tests/unittests' flowId='25286']

##teamcity[testStarted name='createDialogFilters' locationHint='php_qn:///Users/tlamy/git/XXX/tests/unittests/Messaging/Model/RepositoryTest.php::\Tests\unittests\Messaging\Model\RepositoryTest::createDialogFilters' flowId='25286']

##teamcity[testFailed name='createDialogFilters' message='The at() matcher has been deprecated. It will be removed in PHPUnit 10. Please refactor your test to not rely on the order in which methods are invoked.' details=' ' duration='0' flowId='25286']

##teamcity[testFinished name='createDialogFilters' duration='0' flowId='25286']
...

Expected behavior

Use another event name (not testFailed), so PhpStorm does not mark the test case as failed.

@tlamy tlamy added the type/bug Something is broken label Jan 25, 2021
@sebastianbergmann sebastianbergmann added the feature/logging Issues related to logging test results label Jan 25, 2021
@sebastianbergmann
Copy link
Owner

@SvetlanaZem @wbars @epdenouden Can you have a look, please? Thanks!

@sebastianbergmann sebastianbergmann changed the title Teamcity logger logs Warning as "testFailure" TeamCity logger logs warnings as test failures Mar 17, 2021
This was referenced Mar 17, 2021
This was referenced Mar 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/logging Issues related to logging test results type/bug Something is broken
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants