Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report list of failed tests with age #186

Open
db410009 opened this issue Nov 16, 2021 · 3 comments
Open

Report list of failed tests with age #186

db410009 opened this issue Nov 16, 2021 · 3 comments
Labels
enhancement New feature or request

Comments

@db410009
Copy link

When reviewing the report for the failed tests, I miss a "table" including the list of failed test and the "age" for the failure, I mean, how many consecutive failures.

@EnricoMi EnricoMi changed the title Add (optionally) into the report the list of the failed test (class and name) and a counter with failed age Report list of failed tests with age Nov 16, 2021
@EnricoMi EnricoMi added the enhancement New feature or request label Nov 16, 2021
@EnricoMi
Copy link
Owner

So in addition to the test failures listed under annotation on the check page:
grafik

You mean to list them in the pull request comment like skipped tests are:
grafik

I think implementing the age feature would be interesting to achieve, but it gives the wrong impression. There should never be tests failing in master. It feels like supporting an anti-use-case. However, contributions are always welcome.

Adding the failed tests as a list is redundant to the list of annotations, which provides more contextual information than just the test and class names. It is just one click away.

@db410009
Copy link
Author

So in addition to the test failures listed under annotation on the check page: grafik

You mean to list them in the pull request comment like skipped tests are: grafik

I think implementing the age feature would be interesting to achieve, but it gives the wrong impression. There should never be tests failing in master. It feels like supporting an anti-use-case. However, contributions are always welcome.

Adding the failed tests as a list is redundant to the list of annotations, which provides more contextual information than just the test and class names. It is just one click away.

Yeah, I was referring to a table/list showing the test suites. It is specially interesting regarding multi module projects.

And related with the "age" and "anti-use-case", I'm almost 100% agreed with you on the bases.
The issue because I miss this is because working on a large project with many components involved and parallel development, we are detecting some tests failures on integration tests. They are not moving to main, but sometimes is hard to identify which is the commit guilty for the failure.

@AleksanderBrzozowski
Copy link

This could be useful for repository with e2e tests that are "flaky".
For example, in my organization we have a repo with e2e tests that are run every day at night on sandbox environment. We use microservice architecture and sometimes test can fail due to timeout or other reason.
Thus, we do not want to track "single" failures.
When test fails consecutively it means that some functionality might be broken. And this is what we want to track. Bamboo has this functionality - "new test failures" and "existing test failures". This is the reason why we still use bamboo instead of github actions :)

I think that this feature would require state persistence in order to know if the test failed yesterday or not :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants