Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For tests that pass but which were retried, bats returns non-zero exit code #660

Closed
djmcgreal-cc opened this issue Sep 30, 2022 · 1 comment · Fixed by #670
Closed

For tests that pass but which were retried, bats returns non-zero exit code #660

djmcgreal-cc opened this issue Sep 30, 2022 · 1 comment · Fixed by #670
Labels
Component: Bash Code Everything regarding the bash code Priority: Critical Broken behavior in nearly all environments, e.g. wrong test results, internal bats error Size: Small Changes to a few localized lines of code (e.g. same function) Type: Bug

Comments

@djmcgreal-cc
Copy link

To Reproduce

echo '
@test "fail once" {
    BATS_TEST_RETRIES=2
    ((BATS_TEST_TRY_NUMBER > 1)) || false
}
' > tst.bats
bats tst.bats
[ "$?" -eq 0 ] && echo 'reported success' || echo 'reported failure'

Expected behavior

reported success should be printed, as per the test output

Environment (please complete the following information):

  • Bats Version: 1.8
  • OS: MacOS
  • Bash version: [e.g. 5.1] 3.2
@djmcgreal-cc djmcgreal-cc added Priority: NeedsTriage Issue has not been vetted yet Type: Bug labels Sep 30, 2022
@martin-schulze-vireso martin-schulze-vireso added Component: Bash Code Everything regarding the bash code Priority: Critical Broken behavior in nearly all environments, e.g. wrong test results, internal bats error Size: Medium Changes in the same file Status: Unconfirmend No reproducer was provided or the reproducer did not work for maintainers. and removed Priority: NeedsTriage Issue has not been vetted yet labels Oct 1, 2022
@martin-schulze-vireso
Copy link
Member

I was surprised since there are tests in place but I am afraid that they don't check the exit code in isolation.

@martin-schulze-vireso martin-schulze-vireso added Size: Small Changes to a few localized lines of code (e.g. same function) and removed Status: Unconfirmend No reproducer was provided or the reproducer did not work for maintainers. Size: Medium Changes in the same file labels Oct 1, 2022
martin-schulze-vireso added a commit to martin-schulze-vireso/bats-core that referenced this issue Oct 19, 2022
martin-schulze-vireso added a commit to martin-schulze-vireso/bats-core that referenced this issue Oct 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Bash Code Everything regarding the bash code Priority: Critical Broken behavior in nearly all environments, e.g. wrong test results, internal bats error Size: Small Changes to a few localized lines of code (e.g. same function) Type: Bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants