Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add error-exit-code to differentiate from failures
it can be helpful to know if RSpec fails because of an example or if it errors out because it couldn't load a spec file, or there was an issue in the a before(:suite) hook or etc i've named the setting error-exit-code, and tried to add it to all the relevant places, but i don't know rspec codebase well so there was some guessing. in particular i'm not sure what bisect should do. i have it fall back to the failure-exit-code before defaulting to 1 so there's no changes if people don't opt in to the setting. the specific use of this is our CI automatically retries failures using the persistence file, and assumes if they pass on that retry they were flaky. however, the persistence file isn't written to when there's an error outside of examples, so this could mean falsely passing builds. by checking for a different exit code, and then not running the retry we can avoid this issue.
- Loading branch information
Showing
12 changed files
with
197 additions
and
11 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
Feature: error exit code | ||
|
||
Use the `error_exit_code` option to set a custom exit code when RSpec fails outside an example. | ||
|
||
```ruby | ||
RSpec.configure { |c| c.error_exit_code = 42 } | ||
``` | ||
|
||
Background: | ||
Given a file named "spec/spec_helper.rb" with: | ||
"""ruby | ||
RSpec.configure { |c| c.error_exit_code = 42 } | ||
""" | ||
|
||
Scenario: A erroring spec with the default exit code | ||
Given a file named "spec/typo_spec.rb" with: | ||
"""ruby | ||
RSpec.escribe "something" do # intentional typo | ||
it "works" do | ||
true | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/typo_spec.rb` | ||
Then the exit status should be 1 | ||
|
||
Scenario: A erroring spec with a custom exit code | ||
Given a file named "spec/typo_spec.rb" with: | ||
"""ruby | ||
require 'spec_helper' | ||
RSpec.escribe "something" do # intentional typo | ||
it "works" do | ||
true | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/typo_spec.rb` | ||
And the exit status should be 42 | ||
|
||
|
||
Scenario: Success running specs spec with a custom error exit code defined | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require 'spec_helper' | ||
RSpec.describe "something" do | ||
it "works" do | ||
true | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the exit status should be 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters