Add --single-process-tag option to run specific Gherkin scenarios sequentially #810
Conversation
lib/parallel_tests/cli.rb
Outdated
| @@ -197,7 +197,11 @@ def parse_options!(argv) | |||
| (options[:single_process] ||= []) << /#{pattern}/ | |||
| end | |||
|
|
|||
| opts.on("-i", "--isolate", "Do not run any other tests in the group used by --single(-s)") do | |||
| opts.on("--single-process-tag [TAG]", "Run all scenarios with specified tag in the same process") do |tag| | |||
There was a problem hiding this comment.
| opts.on("--single-process-tag [TAG]", "Run all scenarios with specified tag in the same process") do |tag| | |
| opts.on("--single-tag [TAG]", "Run all scenarios with specified tag in the same process") do |tag| |
There was a problem hiding this comment.
also say cucumber/gherkin or so since scenarios makes no sense for rspec/test
There was a problem hiding this comment.
this is a refactor and a new feature ... hard to know what's going on and if all edge-cases are covered :(
I'd prefer if we do:
@scenarios << [[uri, feature_element.source_line].join(":"), scenario_tags]
and keep the old structure so it's clear what is new
... or do the refactor first without new behavior
…arios sequentially
andrii-rymar
force-pushed
the
single-process-tag
branch
from
5aa32e4
to
a5a01bd
Compare
…arios sequentially
andrii-rymar
force-pushed
the
single-process-tag
branch
from
339e4ed
to
d4bc8e7
Compare
| @@ -9,20 +11,16 @@ def by_steps(tests, num_groups, options) | |||
|
|
|||
| def by_scenarios(tests, num_groups, options = {}) | |||
| scenarios = group_by_scenarios(tests, options) | |||
| in_even_groups_by_size(scenarios, num_groups) | |||
| in_even_groups_by_size(scenarios, num_groups, options.slice(*BY_SCENARIOS_SUPPORTED_OPTIONS)) | |||
There was a problem hiding this comment.
could just pass in all options and let it pick what it wants
| single_process_patterns.any? { |pattern| item =~ pattern } | ||
| end | ||
| # add all files/scenarios that should run in a single process to one group | ||
| single_items, items = separate_single_items(items, options) |
| item = item_with_tags?(item) || item_with_size?(item) ? item[0] : item | ||
| options[:single_process].any? { |pattern| item =~ pattern } | ||
| elsif options[:single_process_tag] | ||
| raise "--single-tag option can only be used with '--group-by scenarios'" unless item_with_tags?(item) |
There was a problem hiding this comment.
put the raise into the option parser so it fails early ?
| items.partition { |item| to_single_items?(item, options) } | ||
| end | ||
|
|
||
| def to_single_items?(item, options) |
There was a problem hiding this comment.
| def to_single_items?(item, options) | |
| def single_item?(item, options) |
There was a problem hiding this comment.
this whole "item is item or item and size or item and tags" is kinda funky ... if you have any good ideas how to clean that up lmk or leave a TODO comment on how that can be improved
| @@ -138,7 +153,30 @@ def group_features_by_size(items, groups_to_fill) | |||
| end | |||
|
|
|||
| def items_to_group(items) | |||
There was a problem hiding this comment.
some rough comment on what this method is used/does for would be nice
|
|
||
| def to_single_items?(item, options) | ||
| if options[:single_process] | ||
| item = item_with_tags?(item) || item_with_size?(item) ? item[0] : item |
There was a problem hiding this comment.
| item = item_with_tags?(item) || item_with_size?(item) ? item[0] : item | |
| item = item[0] if item_with_tags?(item) || item_with_size?(item) |
There was a problem hiding this comment.
... maybe just if item.is_a?(Array) ... but not sure
|
@grosser thanks for the comments |
This PR adds new
--single-process-tagoption to run all Gherkin scenarios marked by some tag within single process.It's aimed to deal with the situation when we have some scenarios sharing the same data thus required to run sequentially.
Currently to achieve that we have two options:
--exclude-patternand run them outside parallel_tests-tCucumber option (-t "@tag_to_run and not @tag_not_to_run") and run them separately as well.Both ways require either changes to features which can be undesirable, for example if features are used as a documentation and need to have a specific structure, or to execution logic to have multiple runs performed sequentially instead of one.
Proposed change simplify this process down to one change required - adding a specific tag to all scenarios needed to be run sequentially. Also it's possible to combine it with
--isolateto have a single process only for selected cases.As for the code - it includes some workarounds to store scenario tags both with scenario location in Grouper. Probably it would be helpful in the future to add other scenario-related features.
Would like to know your opinion whether or not we can proceed with that as these changes are pretty major.
Checklist
master(if not - rebase it).code introduces user-observable changes.