support progress bar formatters #755

ivandenysov · 2020-03-20T23:49:04Z

What does this PR add?

new command line option --progress-bar-compatible that makes parallel_tests work nice with fuubar formatter for rspec

Why do we need it?

I tried to use fuubar with parallel_tests, but it doesn't look good: all progress bars are displayed concurrently on the same line. This PR adds an option that makes parallel_tests display every progress bar separately.

I thought that it might be possible to implement a generic solution that will work with any progress bar-like formatters in all supported test framework, but now I doubt it. I think that it will be complicated to achieve unless there will be a clear way to distinguish between the progress bar itself and other output that should be stored and displayed after all threads of parallel_tests finish execution. I want this option to act like --serialize-stdout, but it should display progress bars immediately and refresh them constantly

TODO

Fix blinking of progress bars. I'm probably not using the best way to rewrite content of the terminal.
Fix malformed output if there are pending or failing examples
Don't use ruby's global variables for output storage and mutex
Add tests

Any advice or guidance will be much appreciated

grosser · 2020-03-21T01:10:21Z

lib/parallel_tests/output_rewriter.rb

@@ -0,0 +1,24 @@
+require 'parallel_tests'
+
+module ParallelTests


some comments here would be nice (what it does / how it works etc)

grosser · 2020-03-21T01:12:04Z

lib/parallel_tests/cli.rb

@@ -212,6 +212,7 @@ def parse_options!(argv)
        opts.on("--serialize-stdout", "Serialize stdout output, nothing will be written until everything is done") { options[:serialize_stdout] = true }
        opts.on("--prefix-output-with-test-env-number", "Prefixes test env number to the output when not using --serialize-stdout") { options[:prefix_output_with_test_env_number] = true }
        opts.on("--combine-stderr", "Combine stderr into stdout, useful in conjunction with --serialize-stdout") { options[:combine_stderr] = true }
+        opts.on("--progress-bar-compatible", "Serialize stdout output, but write immediately and rewrite on the fly") { options[:progress_bar_compatible] = true; options[:combine_stderr] = false }


how about store and instance of the rewriter in the options so we don't have to use global vars ?

grosser · 2020-03-21T01:12:53Z

lib/parallel_tests/test/runner.rb

-              unless options[:serialize_stdout]
+
+              if options[:progress_bar_compatible]
+                ParallelTests::OutputRewriter.rewrite(new_group_output: result, group_index: env['TEST_ENV_NUMBER'].to_i)


best pass in the env number via options or a new argument

grosser · 2020-03-21T01:15:02Z

that's some funky hackery :D
... how about make each process output via junit or so and then parse that with a single progessbar so we get an overall progress ?

ivandenysov · 2020-03-27T08:37:30Z

@grosser Thank you for your feedback

that's some funky hackery :D
... how about make each process output via junit or so and then parse that with a single progessbar so we get an overall progress ?

Ok. I can use that approach. Maybe we can then reuse that JUnit parsing to produce a metadata file that will be used to split future test runs into more even groups (execution time wise).

rgaufman · 2020-05-01T14:31:20Z

This looks awesome! -- How would I test this? -- I added this to my Gemfile:

  gem 'parallel_tests', github: 'ivan-denysov/parallel_tests', branch: 'support-progress-bar-formatters'

and I added --progress-bar-compatible to .rspec_parallel

Now it's saying:

$ bundle exec rake parallel:spec
/usr/local/lib/ruby/gems/2.7.0/gems/faraday-0.15.4/lib/faraday/options.rb:166: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
/usr/local/lib/ruby/gems/2.7.0/gems/faraday-0.15.4/lib/faraday/options.rb:166: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
/usr/local/lib/ruby/gems/2.7.0/gems/faraday-0.15.4/lib/faraday/options.rb:166: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
/usr/local/lib/ruby/gems/2.7.0/gems/faraday-0.15.4/lib/faraday/options.rb:166: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
/usr/local/lib/ruby/gems/2.7.0/gems/faraday-0.15.4/lib/faraday/options.rb:166: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
/Users/hackeron/Development/Tether/timeline/vendor/cache/will_paginate-66f3a4209ec5/lib/will_paginate/deprecation.rb:35: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
/Users/hackeron/Development/Tether/timeline/vendor/cache/will_paginate-66f3a4209ec5/lib/will_paginate/deprecation.rb:35: warning: Capturing the given block using Proc.new is deprecated; use `&block` instead
12 processes for 118 specs, ~ 9 specs per process
invalid option: --progress-bar-compatible (defined in .rspec_parallel)
...

Any ideas?

mollerhoj · 2022-01-10T15:39:18Z

@rgaufman you probably figured this out already, but the option goes in the terminal, not in the .rspec_parallel file:

bundle exec parallel_test spec -t rspec --progress-bar-compatible

mollerhoj · 2022-01-10T15:41:57Z

@grosser this works beautifully, and I really like it. Would you be open to this solution if fix the issues described in the TODO section?

@ivan-denysov I imagine you are no longer working on it?

It's actually quite nice to have the progress bars separated out, I gives you a good idea about how balanced the tests are.

grosser

for this to be mergeable I'd like:

testing
docs
bit of cleanup (see suggestions)
ideally also a better name because it sounds super cryptic ... not sure if there is something better

support progress bar formatters

c616270

grosser reviewed Mar 21, 2020

View reviewed changes

grosser reviewed Jan 14, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support progress bar formatters #755

support progress bar formatters #755

ivandenysov commented Mar 20, 2020

grosser Mar 21, 2020 •

edited

grosser Mar 21, 2020

grosser Mar 21, 2020

grosser commented Mar 21, 2020

ivandenysov commented Mar 27, 2020

rgaufman commented May 1, 2020

mollerhoj commented Jan 10, 2022 •

edited

mollerhoj commented Jan 10, 2022 •

edited

grosser left a comment

		@@ -0,0 +1,24 @@
		require 'parallel_tests'

		module ParallelTests

support progress bar formatters #755

Are you sure you want to change the base?

support progress bar formatters #755

Conversation

ivandenysov commented Mar 20, 2020

What does this PR add?

Why do we need it?

TODO

Any advice or guidance will be much appreciated

grosser Mar 21, 2020 • edited

Choose a reason for hiding this comment

grosser Mar 21, 2020

Choose a reason for hiding this comment

grosser Mar 21, 2020

Choose a reason for hiding this comment

grosser commented Mar 21, 2020

ivandenysov commented Mar 27, 2020

rgaufman commented May 1, 2020

mollerhoj commented Jan 10, 2022 • edited

mollerhoj commented Jan 10, 2022 • edited

grosser left a comment

Choose a reason for hiding this comment

grosser Mar 21, 2020 •

edited

mollerhoj commented Jan 10, 2022 •

edited

mollerhoj commented Jan 10, 2022 •

edited