You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When some of the tests from job fails, it would be good to be able to re-run only the failed tests with the same configuration to reproduce the failures. It is more efficient to run only failed tests and skip the passed ones.
Describe the solution you'd like
Let's create avocado configuration option for reproducing failed tests. We can use avocado recipe file to store the active avocado configuration for each failed test. If the configuration will be stored in recipe file, it would be easier to move the configuration to another machine.
This feature should look like this:
Running avocado job:
$ avocado run --store-failed-configuration=/tmp/fail.json /bin/true /bin/false examples/tests/passtest.py examples/tests/failtest.py examples/tests/errortest.py
JOB ID : 2c7b8348c00b3ab711061b76995d8db216d03446
JOB LOG : /home/jarichte/avocado/job-results/job-2022-08-04T14.05-2c7b834/job.log
(3/5) examples/tests/passtest.py:PassTest.test: STARTED
(1/5) /bin/true: STARTED
(1/5) /bin/true: PASS (0.02 s)
(3/5) examples/tests/passtest.py:PassTest.test: PASS (0.01 s)
(2/5) /bin/false: STARTED
(2/5) /bin/false: FAIL (0.01 s)
(4/5) examples/tests/failtest.py:FailTest.test: STARTED
(5/5) examples/tests/errortest.py:ErrorTest.test: STARTED
(4/5) examples/tests/failtest.py:FailTest.test: FAIL: This test is supposed to fail (0.04 s)
(5/5) examples/tests/errortest.py:ErrorTest.test: ERROR: This should end with ERROR. (0.04 s)
RESULTS : PASS 2 | ERROR 1 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : /home/jarichte/avocado/job-results/job-2022-08-04T14.05-2c7b834/results.html
JOB TIME : 2.45 s
Failed tests were stored in /tmp/fail.json
Reproducing failed tests:
avocado run /tmp/fail.json
JOB ID : 97c28bcdccaeb762060a192bad75f73048728823
JOB LOG : /home/jarichte/avocado/job-results/job-2022-08-04T14.08-97c28bc/job.log
(1/3) /bin/false: STARTED
(2/3) examples/tests/failtest.py:FailTest.test: STARTED
(1/3) /bin/false: FAIL (0.01 s)
(3/3) examples/tests/errortest.py:ErrorTest.test: STARTED
(2/3) examples/tests/failtest.py:FailTest.test: FAIL: This test is supposed to fail (0.04 s)
(3/3) examples/tests/errortest.py:ErrorTest.test: ERROR: This should end with ERROR. (0.03 s)
RESULTS : PASS 0 | ERROR 1 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : /home/jarichte/avocado/job-results/job-2022-08-04T14.08-97c28bc/results.html
JOB TIME : 2.12 s
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
When some of the tests from job fails, it would be good to be able to re-run only the failed tests with the same configuration to reproduce the failures. It is more efficient to run only failed tests and skip the passed ones.
Describe the solution you'd like
Let's create avocado configuration option for reproducing failed tests. We can use avocado recipe file to store the active avocado configuration for each failed test. If the configuration will be stored in recipe file, it would be easier to move the configuration to another machine.
This feature should look like this:
Running avocado job:
Reproducing failed tests:
The text was updated successfully, but these errors were encountered: