Produce a cleaner scenario output

[junalmeida]

Description

Produce a cleaner scenario output, by not repeating the entire given/when/then/and line for just state that it was a success or fail.

Progress

• Feature is implemented,
• Ensured backward-compatibility,
• Ensured good debugging experience

[Suremaker]

Hello @junalmeida,

Let me start with the reasoning of current behavior:

The progress notifiers reports two lines per step as they are reporting when given step starts and when ends.

Such approach can be useful, as it shows what is actually going on, especially when:

• step takes more time to execute,
• step comments are used within the step,
• the step method is logging some additional information.

Let's take a look at this code:

using LightBDD.Framework;
using LightBDD.Framework.Scenarios;
using LightBDD.NUnit3;
using NUnit.Framework;
using System;
using System.Threading;

namespace LightBDDTests1
{
    public class My_feature : FeatureFixture
    {
        [Scenario]
        public void Some_scenario()
        {
            Runner.RunScenario(
                Given_I_have_a_basket,
                When_I_add_5_eggs_to_the_basket,
                When_I_add_2_eggs_to_the_basket,
                Then_I_should_have_7_items_in_the_basket);
        }

        private void Given_I_have_a_basket()
        {
            TestContext.Progress.WriteLine("Logging context creation....");
        }

        private void When_I_add_2_eggs_to_the_basket()
        {
            Console.Error.WriteLine("Some console writes...");
        }

        private void When_I_add_5_eggs_to_the_basket()
        {
            StepExecution.Current.Comment("adding more eggs");
        }

        private void Then_I_should_have_7_items_in_the_basket()
        {
            Thread.Sleep(5000);
            TestContext.Progress.WriteLine("Simulated some super long task....");
        }
    }
}

Running dotnet test on the project (netcoreapp2.0 / LightBDD.NUnit3 3.0.1) will produce following results:

Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Fi=000,Fa=000,Pe=000 #   > FEATURE: My feature
Fi=000,Fa=000,Pe=001 #  1> SCENARIO: Some scenario
Fi=000,Fa=000,Pe=001 #  1>   STEP 1/4: GIVEN I have a basket...
Logging context creation....
Fi=000,Fa=000,Pe=001 #  1>   STEP 1/4: GIVEN I have a basket (Passed after 18ms)
Fi=000,Fa=000,Pe=001 #  1>   STEP 2/4: WHEN I add 5 eggs to the basket...
Fi=000,Fa=000,Pe=001 #  1>   STEP 2/4: /* adding more eggs */
Fi=000,Fa=000,Pe=001 #  1>   STEP 2/4: WHEN I add 5 eggs to the basket (Passed after 3ms)
Fi=000,Fa=000,Pe=001 #  1>   STEP 3/4: AND I add 2 eggs to the basket...
Some console writes...
Fi=000,Fa=000,Pe=001 #  1>   STEP 3/4: AND I add 2 eggs to the basket (Passed after <1ms)
Fi=000,Fa=000,Pe=001 #  1>   STEP 4/4: THEN I should have 7 items in the basket...
Simulated some super long task....
Fi=000,Fa=000,Pe=001 #  1>   STEP 4/4: THEN I should have 7 items in the basket (Passed after 5s 001ms)
Fi=001,Fa=000,Pe=000 #  1>   SCENARIO RESULT: Passed after 5s 058ms
Fi=001,Fa=000,Pe=000 #   > FEATURE FINISHED: My feature

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 6.3055 Seconds

Reporting step start and step end separately, gives here more context on what is going on and actually it is printed on console as the scenario progresses.

Please let me know if that makes sense or if that does not give value for you...

---

If you are not happy with the current implementation of the notifiers, you can provide yours. Here is a wiki page describing it: Test Progress Notification > Customizing progress notification.

Unfortunately, there is no way to disable just printing the step start in the current notifiers, so you will have to provide full implementation of it:

• here is an implementation of DefaultProgressNotifier
• here is an example how NUnit3 creates own notifiers: NUnit3ProgressNotifier.

Having said that, I will leave this ticket open to implement an easy way to override the behaviors of the notifiers without need of reimplementing them and will let you know of the progress.

[junalmeida]

Thank you very much for this very clarifying answer, it helped me a lot to implement a custom notifier that produces a more readable version in my use case. Since all steps always execute in order, for me there is no point in repeating step description each comment or in the event of finish.

Anyway, If this feature request makes no sense to the overall community that uses LightBDD, fell free to close it.

Congratulations for the awesome lib.

[Suremaker]

I am glad the response helped you and all is working now fine!

If you could share your implementation of the notifier, I can always take a look at it and perhaps make it available in the library itself, so that the users will be able to pick the one that works better for them.

I will leave this ticket open for a time being because you I believe there is a room for improvement in configuration of the notifiers.

Congratulations for the awesome lib.

Thanks a lot! It is great to hear it 😃

[lemonlion]

Hi,

I've tried overriding the implementation as per the documentation, but it seems to be failing.

internal class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        // some example customization of report writers
        configuration
            .ProgressNotifierConfiguration()
            .Clear() // remove defaults
            .Append(new DefaultProgressNotifier2());
    }
}

Then I copy and pasted to DefaultProgressNotifier, renaming it to DefaultProgressNotifier2. To make it compile I also had to copy and paste the TextTableRenderer and TextTreeRenderer as they are internal (might be useful to make them accessible by making them public or via some other means?). Ran the the test and it failed on the constructor:

As you can see, it's being fed an empty action string array in the constructor (naturally, as I'm newing it up with nothing in the constructor). What am I doing wrong here? Where am I supposed to get the constructor arguments from?

Thanks!

EDIT

---

I've fixed it by passing it this:

    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        // some example customization of report writers
        configuration
            .ProgressNotifierConfiguration()
            .Clear() // remove defaults
            .Append(new DefaultProgressNotifier2(x => ScenarioExecutionContext.GetCurrentScenarioFixtureIfPresent<ITestOutputProvider>()?.TestOutput.WriteLine(x)));
    }

You mentioned it in the documentation, but I guess it wasn't immediately clear from the example. Will leave this here in case anyone else has the same issue.