How to execute particular scenario or feature ?

[ilterpehlivan]

Hi,

Is there any way to run certain scenario by giving a parameter to go run command ?
I followed the example given in README under this section : https://github.com/cucumber/godog#subtests-of-testingt but it did not work!

Thanks for the help
REgards,

[vearutop]

Hello, could you share your command and what did you get as a result?

For example if you run

go test -run ^Test_AllFeaturesRunAsSubtests$/^should_run_a_normal_outline$

in godog repo, it will only execute

  Scenario: should run a normal outline

Please keep in mind, that you need to change spaces to underscores (this is a limitation coming from how subtests are implemented in Go).

If in doubt how scenario name is translated to subtest name, you can run full suite with -v flag, for example

go test -run ^Test_AllFeaturesRunAsSubtests$ -v
=== RUN   Test_AllFeaturesRunAsSubtests
=== RUN   Test_AllFeaturesRunAsSubtests/should_filter_scenarios_having_Y_and_Z_tags
=== RUN   Test_AllFeaturesRunAsSubtests/Support_of_Feature_Plus_Scenario_Outline_With_Tags
=== RUN   Test_AllFeaturesRunAsSubtests/should_filter_scenarios_by_X_tag_not_having_Y
=== RUN   Test_AllFeaturesRunAsSubtests/should_skip_examples_on_background_failure
.......

in the output you will find all scenarios with their fully qualified subtest names.

[ilterpehlivan]

Thanks @vearutop for the answer, it seems 2 mistakes I was doing that you pointed out:

1. I was using Feature name -> it looks like we can only filter scenarios, please confirm
2. I was using spaces inside "" instead of "_"

Since feature level filtering not possible, how can we run only single feature file rather than whole suite ?
One solution would be to use the same pattern for all the scenario names within same file and choose them, but can we use pattern matching ? for example, the below one works ?

go test -run ^Test_AllFeaturesRunAsSubtests$/should_filter*

[vearutop]

Yes, currently subtest name only has scenario title, but now that you described your use case, I think this is unnecessarily limiting.
Maybe we should add feature name to subtest name too to make this control more consistent and useful.

Control of which features to run is also available via

			Paths:       []string{"features"},

that can take a form of

			Paths:       []string{"features/Foo.feature", "features/Bar.feature"},

So, for example, you can define it with command line flags.

  feature := flag.String("feature", "features", "Feature to test")
  flag.Parse()

  suite := godog.TestSuite{
    ScenarioInitializer: func(s *godog.ScenarioContext) {
      // Add step definitions here.
    },
    Options: &godog.Options{
      Format:   "pretty",
      Paths:    []string{*feature},
      TestingT: t, // Testing instance that will run subtests.
    },
  }
  ...

And then run test with

go test -feature features/Foo.feature

[ilterpehlivan]

Ok i tried your recommendation, tests starting to run with given feature file but throwing strange error from test execution

$ go test -feature features/echo.feature -run ^TestFeatures$
2022/03/10 18:16:58 inside TestFeatures: features/echo.feature
unknown shorthand flag: 'f' in -feature
Usage of /var/folders/0d/13721n751vxd7lbt7c6y0q4m0000gp/T/go-build1270607635/b001/tests.test:
      --conf.name string                configuration file name without extension (default "config")
      --conf.path string                configuration file path (default "config")
      --feature string                  Feature to test (default "features/echo.feature")
      --logging.level string            sets the log level (default "INFO")
      --test.bench regexp               run only benchmarks matching regexp
      --test.benchmem                   print memory allocations for benchmarks
      --test.benchtime d                run each benchmark for duration d (default 1s)
      --test.blockprofile file          write a goroutine blocking profile to file
      --test.blockprofilerate rate      set blocking profile rate (see runtime.SetBlockProfileRate) (default 1)
      --test.count n                    run tests and benchmarks n times (default 1)
      --test.coverprofile file          write a coverage profile to file
      --test.cpu list                   comma-separated list of cpu counts to run each test with
      --test.cpuprofile file            write a cpu profile to file
      --test.failfast                   do not start new tests after the first test failure
      --test.list regexp                list tests, examples, and benchmarks matching regexp then exit
      --test.memprofile file            write an allocation profile to file
      --test.memprofilerate rate        set memory allocation profiling rate (see runtime.MemProfileRate)
      --test.mutexprofile string        write a mutex contention profile to the named file after execution
      --test.mutexprofilefraction int   if >= 0, calls runtime.SetMutexProfileFraction() (default 1)
      --test.outputdir dir              write profiles to dir
      --test.paniconexit0               panic on call to os.Exit(0) (default true)
      --test.parallel n                 run at most n tests in parallel (default 8)
      --test.run regexp                 run only tests and examples matching regexp (default "^TestFeatures$")
      --test.short                      run smaller test suite to save time
      --test.shuffle string             randomize the execution order of tests and benchmarks (default "off")
      --test.testlogfile file           write test action log to file (for use only by cmd/go)
      --test.timeout d                  panic test binary after duration d (default 0, timeout disabled) (default 10m0s)
      --test.trace file                 write an execution trace to file
      --test.v                          verbose: print additional output
unknown shorthand flag: 'f' in -feature

Any guess what could this be ?

Thanks

[vearutop]

Ah, sorry, I gave you wrong example of flag instrumentation, cli flags should be defined before test run.

Move feature variable to global scope (outside test function) and remove flag.Parse() (flags are already parsed by Go test runner).

var feature = flag.String("feature", "features", "Feature file to test.")

[ilterpehlivan]

I already did that actually, the problem seems that while running I had to run with 2 dashes --feature

Thanks for the support @vearutop