Inconsistent behaviour with --pattern and specifying which files to test

[imran-iq]

Subject of the issue

If the default pattern is provided (by omitting --pattern or passing it in with its default value --pattern '**{,/*/**}/*_spec.rb')
we are able to filter which tests to run by providing a file via invocation and only that files tests will run

If we change the pattern to something other than default, we lose that behaviour

There is also another issue where the default glob does not behave as expected
In that link all 3 tests files should match, but when rspec is invoked only tests in the spec directory

Some files are not included to be run (in this case under the engine dir)

Your environment

• Ruby version: 2.6.3
• rspec-core version: 3.10

Steps to reproduce

1. Create a directory with the following tree structure:

➜ tree                                                                                                    
.                                                                                                         
├── Gemfile                                                                                               
├── Gemfile.lock                                                                                          
├── engine                                                                                                
│   └── test                                                                                              
│       └── spec                                                                                          
│           └── model                                                                                     
│               └── model_spec.rb                                                                         
└── spec                                                                                                  
    └── model                                                                                             
        ├── another_model_spec.rb                                                                         
        └── model_spec.rb

6 directories, 5 files

The files have the following content:

# spec/model/model_spec.rb
RSpec.describe "model_spec" do
  it "passes" do
  end

  it "passes too" do
  end
end

# spec/model/another_model_spec.rb
RSpec.describe "another_model_spec" do
  it "passes" do
  end
end

# engine/test/spec/model/model_spec.rb
RSpec.describe "engine_model_spec" do
  it "passes" do
  end
end

2. Run rspec --pattern '**{,/*/**}/*_spec.rb' -f d
3. Run rspec --pattern '**{,/*/**}/*_spec.rb' -f d spec/model/model_spec.rb
4. Run rspec --pattern '**{,/*/**}/*_spec.rb' -f d engine/test/spec/model/model_spec.rb
5. Run rspec --pattern '*/**/*_spec.rb' -f d spec/model/model_spec.rb

Expected behavior

2. Should test all files as per the glob
3. Should test spec/model/model_spec.rb only
4. Should test engine/test/spec/model/model_spec.rb only
5. Should test spec/model/model_spec.rb only

Actual behavior

2. Only tests files under spec/
3. Only tests spec/model/model_spec.rb (expected)
4. Only tests engine/test/spec/model/model_spec.rb (expected)
5. Runs all tests

[JonRowe]

So a couple of things, firstly you've made an assumption that the pattern applies to the working directory but it does not, it applies to the configured default directory which is spec (see: https://relishapp.com/rspec/rspec-core/docs/command-line/pattern-option). So the expected behaviour is actually:

2. Only tests files under spec/
3. Only tests spec/model/model_spec.rb (expected)
4. Only tests engine/test/spec/model/model_spec.rb (expected)
5. Should test spec/model/model_spec.rb only

To change the behaviour of 2. to what you desire you can run rspec --pattern '**{,/*/**}/*_spec.rb' . or change the default directory to the working directory.

Now 5. is a bit contradictory, we add the working directory to the default paths to allow relative patterns to work (that is if you specified --pattern 'engine/**/*_spec.rb') to prevent the surprise you encountered with 2., but that probably shouldn't happen when there is already a file present so that is a bug that could be improved.

[iMacTia]

For future readers, I was in a similar situation as @imran-iq since my team is introducing Packwerk into our monolith.
We wanted a way to host rspec tests in the packwerk components, which are inside the /components folder.
I managed to make this work by adding the following to .rspec:

--require ./spec/spec_helper
--default-path "."

And then a bunch of requires broke, because I believe RSpec adds the default-path to the $LOAD_PATH, so you can require support files or similar without having to indicate spec in the path. So to fix that, I also added the following in our spec_helper:

$LOAD_PATH.push __dir__

With this changes, everything works as expected:

1. I can run ALL tests (main application and components) with bundle exec rspec
2. I can run only the main application tests with bundle exec rspec spec
3. And I can run only specific component tests with bundle exec rspec components/my_components

This won't probably work for EVERY setup, but I hope this will be useful if you're trying to achieve something similar!