Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Avoid scanning and building entries for silenced directories #542

Merged
merged 1 commit into from Aug 14, 2021

Conversation

ElMassimo
Copy link
Contributor

@ElMassimo ElMassimo commented Jul 22, 2021

Description πŸ“–

This pull request patches the Record#build method to avoid recursively scanning and creating entries for directories that are being ignored in the Listener configuration.

Background πŸ“œ

Building entries that will be ignored by the listener degrades the initialization performance, and can greatly increase memory consumption.

This is especially noticeable on projects with large .git or node_modules directories.

The Fix πŸ”¨

Use the Silencer configuration to avoid scanning ignored directories.

Benchmarks πŸ“ˆ

The performance difference is significant on startup time.

Using the listen gem in a small Jekyll site:

Before (silenced dirs are scanned)

Record.build(): 0.70397 seconds

After (silenced dirs are ignored)

Record.build(): 0.02439 seconds

Additional Benefits ✨

Directories that are ignored would no longer output symlink warnings since they are no longer being scanned.

Building entries that will be ignored by the listener degrades the
initialization performance, and can greatly increase memory consumption.

Specially on projects with large .git or node_modules directories.
Copy link
Collaborator

@ColinDKelley ColinDKelley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ElMassimo Thanks for the PR. Sorry about the delayed response.

It looks like you filled in a test with an empty silencer. But is there also a test where a silencer is configured, to prove that the short-circuiting return happens?

@ElMassimo
Copy link
Contributor Author

ElMassimo commented Aug 12, 2021

@ColinDKelley Hi!

The silencer always uses some defaults, without this patch .git/FETCH_HEAD would show up in the record tree, and cause the test to fail.

Failures:

  1) Listen::Record#build with subdir containing files builds record
     Failure/Error:
       expect(record_tree(record)).
         to eq(
           'dir1' => {},
           'dir1/foo' => { 'bar' => { mtime: 2.3, mode: 0755, size: 42 } },
           'dir2' => {}
         )

       expected: {"dir1"=>{}, "dir1/foo"=>{"bar"=>{:mode=>493, :mtime=>2.3, :size=>42}}, "dir2"=>{}}
            got: {".git"=>{"FETCH_HEAD"=>{:mode=>493, :mtime=>2.3, :size=>42}}, "dir1"=>{}, "dir1/foo"=>{"bar"=>{:mode=>493, :mtime=>2.3, :size=>42}}, "dir2"=>{}}

       (compared using ==)

       Diff:
       @@ -1 +1,2 @@
       +".git" => {"FETCH_HEAD"=>{:mode=>493, :mtime=>2.3, :size=>42}},
        "dir1" => {},

     # ./spec/lib/listen/record_spec.rb:323:in `block (4 levels) in <top (required)>'

Finished in 0.11717 seconds (files took 0.21683 seconds to load)
33 examples, 1 failure

Failed examples:

rspec ./spec/lib/listen/record_spec.rb:321 # Listen::Record#build with subdir containing files builds record

@ColinDKelley
Copy link
Collaborator

@ElMassimo Aha! Glad it's being tested.
I think it would be an improvement if the test here didn't depend on the defaults of another class. That's an easy change. I'll make it in a PR after I merge this, and tag you.

@ColinDKelley ColinDKelley merged commit 8b17254 into guard:master Aug 14, 2021
@ElMassimo ElMassimo deleted the exclude-silenced-in-record branch August 14, 2021 21:40
@ColinDKelley
Copy link
Collaborator

BTW, I'm expecting to release this in v3.7.0 in the coming week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants