lot of 'is already being watched through' messages when using rails console if using node_modules #32700

vmarquet · 2018-04-23T17:21:54Z

(This is not strictly a bug but more of a medium annoyance.)

Steps to reproduce

create a rails project
install some particular npm packages at the root of the rails app: npm install puppeteer
stopping spring ensures the messages appear: spring stop
try logging into the console: rails c

Expected behavior

I expect to be elegantly logged into the Rails console:

$ rails c
Running via Spring preloader in process 45930
Loading development environment (Rails 5.1.4)
>

Actual behavior

Sometimes, I get a lot of error messages:

$ rails c
        ** ERROR: directory is already being watched! **

        Directory: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current

        is already being watched through: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Helpers

        is already being watched through: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Helpers

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Internet Plug-Ins

        is already being watched through: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Internet Plug-Ins

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Libraries

        is already being watched through: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Libraries

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Resources

        is already being watched through: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Resources

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/XPCServices

        is already being watched through: /Users/vincent/Projets/myapp/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/XPCServices

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
Running via Spring preloader in process 45579
Loading development environment (Rails 5.1.4)
>

System configuration

Rails 5.1.4
Ruby 2.4.2p198

Additional information

I looked at the Guard wiki, but that didn't really help me to solve the problem. I couldn't find how to configure Guard/Listen to ignore the node_modules directory.

Is there a way to not watch that directory and thus prevent error messages from cluttering the Rails console? Or maybe those message could simply be filtered and not sent to the user terminal.

The text was updated successfully, but these errors were encountered:

rafaelfranca · 2018-04-23T18:16:15Z

Is the node_modules directory listed as an autoload directory? Rails and listen only watches directories that are autloaded.

rafaelfranca · 2018-04-23T18:19:14Z

I tried to reproduce this issue and I could not. I don't see those warnings.

Here are the steps that I made

[2.5.0 src]$ rails new foo
[2.5.0 src]$ cd foo/
[2.5.0 foo (master #)]$ npm install puppeteer

> puppeteer@1.3.0 install /Users/rafaelfranca/src/foo/node_modules/puppeteer
> node install.js

Downloading Chromium r549031 - 76 Mb [====================] 100% 0.0s
Chromium downloaded to /Users/rafaelfranca/src/foo/node_modules/puppeteer/.local-chromium/mac-549031
npm notice created a lockfile as package-lock.json. You should commit this file.
+ puppeteer@1.3.0
added 41 packages in 7.329s
[2.5.0 foo (master #)]$ ls
Gemfile           app               db                package-lock.json test
Gemfile.lock      bin               lib               package.json      tmp
README.md         config            log               public            vendor
Rakefile          config.ru         node_modules      storage
[2.5.0 foo (master #)]$ spring stop
Spring is not running
[2.5.0 foo (master #)]$ rails c
Loading development environment (Rails 5.2.0)
>>

vmarquet · 2018-04-23T18:21:02Z

No, I have not added it to the autoloaded paths, and it does not appear when I list them with Myapp::Application.config.autoload_paths.

vmarquet · 2018-04-23T18:29:41Z

Those exact steps get me the error. Maybe you don't have it because you did not use the same versions of Ruby and Rails?

rafaelfranca · 2018-04-23T18:29:44Z

I'm not sure what could be causing that because I could not reproduce and Rails only listen to autoload_paths.

Try ActiveSupport::Dependencies.autoload_paths on console.

vmarquet · 2018-04-23T18:30:29Z

2.4.2 :001 > ActiveSupport::Dependencies.autoload_paths
 => ["/private/tmp/foo/app/assets", "/private/tmp/foo/app/channels", "/private/tmp/foo/app/controllers", "/private/tmp/foo/app/controllers/concerns", "/private/tmp/foo/app/helpers", "/private/tmp/foo/app/jobs", "/private/tmp/foo/app/mailers", "/private/tmp/foo/app/models", "/private/tmp/foo/app/models/concerns", "/private/tmp/foo/test/mailers/previews"]

rafaelfranca · 2018-04-23T18:30:29Z

Does it happens with a Rails 5.2 application? That will help us to check if that is a problem that was already fixed.

vmarquet · 2018-04-23T18:45:38Z

Same problem with Rails 5.2.

Note that I forgot to mention that I'm using Mac OS (Sierra 10.12.6 for now), and this is maybe relevant, I think I read somewhere the creator of the gem Listen saying that Mac OS is causing trouble. Unfortunately I don't have a linux machine to test if this is OS related.

Also, I just had a trouble with my npm install, I used yarn add puppeteer --dev to install instead of npm install puppeteer, but that shouldn't change anything.

rafaelfranca · 2018-04-23T19:01:15Z

I'm on Mac OS 10.13.4 with Ruby 2.5, Rails 5.2 and 5.1 and I could not reproduce this issue.

And given node_modules is not in the autoload_paths I'm not sure how and why you are getting that behavior.

vmarquet · 2018-04-23T19:11:56Z

I have no idea :(

Maybe that would help if someone with a version of Mac OS closer to mine could test this. 10.13.4 is maybe too different from 10.12.6.

Also for the record, I'm using yarn 1.6.0, and the version of puppeteer is 1.3.0.

pixeltrix · 2018-04-23T21:05:35Z

Is it possible there's a symlink to node_modules inside one of the autoload paths?

vmarquet · 2018-04-23T21:13:49Z

@pixeltrix I haven't created any symlink.

matthewd · 2018-04-23T21:49:53Z

I believe this (watching everything inside the app root) is a known symptom of one of the watched-by-default directories being missing.

I feel like the previous report involved an app that was using rspec, and therefore didn't have a test directory, and that being significant?

@vmarquet do you have a .railsrc or similar that could be affecting your rails new command? I'm much more confused that you're getting different results from nominally-identical reproduction steps than I am about the actual results.

vmarquet · 2018-04-23T21:54:07Z

@matthewd No I don't have any .railsrc, I did not even knew that existed.

davidrunger · 2018-05-07T01:12:23Z

Thanks!

(First of all, a heartfelt thank you @rafaelfranca and @matthewd for all of your work in Rails!, and for taking a look at this issue in particular, and @vmarquet for reporting it.)

Thanks in advance for your patience with this lengthy comment, as well!

I am also affected

This issue is affecting me and I am able to reproduce it by following the steps that you demonstrated @rafaelfranca (although, mysteriously, those reproduction steps did not work for you).

View my full reproduction log

~/workspace   12:51:25 PM
❯ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.13.4
BuildVersion:	17E202

~/workspace   12:51:28 PM
❯ ruby --version
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17]

~/workspace   12:51:35 PM
❯ rails --version
Rails 5.2.0

~/workspace   12:51:40 PM
❯ rails new foo
      create
      create  README.md
      create  Rakefile
      create  .ruby-version
      create  config.ru
      create  .gitignore
      create  Gemfile
         run  git init from "."
Initialized empty Git repository in /Users/david/workspace/foo/.git/
      create  package.json
      create  app
      create  app/assets/config/manifest.js
      create  app/assets/javascripts/application.js
      create  app/assets/javascripts/cable.js
      create  app/assets/stylesheets/application.css
      create  app/channels/application_cable/channel.rb
      create  app/channels/application_cable/connection.rb
      create  app/controllers/application_controller.rb
      create  app/helpers/application_helper.rb
      create  app/jobs/application_job.rb
      create  app/mailers/application_mailer.rb
      create  app/models/application_record.rb
      create  app/views/layouts/application.html.erb
      create  app/views/layouts/mailer.html.erb
      create  app/views/layouts/mailer.text.erb
      create  app/assets/images/.keep
      create  app/assets/javascripts/channels
      create  app/assets/javascripts/channels/.keep
      create  app/controllers/concerns/.keep
      create  app/models/concerns/.keep
      create  bin
      create  bin/bundle
      create  bin/rails
      create  bin/rake
      create  bin/setup
      create  bin/update
      create  bin/yarn
      create  config
      create  config/routes.rb
      create  config/application.rb
      create  config/environment.rb
      create  config/cable.yml
      create  config/puma.rb
      create  config/spring.rb
      create  config/storage.yml
      create  config/environments
      create  config/environments/development.rb
      create  config/environments/production.rb
      create  config/environments/test.rb
      create  config/initializers
      create  config/initializers/application_controller_renderer.rb
      create  config/initializers/assets.rb
      create  config/initializers/backtrace_silencers.rb
      create  config/initializers/content_security_policy.rb
      create  config/initializers/cookies_serializer.rb
      create  config/initializers/cors.rb
      create  config/initializers/filter_parameter_logging.rb
      create  config/initializers/inflections.rb
      create  config/initializers/mime_types.rb
      create  config/initializers/new_framework_defaults_5_2.rb
      create  config/initializers/wrap_parameters.rb
      create  config/locales
      create  config/locales/en.yml
      create  config/master.key
      append  .gitignore
      create  config/boot.rb
      create  config/database.yml
      create  db
      create  db/seeds.rb
      create  lib
      create  lib/tasks
      create  lib/tasks/.keep
      create  lib/assets
      create  lib/assets/.keep
      create  log
      create  log/.keep
      create  public
      create  public/404.html
      create  public/422.html
      create  public/500.html
      create  public/apple-touch-icon-precomposed.png
      create  public/apple-touch-icon.png
      create  public/favicon.ico
      create  public/robots.txt
      create  tmp
      create  tmp/.keep
      create  tmp/cache
      create  tmp/cache/assets
      create  vendor
      create  vendor/.keep
      create  test/fixtures
      create  test/fixtures/.keep
      create  test/fixtures/files
      create  test/fixtures/files/.keep
      create  test/controllers
      create  test/controllers/.keep
      create  test/mailers
      create  test/mailers/.keep
      create  test/models
      create  test/models/.keep
      create  test/helpers
      create  test/helpers/.keep
      create  test/integration
      create  test/integration/.keep
      create  test/test_helper.rb
      create  test/system
      create  test/system/.keep
      create  test/application_system_test_case.rb
      create  storage
      create  storage/.keep
      create  tmp/storage
      create  tmp/storage/.keep
      remove  config/initializers/cors.rb
      remove  config/initializers/new_framework_defaults_5_2.rb
         run  bundle install
The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`.
Fetching gem metadata from https://rubygems.org/..............
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies....
Fetching rake 12.3.1
Installing rake 12.3.1
Using concurrent-ruby 1.0.5
Using i18n 1.0.1
Fetching minitest 5.11.3
Installing minitest 5.11.3
Using thread_safe 0.3.6
Using tzinfo 1.2.5
Using activesupport 5.2.0
Using builder 3.2.3
Using erubi 1.7.1
Using mini_portile2 2.3.0
Using nokogiri 1.8.2
Using rails-dom-testing 2.0.3
Using crass 1.0.4
Using loofah 2.2.2
Using rails-html-sanitizer 1.0.4
Using actionview 5.2.0
Using rack 2.0.5
Using rack-test 1.0.0
Using actionpack 5.2.0
Using nio4r 2.3.1
Using websocket-extensions 0.1.3
Using websocket-driver 0.7.0
Using actioncable 5.2.0
Using globalid 0.4.1
Using activejob 5.2.0
Using mini_mime 1.0.0
Using mail 2.7.0
Using actionmailer 5.2.0
Using activemodel 5.2.0
Using arel 9.0.0
Using activerecord 5.2.0
Using mimemagic 0.3.2
Using marcel 0.3.2
Using activestorage 5.2.0
Fetching public_suffix 3.0.2
Installing public_suffix 3.0.2
Fetching addressable 2.5.2
Installing addressable 2.5.2
Fetching io-like 0.3.0
Installing io-like 0.3.0
Fetching archive-zip 0.11.0
Installing archive-zip 0.11.0
Fetching bindex 0.5.0
Installing bindex 0.5.0 with native extensions
Fetching msgpack 1.2.4
Installing msgpack 1.2.4 with native extensions
Fetching bootsnap 1.3.0
Installing bootsnap 1.3.0 with native extensions
Using bundler 1.16.1
Fetching byebug 10.0.2
Installing byebug 10.0.2 with native extensions
Fetching xpath 3.0.0
Installing xpath 3.0.0
Fetching capybara 3.0.3
Installing capybara 3.0.3
Fetching ffi 1.9.23
Installing ffi 1.9.23 with native extensions
Fetching childprocess 0.9.0
Installing childprocess 0.9.0
Fetching chromedriver-helper 1.2.0
Installing chromedriver-helper 1.2.0
Fetching coffee-script-source 1.12.2
Installing coffee-script-source 1.12.2
Fetching execjs 2.7.0
Installing execjs 2.7.0
Fetching coffee-script 2.4.1
Installing coffee-script 2.4.1
Using method_source 0.9.0
Using thor 0.20.0
Using railties 5.2.0
Fetching coffee-rails 4.2.2
Installing coffee-rails 4.2.2
Fetching multi_json 1.13.1
Installing multi_json 1.13.1
Fetching jbuilder 2.7.0
Installing jbuilder 2.7.0
Fetching rb-fsevent 0.10.3
Installing rb-fsevent 0.10.3
Fetching rb-inotify 0.9.10
Installing rb-inotify 0.9.10
Fetching ruby_dep 1.5.0
Installing ruby_dep 1.5.0
Fetching listen 3.1.5
Installing listen 3.1.5
Fetching puma 3.11.4
Installing puma 3.11.4 with native extensions
Using sprockets 3.7.1
Using sprockets-rails 3.2.1
Using rails 5.2.0
Fetching rubyzip 1.2.1
Installing rubyzip 1.2.1
Fetching sass-listen 4.0.0
Installing sass-listen 4.0.0
Fetching sass 3.5.6
Installing sass 3.5.6
Fetching tilt 2.0.8
Installing tilt 2.0.8
Fetching sass-rails 5.0.7
Installing sass-rails 5.0.7
Fetching selenium-webdriver 3.11.0
Installing selenium-webdriver 3.11.0
Fetching spring 2.0.2
Installing spring 2.0.2
Fetching spring-watcher-listen 2.0.1
Installing spring-watcher-listen 2.0.1
Fetching sqlite3 1.3.13
Installing sqlite3 1.3.13 with native extensions
Fetching turbolinks-source 5.1.0
Installing turbolinks-source 5.1.0
Fetching turbolinks 5.1.1
Installing turbolinks 5.1.1
Fetching uglifier 4.1.10
Installing uglifier 4.1.10
Fetching web-console 3.6.2
Installing web-console 3.6.2
Bundle complete! 18 Gemfile dependencies, 78 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
         run  bundle exec spring binstub --all
* bin/rake: spring inserted
* bin/rails: spring inserted

~/workspace   12:52:47 PM
❯ cd foo/

~/workspace/foo  master ✗ 12:52:51 PM
❯ npm install puppeteer

> puppeteer@1.3.0 install /Users/david/workspace/foo/node_modules/puppeteer
> node install.js

Downloading Chromium r549031 - 76 Mb [====================] 100% 0.0s
Chromium downloaded to /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031
npm notice created a lockfile as package-lock.json. You should commit this file.
+ puppeteer@1.3.0
added 41 packages in 17.496s

~/workspace/foo  master ✗ 12:53:48 PM
❯ ls
Gemfile           README.md         app               config            db                log               package-lock.json public            test              vendor
Gemfile.lock      Rakefile          bin               config.ru         lib               node_modules      package.json      storage           tmp

~/workspace/foo  master ✗ 12:53:58 PM
❯ spring stop
Spring is not running

~/workspace/foo  master ✗ 12:54:05 PM
❯ rails c
        ** ERROR: directory is already being watched! **

        Directory: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current

        is already being watched through: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Resources

        is already being watched through: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Resources

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Libraries

        is already being watched through: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Libraries

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Internet Plug-Ins

        is already being watched through: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Internet Plug-Ins

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/XPCServices

        is already being watched through: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/XPCServices

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
        ** ERROR: directory is already being watched! **

        Directory: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Helpers

        is already being watched through: /Users/david/workspace/foo/node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A/Helpers

        MORE INFO: https://github.com/guard/listen/wiki/Duplicate-directory-errors
Running via Spring preloader in process 11287
Loading development environment (Rails 5.2.0)
irb(main):001:0>

~/workspace/foo  master ✗ 12:54:24 PM
❯ ls -lh node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium\ Framework.framework/Versions/
total 0
drwxr-xr-x  8 david  staff   256B May  6 12:53 A
lrwxr-xr-x  1 david  staff     1B May  6 12:53 Current -> A

~/workspace/foo  master ✗ 01:01:31 PM
❯

My system

I have:

Mac OS X High Sierra (version 10.13.4, build 17E202)
Ruby 2.5.1 (p57) (issue also occurred with Ruby 2.4.2 (p198))
Rails 5.2.0
node 8.9.3
npm 5.6.0
yarn 1.5.1

Problem: symlinks within `node_modules/puppeteer/`

In my case, as with @vmarquet, the problem is caused by my usage of the puppeteer JavaScript package. Installing that package (whether via npm or yarn) produces several symlinks within the node_modules/puppeteer/ directory, for example:

~/workspace/foo  master ✗ 12:54:24 PM
❯ ls -lh node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium\ Framework.framework/Versions/
total 0
drwxr-xr-x  8 david  staff   256B May  6 12:53 A
lrwxr-xr-x  1 david  staff     1B May  6 12:53 Current -> A

This symlink seems to be what is causing the ERROR: directory is already being watched! warnings.

So, the question becomes: why is node_modules/ being watched in the first place?

`node_modules` is being watched because `spring-watcher-listen` listens to `root`

I think that node_modules/ is being watched because of the spring-watcher-listen gem. We see here that spring-watcher-listen listens to "base_directories" ...

@listener = ::Listen.to(*base_directories, latency: latency, &method(:changed))

... and in that same file we see that the definition of base_directories includes root (i.e. the Rails project root):

def base_directories
  ([root] +
    files.reject       { |f| f.start_with? "#{root}/" }.map { |f| File.expand_path("#{f}/..") } +
    directories.reject { |d| d.start_with? "#{root}/" }
  ).uniq.map { |path| Pathname.new(path) }
end

... and, of course, node_modules/ is within the project root, so we are watching node_modules/.

A possible fix -- but it breaks listening for changes to files in the root of the Rails project

The warnings go away for me if I modify the Spring::Watcher::Listen#base_directories method to not listen to the Rails project root and instead to instead listen specifically to each directory within root -- except for node_modules -- plus directories that might be external to the Rails project (as before).

The following illustrates this proposed change to spring-watcher-listen (in that gem's lib/spring/watcher/listen.rb file).

module Spring
  module Watcher
    class Listen < Abstract
      IGNORED_DIRECTORIES = %w[. .. node_modules].map(&:freeze).freeze

      # ...

      def base_directories
        project_subdirectories =
          Dir.entries(root)
            .reject { |entry| IGNORED_DIRECTORIES.include?(entry) }
            .map { |entry| File.join(root, entry) }
            .select { |entry| File.directory?(entry) }

        (
          project_subdirectories +
          files.map { |f| File.expand_path("#{f}/..") } +
          directories.to_a
        ).uniq.map { |path| Pathname.new(path) }.reject { |path| path == Pathname.new(root) }
      end
    end
  end
end

This implementation is not at feature parity with the current implementation, though, because this proposed implementation does not allow for listening to changes to files that are at the root of the Rails project directory. For example, this implementation breaks listening for changes to .ruby-version and .rbenv-vars, which Rails projects try to listen for changes to by default.

A fix that continues to allow for listening to changes to files in the root of the Rails project might be difficult

Given the current state of listen, I'm not sure that there is any way to fix the issue that @vmarquet and I are experiencing without also breaking the ability to listen for changes to files in the root of the Rails project.

Specifically, I believe this because of these statements from @e2 on guard/listen#363 (comment):

Basically the problem is OSX. OSX adapter (rb-fsevent) watches directories recursively, period.

But, if you want to watch the project root directory, on OSX you're forced to watch everything.

In other words, in order to listen for changes to files in the project root (which requires listening to the project root directory), I believe that listen will also unavoidably listen to node_modules/, and thus produce the warnings that are the subject of this issue.

Hacky workaround

UPDATE: Although this does fix the warnings when running rails c in the foo project that I was experimenting in, when I tried to port this solution back to my "real" project, I found that it breaks puppeteer and I am no longer able to run my JavaScript tests via puppeteer ... so ... not a great fix. 🙃 😛

~~As a hacky workaround, I've added to following postinstall script to my package.json ...~~

"scripts": {
  "postinstall": "bin/remove-puppeteer-symlinks"
}

... and added a bin/remove-puppeteer-symlinks script (make it executable with chmod +x bin/remove-puppeteer-symlinks) that removes the symlinks and replaces them with the actual directories that the symlinks had been pointing to:

View the script

#!/usr/bin/env bash

rm "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current"
mv "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/A" "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current"

rm "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Resources"
mv "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current/Resources" "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Resources"

rm "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Libraries"
mv "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current/Libraries" "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Libraries"

rm "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Internet Plug-Ins"
mv "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current/Internet Plug-Ins" "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Internet Plug-Ins"

rm "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/XPCServices"
mv "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current/XPCServices" "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/XPCServices"

rm "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Helpers"
mv "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Versions/Current/Helpers" "node_modules/puppeteer/.local-chromium/mac-549031/chrome-mac/Chromium.app/Contents/Versions/67.0.3391.0/Chromium Framework.framework/Helpers"

A major weakness of this approach is that this bin/remove-puppeteer-symlinks script contains some hardcoded values that are sure to change soon (e.g. the Chrome version, 67.0.3391.0). With a little better bash fu, I imagine that the script could be improved to be more generic/future-proof, though.

davidrunger · 2018-05-07T02:18:50Z

@rafaelfranca I wonder if you have DISABLE_SPRING set in your env somewhere? I notice that, when you booted rails c, no line was printed stating Running via Spring preloader in process <process id here>. Maybe this is why you could not reproduce the issue?

vmarquet · 2018-05-07T10:21:17Z

Another hacky workaround would be to prevent outputting the message to the user console, maybe something like:

module Listen
  class Record
    class SymlinkDetector

      private

      def _fail(symlinked, real_path)
        unless symlinked.include? '/node_modules/puppeteer/'
          STDERR.puts format(SYMLINK_LOOP_ERROR, symlinked, real_path)
        end

        fail Error, 'Failed due to looped symlinks'
      end
    end
  end
end

cf https://github.com/guard/listen/blob/4c15bb72984bc67f9200109e28e5308ba3ade80e/lib/listen/record/symlink_detector.rb

rails-bot · 2018-08-05T11:53:06Z

This issue has been automatically marked as stale because it has not been commented on for at least three months.
The resources of the Rails team are limited, and so we are asking for your help.
If you can still reproduce this error on the 5-2-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.

jalkoby · 2018-08-09T11:26:29Z

In my case the reason is that "app/javascript" & "app/styles" includes symlinks to a folders in "node_modules". Removing "app/javascript" & "app/styles" from eager_load_paths didn't help

mobilutz · 2018-08-13T12:47:00Z

Is there any new news on this?
We just started using puppeteer with our rails app, and these warnings keep popping up once and again.

y-yagi · 2018-08-19T01:00:34Z

@mobilutz Unfortunately not solved it yet.
Are you using spring-watcher-listen gem? If so, what happens if delete spring-watcher-listen from Gemfile?

rjurado01 · 2018-08-31T12:15:22Z

In my case, the method directories_to_watch of ActiveSupport::EventedFileUpdateChecker is returned the root directory. I have fix the issue overwriting this method:

# config/initializers/fixes.rb

module ActiveSupport
  class EventedFileUpdateChecker
    private

    def directories_to_watch
      dtw = (@files + @dirs.keys).map { |f| @ph.existing_parent(f) }
      dtw.compact!
      dtw.uniq!

      normalized_gem_paths = Gem.path.map { |path| File.join path, "" }
      dtw = dtw.reject do |path|
        normalized_gem_paths.any? { |gem_path| path.to_s.start_with?(gem_path) }
      end

      dtw.delete Rails.root # added for me

      @ph.filter_out_descendants(dtw)
    end
  end
end

y-yagi · 2018-09-01T09:52:15Z

@rjurado01 Can you please provide a sample application that reproduces?

rjurado01 · 2018-09-06T06:46:05Z

@y-yagi Yes, I have uploaded a sample rails api project with this bug: https://github.com/rjurado01/bug_sample.

It contains a client folder with node project inside.

y-yagi · 2018-09-07T08:51:10Z

@rjurado01 Thanks! I will investigate.

`EventedFileUpdateChecker` will search the parent directory if the specified directory does not exist. Since `test/mailers/previews` is included in the watch target by default, if there is no test directory (e.g. using `rspec`), the Rails root directory will be included in the watch target. ``` $ rails new app $ cd app $ ./bin/rails r "p Rails.application.reloaders.last.send(:directories_to_watch).include?(Rails.root)" false $ rm -rf test $ ./bin/rails r "p Rails.application.reloaders.last.send(:directories_to_watch).include?(Rails.root)" true ``` This causes `node_modules` to be included in watch target. Adding parent directories to watch target may include unexpected directories. In order to avoid this, fixed that parents of nonexistent directories are not added to the watch targets, instead checking that the directory exists when checking changes. Related to rails#32700. [Matthew Draper & Yuji Yaginuma]

y-yagi · 2018-12-23T11:55:16Z

Fixed by #33822.

Currently, `factories`, `test/factories` and `spec/factories` are specified by default as reload watching paths. In many applications, there are directories which do not exist (perhaps do not have` spec` if using minitest and do not have` spec` if using minitest). In Rails 5 series, if specify a path that does not exist in `EventedFileUpdateChecker`, its parent is added to the watching path. As a result, `node_modules` is also included in the watching path, and unexpectedly many directories and files are included in listen's watching targets. Also, if symlink is included in `node_modules`, it also causes warning of listen. This issue is solved in Rails 6. However, since many applications use this gem in Rails 5 and below, it is good not to add directories that do not exist on the gem side. Ref: rails/rails#32700

…ed」の対応 see: rails/rails#32700 (comment)

justanotherkevin · 2019-03-22T00:52:33Z

on Feb 5 master been merged? I am still running into this issue March 21;

ruby '2.4.1'
gem 'rails', '~> 5.2.2', '>= 5.2.2.1'

Created rails app with rails new react-app --webpack=react

Sorry new to Ruby and Rails
as for using rjurado01 way how do you then import the fixes.rb
Assuming you create the fixes.rb since theres no such file.

BintLopez · 2019-03-22T20:30:21Z

I also just encountered this while setting up a new rails 6 app.

ruby '2.5.3'
rails '6.0.0.beta3'

I didn't start getting the error until I included a few test gems and isolated the break to the addition of the factory_bot_rails gem (going to post in their issues and link to this).

gem 'factory_bot_rails', '~> 5.0', '>= 5.0.1'

coaxial · 2019-04-02T17:55:37Z

Also having this issue running rails 5.2.2.1 and after manually adding the webpacker gem to an existing Rails app.

abuisman · 2019-04-21T21:44:39Z

I have this issue with:

ruby '2.6.2'
gem 'rails', '~> 6.0.0.beta3'

benlayton · 2019-05-19T08:21:54Z

I have this issue on 6.0.0.rc1. Can provide more info if needed.

y-yagi · 2019-05-19T22:30:44Z

Can you please provide a sample application that reproduces the issue with Rails 6.0?

jordan-brough · 2019-05-30T16:51:09Z

@y-yagi here's a repro for Rails 6.0.0.rc1: https://github.com/jordan-brough/example-already-watched

^ See the README for repro

Summary: I get the warnings just by doing this:

$ rails new example-already-watched
$ cd example-already-watched
$ rails console

y-yagi · 2019-05-31T00:39:06Z

@jordan-brough Thanks for your application. This is a spring-watcher-listen's issue(Ref : #32700 (comment)). For now, please remove spring-watcher-listen from Gemfile. Thanks.

jordan-brough · 2019-05-31T14:38:16Z

For now, please remove spring-watcher-listen from Gemfile

👍 Will do, thanks!

@y-yagi will this be fixed in some way for Rails 6? Is there an open issue anywhere (in rails/rails or jonleighton/spring-watcher-listen or wherever) about this, or should this issue be re-opened?

I don't see this problem when using Rails 5.2.3. I assume it would be undesirable to ship Rails 6 with this?

i.e. this works fine for me with no warnings:

rails _5.2.3_ new rails5test
cd rails5test
rails console

but this generates a bunch of warnings: (and again whenever spring restarts)

rails _6.0.0.rc1_ new rails6test
cd rails6test
rails console

`spring-watcher-listen` watch application root by default. https://github.com/jonleighton/spring-watcher-listen/blob/c4bfe15805867e229588e4b776a7b87438137094/lib/spring/watcher/listen.rb#L58 This is necessary to watch the file (e.g. `.ruby-version`) in the application root. By this `node_modules` also be watched, and it is a possibility to be shown a warning by `listen`. Related to rails#32700, rails#34912, rails/webpacker#1990. `listen` watches directory recursive by default, and it cannot avoid it. guard/listen#111 So If this warning happens, the only workaround the user can do is remove the gem. The issue is likely to occur more frequently in Rails 6 because `rails new` runs `webpacker:install` by default. Because of such a state, I think that we should not recommend to use `spring-watcher-listen`. Spring has polling watcher, restart process works without this `spring-watcher-listen`. Because of polling base, CPU load may be higher than listen base. Still I think that it is better than the warning comes out.

rafaelfranca added the railties label Apr 23, 2018

rails-bot bot added the stale label Aug 5, 2018

rails-bot bot removed the stale label Aug 9, 2018

y-yagi mentioned this issue Sep 8, 2018

Do no watch parent directory of dirs #33822

Merged

y-yagi closed this as completed Dec 23, 2018

tabishiqbal mentioned this issue Jan 10, 2019

** ERROR: directory is already being watched! ** #34912

Closed

y-yagi mentioned this issue Feb 5, 2019

Only add exist paths to reloading target thoughtbot/factory_bot_rails#317

Merged

ArtefactGitHub added a commit to ArtefactGitHub/Re_T_Todo that referenced this issue Mar 16, 2019

rails g model <hoge> の際に出たエラー「ERROR: directory is already being watch…

9ea60b4

…ed」の対応 see: rails/rails#32700 (comment)

cyrusstoller mentioned this issue Mar 18, 2019

Directory is already being watched rails/webpacker#1990

Closed

This was referenced Mar 22, 2019

Reloading Issue Triggering #35716

Closed

Reloading Issue Triggering Resolved Rails Bug thoughtbot/factory_bot_rails#328

Closed

y-yagi mentioned this issue Jun 2, 2019

Remove spring-watcher-listen from default Gemfile #36377

Merged

ndbroadbent mentioned this issue Dec 7, 2019

[Solved: Upgrade to Rails 6] Spring app starts to use 100% CPU after I run some specs rails/spring#608

Closed

ahukkanen mentioned this issue Mar 9, 2022

Update rails to 6.1 decidim/decidim#8411

Merged

lot of 'is already being watched through' messages when using rails console if using node_modules #32700

lot of 'is already being watched through' messages when using rails console if using node_modules #32700

Comments

vmarquet commented Apr 23, 2018 • edited

Steps to reproduce

Expected behavior

Actual behavior

System configuration

Additional information

rafaelfranca commented Apr 23, 2018

rafaelfranca commented Apr 23, 2018

vmarquet commented Apr 23, 2018

vmarquet commented Apr 23, 2018

rafaelfranca commented Apr 23, 2018

vmarquet commented Apr 23, 2018

rafaelfranca commented Apr 23, 2018

vmarquet commented Apr 23, 2018 • edited

rafaelfranca commented Apr 23, 2018

vmarquet commented Apr 23, 2018

pixeltrix commented Apr 23, 2018 • edited

vmarquet commented Apr 23, 2018

matthewd commented Apr 23, 2018

vmarquet commented Apr 23, 2018

davidrunger commented May 7, 2018 • edited

Thanks!

I am also affected

My system

Problem: symlinks within node_modules/puppeteer/

node_modules is being watched because spring-watcher-listen listens to root

A possible fix -- but it breaks listening for changes to files in the root of the Rails project

A fix that continues to allow for listening to changes to files in the root of the Rails project might be difficult

Hacky workaround

davidrunger commented May 7, 2018 • edited

vmarquet commented May 7, 2018

rails-bot bot commented Aug 5, 2018

jalkoby commented Aug 9, 2018 • edited

mobilutz commented Aug 13, 2018

y-yagi commented Aug 19, 2018

rjurado01 commented Aug 31, 2018

y-yagi commented Sep 1, 2018

rjurado01 commented Sep 6, 2018

y-yagi commented Sep 7, 2018

y-yagi commented Dec 23, 2018

justanotherkevin commented Mar 22, 2019 • edited

BintLopez commented Mar 22, 2019

coaxial commented Apr 2, 2019

abuisman commented Apr 21, 2019

benlayton commented May 19, 2019

y-yagi commented May 19, 2019

jordan-brough commented May 30, 2019

y-yagi commented May 31, 2019

jordan-brough commented May 31, 2019 • edited

vmarquet commented Apr 23, 2018 •

edited

vmarquet commented Apr 23, 2018 •

edited

pixeltrix commented Apr 23, 2018 •

edited

davidrunger commented May 7, 2018 •

edited

Problem: symlinks within `node_modules/puppeteer/`

`node_modules` is being watched because `spring-watcher-listen` listens to `root`

davidrunger commented May 7, 2018 •

edited

jalkoby commented Aug 9, 2018 •

edited

justanotherkevin commented Mar 22, 2019 •

edited

jordan-brough commented May 31, 2019 •

edited