How to pause while responding to file changes?

[beechnut]

I'm using Listen to watch a directory for changes, and sometimes the response to a change modifies one of the files being watched. I don't want the listener to respond to the automatic file modification. (It is not possible, within my design, to exclude such files from being watched.)

Another way to say this: I want to pause the listener while the block that's responding to changes gets executed.

I'm looking for a way to do something like:

listener = Listen.to('/src') do |modified, added, remaining|
  # pause the listener
  maybe_make_changes_to_files(modified, added)
  # unpause the listener
end

Attempted solutions

• I've tried setting various options, including latency and the like. None of these stops the listener from triggering when the files in question change, it just changes the timing.
• I've tried passing listener to maybe_make_changes_to_files as a parameter and have that method call listener.pause. I get a message that says that's not allowed.

[ColinDKelley]

@beechnut Can you try

listener.pause
...
listener.start

? I haven't used it myself, but the README documents this usage. (Although it refers to listener.unpause, which has been removed. That's being fixed in #550.

[beechnut]

@ColinDKelley I've tried that a few different ways, and I can't figure out how to have it pause while the listen job is running. (I just edited my second bullet to be more clear about that.)

The main issue, as I see it, is that there is no way to access the listener from within the block.

listener = Listen.to('/src') do |modified, added, remaining|
  listener.pause # `listener` isn't defined, so I can't call this here.
  maybe_make_changes_to_files(modified, added)
  listener.start # same issue as 2 lines up
end

Ideally I'd be able to write something like:

my_listener = Listen.to('/src') do |event|
  event.listener.pause
  maybe_make_changes_to_files(event.modified, event.added)
  event.listener.start
end

[ColinDKelley]

Hi @beechnut, are you sure there is no way to access listener from inside the block? From a simple test I would assume it could be accessed. The only scenario I could think of where it wouldn't be visible is if the gem were to call back to your block before returning from initialize. I don't believe that ever happens. In fact, the documented usage always has listener.start call after the call to Listen.to, which seems to me to make such a race condition impossible.

Can you double-check your assertion here? If you still get a failure, which version of Ruby are you using? I tested with 2.6.1.

# `listener` isn't defined, so I can't call this here.

[beechnut]

From a simple test I would assume it could be accessed.

Can you explain further what you mean by "test" here? Did you run a test or have a working code sample that can access listener from inside the block? If so, I'd appreciate seeing it, since I've tried this from many angles with no luck.

[ColinDKelley]

@beechnut I just tried this code and it worked exactly as expected, using Ruby 2.6.1 and listen 3.4.0:

listener = Listen.to("/tmp/") { |*args| puts args.inspect; listener.pause }
listener.start

The expected behavior is that the block runs the first time anything is changed in /tmp. It then pauses. When I make more changes in /tmp, the block doesn't execute. Until I run listener.start again, at which point the queued changes are yielded back to the block.

Does this code work for you? If not, can you include the specific error you get?

[ColinDKelley]

@beechnut Any update here?

[beechnut]

So, that particular code snippet does technically work for me, and I got another more complicated example to work:

listener = Listen.to("/path/to") do |*args|
  puts args.inspect
  listener.pause
  puts "doing some work:"
  %x( touch /path/to/file ) ; puts "\tmade a file"
  sleep 1
  %x( rm /path/to/file ) ; puts "\tdeleted a file"
  sleep 0.5
  listener.start
end

However, this has helped me realize that I'm trying to pause listening, not responding, and that's why I referred to #pause. However, #pause keeps collecting file changes, which is why my listener keeps getting retriggered once it's triggered the first time. What I'm really needing is to be able to call listener.stop, process some file changes, and restart.

listener = Listen.to("/path/to") do |*args|
  puts args.inspect
  listener.stop # <-- changed from `pause` to get it to stop listening to changes
  puts "doing some work:"
  %x( touch /path/to/file ) ; puts "\tmade a file"
  sleep 1
  %x( rm /path/to/file ) ; puts "\tdeleted a file"
  sleep 0.5
  listener.start
end

This results in the following output, printing the changed files and then throwing a ThreadError:

[[], ["/path/to/hello_"], ["/path/to/hello"]]
E, [2022-01-24T16:55:19.083667 #4625] ERROR -- : Exception rescued in _process_changes:
ThreadError: Target thread must not be current thread

I assume that the issue here is calling #stop on the listener that is currently running.

The options I'm seeing are:
(a) Kill the current thread (#stop) or
(b) Pause processing, but continue collecting file changes.
I'm in need of option (c): Keep the current thread but pause processing AND pause collecting file changes.

[ColinDKelley]

However, this has helped me realize that I'm trying to pause listening, not responding

Can't you achieve what you want with a level of indirection, where the callback sometimes delegates to a proc/lambda and sometimes doesn't? As in:

pausable_listen_block = lambda do |*args|
  puts "Pausable listen block got: #{args.inspect}"
end

listen_block = pausable_listen_block

listener = Listen.to("/path/to") do |*args|
  puts "Listen.to got #{args.inspect}"
  listen_block&.call(*args)
  listen_block = nil
  puts "doing some work:"
  `touch /path/to/file`
  puts "    made a file"
  sleep 1
  `rm /path/to/file`
  puts "    deleted a file"
  sleep 0.5
  listen_block = pausable_listen_block
end

But this example is inherently confusing because the state changes are inside the callback...which can run recursively. I think it would be simpler to demonstrate without recursive nesting.

[ColinDKelley]

@beechnut Any objections if I close out this issue and you can file a fresh one with your refined concern?

[LouisaNikita]

我已收到你的邮件，谢谢！  

[beechnut]

@ColinDKelley I'd like to keep this open for now, because reading back through the history, I think the refined concern is still essentially the same as it was in the first post:

Another way to say this: I want to pause the listener while the block that's responding to changes gets executed.
I'm trying to pause listening, not responding

I've been swamped and haven't had time to try the last solution you posted—thanks for posting that, and 'll give that a try early next week and reply with results.

I'm still interested in the idea of:

option (c): Keep the current thread but pause processing AND pause collecting file changes.

Is that something that seems architecturally possible? I haven't dug into the internals of Listen much, you'll know better than me.

[ColinDKelley]

@beechnut

option (c): Keep the current thread but pause processing AND pause collecting file changes.

That sounds achievable with a level of indirection. I posted about that above, on Jan 31. Will that approach work for you?

[LouisaNikita]

我已收到你的邮件，谢谢！  

[beechnut]

I haven't been free to focus on this in a while, but I'm getting the impression that there's not much interest in fully understanding or supporting this use case, so I'm just going to close the issue.