inotify: Flush All Events Before Close

[horahoradev]

Is your feature request related to a problem? Please describe.
This is a feature request, but relates to a problem. The current code implementing fsnotify's event reads and sends for Linux does not flush new events on close, and may thus drop events; a relevant snippet is as follows:

			if !event.ignoreLinux(mask) {
				select {
				case w.Events <- event:
				case <-w.done:
					return
				}
			}

If w.done can be received from and w.events can be sent to, then the select statement will pick one of the above cases at random, potentially dropping the current event (and all other events).

Describe the solution you'd like
I'd like a configuration option to specify whether events should be flushed on watcher close, with a configurable timeout on flushing those events.

The library would attempt to flush all events to the client within a timeout, and if it couldn't flush all events in time, then it should return an error to that effect.

Describe alternatives you've considered
In the below problem, if the client is aware of the library's behavior and that the main process has exited, it could receive on the events channel with a timeout, but that's awkward, and the shutdown time could be shorter if we exit immediately after ensuring that the inotify fd isn't readable.

Additional context
Here's a contrived example: a program is deployed alongside a sidecar which monitors the files that it writes. If the main program exits, the sidecar exits as well, but how can the sidecar be sure that it has received and processed every event before exiting? Flushing the events on close would be an elegant solution here.

I can make a PR for this change, just let me know if you approve of the content of my proposal.