Skip to content

Leaks in the abstraction

Russell Davis edited this page Oct 17, 2019 · 4 revisions

Leaks in the abstraction

Filesystem notification platforms are each very different from each other. They are each designed with a different idea of their use and purpose, offer wildly varying tradeoffs, and the behaviour of one can be surprising to someone used to another. Nevertheless, this library attempts to abstract away a common interface on top of each, and more than that: an interface to use several of these platforms at the same time, working together.

Naturally, as is law, this abstraction leaks.

This page is not a comprehensive list, because no such list exists, but it tries. It includes leaks from both built-in and external components, and even includes details on components not yet written (so take care while reading: just because a section might indicate a solution to your problem, doesn’t mean the component providing that solution is available right now).

You need not read through all of them, but if you’re building a tool using Notify, it is almost guaranteed that at some point or other, one of your users will encounter a leak. You may as well be prepared.

General edge cases

Behaviour when watched paths are themselves changed

TBD ref #165, #166, op:: docs

Available events

Every platform and backend has a specific set of filesystem events it supports. These are translated to the Notify classification as best they can be. If a platform is said to not support an event here, no recourse is available other than using another platform (if available) or offering patches upstream (if at all possible).

Rather than listing every event supported for every platform, here a note is provided only for a lack or a presence that is of interest.

Metadata events

Linux: inotify

Only has one metadata event: IN_ATTRIB. This covers ownership, permissions, extended attributes, etc. It is not possible to discern what changed, or which change prompted the event.

Linux: fanotify

Has no support for metadata events.

Windows

Emits a "data write" event for metadata changes, so the two are not discernible.

Linux access events: inotify & fanotify

When emitting events for opens and closes, they are able to discern between write-mode or read-mode when the file is closed, but not when the file is opened.

Specific to Linux

fanotify requires superuser

Fanotify requires CAP_SYS_ADMIN, which is equivalent in most ways to root. While it is possible to grant CAP_SYS_ADMIN to a non-root process, it is explicitly recommended against to do so to use fanotify, for security and privacy reasons.[man]

Specific to macOS

Choosing the right backend

macOS has two native platforms: FSEvent and kqueue. By default, Notify uses kqueue. FSEvent has to be explicitly opted into.

The FSEvent platform is designed "for passively monitoring a large tree of files for changes." [apple] Events may be batched, missed, delivered out of order. This is not a mere possibility: this is how this platform works, and there is no workaround. The way FSEvent is meant to be used is as an advisory that something has changed, prompting a manual rescan of the watched directory or tree.

The kqueue platform is a much older interface, originating in BSD systems, [wiki] more powerful but also more demanding. With kqueue, a handle is obtained for every object to be watched, and an event stream is requested to be directed to a particular "queue" through the kernel. When an object (a file or directory) changes, events are immediately emitted. There are no platform facilities for automatically watching a tree of objects: Notify has to do this by itself.

Taking these characteristics into account, FSEvent does not function as a typical user of this library would expect at all. Thus, Notify’s choice of kqueue. If your usecase would benefit from FSEvent’s way of things, you can opt into it, through the notify-backend-fsevent crate.

FSEvent security model

Due to the inner security model of FSEvents, [apple] some events cannot be observed easily when trying to follow files that do not belong to the running user.