New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add fanotify support #114
Comments
That would be great. The only thing is, how does someone using Linux choose between them? We may need something along the lines of #104 first. |
Just wanted to get the approximate ACK, so that I can point people here if they're interested in fanotify benefits. Thanks! |
@purpleidea I think it would be best to build fanotify Go wrapper out as a separate repo and then look at integrating after that. |
@amir73il is working on a "super block watch" for Linux, providing "the ability to set a single (fanotify) watch on a root directory and get notified on all the legacy inotify events without the need to recursively add watches on all directories." https://lkml.org/lkml/2016/12/20/312 This could avoid the need for a user-space recursive watcher (#16) on modern Linux kernels. |
Well the patches are out there already in my github (applied to kernel 4.9), but for those of you hoping for this functionality to get upstream, I suggest to be patient. I have no doubt it is going to be some time before this feature can be This is were you guys can be of help. When promoting a feature for upstream it is important to bring solid use cases that require the feature and argue that the same cannot be achieved by user library code and existing kernel functionality. However, if you can't test my work on a distro kernel then it is going to be harder to claim that it is beneficial for your use cases. I cannot guaranty when I will get to providing this level of installation though, so if there are any of you out there not afraid of building a custom kernel, I will gladly assist you if you want to test my patches. Cheers. |
Thanks Amir. Perhaps another option to make the patched kernel available would be to maintain a Vagrant box built with Packer. That way we could test fanotify super block using a VirtualBox VM from any operating system. |
Yes, that could work. And I promise to assist the person who volunteers to work on this setup. |
Amir, which kernel version you would like to target ? |
@amir73il I have pinged some kernel engineers at my company to look into your patch. In the meantime, if you have a moment, could you look into and recommend an algorithm or suggest an improvement to the recursive file watching which I've implemented for mgmt? The code is available here: Cheers! |
@tiwaana question is moot. I would like to target the earliest kernel version possible, but since this is not a bug fix nor a trivial improvement, some things have to happen first not all of them depend on me, not necessarily in that order:
|
@purpleidea thanks for the ping. If your company will show interest in the super block watch, that can be a game changer. wrt your recursive watcher, I am new to golang and have zero knowledge about fsnotify library, but it appears your code is not calling addSubFolders() recursively from Init() more than 1 level of depth, so if you never get events on the direct sub folders you will never add watchers for level 2 subdirs, but I may be missing something. Also I don't see any handling of Move events for dirs, unless it is handled in lib by generating Rename/Create event pair. |
On Wed, Feb 1, 2017 at 3:26 AM, Amir Goldstein ***@***.***> wrote:
@purpleidea thanks for the ping. If your company will show interest in the super block watch, that can be a game changer. wrt your recursive watcher, I am new to golang and have zero knowledge about fsnotify library, but it appears your code is not calling addSubFolders() recursively from Init() more than 1 level of depth, so if you never get events on the direct sub folders you will never add watchers for level 2 subdirs, but I may be missing something. Also I don't see any handling of Move events for dirs, unless it is handled in lib by generating Rename/Create event pair.
I think I am having an algorithmic mind block, as IIRC I think some
cases didn't work all the time. In any case, golang is very similar to
C if you get bored :)
|
You do know, that fanotify supports recursive watch on (any, even bind) mountpoint with FAN_MARK_MOUNT, right? |
@isage focus on the part 'all the legacy inotify event', namely, create/move/delete. |
Linux fanotify added directory events (move/delete/etc) back in 2017: |
@pabs3 I was not aware of any distro that picked up the patch you mentioned, @nathany, sorry I forgot to update you when the feature got merged upstream: Man pages were already updated: On my github, you can find demo conversion of inotifywait tool to use fanotify super block watch instead of a recursive inotify watch: Please note that at this time, the feature enables user to listen on ALL directory events in the filesystem and any sort of filtering by subtree would have to be implemented in user space. Let me know if you are interested in using fanotify and if you have any questions. |
Other than requiring a newer Linux kernel, is there any disadvantage to using fanotify? Could we detect support for fanotify and fallback to inotify if not available? Would two or more people be interested in building out a stand-alone fanotify module/package, either in a separate repository or a subfolder of fsnotify? Then we could look at integrating it into fsnotify after that. |
To detect support just need to execute fanotify_init(FAN_REPORT_FID, 0). The disadvatage compared to recursive inotify is that there is no subtree level filterting in the kernel. At the moment, directory modification events are NOT supported along with FAN_MARK_MOUNT due to Linux vfs implementation constrains. |
@amir73il any update on this issue? |
@s3rj1k which updates are you expecting? The way I see it, the kernel code is ready and waiting for volunteers to implement the userspace recursive watcher. I even provided sample C code. I forgot to mention in the answer to @nathany, that unlike inotify, fanotify requires SYS_CAP_ADMIN. Not sure if that is a problem for fsnotify. |
@amir73il Hi, basic support for fanotify in fsnotify. |
@amir73il I actually plan on using it for Directory backend. |
@s3rj1k Are you interested only in open/close events? |
@amir73il Open/Close recursively, for antivirus on demand scanner. |
@s3rj1k I don't understand the question. Fanotify could always do that. If you are asking about fsnotify lib support the no, but you are talking to the wrong person |
This comment was marked as spam.
This comment was marked as spam.
I played a bit with fanotify and go, here is a working example |
@amir73il are there any examples with newer fanotify functionality? |
@s3rj1k see link above to inotofy-tools global watch demo |
@amir73il The latter example actually helps a lot for none-C people. |
@s3rj1k you will have to be more specific than "similar functionality". |
@amir73il yes. I've set |
@nathany basic (pre 5.1 kernel) fanotify support can be added using |
Would be nice to have someone with more knowledge of C to port those new fanotify structures to golang. |
@amir73il You mentioned earlier
Can you make this patch public? |
For the record, the remaining bits of fanotify filesystem watch have been merged to kernel v5.9: Man pages were updated for using modes like FAN_REPORT_DFID_NAME, which most closely resembles the inotify event information: |
Would there be any objections if someone sent in patches to add support for fanotify?
The text was updated successfully, but these errors were encountered: