New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add an experimental built-in FSMonitor #3082
Changes from all commits
21f4ba1
e194809
2b4dd0c
ebf106b
a6c0c42
e24c74b
c6e4e01
0213ed9
ae8103d
e287fcb
5b0f4e9
d74719a
c4bd6a0
ff57cc0
6202d0e
db970f9
5162e1f
f783c65
7c17e4f
88bfdc9
4654a3e
5c6a9f8
1c9f8ec
c1cf306
3a7659b
fce149c
d49f164
6b4070e
3ffa3f6
7952e18
581ab48
4d728d1
bd9d11c
6cc4f89
bbf2a4f
df6bf08
5fa93cf
f9046e5
fe048bf
e666cc2
f049558
e758ff7
44234a2
60deac2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -66,18 +66,45 @@ core.fsmonitor:: | |
will identify all files that may have changed since the | ||
requested date/time. This information is used to speed up git by | ||
avoiding unnecessary processing of files that have not changed. | ||
See the "fsmonitor-watchman" section of linkgit:githooks[5]. | ||
+ | ||
See the "fsmonitor-watchman" section of linkgit:githooks[5]. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ?? ... and see There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm. I don't think we need this here, but you're right, I need to mention the |
||
+ | ||
Note: FSMonitor hooks (and this config setting) are ignored if the | ||
(experimental) built-in FSMonitor is enabled (see | ||
`core.useBuiltinFSMonitor`). | ||
|
||
core.fsmonitorHookVersion:: | ||
Sets the version of hook that is to be used when calling fsmonitor. | ||
There are currently versions 1 and 2. When this is not set, | ||
version 2 will be tried first and if it fails then version 1 | ||
will be tried. Version 1 uses a timestamp as input to determine | ||
which files have changes since that time but some monitors | ||
like watchman have race conditions when used with a timestamp. | ||
Version 2 uses an opaque string so that the monitor can return | ||
something that can be used to determine what files have changed | ||
without race conditions. | ||
Sets the version of hook that is to be used when calling the | ||
FSMonitor hook (as configured via `core.fsmonitor`). | ||
+ | ||
There are currently versions 1 and 2. When this is not set, | ||
version 2 will be tried first and if it fails then version 1 | ||
will be tried. Version 1 uses a timestamp as input to determine | ||
which files have changes since that time but some monitors | ||
like watchman have race conditions when used with a timestamp. | ||
Version 2 uses an opaque string so that the monitor can return | ||
something that can be used to determine what files have changed | ||
without race conditions. | ||
+ | ||
Note: FSMonitor hooks (and this config setting) are ignored if the | ||
built-in FSMonitor is enabled (see `core.useBuiltinFSMonitor`). | ||
|
||
core.useBuiltinFSMonitor:: | ||
(EXPERIMENTAL) If set to true, enable the built-in filesystem | ||
event watcher (for technical details, see | ||
linkgit:git-fsmonitor--daemon[1]). | ||
+ | ||
Like external (hook-based) FSMonitors, the built-in FSMonitor can speed up | ||
Git commands that need to refresh the Git index (e.g. `git status`) in a | ||
worktree with many files. The built-in FSMonitor facility eliminates the | ||
need to install and maintain an external third-party monitoring tool. | ||
+ | ||
The built-in FSMonitor is currently available only on a limited set of | ||
supported platforms. | ||
+ | ||
Note: if this config setting is set to `true`, any FSMonitor hook | ||
configured via `core.fsmonitor` (and possibly `core.fsmonitorHookVersion`) | ||
is ignored. | ||
|
||
core.trustctime:: | ||
If false, the ctime differences between the index and the | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
git-fsmonitor--daemon(1) | ||
======================== | ||
|
||
NAME | ||
---- | ||
git-fsmonitor--daemon - (EXPERIMENTAL) Builtin file system monitor daemon | ||
|
||
SYNOPSIS | ||
-------- | ||
[verse] | ||
'git fsmonitor--daemon' --start | ||
'git fsmonitor--daemon' --run | ||
'git fsmonitor--daemon' --stop | ||
'git fsmonitor--daemon' --is-running | ||
'git fsmonitor--daemon' --is-supported | ||
'git fsmonitor--daemon' --query <token> | ||
'git fsmonitor--daemon' --query-index | ||
'git fsmonitor--daemon' --flush | ||
|
||
DESCRIPTION | ||
----------- | ||
|
||
NOTE! This command is still only an experiment, subject to change dramatically | ||
(or even to be abandoned). | ||
|
||
Monitors files and directories in the working directory for changes using | ||
platform-specific file system notification facilities. | ||
|
||
It communicates directly with commands like `git status` using the | ||
link:technical/api-simple-ipc.html[simple IPC] interface instead of | ||
the slower linkgit:githooks[5] interface. | ||
|
||
OPTIONS | ||
------- | ||
|
||
--start:: | ||
Starts the fsmonitor daemon in the background. | ||
|
||
--run:: | ||
Runs the fsmonitor daemon in the foreground. | ||
|
||
--stop:: | ||
Stops the fsmonitor daemon running for the current working | ||
directory, if present. | ||
|
||
--is-running:: | ||
Exits with zero status if the fsmonitor daemon is watching the | ||
current working directory. | ||
|
||
--is-supported:: | ||
Exits with zero status if the fsmonitor daemon feature is supported | ||
on this platform. | ||
|
||
--query <token>:: | ||
Connects to the fsmonitor daemon (starting it if necessary) and | ||
requests the list of changed files and directories since the | ||
given token. | ||
This is intended for testing purposes. | ||
|
||
--query-index:: | ||
Read the current `<token>` from the File System Monitor index | ||
extension (if present) and use it to query the fsmonitor daemon. | ||
This is intended for testing purposes. | ||
|
||
--flush:: | ||
Force the fsmonitor daemon to flush its in-memory cache and | ||
re-sync with the file system. | ||
This is intended for testing purposes. | ||
|
||
REMARKS | ||
------- | ||
The fsmonitor daemon is a long running process that will watch a single | ||
working directory. Commands, such as `git status`, should automatically | ||
start it (if necessary) when `core.useBuiltinFSMonitor` is set to `true` | ||
(see linkgit:git-config[1]). | ||
|
||
Configure the built-in FSMonitor via `core.useBuiltinFSMonitor` in each | ||
working directory separately, or globally via `git config --global | ||
core.useBuiltinFSMonitor true`. | ||
|
||
Tokens are opaque strings. They are used by the fsmonitor daemon to | ||
mark a point in time and the associated internal state. Callers should | ||
make no assumptions about the content of the token. In particular, | ||
the should not assume that it is a timestamp. | ||
|
||
Query commands send a request-token to the daemon and it responds with | ||
a summary of the changes that have occurred since that token was | ||
created. The daemon also returns a response-token that the client can | ||
use in a future query. | ||
|
||
For more information see the "File System Monitor" section in | ||
linkgit:git-update-index[1]. | ||
|
||
CAVEATS | ||
------- | ||
|
||
The fsmonitor daemon does not currently know about submodules and does | ||
not know to filter out file system events that happen within a | ||
submodule. If fsmonitor daemon is watching a super repo and a file is | ||
modified within the working directory of a submodule, it will report | ||
the change (as happening against the super repo). However, the client | ||
should properly ignore these extra events, so performance may be affected | ||
but it should not cause an incorrect result. | ||
|
||
GIT | ||
--- | ||
Part of the linkgit:git[1] suite |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
Simple-IPC API | ||
============== | ||
|
||
The Simple-IPC API is a collection of `ipc_` prefixed library routines | ||
and a basic communication protocol that allow an IPC-client process to | ||
send an application-specific IPC-request message to an IPC-server | ||
process and receive an application-specific IPC-response message. | ||
|
||
Communication occurs over a named pipe on Windows and a Unix domain | ||
socket on other platforms. IPC-clients and IPC-servers rendezvous at | ||
a previously agreed-to application-specific pathname (which is outside | ||
the scope of this design) that is local to the computer system. | ||
|
||
The IPC-server routines within the server application process create a | ||
thread pool to listen for connections and receive request messages | ||
from multiple concurrent IPC-clients. When received, these messages | ||
are dispatched up to the server application callbacks for handling. | ||
IPC-server routines then incrementally relay responses back to the | ||
IPC-client. | ||
|
||
The IPC-client routines within a client application process connect | ||
to the IPC-server and send a request message and wait for a response. | ||
When received, the response is returned back the caller. | ||
|
||
For example, the `fsmonitor--daemon` feature will be built as a server | ||
application on top of the IPC-server library routines. It will have | ||
threads watching for file system events and a thread pool waiting for | ||
client connections. Clients, such as `git status` will request a list | ||
of file system events since a point in time and the server will | ||
respond with a list of changed files and directories. The formats of | ||
the request and response are application-specific; the IPC-client and | ||
IPC-server routines treat them as opaque byte streams. | ||
|
||
|
||
Comparison with sub-process model | ||
--------------------------------- | ||
|
||
The Simple-IPC mechanism differs from the existing `sub-process.c` | ||
model (Documentation/technical/long-running-process-protocol.txt) and | ||
used by applications like Git-LFS. In the LFS-style sub-process model | ||
the helper is started by the foreground process, communication happens | ||
via a pair of file descriptors bound to the stdin/stdout of the | ||
sub-process, the sub-process only serves the current foreground | ||
process, and the sub-process exits when the foreground process | ||
terminates. | ||
|
||
In the Simple-IPC model the server is a very long-running service. It | ||
can service many clients at the same time and has a private socket or | ||
named pipe connection to each active client. It might be started | ||
(on-demand) by the current client process or it might have been | ||
started by a previous client or by the OS at boot time. The server | ||
process is not associated with a terminal and it persists after | ||
clients terminate. Clients do not have access to the stdin/stdout of | ||
the server process and therefore must communicate over sockets or | ||
named pipes. | ||
|
||
|
||
Server startup and shutdown | ||
--------------------------- | ||
|
||
How an application server based upon IPC-server is started is also | ||
outside the scope of the Simple-IPC design and is a property of the | ||
application using it. For example, the server might be started or | ||
restarted during routine maintenance operations, or it might be | ||
started as a system service during the system boot-up sequence, or it | ||
might be started on-demand by a foreground Git command when needed. | ||
|
||
Similarly, server shutdown is a property of the application using | ||
the simple-ipc routines. For example, the server might decide to | ||
shutdown when idle or only upon explicit request. | ||
|
||
|
||
Simple-IPC protocol | ||
------------------- | ||
|
||
The Simple-IPC protocol consists of a single request message from the | ||
client and an optional response message from the server. Both the | ||
client and server messages are unlimited in length and are terminated | ||
with a flush packet. | ||
|
||
The pkt-line routines (Documentation/technical/protocol-common.txt) | ||
are used to simplify buffer management during message generation, | ||
transmission, and reception. A flush packet is used to mark the end | ||
of the message. This allows the sender to incrementally generate and | ||
transmit the message. It allows the receiver to incrementally receive | ||
the message in chunks and to know when they have received the entire | ||
message. | ||
|
||
The actual byte format of the client request and server response | ||
messages are application specific. The IPC layer transmits and | ||
receives them as opaque byte buffers without any concern for the | ||
content within. It is the job of the calling application layer to | ||
understand the contents of the request and response messages. | ||
|
||
|
||
Summary | ||
------- | ||
|
||
Conceptually, the Simple-IPC protocol is similar to an HTTP REST | ||
request. Clients connect, make an application-specific and | ||
stateless request, receive an application-specific | ||
response, and disconnect. It is a one round trip facility for | ||
querying the server. The Simple-IPC routines hide the socket, | ||
named pipe, and thread pool details and allow the application | ||
layer to focus on the application at hand. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we qualify this a little?