lock domain socket and remove on last arbiter exit

[paulnivin]

This fixes a bug where during a hot restart, any unix domain sockets are removed when the original arbiter is TERM'd.

Sequence:

• First arbiter starts (socket file created)
• New arbiter starts via USR2 (all is well)
• First arbiter exits via TERM (socket file is removed)

For the fix, every arbiter grabs a read lock on startup. On exit, every arbiter attempts to grab a write lock -- if successful, we're the only arbiter running and the socket file can be safely removed.

[benoitc]

the fix looks fine for me. Any idea how we could have a test for it though? Would be better.

[benoitc]

@tilgovi @berkerpeksag ok for you?

[paulnivin]

I haven't had time to look into how we could write a test for this case, but I can do so within the next week or so.

[tilgovi]

Probably a new file, tests/test_socket.py, with a simple test would suffice. Mock out fcntl and unlink and verify that fcntl is called during the constructor and that unlink is not called if fcntl raises.

👍 this is a good patch

[tilgovi]

Closing in favor of #1220, which adds tests to this.