Realtime sync proof of concept

[richardolsson]

Description

This PR is a proof of concept for how "realtime sync" could be implemented, meaning that the interfaces for two users (or the same user in two browser windows) would be immediately updated in both places when data changes.

Architecture

This feature uses websockets (using the Socket.IO library) to communicate rapidly and push updates from server to client. See diagram and description below.

Backend

Updates are triggered by the API proxy on the backend, meaning that when an API request is made that modifies data, it may trigger a message to be broadcast to all connected users (see "Security Model" below). This means that to use this feature, a developer does not need to write any custom frontend logic, but may need to hook into the API proxy to define what messages should trigger.

Frontend

When a message is received on the frontend, a Redux action is dispatched. This means that all store slices can react to the message using extraReducers and update as needed. In the typical case, the relevant part of the store should be invalidated (isStale = true) to prompt the repo to re-fetch the data the next time it's used. In other cases, modifying the data directly might make sense (such as when something is deleted).

In a future version, the useSocket() hook (or some wrapper) could be used by the frontend to also trigger updates, i.e. updates that are not related to API CRUD operations, such as selecting a view cell.

Security Model

Security is severely lacking in this POC. All changes in any organization are broadcast to all users who have managed to connect to the socket (which could potentially happen even without authenticating).

For this feature to be secure I have envisioned three measures.

Per-org namespaces

Instead of a global socket, there should be namespaces per organization, so that users are subscribed only to updates related to data in their organization.

Authorization on connect

Whenever a user tries to connect to the socket, their privileges should be verified against the Zetkin Platform API. This can be done by making a simple request to /users/me/memberships and verifying their access to the organization they are trying to subscribe to.

Minimum-disclosure policy

Sockets should never be used to transfer full data object. Instead, only IDs should be included, and the receiving frontends can use that information to invalidate their caches for relevant objects, prompting up-to-date to be retrieved from the API (where security is more strictly per-request) the next time it's needed.

Potential improvements

• Implement full security model
• Create generalized way of defining event triggers in the code for each feature
• Design UX/UI that explains to user what is happening when updates are synced

Screenshots

realtime-view-poc.mov

Changes

• Modifies the system to run NEXT in a custom server
• Adds the Socket.IO library (for websocket communication) to the custom server
• Creates a SocketContext provider and hook
  • The provider also triggers store updates (not an ideal place for this, but good enough for POC)
  • The useSocket() hook is unused, but could be used to trigger updates from the UI
• Implements add and remove events for view rows

Notes to reviewer

Try if you want! Right now I'm mostly looking for feedback.

Related issues

None