memory friendly bijection for async-flow membrane

[mhofman]

What is the Problem Being Solved?

The async-flow (#9302) membrane as implemented by #9097 uses a pair of heap WeakMap for the bijection between host and guest objects. That means any reachable guest object will keep the host representative pinned in memory, which is unfortunate because host objects are durable compatible, and as such virtual (they can be reconstituted as necessary). Furthermore because the WeakMap exposed to userland is virtualized by liveslots, all guest objects remain reachable through the mapping from host objects to guests (they are effectively stored in a Map using the host object's vref as key). The WeakMap virtualization also has significant overhead for something effectively behaving as a Map.

Description of the Design

Without some new liveslots supported mechanism we cannot avoid pinning the guest object to memory.

To avoid pinning the host side to memory, the bijection could be split into a c-list like system where the asyncFlow instance associates an "href" to all host remotables and vows. The heap bijection would then keep a (strong) mapping from guest object to "href", and resolve the actual host object when needed.

Another use of this mapping beyond relieving memory usage may be to remember that a vow has already been watched, but that may not work because of failed replays (see #9097 (comment)). Beyond that, the mapping probably does not have any other uses. The replay membrane does not use marshal to save the log, so there is no space to encode and differentiate clist entries from other data.

Alternate liveslots supported membrane

Some far in the future optimization possibilities

Technically the guest function execution may no longer reference certain guest objects after some `await` points. To avoid pinning guest objects to memory during the whole activation, the bijection would need a `WeakRef` to the guest object, which is restricted to privileged liveslots code. To avoid exposing GC to userland, virtual objects are implemented in such a way that no user code runs when a representative is (re)created.

Virtual objects cannot easily be used for guests because the shape is dynamic (although the number of shapes is low cardinality since the host objects must be durable compatible and as such have a shape defined virtually already).

The guest object is not particularly hard to implement, but it does have a couple baked in notions:

• it currently reflects the exact shape of the host object
• only local objects and operations are supported (no presences or eventual send, see asyncFlow should directly support E via promise handlers #9299
• it's a Remotable

Liveslots could technically implement a ProxySet API which automatically creates guest objects as needed:

interface ProxySetHooks {
  handleCall(guestTarget: object, methodName: undefined | string, args: unknown[]): unknown;
  handleSend(guestTarget, object, methodName: undefined | string, args: unknown[], returnedP: Promise<unknown>): unknown;
}

declare class ProxySet {
  constructor(hooks: ProxySetHooks);
  /** returns or creates the guest object for a given host object */
  guestFor(hostObject: object): object;
  /** returns whether the object is known as a guest object of this set */
  hasGuest(guestObject: object): boolean;
  /** returns the host object corresponding to the guest object */
  hostOf(guestObject): object | undefined;
  /** revokes any outstanding guest objects of this set */
  clear(): void;
}

Liveslots would need to support these guest objects in virtualized WeakMap, likely as a special derived ref of the host object (tied to some kind of "generation number" which is incremented every time the set is cleared.

Security Considerations

The userland approach has none

Any solution involving liveslots and WeakRef powers must be careful to not expose GC to userland.

Scaling Considerations

The mapping would result in 2 durable stores to be created for each async flow instance / activation. There is already one mapStore for each async flow for the log, so the order of magnitude of these stores would remain the same. We would need something like #5263 to reduce their number, which may not be worth it since collections don't have a huge overhead)

Test Plan

TBD. Likely some test logging in liveslots the creation and finalization of presences and virtual object representatives, showing that active flow let these be finalized

Upgrade Considerations

If done after async flow helper starts getting used, must be backwards compatible with flows defined before this change (instance state shape would likely be different).