Live variables (signal-slot paradigm) and ECS compatibility (suitability)

[dumblob]

As follow-up on bevyengine/bevy#254 (comment) .

@Pauan could you share your opinion why futures-signals or better the whole signal-slot pattern (which implements most of the "double pub-sub" functionality I had in mind in bevyengine/bevy#254 (comment) ) is and I quote a "fundamentally different paradigm" than ECS systems (bevyengine/bevy#254 (comment) )?

I'd say signal-slot ("live variables") actually fits the ECS paradigm very well - see e.g. one of my favourite signal-slot libs out there:

1. https://github.com/TheWisp/signals/
2. or even https://github.com/skypjack/entt/wiki/Crash-Course:-events,-signals-and-everything-in-between

[Pauan]

From what I can tell reading those links, "signal slots" doesn't look like FRP at all, it's basically just an event system.

FRP is very different from an event system, it represents time-varying values as a first-class concept, and provides semantically correct functional ways to compose those values together.

There are a lot of problems with event systems, including the "diamond update" problem (which FRP solves). Another issue that event systems have is that they need to allocate for every individual emitter (including intermediate emitters). Whereas futures-signals completely avoids allocation for Signals (no matter how deeply nested the dependency chain is).

It might be a good idea to read the tutorial to familiarize yourself on the API for Signals (though the tutorial doesn't explain the implementation or the design choices).

As for your question, these are the problems I see with integrating an FRP system with ECS:

• Since FRP is based around a value which changes over time, where will that value be stored? In a component? In a local system? In a global system?
• When a system uses a Signal, it needs to be notified when the Signal changes. That means it needs to spawn a listener, and it needs to cleanup that listener when it's done, and it needs to prevent duplicate listeners from being spawned.
• How will the async Signal listeners be integrated into the ECS event loop?
• Since Signals are async, they cannot access any & or &mut references, so how can a Signal update a component, or run queries?

Those problems can likely be solved, but I don't have any immediate solutions.

There is one way in which Signals could be very easily integrated into ECS: components can contain Mutables, and systems can update the value of those Mutables, however systems cannot listen to Signal changes. And the Signals are only used outside of the ECS system (e.g. with a hierarchical UI).

In that situation everything works perfectly fine, because Signals aren't being used inside of systems, instead Signals are only used as a glue to connect systems to the UI.

[dumblob]

From what I can tell reading those links, "signal slots" doesn't look like FRP at all, it's basically just an event system.

Correct. Achieving FRP wasn't my intention in the Bevy thread and I'm sorry it wasn't clear from the separation into "composition in space" versus "composition in time". Signal-slot/live_variables wasn't meant to be the only mechanism to achieve "composition in time" - assuming we were talking about ECS which on its own has a (great) mechanism for composition in time but still allows for other mechanisms to be combined with.

This is unlike FRP which can't be "combined" with any other "composition in time" mechanism (not counting the case of running FRP separately parallel to some other "composition in time" mechanism).

FRP is very different from an event system, it represents time-varying values as a first-class concept, and provides semantically correct functional ways to compose those values together.

Yes, that's the difference between a "live variable" (as I call it; it's just another way of modeling FRP) and a "signal-slot" mechanism (which though could be used as one of the building blocks for "live variables"). So yes, I've mixed these two and I probably shouldn't have done that. I was under the impression signal-slot would be "good enough" for the intended purpose in Bevy.

There are a lot of problems with event systems, including the "diamond update" problem (which FRP solves). Another issue that event systems have is that they need to allocate for every individual emitter (including intermediate emitters). Whereas futures-signals completely avoids allocation for Signals (no matter how deeply nested the dependency chain is).

It might be a good idea to read the tutorial to familiarize yourself on the API for Signals (though the tutorial doesn't explain the implementation or the design choices).

As for your question, these are the problems I see with integrating an FRP system with ECS:

• Since FRP is based around a value which changes over time, where will that value be stored? In a component? In a local system? In a global system?

In the biggest serially-behaving part of the system. It might be a thread-local storage if the biggest serially behaving part is an os thread. It might be a closure-local place (in case the closure is akin to Erlang process). It might even be global if there is no true concurrency but only parallelism.

This should be decided by the compiler and not the programmer. That's why I wrote we might need a programming paradigm (and thus programming language) extension. Or maybe not - I don't know, I didn't see any full featured "live variable" implementation in any "traditionally" imperative language yet (I know only about Mech lang which is not easily classifiable as it lies somewhere between functional-imperative-actor space). There are things quite close to that, but they always have at least one nasty side effect which makes them unusable in practice (see e.g. HigherOrderCO/Kind1#169 (comment) and HigherOrderCO/Kind1#169 (comment) ).

• When a system uses a Signal, it needs to be notified when the Signal changes. That means it needs to spawn a listener, and it needs to cleanup that listener when it's done, and it needs to prevent duplicate listeners from being spawned.

Well, from my understanding of ECS this would be a wrong approach as I consider each "System" in ECS to be atomic. I.e. either run as a whole or not run at all. In some (many?) cases there are even dependencies between "Systems" so these systems would need to run as well. But this is no more a signal-slot mechanism but the live variable mechanism which ensures the whole graph of dependencies is executed.

• How will the async Signal listeners be integrated into the ECS event loop?

IMHO there is already quite some knowledge from many projects and languages how to integrate such "interrupts" into (hierarchical) event loops of all kinds. So this doesn't sound like a big issue.

• Since Signals are async, they cannot access any & or &mut references, so how can a Signal update a component, or run queries?

This would need to be worked out. Maybe some language extension as I said above to maintain the guarantees?

Those problems can likely be solved, but I don't have any immediate solutions.

I fully agree. I have the very same feeling.

There is one way in which Signals could be very easily integrated into ECS: components can contain Mutables, and systems can update the value of those Mutables, however systems cannot listen to Signal changes. And the Signals are only used outside of the ECS system (e.g. with a hierarchical UI).

In that situation everything works perfectly fine, because Signals aren't being used inside of systems, instead Signals are only used as a glue to connect systems to the UI.

Admittedly this was my primary motivation of proposing signal-slot for Bevy. It just allows for more decoupling. But it doesn't aim to solve the whole "composition in time" problem (it also can't if we have an ECS which also bites off part of the "composition in time" cake).

On the other hand there is also an ongoing work (Polyphony) on finding the right abstractions for fully interactive animated hierarchical GUI in pure ECS (so no trees, no graphs, no signal-slot, nothing like that - only components, entities and systems). And I really like this pure approach despite having the fear that in the end the most efficient way will be some hybrid approach (which the authors of Polyphony admit as a viable way forward if the pure approach proves cumbersome despite it being the other way round so far).

[Pauan]

This is unlike FRP which can't be "combined" with any other "composition in time" mechanism

I don't think that's true, FRP combines perfectly with Futures and Streams. It can also combine perfectly with any event system.

The reason why it is difficult to combine with ECS is because ECS fragments state into flat linear components and synchronous systems (rather than a tree of state and asynchronous updates with FRP).

In other words, ECS is hard to integrate because it is synchronous, and so it doesn't give enough control over time.

In the biggest serially-behaving part of the system. It might be a thread-local storage if the biggest serially behaving part is an os thread. [...] This should be decided by the compiler and not the programmer.

I don't understand this at all, I think we are talking past each other. I am talking about where the programmer's UI state should be stored. That is decided entirely by the programmer and it cannot be decided by the compiler.

Well, from my understanding of ECS this would be a wrong approach as I consider each "System" in ECS to be atomic. I.e. either run as a whole or not run at all.

Once again I think we are talking past each other, of course systems are atomic (and synchronous). The question is how we can specify that a system depends on a Signal, so that way it will automatically rerun the system when the Signal changes.

IMHO there is already quite some knowledge from many projects and languages how to integrate such "interrupts" into (hierarchical) event loops of all kinds. So this doesn't sound like a big issue.

I didn't say it was a big issue, just a problem that needs to be solved. I am well aware of how to integrate FRP with event loops in general, but we're talking specifically about Bevy's event loop.

This would need to be worked out. Maybe some language extension as I said above to maintain the guarantees?

No, this is a fundamental limitation in Rust which can never change: & and &mut cannot escape the stack frame, ever, because they are stack-local references. If you want to use heap allocated references then you must use Box or Rc or Arc (or a static / thread_local).

This issue is not exclusive to FRP, it is an issue with all asynchronous code, including Futures and Streams. So Bevy will need to deal with this issue at some point in the future regardless of whether it uses FRP or not.

On the other hand there is also an ongoing work (Polyphony) on finding the right abstractions for fully interactive animated hierarchical GUI in pure ECS

That is quite interesting, I will look into that.

[dumblob]

This is unlike FRP which can't be "combined" with any other "composition in time" mechanism

I don't think that's true, FRP combines perfectly with Futures and Streams. It can also combine perfectly with any event system.

The reason why it is difficult to combine with ECS is because ECS fragments state into flat linear components and synchronous systems (rather than a tree of state and asynchronous updates with FRP).

In other words, ECS is hard to integrate because it is synchronous, and so it doesn't give enough control over time.

Interesting - I had exactly the same in mind, but I came to an opposite conclusion. My observation is, that if there is a synchronous system, it's very difficult to integrate asynchronous stuff into that system (with "integrate" I understand convenient "interleaving") unless awaiting the future/async/... result immediately after dispatch. Quite contrary I also observe, that if there is an asynchronous system (e.g. FRP) and I want to integrate synchronous stuff into it, it's quite easy. The former case is what I meant with "FRP can't be combined with any other composition in time mechanism" (I implicitly assumed we have a synchronous system and wanted to integrate FRP into it while keeping the system synchronous without turning it into an async one).

In the biggest serially-behaving part of the system. It might be a thread-local storage if the biggest serially behaving part is an os thread. [...] This should be decided by the compiler and not the programmer.

I don't understand this at all, I think we are talking past each other. I am talking about where the programmer's UI state should be stored. That is decided entirely by the programmer and it cannot be decided by the compiler.

Oh, sorry, I misunderstood. My answer to your question is that the value wouldn't be stored anywhere inside the ECS system - it'd be read by ECS pipeline from an "input" buffer.

Well, from my understanding of ECS this would be a wrong approach as I consider each "System" in ECS to be atomic. I.e. either run as a whole or not run at all.

Once again I think we are talking past each other, of course systems are atomic (and synchronous). The question is how we can specify that a system depends on a Signal, so that way it will automatically rerun the system when the Signal changes.

I think this should be modeled as any other asynchronous event ECS systems nowadays support. I.e. the same as keyboard input, mouse input, etc. is being buffered & read from in each tick (or alternatively ECS pipeline is waked up by each keyboard press or mouse move without waiting for next tick).

In which case, the answer is "there is no need to specify any dependency etc. because we (writer to the Signal) see the whole ECS pipeline as one big hollistic consumer of the Signal and don't care about how the ECS pipeline actually uses it internally". Of course with all the consequences it has in a synchronous system (e.g. one tick delay, etc.).

IMHO there is already quite some knowledge from many projects and languages how to integrate such "interrupts" into (hierarchical) event loops of all kinds. So this doesn't sound like a big issue.

I didn't say it was a big issue, just a problem that needs to be solved. I am well aware of how to integrate FRP with event loops in general,

Do you mean event loops of synchronous systems or asynchronous systems? The former case would interest me a lot as that's what I can't imagine without breaking anything or without turning it upside down and making FRP the primary concept/loop and then actually integrating the synchronous system into it.

but we're talking specifically about Bevy's event loop.

I'm a newbie with Bevy, so this one I can't comment on. I'd be though really interested in what makes Bevy's event loop to stand out so much, that even a highly skilled and experienced person as you says that it's not straightforward as with other event loops of synchronous systems.

This would need to be worked out. Maybe some language extension as I said above to maintain the guarantees?

No, this is a fundamental limitation in Rust which can never change: & and &mut cannot escape the stack frame, ever, because they are stack-local references. If you want to use heap allocated references then you must use Box or Rc or Arc (or a static / thread_local).

This issue is not exclusive to FRP, it is an issue with all asynchronous code, including Futures and Streams. So Bevy will need to deal with this issue at some point in the future regardless of whether it uses FRP or not.

The "async as keyboard input" approach as I mentioned above could be one of those "worked out" solutions. Same goes for "async" output - it'd simply need to be buffered/postponed/deferred until the end of the ECS pipeline to not interrupt the pipeline.

On the other hand there is also an ongoing work (Polyphony) on finding the right abstractions for fully interactive animated hierarchical GUI in pure ECS

That is quite interesting, I will look into that.

I wrote to Polyphony authors and they're looking for someone (or better a team) who would want to rewrite it in a sane language (I was told JS was a big pain), extend it with missing stuff (e.g. layouting incl. hierarchical stuff, visuals in general, etc.) and promote its use in general (incl. 3D games & frameworks). If you know of someone who would want to try, just let me or directly the authors know.

[Pauan]

My observation is, that if there is a synchronous system, it's very difficult to integrate asynchronous stuff into that system

Yes, because async code is a strict superset of sync code: all sync code can be async, but not vice versa. That's why async is "infectious", as soon as you make one function async you must make all of the callers async too.

I think this should be modeled as any other asynchronous event ECS systems nowadays support. I.e. the same as keyboard input, mouse input, etc. is being buffered & read from in each tick (or alternatively ECS pipeline is waked up by each keyboard press or mouse move without waiting for next tick).

Okay, that's an interesting design. So it's using a buffer as an intermediary to connect systems to Signals. That might work.

In which case, the answer is "there is no need to specify any dependency etc. because we (writer to the Signal) see the whole ECS pipeline as one big hollistic consumer of the Signal and don't care about how the ECS pipeline actually uses it internally".

Note that there can be hundreds (or thousands, or millions) of Signals, and the web of Signals can dynamically change (adding and removing Signals at runtime), there isn't just "one" Signal.

And the whole point of Signals is that they know what their dependencies are, so they can be efficiently recalculated without needing to recalculate everything. So I think some sort of dependency system would be necessary (or at least desirable), similar to how Changed works today.

Do you mean event loops of synchronous systems or asynchronous systems?

I'm not sure what you mean, event loops are always asynchronous by nature. Of course a single tick of the event loop can be synchronous, that is quite common. If you're using wasm-bindgen-futures then each event loop tick is already fully synchronous.

I'd be though really interested in what makes Bevy's event loop to stand out so much, that even a highly skilled and experienced person as you says that it's not straightforward as with other event loops of synchronous systems.

I just haven't looked into Bevy's event loop implementation at all, so I have no idea how easy or difficult it would be. I know they use wasm-bindgen-futures, but I don't know anything beyond that.

The "async as keyboard input" approach as I mentioned above could be one of those "worked out" solutions. Same goes for "async" output - it'd simply need to be buffered/postponed/deferred until the end of the ECS pipeline to not interrupt the pipeline.

Storing the data in buffers and then consuming it in a synchronous loop fixes the Signal -> system problem, but it doesn't fix the system -> Signal problem.

But maybe the system -> Signal problem doesn't really need to be solved, since systems can just update the Mutables.

If you know of someone who would want to try, just let me or directly the authors know.

I don't know, but maybe somebody working on Bevy would be interested.