Do the benefits of signals as a language feature outweigh the costs of solidification into a standard?

[devmachiine]

I think its great that there is a drive towards standardization of signals, but that it is too specialized to be standardized here

Q: Why are Signals being proposed in TC39 rather than DOM, given that most applications of it are web-based?

A: Some coauthors of this proposal are interested in non-web UI environments as a goal, but these days, either venue may be suitable for that, as web APIs are being more frequently implemented outside the web. Ultimately, Signals don't need to depend on any DOM APIs, so either way works. If someone has a strong reason for this group to switch, please let us know in an issue. For now, all contributors have signed the TC39 intellectual property agreements, and the plan is to present this to TC39.

(please let us know in an issue - that's what this issue is for)

Signals is a interesting way to approach state management, but it is much more complicated of a concept compared to something like a PriorityQueue/Heap/BST etc, which I think would be more generally useful as part of javascript itself.

What problem domains besides some UI frameworks would benefit out of it? Are there examples of signals as part of a language feature in other programming languages ?

What would be the benefit of having signals baked-in as a language feature over having a library for doing it?

When something is part of a standard, it's more work & time involved to make changes/additions, than if it was a stand-alone library. For signals to be a part of javascript, I think there would have to be a big advantage over a library.

I can imagine some pros being

• Increased performance
  • Higher than an order of magnitude?
  • How much CPU does signal-related processing consume on a website?
• Easier for a few web frameworks and "Some coauthors" to use signals

Benefits being part of javascript, the same being true if it is part of the DOM api instead

• Less code to ship
• Better debugging posibilities

I can imagine some cons being

• Not being generally useful for the vast majority of javascript programmers
• Changes and improvements take longer to implement across all browsers
• Not being used after a decade, when we discover other approaches to reactive state mangement

Q: Isn't it a little soon to be standardizing something related to Signals, when they just started to be the hot new thing in 2022? Shouldn't we give them more time to evolve and stabilize?

A: The current state of Signals in web frameworks is the result of more than 10 years of continuous development. As investment steps up, as it has in recent years, almost all of the web frameworks are approaching a very similar core model of Signals. This proposal is the result of a shared design exercise between a large number of current leaders in web frameworks, and it will not be pushed forward to standardization without the validation of that group of domain experts in various contexts.

I think in the very least a prerequisite for this as a language feature should be that almost all of the web frameworks use a shared library for their core model of Signals, then it would be proven that there is a use-case for signals as a standard, and much easier to use that shared library as an API reference for a standard implementation.

If anyone could please elaborate more on why signals should be a language feature instead of a library, this issue could serve as a reference for motivation to include it in javascript. 🙃

[mlanza]

Although it may have moved away from the term, Elm as a language had signals baked in from its inception.

Clojure, rather than signals, adopted CSP (core/async). So, yes, some languages have attempted to standardize message relay.

But, like you, I share some apprehension about someone codifying their best idea of what makes a good signal library. RxJS and Bacon are popular but different. And RxJS is quite large. From my own experience, I have gotten by with signals with just a few primitives. I have not needed the sophisticated variations provided by RxJS.

I am smitten with signal goodness, but I'm content to lean on standalone libraries (namely, my own), such is the variety in the implementations.

[dead-claudia]

• Increased performance
  • Higher than an order of magnitude?
  • How much CPU does signal-related processing consume on a website?
• Easier for a few web frameworks and "Some coauthors" to use signals

Benefits being part of javascript, the same being true if it is part of the DOM api instead

• Less code to ship
• Better debugging posibilities

I can imagine some cons being

• Not being generally useful for the vast majority of javascript programmers
• Changes and improvements take longer to implement across all browsers
• Not being used after a decade, when we discover other approaches to reactive state mangement

Q: Isn't it a little soon to be standardizing something related to Signals, when they just started to be the hot new thing in 2022? Shouldn't we give them more time to evolve and stabilize?

A: The current state of Signals in web frameworks is the result of more than 10 years of continuous development. As investment steps up, as it has in recent years, almost all of the web frameworks are approaching a very similar core model of Signals. This proposal is the result of a shared design exercise between a large number of current leaders in web frameworks, and it will not be pushed forward to standardization without the validation of that group of domain experts in various contexts.

I think in the very least a prerequisite for this as a language feature should be that almost all of the web frameworks use a shared library for their core model of Signals, then it would be proven that there is a use-case for signals as a standard, and much easier to use that shared library as an API reference for a standard implementation.

If anyone could please elaborate more on why signals should be a language feature instead of a library, this issue could serve as a reference for motivation to include it in javascript. 🙃

@devmachiine @mlanza You might be surprised to hear that there is a lot of precedent elsewhere, even including the name "signal" (almost always as an analogy to physical electrical signals, if such a physical signal isn't being processed directly). I detailed an extremely incomplete list in #222 (comment), in an issue where I'm suggesting a small variation of the usual low-level idiom for signal/intereupt change detection.

It's slow making its way to the Web, but if you squint a bit, this is only a minor variation of a model of reactivity informally used before even transistors were invented in 1926. Hardware description languages are likewise necessarily based on a similar model.

And this kind of logic paradigm is everywhere in control-oriented applications, from robotics to space satellites. And almost out of necessity.

Here's a simple behavioral model of an 8-bit single-operand adder-subtractor in Verilog, to show the similarity to hardware.

And yes, this is fully synthesizable.

// Adder-subtractor with 4 opcodes:
// 0 = no-op (no request)
// 1 = stored <= in
// 2 = out <= stored + in
// 3 = out <= stored - in
module adder_subtractor(clk, rst, op, in, out);
  input clk, rst;
  input[1:0] op;
  input[7:0] in;
  output reg[7:0] out;

  reg[7:0] stored = 0;
  always @ (posedge clk) begin
    if (rst)
      stored <= 0;
    else case (op)
      2'b00 : begin
        // do nothing
      end
      2'b01 : begin
        stored <= in;
      end
      2'b10 : begin
        out <= stored + in;
      end
      2'b01 : begin
        out <= stored - in;
      end
    endcase
  end
endmodule

Here's an idiomatic translation to JS signals, using methods instead of opcodes:

class AdderSubtractor {
    #stored = new Signal.State(0)

    reset() {
        this.#stored.set(0)
    }

    nop() {}

    set(value) {
        this.#stored.set(value & 0xFF)
    }

    add(value) {
        return (this.stored.get() + value) & 0xFF
    }

    subtract(value) {
        return (this.stored.get() - value) & 0xFF
    }
}

To allow for external monitoring in physical circuits, you'll need two pins:

• A "notify" output signal raised high whenever connected circuits need alerted.
• An optional "notify ack" input that clears that output signal when raised high. (Sometimes needed, but not always.)

Then, consuming circuits can detect the output's rising edge and handle it accordingly.

This idiom is very common in hardware and embedded. And these aren't always one-to-one connections.

Here's a few big ones that come to mind:

• Reset buttons are normally connected to several circuits in parallel. And these are often wired up to both main power (with an in-circuit delay for stability) and dedicated reset buttons, making it a many-to-many connection. When this rises, the circuit is reset to some default state.
• SMBus has a one-way host clock wire, but also a two-way data wire that both a host and connected devices can drive. This two-way data wire could be thought of as a many-to-many connection as sometimes (though rarely) you even see such busses have more than one host on them, complete with the ability to drive them. And the spec does provide for a protocol for one host to take over for another.
• SMBus interrupt signals (SMBALERT#) are normally joined together (a "wired OR" connection in electronics jargon) and connected to an alert input/GPIO pin in a host MCU. This lets connected sensors tell the host to re-poll them while ensuring the host continues to retain full control over the bus's clock. (This side channel is needed to avoid the risk of high-address devices becoming unable to notify a host due to losing arbitration to frequent host or low-address talk.)

It's not as common as you might think inside a single integrated circuit, though, since you can usually achieve what you want through simple boolean logic and (optionally) an internal clock output pin. It's between circuits where it's most useful.

Haskell's had similar for well over a decade as well, though (as a pure functional language) it obviously did signal composition differently: https://wiki.haskell.org/Functional_Reactive_Programming

And keyboard/etc events are much easier to manage performantly in interactive OpenGL/WebGL-based stuff like simple games if you convert keyboard events to persistent boolean "is pressed" states, save mouse position updates to dedicated fields to then handle deltas next frame, and so on. In fact, this is a very common way to manage game state, and the popularity of just rendering every frame like this is also why Dear Imgui is so popular in native code-based games. For similar reasons, that library also has some traction in highly interactive, frequently-updating native apps that are still ultimately window- or box-based (like most web apps).

If anything, the bigger question is why it took so long for front end JS to realize how to tweak this very mature signal/interrupt-based paradigm to suit their needs for more traditional web apps.

[dead-claudia]

As for other questions/concerns:

• Signal performance and memory usage both would be improved with a native implementation.

  • Intermediate array/set objects could be avoided, saving about 12 bytes per signal in V8.
  • Advance knowledge that it's just object reference equality and the fact it's so few values means the watcher set can just be an array, and array search is just a matter of finding a 32-bit value. This also saves 4 bytes per active watcher, at the cost of making it slower for large numbers of watchers (though after a certain size, like maybe 8 watchers, it could be promoted to a proper Set data anyways).
  • With the above, watcher add, remove, and notify would only need one load to get the backing array to iterate. Further, iteration on set doesn't need to go through the ceremony of either set.forEach or set.values(). If you split between small array and large set, you could even use the same loop for both and just change the increment count and start offset (and skip on hole), a code size (and indirectly performance) optimization you can't do in userland.
  • For my proposal in Watcher simplification #222, one could save an entire resolver function worth of overhead, speeding up the notification process to be about the same as queueMicrotask(onNotify). You don't even need to allocate resolver functions at all, just the promise and its associated promise state.

• As for utility, it's not generally useful to most server developers. This is true. It's also of mixed utility to game developers. It is somewhat niche. But there's two points to consider:

  1. It would be far from the first niche proposal to make it in, and there's functionality even more niche than this. Atomics are very niche in the world of browsers. And the receiving side of generators (especially of async generators) is also very niche. I've never used the first in the wild, and I almost completely stopped using the second when async/await became widely supported.
  2. Just re-read the second paragraph of this comment. That history suggests to me that this has a lot more staying power than you'd think at first glance. It's similar to the preference of message passing over shared memory for worker threads in both Node and the Web - it took off as a fad on the server with microservices, but those microservices became the norm as, for most, it's objectively easier both to secure and to scale. (Only caveat: don't prematurely jump to Kubernetes.) In browsers, Redux is essentially this same kind of message passing system, but for state management.

• The proposal is intentionally trying to stay minimal, but still good enough to make direct use possible. Contrast this with URL routing, where on the client side, HTML spec writers only added a very barebones history API and all but required a client-side framework to fill in the rest.

  • They recently added a URL pattern implementation, but most routers' code is in other areas, and URL matching is nobody's bottleneck.
  • This was far from the only Web API addition to rely on magic framework pixie dust to save the day - web components, service workers, and HTTP/2 push are a few more that come to mind. Even the designers behind Trusted Types left non-framework use to a near afterthought.

I think in the very least a prerequisite for this as a language feature should be that almost all of the web frameworks use a shared library [...]

A single shared library isn't on its own a reason to do that. And sometimes, that library idiom isn't even the right way to go.

Sometimes, it is truly one library, and the library has the best semantics: async/await's semantics came essentially from the co module from npm, and almost nothing else came close to it in popularity. Its semantics were chosen as it was the simplest and soundest, though other mechanisms were considered. (The syntax choice was taken from C# due to similarity.) But this is the exception.

Sometimes, it's a few libraries dueling it out, like Moment and date-fns. The very heavy (stage 3) temporal proposal was created to ultimately subsume those with a much less error-prone framework for dates and times that's clearly somewhat influenced by the Intl APIs. This is still not the most common case, though.

Sometimes, it's numerous libraries offering the same exact utility, like Object.entries and Object.fromEntries both being previously implemented in Lodash, Underscore, jQuery, Ramda, among so many others, I gave up years ago even trying to keep track of the "popular" ones with such helpers. In fact, both ES5's Object.keys and all the Array prototype methods added from ES5 to today were added while citing this same kind of extremely broad library and helper precedent. CoffeeScript of old even gave syntax for part of that - here's each of the main object methods (roughly) implemented in it:

Object.keys = (o) ->
    (k for own k, v in o)

Object.values = (o) ->
    (v for own k, v in o)

Object.entries = (o) ->
    ([k, v] for own k, v in o)

Object.fromEntries = (entries) ->
    o = {}
    for [k, v] in entries
        o[k] = v
    o

Speaking of CoffeeScript, that's even inspired additions of its own. And there's been many cases of that and/or other JS dialects inspiring additions.

• The decorators proposal was first experimented on using a custom Angular-specific JS superset called AtScript, and after that got folded into Babel, it later eventually found its way into TypeScript under a compiler option.
• Classes were inspired by a mix of CoffeeScript and early TypeScript. TypeScript clearly inspired the syntax (few changes occurred), but CoffeeScript heavily influenced the semantics (as shown by the relatively few keywords). This inspiration continued even into private fields, where instead of using the private reserved word, a sigil was used instead.
• Optional chaining and nullish coalescing was in CoffeeScript back in 2010 (Groovy, a JVM language, beat it to the punch in 2007, but I couldn't find any others before CoffeeScript), with literally the same exact syntax (and semantics) it has in JS today. Even optional function calls and assignment have a direct equivalent. Only differences are JS uses a ?? b and a ??= b where CoffeeScript uses a ? b and a ?= b, and JS uses ?. for calls and bracketed accesses as well (to avoid ambiguity with ternary expressions). No a? shorthand was added for a == null, though - that was explicitly rejected.

There's also other cases (not just temporal) where existing precedent was more or less thrown away for a clean sheet re-design. For one glaring example, iterables tossed most existing precedent. Anything resembling symbol-named methods are used by nobody else. Nobody had .throw() or .resume() iterator methods. .next() returns a special data structure like nobody else. (Most use "has next" and "get next and advance", Python uses a special exception to stop, and many others stop on null.) Library precedent centered around .forEach and lazy sequences, which was initially postponed with some members at the time rejecting it (this has obviously since changed). JS generators are full stackless coroutines able to have both yield and return values, but do did Python about a decade prior, so that doesn't explain away the modeling difference.

[mlanza]

It's not that I don't think signals are a staple of development. They most certainly are, at least for me. I just think you might get some pushback on what makes sense for the primitives.

JavaScript pipelines are also a necessity and what appeared as straightforward, because Babel had implemented F# pipelines long ago, ended up becoming so extremely fragmented in disagreement as to hit a standstill. And I see signals as far more sophisticated and varied than pipelines.

For example, take this statement from the proposal:

Computed Signals work by automatically tracking which other Signals are read during their evaluation.

This is one variety of how signals can be implemented. I don't like magic. When I was heavily into Ruby I grew to have a strong distaste for the amount of things which had to be understood as happening behind the curtain (e.g. magically). I came to prefer explicit.

So there are a lot of ways signals might be implemented. And if the way the primitives are implemented is distasteful to a group of devs who have other priorities/philosophies for how it should be done, they'll end up using their preferred third-party library. And native signals library will become an extraneous cost. It gets loaded, like it or not, because it's part of the runtime.

[dead-claudia]

It's not that I don't think signals are a staple of development. They most certainly are, at least for me. I just think you might get some pushback on what makes sense for the primitives.

For context, I myself coming in was hesitant to even support the idea of signals until I saw this repo and dug deeper into the model to understand what was truly going on.

And yes, there's been some pushback. In fact, I myself have been pushing back on two major components of the current design:

• I currently find the justification of why the Signal.subtle namespace exists pretty flimsy, and I feel it should be flattened out instead. (TL;DR: the crypto.subtle analogy is weak, and all other justifications I've seen attempted are even less persuasive for me.)
• The change detection mechanism's current registry-based design (using a Watcher class) has a number of issues, both technical and ergonomic. (I'll omit specifics here for brevity - they are very nuanced.)

I also pushed back against watched/unwatched hooks on computeds for a bit, but since backed off from that

I've also been pushing hard for the addition a secondary tracked (and writable) "is pending" state to make async function-based signals definable in userland.

[mlanza]

I don't know what the community sentiment is, but is there a library in JS land which is already considered the de facto standard? Would it be RxJS?

If yes, wouldn't that library form the basis of this proposal? If no, doesn't the lack of a de facto standard seem to suggest a lack of consensus about what signals should be? I mean, why would this newish proposal/api more likely get it right over, say, a mature, well-known library?

I am not arguing against the proposal. I appreciate signals and use them in most UIs I write. I just struggle to understand how a new proposal is is likely to get right what one of the existing libraries hasn't already gotten right.

I recall there is a proposal for adding Observables to the runtime, which is directly related to signals. It's just that particular proposal offers a basic enough primitive/api to reach a consensus. If this one can likewise come up with the right primitives/api, great! I appreciate the effort.

The one thing the Clojure team had going for it when it authored core/async, which is CSP rather than FRP, but close enough, is it appeared to model what Golang had done. In the end, it wasn't design by committee. It was feature complete and so good enough. But then came spec and I guess it made mistakes, because was there spec v2? The point I'm trying to make is that you don't always know how solid a design is until you've had a chance to evaluate it in practice.

And since JS is baked into runtimes, whatever library is adopted becomes an irreversible choice. So this new library should be held as best of breed (by a community majority) after it's designed and used by those having tried the alternatives (RxJS, Bacon, xstream). If that happens, I guess it'd be ready for adopting into the core.

I don't think it need be robust (e.g. all the signal varieties offered by RxJS). It just needs to define/showcase the right primitives/api. The robustness can be tacked on with optional libraries. The aim of getting native primitives is in the optimization, not the robustness.

[dead-claudia]

@mlanza Welcome to the world of the average new stage 1 proposal, where everything is wildly underspecified, somehow both hand-wavy and not, and extremely under flux. 🙃

https://tc39.es/process-document/ should give an idea what to expect at this stage. Stage "0" is the wildest dreams, and stage 1 is just the first attempt to bring a dose of reality into it.

Stage 2 is where the rubber actually meets the road with most proposals. It's where the committee has solidified on a particular solution.

Note that I'm not a TC39 member. I happen to be a former Mithril.js maintainer who's still somewhat active behind the scenes in that project. I have some specific interest in this as I've been investigating the model for a possible future version of Mithril.js.

[devmachiine]

A single shared library isn't on its own a reason to do that. And sometimes, that library idiom isn't even the right way to go.

Good point, I agree. I can see the similarity between game designers and gamers who propose balances/changes which wouldn't benefit the game(rs) as a whole and/or have unintended consequences.

everywhere in control-oriented applications .. very common in hardware and embedded

Interesting. Especially the circuitry example! Because X exists in Y isn't enough justification on its own to include X in Z. I don't think javascript is geared towards those use cases, it's more in the domain of c/zig.

As for utility, it's not generally useful to most server developers. This is true. It's also of mixed utility to game developers. It is somewhat niche. But there's two points to consider: It would be far from the first niche proposal to make it in, and there's functionality even more niche than this. Atomics are very niche in the world of browsers.

Good point! I found the same to be true with the Symbol primitive. For years, I didn't really get it, but once I had a use case, I loved it.

I reconsidered some of the pro's of signals being baked into the language(or DOM api) which I stated
Less code to ship
Even if signals usage is ubiquitous, technically less bytes are sent over the wire, but practically not.
Performance
Technically yes, but practically? If a substantial portion of compute is taken up by signals processing, its probably a simulation or control-oriented application, and here I think it's out of domain scope for javascript again.

recall there is a proposal for adding Observables to the runtime, which is directly related to signals
https://github.com/tc39/proposal-observable

There were similar concerns regarding the conclusion of not moving forward with the Observable proposal

I think there will be a lot of repeat discussion:

Why does this need to be in the standard library? No answer to that yet.
Where does this fit in? The DOM
Are there use cases in Node.js?
(examples of DOM apis that make sense in DOM and Node, but not in language)
Concerns about where it fits in host environments
Stronger concerns: will this be the thing actually _used_ in hosts?

I can appreciate the standardization of signals, but I'm not convinced that tc39 is the appropriate home for signals. The functionality can be provided via a library, which is much easier to extended and improve across contexts.

[dead-claudia]

Technically yes, but practically? If a substantial portion of compute is taken up by signals processing, its probably a simulation or control-oriented application, and here I think it's out of domain scope for javascript again.

@devmachiine You won't likely see .set show up on performance profiles, but very large DOM trees (I've heard of trees in the wild as large as 50k elements, and had to direct someone with 100k+ SVG nodes to switch to canvas once) using signals and components heavily could see .get() showing up noticeably.

But just as importantly, memory usage is a concern. If you have 50k signals in a complex monitoring app (say, 500 items, with 15 discrete visible text fields, 50 fields across 4 dropdowns, 20 error indicators, and 5 inputs), and you can shave off an average of about 20 bytes of each of those signals by simply removing a layer of indirection (2x4=8 bytes) and not allocating arrays for single-reference sets (2x32=64 bytes per impacted object, conservatively assumes about 20% are single-listener + single-parent), you could've shaved off around entire entire megabyte of memory usage. And that could be noticeable.