Opinions on worker/main thread communication design

[SpanishPear]

Hello!

This is a long question on how to best design the communication between worker and main thread when it comes to lots of state - I've tried to lay out the probelem space well but it's late and I'm tired so please let me know if you'd like more details!

I'm trying to refactor a large yew application with a web worker to be more idiomatic and nicer deisgned...

There is a large structure that holds the state of this application that has previously lived in both the application, and the worker itself, i.e. sometimes, fields inside that struct need to be updated, and so I would send a request to the worker to perform some calculations, and send back the new "updated" struct, so that the app could have a new updated copy. I'll note also that the whole struct (or most of it) needs to be sent, as certain calculations to update 1 field, would make use of lots of the fields.

This started becoming more and more unideal the larger the struct became, and also means that for some things (i.e. a web component that renders one field of the struct, i.e. vec of numbers) - to update would have to send a message to the worker, and break the flow (i.e. most of the time the component can't send a message and wait for a response from that message, control flow goes back to the worker callback).

I've attempted one new approach:

• create an async_channel at the root and put it inside a context provider
• inside the worker callback (i.e. the callback that determines what to do with any given response FROM a worker), if the message is a particular type then send that message into the channel (in this case, a vec of numbers)
• Inside the component that renders the vec of numbers, when it is first rendered, .await on the message which contains the vec of numbers

And this does almost work - with the exception that I can't figure out how to subscribe to the channel for new updates, and re-render if (and only if) a new vec of numbers is in the channel. It also has the issue of what happens if in future when multiple types of messages are sent (a vec of images per say) and only the component that renders a vec of images should recieve the vec of images, not the numbers component - it will end up dependant on rendering order, whoever .recv().await first!

I.e. - if a button press adds to the vec of numbers, what I would hope to happen is:

• request to worker to update the vec of numbers is sent
• worker updates vec of numbers, sends response back
• worker response callback sends the message into the channel
• the vec component sees a new message in the channel, sees that it's a different vec of numbres, and re-renders

another example:

• vec component renders
• it sends a request to the web_worker for the latest state it has of the vec
• worker sends response back
• worker response callback sends the message into the channel
• it listens to the channel for a vec
• upon arriving it renders the content

But, I'm not quite sure how possible this is.

It's possible that channels are not even the right for this type of sending data back/forth a lot that determines the data for which structs are rendered, etc, and maybe a big struct at the application root is the best way :/

Let me know if a minimal example of the old method would help

EDIT:
Since the original posing of this question, I've initially gone with:

• create a state at app root
• At some point in time when I need some new state from worker, send a message to worker requesting for updated state
• Worker calculates the contents of the updated state
• In the worker response callback, when recieve update that is the correct type, update said state
• the update of said state would obviously trigger renders in the correct spot

[futursolo]

create an async_channel at the root and put it inside a context provider

JavaScript (and Agent v1) uses a push-based design where messages are pushed to the receiver, I think channel is the correct way to convert this to a poll-based API.

• vec component renders
• it sends a request to the web_worker for the latest state it has of the vec
• worker sends response back
• worker response callback sends the message into the channel
• it listens to the channel for a vec
• upon arriving it renders the content

This design is very similar to Agent v2 (#2773), which is blocked by rustwasm/gloo#251.
Maybe you can have a look at rustwasm/gloo#251 and the pull request rustwasm/gloo#267 and see if it helps.

@hamza1311

I believe a M:N worker message handle should implement Stream + Sink (See Warp and Axum's API).

Is it possible to proceed rustwasm/gloo#251 with the current design?