Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming API refinements #223

Open
4 tasks
goodboy opened this issue Jul 31, 2021 · 0 comments
Open
4 tasks

Streaming API refinements #223

goodboy opened this issue Jul 31, 2021 · 0 comments

Comments

@goodboy
Copy link
Owner

goodboy commented Jul 31, 2021

As a follow up to the core work done for #53 in #219 (originally #209) and the sidekick debugger improvements in #220 this is the list of outstandings and would like to haves that deserve at least further discussion and possibly implementation for next release.


API and internals related:

  • possibly a better internal design for tracking bidir vs unidir context usage to avoid hacky if not self._ctx._portal checking inside ReceiveMsgStream.aclose()

  • move to a split SendMsgStream / ReceiveMsgStream type set and staple them together using a channel/messaging equivalent of trio.StapledStream?

    • one approach would be to make all streams a MsgStream and just don't allow send on receive only (which should be minority use case i'd imagine eventually).
    • currently this would allow making the stream arg to @stream funcs a MsgStream instead of a Context and we shouldn't have to do anything on the caller side other then prevent .send() from existing or going with the split types approacah.
    • turns out anyio has something similar: StapledObjectStream (obvs we won't use *Object* since we're msgpack type contrained). Actual code is here.
  • from Ems to bidir streaming pikers/piker#190

    would be nice if in tractor we can require either a ctx arg, or a named arg with ctx in it and a type annotation of tractor.Context instead of strictly requiring a ctx arg.


docs and tutorials

we obviously need extensive docs and a full tutorial on both receive only and 2 way streaming.
This is something to work out in detail as part of #175. Content basis will likely be from our test set.


theory / design-y questions:

The main question was moreso about cancellation race conditions that can arise where the local channel is killed after it's sent the stop and whether or not we should wait / shield the mem chan until the msg is processed later (also presumably this is all before the sub-actor is killed).

  • this really has to do with whether or not we want a channel teardown transaction eventually. I personally think right now it's not a requirement (and makes facing 2-general's more immediate). the more sane thing to address first is the naive discovery issues we have as per Multi-root discovery: pragmatic, simple consensus. #216.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant