Add @stream directive

[gopeter]

Since we already have a @defer directive, a @stream one would be great too. I have a lot of use cases where I query hundreds of rows, and this could be speeded up by using a @stream, since infinite loading isn't always possible (for example: display a spreadsheet).

Here's some more information about @stream:

• https://graphql.org/blog/2020-12-08-improving-latency-with-defer-and-stream-directives/
• https://summit.graphql.com/slides/directives.pdf

Libraries like urql and relay are already supporting @stream. I think you had that in your docs a few years ago also 🤔 But maybe I'm mixing it up too.

[rmosolgo]

Hey, thanks for opening this discussion. For my part, I have considered (even attempted!) to build an implementation on the same code as @defer in GraphQL-Pro. If you'd be interested in trying it out, I'll try to get it released soon. Are you using a client with support for @stream? (Checking Relay's website, I just see "TODO"! https://relay.dev/docs/glossary/#stream)

[gopeter]

Whoa, that's great! Would be a pleasure to try this out, since I have a few apps with valid use cases (and a Pro subscription) 😄 I'm using urql which supports it.

Relay's support is still experimental, that's why is not mentioned in the docs (except @stream_connection).

I just wanted to add that I'm very impressed by your support. I'm using this gem for years and every issue I've opened was closed and resolved within days or even hours. I really appreciate this!

[rmosolgo]

Thanks for your kind words!

within days or even hours

This one might take a tad longer 😅 . I'll follow up here when I have an update!

[gopeter]

Haha, for sure, no worries 😂

[rmosolgo]

👋 I just released GraphQL-Pro 1.21.0 which includes GraphQL::Pro::Stream (doc). It required some improvements in GraphQL-Ruby which I released in graphql 1.13.6. Please give it a try and let me know what you think!

[rmosolgo]

hasNext

Ah, I think you're right! I wonder if that was added to the spec since I did @defer, I haven't seen it before. I'll update the code to include it!

As for initialCount, I don't see any restrictions on it in https://github.com/graphql/graphql-wg/blob/main/rfcs/DeferStream.md, so I bet hasNext will do the trick...

[gopeter]

Great! Would you mind to implement initialCount as well? I want to try this one for my 3500 items list, and it would be good to show the first 50-100 results initially.

[rmosolgo]

I expect initialCount to work, what happened when you tried it? (I see that the spec describes it as optional (Int, not Int!), so I implemented it with a default value of 0.)

I just shipped graphql-pro 1.21.1 which includes hasNext: true|false -- let me know if it works any better with that version.

[gopeter]

Oh sorry, I thought that there is no initialCount because it was not mentioned in the docs :-)
It's working now, but it seems that my list is waiting for the full response before it renders an item. I'll check this again and come back to you. Am I able to delay the response for each row for x seconds?

[gopeter]

Yes, a little console.log debugging shows the difference. urql example of nodes.length:

And in my app:

I've set initialCount to 1.

[rmosolgo]

Ok... I spent today trying to get urql's streaming example working exactly the same with a Ruby server, and I pretty much got there: https://github.com/rmosolgo/graphql-ruby-urql-stream-example There are some caveats, but it works as it should, IMO.

The biggest thing is that urql seems to expect the first flush to contain only ---, which Ruby doesn't do by default. So I wrote some manual streaming transport code here: https://github.com/rmosolgo/graphql-ruby-urql-stream-example/blob/master/server/server.rb#L101 (I'm pretty sure that clients aren't supposed to assume a one-to-one mapping between flushes and multipart batches, but I haven't found an authoritative source for that yet 🤷 .)

As for timing, I think this is because of debouncing on the client somewhere. If you want to add a pause, I'd recommend doing it inside the controller. For example, for Rails ActionController::Live, you might do something like this:

if (deferred = result.context[:defer])
  # Stream an urql-friendly response over ActionController::Live
  # like https://github.com/FormidableLabs/urql/blob/main/examples/with-defer-stream-directives/server/index.js
  response.headers["Content-Type"] = "multipart/mixed; boundary=\"#{delimiter}\""
  # urql expects a leading boundary marker: 
  response.stream.write("---")
  deferred.deferrals.each do |deferral|
    patch = [
      "",
      "Content-Type: application/json; charset=utf-8",
      "",
      deferral.to_h.to_json
    ]
    if deferral.has_next?
      patch << "---"
    end 
    patch_str = patch.join("\r\n")
    response.stream.write(patch_str)
    # Sleep here to let the view render between flushes:
    sleep 0.1
  end
  
  response.stream.write("-----")
end 

Want to give something like that a try?

[gopeter]

But maybe the sleep just does not work there. Running the client without, I always get multiple node lengths, which means that it renders parts of the list until new data has arrived:

I think that this comes from urql's batching. So it seems to work somehow. But looking at my big query (3726 rows with 34 columns), it "waits" until ~3000 articles are loaded:

But I think that this is related to urqls implementation now. graphql-ruby seems to stream everything correctly 👍 Many many thanks for you efforts! I'll walk to the maintainers of urql now, since your findings are very valuable for their work too!

[rmosolgo]

It was a Rails bug 😖 . I added a Rails implementation to my example repo and I found the same behavior you described. It just wasn't streaming the response. Turns out it's a Rack bug: rack/rack#1619

Adding the suggested workaround made my example work:

response.headers['Last-Modified'] = Time.now.httpdate

So I updated the docs in 6c56cbc. Does adding that header make your app work?

[kitten]

Regarding this section:

The biggest thing is that urql seems to expect the first flush to contain only ---, which Ruby doesn't do by default.

We're basically directly based on the reference implementation for Incremental Delivery as part of the GraphQL over HTTP spec ✌️ https://github.com/graphql/graphql-over-http/blob/main/rfcs/IncrementalDelivery.md#content-type-multipartmixed

There the spec basically states:

An initial boundary is sent marking the end of the preamble area.

So, I hope that clarifies why we're not looking for data before the first boundary. We could potentially try to detect whether the first boundary is "missing", however, I'm not sure whether that could be a spec-compliant addition 🤔

[gopeter]

It was a Rails bug 😖 . I added a Rails implementation to my example repo and I found the same behavior you described. It just wasn't streaming the response. Turns out it's a Rack bug: rack/rack#1619

Adding the suggested workaround made my example work:

response.headers['Last-Modified'] = Time.now.httpdate

So I updated the docs in 6c56cbc. Does adding that header make your app work?

Wow, yes, that did the trick! Thanks again!

[rmosolgo]

Thanks for clarifying, @kitten 👍 I went back to the example to replicate the scenario I was having trouble with a few days back and ... I couldn't! I found that I was able to greatly simplify my Ruby servers (Sinatra, Rails) and they both worked beautifully.

I honestly have no idea what was going on the other day 😩. I must have gotten something else confused along the way. I removed my comment about that boundary issue from the readme, anyhow -- evidently it was my bad!