That's because it isn't currently an official rack spec. However, considering the support in falcon, puma, and unicorn on the webserver end and rails, roda, and hanami on the web framework end, it seems reasonable to add to the spec. Can you submit a pull request to update lib/rack/lint.rb appropriately for early hints (SPEC.rdoc is generated from lib/rack/lint.rb, see rake spec task).

Can you submit a pull request

I'll try my hand at it early next week.

I have some issues with how early hints is currently implemented. So, I want to ensure that this is implemented in a generic and useful way. Please make sure there is time for me to review before merging.

When I implemented this in falcon, I found the interface defined by Puma somewhat HTTP/1 specific.

https://www.codeotaku.com/journal/2019-02/falcon-early-hints/index

Can you check what I wrote?

Can you check what I wrote?

I did when I added Early Hints support to Unicorn.

Quick answer because I'm on mobile, I can elaborate more tomorrow if needed.

I think your blog post assume that 103 Early Hints sole goal is to be translated into a push promise by the reverse proxy (h2o most likely).

It is true that today that's pretty much all you can do with it.

However since HTTP/2 server push has been implemented, there has been a lot of experimentations that showed that because the server doesn't know the state of the browser cache, often pushing the assets is slower than letting the browser request it as the browser can't cancel the push if it's useless.

So I believe that when browsers will start to support 103 early hints, these will likely exhibit better performance than push promises.

So maybe what you are looking for is a distinct API specifically for pushes.

That all makes sense.

So with that in mind, what I'm saying is, we should not make an API which is tied to the format of the HTTP/1 headers. It seems like we are in agreement here?

I'm less concerned about functionality. Yes great to support both push and 1xx informal responses. That being said, I'd need to actually implement it in HTTP/2 to provide feedback. Do any clients actually support it?

https://bugs.chromium.org/p/chromium/issues/detail?id=671310

In my experience, things like 1xx break quite a few assumptions about the request -> response logic. What you end up with is request -> 1xx response -> 1xx response -> 200 response and most clients are unprepared for that. I deliberately didn't support it in async-http because I can't actually see any valid use case that isn't better handled by explicit link headers, as in request -> response headers -> response-body.

That doesn't mean we shouldn't support it in rack. I just think we need to be careful about introducing an interface where very few clients actually support it (including Chrome) which makes testing and high level design more tricky, since it can be hard to see how all the bits fit together.

we should not make an API which is tied to the format of the HTTP/1 headers. It seems like we are in agreement here?

I still don’t understand what you mean by HTTP/1 headers. Early Hints responses are exactly the same in HTTP/1 and 2.

Do any clients actually support it?

As you probably saw in the Chrome thread, there a bit of a chicken and egg problem. Chrome devs seem to be waiting for a usage to take of before implementing it, and obviously nobody sees the point of sending these until some major browser supports it. Funnily enough the puma/rack/rails support is the most cited argument in that thread.

We do have people very interested by Early Hints at Shopify that is why I added support in Unicorn, I hope we can help move things forward. The tricky part is they do break some old browsers.

In my experience, things like 1xx break quite a few assumptions about the request -> response logic

Where they ever valid assumptions though? 100 Continue has existed for ages.

I can't actually see any valid use case that isn't better handled by explicit link headers

But the whole point of Early Hints is to be able to send link headers sooner. In many cases you know some assets will be used super early in the request cycle (e.g. request routed to an AdminController, tell the browser the preload the admin assets), however to send the response code and response body you need to process the request entirely (query DB, render templates etc). That is what makes the Early Hints RFC so elegant, efficient and future proof, it allows you to send arbitrary headers before you are ready for the response.

I still don’t understand what you mean by HTTP/1 headers. Early Hints responses are exactly the same in HTTP/1 and 2.

Semantically they might be the same, but the way they are handled internally can be entirely different. As long as we don't end up in a situation like we have with full hijack (with users expecting to write HTTP/1 headers to a socket), I'm okay with it. But as it stands now, early hints -> push promises requires parsing the data provided to the interface.

Where they ever valid assumptions though? 100 Continue has existed for ages.

Sure it may have. But have you tested popular clients to see if they actually work?

But the whole point of Early Hints is to be able to send link headers sooner.

I don't really understand this.

What's stopping someone from doing the following:

# generate link headers
write_response(200)
write_header(link_headers)
flush

# generate application response

write_headers(...)
write_body(...)

in comparison to

# generate link headers
write_response(100, :early_hints)
write_header(link_headers)
flush

# generate application response

write_response(200)
write_headers(...)
write_body(...)

(which as it seems you agree, is probably broken on most browsers/proxies, etc).

By the way, I also agree push promises are pretty pointless in practice, as far as I can tell, they don't deliver a huge amount of value commensurate to the level of complexity they add to the protocol.

That is what makes the Early Hints RFC so elegant, efficient and future proof, it allows you to send arbitrary headers before you are ready for the response.

I cannot see why this impacts HTTP/1 where you can just write the headers at any time.

I can see some value in HTTP/2 where the headers frame must be buffered and send in it's entirety.

Regarding push promises, some servers implement a "session hash" to keep track of what assets clients have already downloaded. It seems like a crazy level of complexity to me personally. The normal solution is for the client to push the assets, and for the client to cancel the stream. But frankly, push promises just seem like a failed experiment. I don't think any browser actually receives them correctly - I know that at least the last time I checked Safari, they didn't even update the local cache.

Interesting summary: https://www.fastly.com/blog/faster-websites-early-priority-hints - seems like very little consensus.

What's stopping someone from doing the following:

Because if you write_response(200) early to be able to stream headers, then you can no longer return a 404, or 401, or 500 if you later realize you should have.

The overwhelming majority of web applications (not just Ruby ones) works likes this:

def call(env)
  query_the_database
  check_authentication_and_permissions_and_such
  body = render_content
  write_response(200) # Everything went fine so 200
  write_headers(...)
  write_body(body)
rescue => error
  write_response(302 / 404 / 401 / 500) # based on what went wrong
end

which as it seems you agree, is probably broken on most browsers/proxies, etc)

Again, chicken and egg problem.

Regarding push promises, some servers implement a "session hash" to keep track of what assets clients have already downloaded.

Yes, that's what they are trying to implement to "rescue" the server push protocol. My opinion is that they're pilling even more complexity on top of an already very complex part of the protocol, and it requires some stateful tracking on the server side which is a huge PITA. It might be a OK thing for behemoths like Google, but for the vast majority of the companies out there, it's a silly idea.

Early Hints are a much more down to earth, simple, elegant and efficient solution to the problem.

So in my opinion the current API is just fine for Early Hints. If you wish to translate Early Hints into push promises in Falcon, well fine, but I don't see it as an argument to change the interface to make that use case easier.

Just for clarity, falcon does not support HTTP/1 early hints, it only supports push promises.

Regarding early hints, it looks like fastly wants this to work with more than just link headers. Can you clarify if we should support more than just link headers? i.e. is this a general way to send response headers before the response itself?

In terms of performance, I'd really like to see some numbers that show this is a net benefit, comparing push promises and normal headers. Once clients have the resource, it doesn't need to be downloaded again, so this only makes an impact on the first request right? After that, the latency could actually be a net negative right? Since the client will now need to receive more than one packet for a response. It's effectively the same downside as push promises.

In addition, do we need to implement this in rack? Surely the load balancer/proxy can figure out (1) whether client can support it and (2) translate cached responses into 103 early hints if it makes sense. Do we need to hoist all of this logic into applications? I can see pros and cons.

Looks like there might be some standardised solution to server push using a proposed cache digest: https://www.fastly.com/blog/optimizing-http2-server-push-fastly

I've been thinking about clients (and servers). How is one supposed to consume early hints?

response = client.request(...)
# what does response contain? Does it contain early hints? A full response? Do we need to read multiple times:
if response.continue?
  response = response.continue
end

To me, even thought this is not part of rack, it's part of the puzzle we need to solve. If we encourage app developers to do this, we need to be sure that we aren't imposing a huge design burden on client authors.

I don't personally have any issue with extending the spec, I just want to make sure we are making the right decisions before we put this in front of every Ruby web application as a thing we both endorse and need to support for the next 10 years. I don't personally feel Rack should become a polyglot wrapper around every single standard available - the key point is we should try to build a cohesive interface for app developers. That bar is pretty high and tricky to establish. I appreciate your efforts.

Can you clarify if we should support more than just link headers?

I believe we should yes.

i.e. is this a general way to send response headers before the response itself?

To be pedantic, this is a general way to hint the user agent, that some headers are very likely to appear in the final response. But they might not appear in the final response, in which case they should be disregarded.
For Link: headers, it doesn't matter, as you can start to preload them at the Early Hints stage, there's pretty much no downside to that.

I haven't heard of thought of another header that could be helpful to hint at early like this, but that's also what makes that RFC so clean. It didn't just add an ad hoc response type for assets, it added a new generic capability that might be useful in the future, and we should implement that rather than the very narrow and specific way it has been used.

If anything our discussion convinced me to go in the opposite direction than the one you suggest.

I now think that env["rack.early_hints"].call(headers) is too narrow and specific on an API. What if I'm implementing a Rack based WebDAV server and I want to send 102 Processing? Or any future 1xx range response. Or maybe I want to implement a proprietary protocol based on partial responses between my app server and my reverse proxy?

env['rack.informational_response'].call(1xx, headers) or env['rack.provisional_response'].call(1xx, headers) would have been more generic and forward thinking.

@nateberkopec I think your input as Puma maintainer could be useful in this discussion.

I agree all your points... we also should try to align up with headers in the early hints as well as headers in the response array (Hash instance).

we also should try to align up with headers in the early hints as well as headers in the response array (Hash instance).

I'm afraid I don't understand what you mean by "align up". Are you suggesting to enforce the same interface than for full response headers? e.g. call check_headers with them?

Yes, that would make sense to me.

I now think that env["rack.early_hints"].call(headers) is too narrow and specific on an API. What if I'm implementing a Rack based WebDAV server and I want to send 102 Processing? Or any future 1xx range response. Or maybe I want to implement a proprietary protocol based on partial responses between my app server and my reverse proxy?

Specifying the response status in addition to the headers seems fine. We can implement the current use case in terms of that. My only concern is premature optimization of the API (IOW, making it too flexible when there's no reason). I think adding a status code is really no big deal though.

+1 on calling check_headers in lint.

@kazuho can we please get your input on this issue?

I've been strongly in the camp of using streaming rather than early hints. This blog post gives some examples of streaming.

https://medium.com/airbnb-engineering/improving-performance-with-http-streaming-ba9e72c66408

It also showcase all the problems with streaming:

• Reverse-Proxy buffering
• Can no longer changes headers
• Can no long change status code
• Implementation complexity

103 Early Hints is comparatively trivial to implement and give you 80% of the benefits.

And yet the conclusion appears at face value to be strongly in favour of streaming, despite the challenges. I actually don't think 103 Early Hints is that trivial to implement in practice as it requires knowing ahead of time what resources are required for what pages. It also potentially introduces more network latency (at least one extra round trip). It would be interesting to compare the two approaches in practice.

And yet the conclusion appears at face value to be strongly in favour of streaming

I don't get what makes you say that. The article never mention early hints, it's unclear whether they were even considered.

I actually don't think 103 Early Hints is that trivial to implement in practice as it requires knowing ahead of time what resources are required for what pages.

Absolutely not. You can emit as many 103 responses as you want, as late as you want. See how it's implemented in Rails.

It also potentially introduces more network latency (at least one extra round trip).

No clue what makes you say that. There's no roundtrip, they are streamed just like the response.

No clue what makes you say that. There's no roundtrip, they are streamed just like the response.

If you start streaming, your network packets are directly relate to the response body. Early hints create extra network overhead.

You can emit as many 103 responses as you want, as late as you want.

Not sure how that's relevant to my point.

I quickly looked at the Airbnb article (interesting). Notice that they used the term 'chunk'. It would seem that everything they're discussing could be implemented with chunked encoding. Not.sure. So, 'streaming' could be used, but isn't a requirement.

'Early hints' is more a mechanism for allowing the browser to handle external dependencies.

Assuming current/recent browsers make use of 'Early hints', some mention of it in Rack might be helpful. Re Puma, I don't recall any recent issues/questions about 'Early hints', not sure if that implies anything...

Early hints create extra network overhead.

It's extremely negligible. Still don't understand where you see a roundtrip....

You can emit as many 103 responses as you want, as late as you want.

Not sure how that's relevant to my point.

You said Early Hints requires knowing ahead of time what resources are required for what pages. Which is incorrect, as whenever you discover the client requires a resource, you can directly emit a 103 provisional response. For instance Rails has an option to emit an early hint whenever you call javascript_include_tags &co: https://github.com/rails/rails/blob/7c70791470fc517deb7c640bead9f1b47efb5539/actionview/lib/action_view/helpers/asset_tag_helper.rb#L118-L120

So they're really trivial to implement, and extremely easy to retrofit on existing architecture, whereas streaming require to build you application around it, as well as to have some client side code to handle unexpected errors.

Also like this comment notes, streaming is how thing were done back in PHP times, and that's why dealing with an unexpected error after headers have been sent is a PITA in PHP and generally end with a truncated HTML document.

It's extremely negligible. Still don't understand where you see a roundtrip....

Early Hints only has any potential benefit on the first request before any cache is established for a site. It's the same problem as push promises, which have effectively been deprecated.

After the first request, there is very little benefit to early hints (and push promises), and they can even have some negative impact: it's pure network overhead if the linked assets are already cached by the browser. In contrast, streaming continues to have value even after the first request as it simply reduces the TTFB which is always beneficial.

I agree, on the first request, there may be some benefit, but the benefit also comes with a cost, which is increased complexity: (1) in the code that generates the website (2) in the server which has to provide an interface for generating provisional responses (3) in the client that has to read and handle the provisional response. This is why I have not implemented support for it in Falcon/Async::HTTP because doing it properly breaks the simple request/response model of HTTP (that most people expect), in effect it's now possible that you receive MULTIPLE responses for a single request (0 or more provisional responses followed by a final non-provisional response). Modelling this is a significant complexity on the interface - the same problem applied to push promises which were fairly horrendous to implement and expose to the client and server in a symmetrical way.

You said Early Hints requires knowing ahead of time what resources are required for what pages. Which is incorrect

You then go on to say "emit an early hint whenever you call javascript_include_tags". This is exactly my point, you need to know ahead of time what resources are required. The fact that there is an API for this in Rails is great, but it's not universally true.

dealing with an unexpected error after headers have been sent is a PITA

Actually I think Rails was a step backwards in regards to buffering the response. It's true that PHP did not have the best error handling, but I'd say that's a problem of the protocol that was solved in HTTP/2 with STREAM_RST. I agree, it's not perfect, but I suppose it's impossible to have real time rendering of the response body AND robust error handling.

In any case, is it still true that 103 Early Hints is only supported by Chrome, and only over HTTP/2? Because streaming can work on all browsers, all HTTP versions, has a wider impact.

If the impact of provisional responses to the request/response model wasn't so significant (e.g. some kind of side channel for communicating provisional responses), I probably wouldn't feel as strongly about it. In the case of rack, we can conveniently ignore the complexity on the client side, but as Async::HTTP implements both client and server (and proxying) I have no such luxury and need to consider both sides and the symmetry of the interface along with concurrency issues that go along with potentially having multiple responses to a single request, etc.

As it stands, the only way to make this work would be to use a server that supports it, fronted by a HTTP/2 load balancer that understands provisional responses -> HTTP/2 -> Chrome.

Early Hints only has any potential benefit on the first request before any cache is established for a site. It's the same problem as push promises, which have effectively been deprecated.

Absolutely not. If the client already has the resource in cache, the early hint is just ignored. With server push, the client had to download the resource regardless.

it's pure network overhead if the linked assets are already cached by the browser.

It's extremely small, and the idea is that they are sent while the response is being generated anyway.

the benefit also comes with a cost, which is increased complexity

As explained before, it's much less complexity. It took only 50 lines of code to implement transparent support for it in Rails. I don't get how you can see this as complex in comparison of dealing with all the problem streaming brings.

You then go on to say "emit an early hint whenever you call javascript_include_tags". This is exactly my point, you need to know ahead of time what resources are required.

That's absolutely not ahead of time. It's discovered while the response is being rendered... You can do javascript_include_tag "https://example.com/#{rand}.js" if you so chose.

I'd say that's a problem of the protocol that was solved in HTTP/2 with STREAM_RST

STREAM_RST absolutely doesn't solve this problem. STREAM_RST allow the client to tell the server to cancel the request. It doesn't allow the server to say: "Oops sorry I gave you a 200, but I actually mean 500 because I lost the connection to the database".

is it still true that 103 Early Hints is only supported by Chrome

Firefox nightly has it, not sure if/when Safari plans to add it, but yeah, support is relatively limited. Even though for better or worse, Chrome is a very significant portion of clients. Big enough that all Shopify storefronts send early hints today with very good results.

If the impact of provisional responses to the request/response model wasn't so significant

The funny thing is that this isn't anything new. From the very beginning of HTTP 1xx responses were meant to be provisional responses, just like 100 continue. Meaning that when you receive a 1xx you just continue reading until you get a final response.

with concurrency issues that go along with potentially having multiple responses to a single request

I fail to see any concurrency issue, the multiple response arrive sequentially. Also It's perfectly valid for an HTTP client to just ignore 1xx responses and keep reading. The only behavior that would be invalid would be to stop reading.

And if you wish to expose provisional responses to the user, it's actually much easier for an async API than for a sync one.

Anyway, I'm not sure why we're having this conversation on the merits of early hints. This ticket is about whether we want a spec for it in Rack, it doesn't matter how we feel about it, what matters is that they are used in the wild.

IMO given that the servers that represent the overwhelming majority of Rack deployments supports it (82% according to the latest hosting survey), it would make sense to spec it. But that's just my 2 cents.

Anyway, I'm not sure why we're having this conversation on the merits of early hints. This ticket is about whether we want a spec for it in Rack, it doesn't matter how we feel about it, what matters is that they are used in the wild.

IMO given that the servers that represent the overwhelming majority of Rack deployments supports it (82% according to the latest hosting survey), it would make sense to spec it. But that's just my 2 cents.

I agree completely. rack.early_hints is already de facto spec because popular web servers and frameworks already support it. As both frameworks and servers already agree on how this should work, there should have to be overwhelming reasons to deviate from the de facto spec when adding support for this officially to the Rack SPEC.