Support hpack.NeverIndexedHeaderTuple and hpack.HeaderTuple

[squirrel532]

Hi there,

I want to send header without indexing, after digging into codebase I found that SyncHTTP2Stream.send_headers always extract and build a native tuple. Is there any plan to acept hapck.HeaderTuple directly ?

https://github.com/encode/httpcore/blob/4b662b5c42378a61e54d673b4c949420102379f5/httpcore/_sync/http2.py#L334-L347

hpack.NeverIndexedHeaderTuple

[tomchristie]

Hi there 👋

Could you describe in a bit more detail what it is you're trying to do, and why?
What is "sending a header without indexing", and when and why is that a useful thing to do in HTTP/2?
Are there any good resources you can point to around this?

[squirrel532]

I'm trying to build a libray for Apple Push Notification service (APNs) with the minimum footprint. There's some APNs library on pip already, but most of them have disadvantages. Thus I decided build my own one.

As per APNs document, it requires client not to index authorization header:

APNs requires the use of HPACK (header compression for HTTP/2), which prevents repeatedly storing header keys and values. APNs maintains a small dynamic table for HPACK. To avoid filling up that table, encode your headers in the following way—especially when using many streams:

• Encode the :path and authorization values as literal header fields without indexing.
• Encode the apns-id, apns-expiration, and apns-collapse-id values differently based on whether this is an initial or subsequent request.
  • The first time you send these headers, encode them with incremental indexing to add the header fields to the dynamic table.
  • For subsequent requests, encode these headers as literal header fields without indexing.
• Encode all other fields as literal header fields with incremental indexing.

However, it seems like nothing goes wrong without doing so. Maybe error only happens at large scale.

[tomchristie]

RFC reference... https://tools.ietf.org/html/rfc7541#section-7.1.3

Support in h2... https://github.com/python-hyper/h2/blob/94b89b88edd587abb3ee0e72f1c961a5be7e132d/src/h2/connection.py#L690

We could add optional support for this, yes. We'd need to do this via the ext extensions argument, so eg...

ext={"never_indexed_headers": [b":path", b"authorization"]}

which would take effect here...

https://github.com/encode/httpcore/blob/bd88bf8547a0fae216ae2d18b0d1cb0de81a8707/httpcore/_async/http2.py#L334-L350

[squirrel532]

How about checks instance type when iterate through headers ? Just leave hapck.HeaderTuple and hpack.NeverIndexedHeaderTuple unchange.

I'm not sure how much it would cause to performance loss.

[tomchristie]

So, the reason we wouldn't do that is because it changes the core API.

We're very specific about the contract that we provide, which is the basis of the HTTPX Transport API... https://github.com/encode/httpx/blob/master/docs/advanced.md#writing-custom-transports

An important aspect of that is that anything that falls outside the core request/response API can still be provided for, but is done in the form of an extension. We have a "leaky abstraction", but we make sure that it only leaks in the right places.