GREASE incompatible with Cloudflare

[daxpedda]

I noticed in hickory-dns/hickory-dns#1987 that Cloudflare's HTTP/3 implementation seems to be incompatible (or there might be a bug?) with h3.

I made a "minimal" reproducible example here: https://github.com/daxpedda/doh3-test.
But to summarize:
This sends a DoH3 query over POST for www.example.com to Cloudflare's 1.1.1.1 DNS server, which responds with 400 Bad Request and a body with Invalid query..

Couple of observations I already made:

• Normal HTTP/3 connections to Cloudflare in general work, e.g. I tried connecting to Cloudflare Workers over HTTP/3 and it worked with GREASE.
• cURL doesn't use GREASE which might be why it worked.
• Comparing cURL and h3 on Wireshark, they look quite different, so it might not be an issue with GREASE, but some HTTP/3 incompatibility that is triggered by GREASE.
• This is unlikely to be a DNS issue, other DNS servers with DoH3 support, e.g. Google, don't have an issue here.

I didn't dig deeper into this as I'm not familiar with the HTTP/3 protocol and this I "easily" solved for me by just deactivating the GREASE.
I'm happy to dig deeper into it with some guidance.

Related #205.

[seanmonstar]

The goal of grease is to prevent ossification of mechanisms that should allow for protocol extension. Cloudflare's implementation does try to support all kinds (that's where I learned of the kinds of grease from in the first place). It could be:

• We are sending grease wrong.
• We are doing something else wrong when grease is enabled.
• (Unlikely, but I guess possible) We found a case where Cloudflare's quiche doesn't handle grease right.

[Ruben2424]

I tried it out and got the same error as you.

In general there are 3 of grease types. One for settings-id, frame type and stream type.

My guess was that the peer received a wrong body, because we got a http response (which says "400 Invlid query") an not a connection closure with a http3 error.
The grease frame type will be sent direct after the data frame with the body. So my thought was that this specific grease type is probably causing the problem.
When I commented out the sending the grease frame type it worked (most of the time).
https://github.com/hyperium/h3/blob/dccb3cdae9d5a9d720fae5f774b53f0bd8a16019/h3/src/connection.rs#L845C15-L847C55

I also cloned and modified the quiche http3 example server a bit to log the received body. When sending the post from your "minimal" reproducible example" i received the same body with and without grease.

Also important to say, i got the error one time when sending no grease frame (with the setting to true but without this https://github.com/hyperium/h3/blob/dccb3cdae9d5a9d720fae5f774b53f0bd8a16019/h3/src/connection.rs#L845C15-L847C55 )

All of this and the fact that the google server worked lets me guess that there is a race condition which get maybe triggered by the way the cloudflare server responds. And which is more likely to happen when sending the grease frame.

But these are just guesses. If I have the time i will look at it tomorrow again to see if i can find out more.