Add Brotli support

[zachleat]

Hey, I am loving Bridgy thanks for this service. Although recently I’ve noticed that I stopped receiving webmentions from the service a few weeks ago? Can you help me out? I use webmention.io.

This is the post it can’t find webmention support for:
https://www.zachleat.com/web/css-tricks-web-fonts/

I have the site linked up from my Twitter bio https://twitter.com/zachleat

I’ve tried recrawling my site using the Crawl now button and tried using Resend for post with the https://www.zachleat.com/web/css-tricks-web-fonts/ url.

AHA I found #829 which made a lot of sense to me. I did add brotli support recently and it would seem to coincide with this webmention regression.

However my Accept-Encoding headers seem to be working correctly.

What headers are you sending to opt-in to brotli encoding? My server only returns it if br is in Accept-Encoding. Are you using br;q=0 like @aaronpk suggested here? #829 (comment)

[snarfed]

hey, thanks for the kind words! and ugh, sorry for the trouble. this is due to an App Engine bug: it incorrectly adds br to Accept-Encoding in all outbound HTTP requests, regardless of the initial value.

the bug report has been open for almost a year, and acknowledged, but that's it. sigh. i'll nudge it again. i can also look into adding brotli support to bridgy...but in the meantime, your best bet is to disable brotli on your site. apologies again!

[snarfed]

a bit more background: requests supports brotli automatically when the brotli pip package is installed, but sadly that package uses a C native module, which App Engine Python 2.7 standard doesn't allow. i'm looking forward to upgrading Bridgy to the Python 3 standard runtime, which allows native modules, but they don't yet have a migration path from the existing APIs, which we'd need.

there is a pure python implementation of brotli, but its description, super slow pure python brotli decoder, please ignore, gives me pause, not that it might be slow, but that it sounds immature at best.

[zachleat]

If adding support for Brotli is a medium to larger scope item, maybe it would be worth investing in better error messaging for this issue first? No webmention support could be a bit more clear 😇

[snarfed]

sure! i've actually already added an explanation of this bug (and workaround) to the docs, and linked the error message to it.

brotli is still a very small subset of all the possible causes of this error - this is only the second instance i've seen so far - so if i added anything more to the UI itself, i'd lean toward the other more common causes first. hopefuly the link and mention in the docs are a good start though!

[zachleat]

Awesome—that’s great to hear.

[Vacilando]

Same problems here; bridgy not working for our site since Netlify added Brotli to their CDN (https://www.netlify.com/blog/2020/05/20/gain-instant-performance-boosts-as-brotli-comes-to-netlify-edge/) :-(

[snarfed]

Fixed! Bridgy should now support Brotli. I'm not 100% confident in my testing though, so @Vacilando @zachleat et al, feel free to test and let me know if anything still seems unhappy.

Thanks for your patience!