Skip to content

Commit

Permalink
WebClient: Send Accept-Encoding
Browse files Browse the repository at this point in the history
Guzzle does not send `Accept-Encoding` header by default.
That is equivalent to sending `Accept-Encoding: *`:
https://www.rfc-editor.org/rfc/rfc9110#field.accept-encoding>
Most servers will probably return an uncompressed body
in response to that, which can be considered wasteful,
and can trigger crawler detection systems:
#1481
Others might even opt to use a compression method
that is not supported by the system
(e.g. when libcurl is not compiled with brotli support).

Let’s force Guzzle to let curl send `Accept-Encoding`
header reflecting which compression methods it supports:
guzzle/guzzle#3215
  • Loading branch information
jtojnar committed Apr 13, 2024
1 parent cc629a1 commit 2429274
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 0 deletions.
1 change: 1 addition & 0 deletions NEWS.md
Expand Up @@ -39,6 +39,7 @@
- Content Extraction spout will no longer try to extract content we have already extracted. ([#1413](https://github.com/fossar/selfoss/pull/1413))
- Source filters are stricter, they need to start and end with a `/`. ([#1423](https://github.com/fossar/selfoss/pull/1423))
- OPML importer has been merged into the React client. ([#1442](https://github.com/fossar/selfoss/pull/1442))
- Web requests will send `Accept-Encoding` header. ([#1482](https://github.com/fossar/selfoss/pull/1482))

## 2.19 – 2022-10-12
**This version requires PHP ~~5.6~~ 7.2 (see known regressions section) or newer. It is also the last version to support PHP 7.**
Expand Down
6 changes: 6 additions & 0 deletions src/helpers/WebClient.php
Expand Up @@ -62,6 +62,12 @@ public function getHttpClient(): GuzzleHttp\Client {
],
'handler' => $stack,
'timeout' => 60, // seconds
'curl' => [
// Guzzle will not send Accept-Encoding by default.
// https://github.com/guzzle/guzzle/pull/3215
// Delegate choosing compression method to curl.
\CURLOPT_ENCODING => '',
],
]);

$this->httpClient = $httpClient;
Expand Down

0 comments on commit 2429274

Please sign in to comment.