Discussion: Enable request interception and caching at the same time

[liweixi100]

Caching is intentionally disabled with page.setRequestInterception(true):
https://github.com/GoogleChrome/puppeteer/blob/d17708ba1fc8411941bfd07f633aaf2fb5cb671e/lib/NetworkManager.js#L154

As a result, all requests sent by Puppeteer carry the headers Cache-Control: no-cache; Pragma: no-cache, which disallows any external proxies to serve cached responses.

But our application leverages a caching proxy to save bandwidth and improve performance. It would be very important for us to have the caching enabled.

The only I way I found that would make request interception and caching work at the same time is to hack NetworkManager.js and remove || this._protocolRequestInterceptionEnabled at L154. (I think that was also the old behavior until the change introduced by #1154).

My questions are:

1. Is there a better way to solve this dilemma?
2. Exactly which parts of the interception feature would be broken if we were to do the hack? If it were only a few edge cases, we may want to choose to live with them.

@aslushnikov

[kblok]

Duplicated of #4620

[liweixi100]

I did tests with or without this hack for over 600 URLs and found something very shocking:
with || this._protocolRequestInterceptionEnabled removed, it ended up with a timeout in 70% of the cases while waiting for the load event.

So to answer partially of question 2 --- the hack simply will not work in the majority of the cases.

Is there any way to make request interception and caching work at same time?

Header overrides obviously did not work as many (including myself) have tried: #3403 #4055

[28141932]

我也需要

[Androbin]

@liweixi100 Would you mind sharing the details of your tests? Ideally, we would have a unit test covering this.

[Androbin]

The issue described by @liweixi100 is described in #7038 and solved by #7060.

[liweixi100]

This is a great enhancement. Thank you @Androbin @starrify