fetch.FulfillRequest when body too big，the chromium crashed.

[GuangxinZhang]

What versions are you running?

$ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.5.3
$ google-chrome --version
Chromium 78.0.3870.0 stable
$ go version
go version go1.14.1 linux/amd64

What did you do? Include clear steps.

I use fetch.FulfillRequest to modify the http response:

err := fetch.FulfillRequest(ev.RequestID, int64(status)).
    WithBody(base64.StdEncoding.EncodeToString([]byte(respBody))).
    WithResponseHeaders(headers).
    WithResponsePhrase("OK").
    Do(ctx)

but when the respBody is too big (961kb), the chromium crashed!

2020/12/04 15:23:38 ERROR: write tcp 127.0.0.1:53546->127.0.0.1:37910: write: broken pipe

What did you expect to see?

What did you see instead?

[ysmood]

From Rod's doc:

Chromedp uses third part WebSocket lib which has 1MB overhead for each cdp client, if you want to control thousands of remote browsers it can become a problem. Because of this limitation, if you evaluate a js script larger than 1MB, Chromedp will crash.

Rod doesn't have this limitation: https://gist.github.com/ysmood/0d5b2c878ecbdb598776af7d3d305b79

Here's how Rod solves it, might help chromedp to improve the way it uses websocket: https://github.com/go-rod/rod/blob/master/lib/cdp/websocket.go

[GuangxinZhang]

From Rod's readme:

Chromedp uses third part WebSocket lib which has 1MB overhead for each cdp client, if you want to control thousands of remote browsers it can become a problem. Because of this limitation, if you evaluate a js script larger than 1MB, Chromedp will crash.

Rod doesn't have this limitation: https://gist.github.com/ysmood/0d5b2c878ecbdb598776af7d3d305b79

Here's how Rod solves it, might help chromedp to improve the way it uses websocket: https://github.com/go-rod/rod/blob/master/lib/cdp/websocket.go

Oh, thank you very much.

[molon]

This is a serious problem, it will prevent us from hijack requests safely

[bpingris]

Is there a way to solve this? Any workaround or something ?

[GuangxinZhang]

I determine the body size, if it is larger than 1MB, use Fetch. continueRequest

[diogogmt]

Hitting the same error when trying to fulfill a response with size of 978KB

write tcp 127.0.0.1:61839->127.0.0.1:61836: write: broken pipe

[ZekeLu]

Please check this PR: #782.

It's not ready for merge yet, but you can adopt the workaround with the replace directive in your go.mod if it's urgent.

[ZekeLu]

The PR has been merged. Closing.