The non-obvious effect of Response.clone() on getting the response body

[broyde]

const fetchWithRedirect = async (...args) => {
  const response = await fetch(...args);
  const clone = response.clone();
  // I was surprised by nodejs process exit at this line (for ~64 kB response):
  const data = await clone.json();

  if (data.redirect) {
    redirect(data.redirect);
    return new Promise(() => {});
  }

  return response;
};

This function works with small response size, but large result in an unexpected nodejs process exit without any error.
I've also posted a nodejs bug about this problem: nodejs/help#3851, but seems it's not a bug.

Is this a node-fetch bug or just an example of bad response.clone() usage?

[jimmywarting]

We use node:streams under the hood so it might not work exactly the same way as in the browser.
i'm only guessing right now, but i think that if you do not consume the original stream in parallel as the cloned response then the first stream will fill it's bucket (b/c it's not being consumed) and in turn make both of the stream unacceptable for reading more data from the original http.incomingResponse

it may violate the browser spec, but i would also argue that you also used response.clone() in a bad way...
you let the original stream get exhausted without draining it and taking up memory that are not being used for something.

This would be more acceptable:

const clone = response.clone()

// consume both streams at the same time
response.pipe(process.stdout) 
const data = await clone.json()

or just

const [text, json] = await Promise.all([response.text(), clone.json()])
// but if that where the case then i would just do
text = await response.text()
json = JSON.parse(text)

[broyde]

But I need process the response outside of fetchWithRedirect only if it has no redirect.

text = await response.text()
json = JSON.parse(text)

Ok, I can then return custom Response implementation like this:

return {
  text: async () => text,
  json: async () => json,
  ok: response.ok,
  // and some other properties
}

as a workaround to avoid response.clone() usage, but it looks ugly.