if cache pack is too big, we should batch writing

[vankop]

What kind of change does this PR introduce?

fixes #14907

Did you add tests for your changes?

there are existing tests in #15367

Does this PR introduce a breaking change?

no

What needs to be documented once your changes are merged?

nothing

[webpack-bot]

For maintainers only:

• This needs to be documented (issue in webpack/webpack.js.org will be filed when merged)
• This needs to be backported to webpack 4 (issue will be created when merged)

[vankop]

previous version

const batchWrite = err => {
	if (i === len) {
		stream.end();
		return;
	}
	// will be handle in "on" handler
	if (err) return;
	const start = i;
	let batchSize = 0;
	while (i < len && batchSize < _1_5_GB)
		batchSize += content[i++].length;
	return stream.write(
		Buffer.concat(content.slice(start, i), batchSize),
		batchWrite
	);
};

hm.. seems like

const batchWrite = err => {
	if (i === len) {
		stream.end();
		return;
	}
	// will be handle in "on" handler
	if (err) return;
	stream.write(content[i++], batchWrite);
};

is faster... I have checked I/O and data + number of I/O per sec seems roughly equal

( red lined batched version )
maybe this will be better I decided to use this version

[vankop]

if (size < WRITE_LIMIT_CHUNK) {
   for (const b of content) stream.write(b);
   return stream.end();
}

I think we still can use this

[sokra]

We could, but that doesn't really makes a difference...

[vankop]

for small builds we will skip chunking..