Improve performance when sending large messages

[raubitsj]

Description

When sending and receiving large messages over the service socket we were doing suboptimal handing of the data.

Part of the solution is to avoid copying each buffer everytime a new chunk is received. The offending code was:

self._data += data

The new code path instead keeps track of the receive buffers in a list and only assembles the list when they are needed. The new logic is contained in a SockBuffer class.

Testing

Create a bunch of files:

#! /bin/bash
set -e
mkdir -p data/
cd data/
for n in `seq 600000`; do
    dd if=/dev/urandom of=file$( printf %08d "$n" ).bin bs=1 count=$(( (RANDOM % 255) + 1 )) 2>/dev/null
    if [ $(( $n % 1000 )) == 0 ]; then
        echo "$n"
    fi
done

Run cvp's artifact reference script pointing to this data directory.
I could not reproduce problems or a great speedup with this change on my machine. 600k files takes ~40 seconds to load. Is that good?

[codecov]

Codecov Report

Merging #4119 (a1cc023) into master (7eb62d4) will increase coverage by 0.03%.
The diff coverage is 97.19%.

@@            Coverage Diff             @@
##           master    #4119      +/-   ##
==========================================
+ Coverage   82.68%   82.72%   +0.03%     
==========================================
  Files         256      256              
  Lines       32508    32561      +53     
==========================================
+ Hits        26880    26936      +56     
+ Misses       5628     5625       -3     

Flag	Coverage Δ
functest	55.06% <90.65%> (+0.09%)	⬆️
unittest	73.48% <96.26%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
wandb/docker/__init__.py	81.39% <0.00%> (ø)
wandb/wandb_torch.py	61.29% <94.11%> (+1.35%)	⬆️
wandb/sdk/lib/sock_client.py	92.26% <100.00%> (+2.33%)	⬆️
wandb/sdk/service/service.py	90.36% <100.00%> (+0.48%)	⬆️
wandb/sdk/wandb_watch.py	81.48% <100.00%> (+6.93%)	⬆️
wandb/integration/tensorboard/monkeypatch.py	90.12% <0.00%> (-2.47%)	⬇️
wandb/filesync/step_prepare.py	93.67% <0.00%> (-1.27%)	⬇️
wandb/sdk/wandb_run.py	90.94% <0.00%> (-0.25%)	⬇️
wandb/sdk/lib/git.py	79.01% <0.00%> (ø)
wandb/sdk/internal/meta.py	90.79% <0.00%> (+3.06%)	⬆️

[vanpelt]

So hot, thanks for cranking on this! I'm gonna do some testing locally now but the code looks great!

[dmitryduev]

🔥