Support BBR congestion algorithm

[aggresss]

Reference to #341 and #1888
This PR implements the BBR congestion algorithm.
For reference only.

BBR Version: https://github.com/google/quiche/blob/e7872fc9e12bb1d46a118949c3d4da36de58aa44/quiche/quic/core/congestion_control/bbr_sender.cc

Source File	Performance
utils/ringbuffer.go	100%
utils/ringbuffer_test.go	100%
utils/windowed_filter.go	100%
utils/windowed_filter_test.go	100%
congestion/packet_number_indexed_queue.go	100%
congestion/packet_number_indexed_queue_test.go	100%
congestion/bandwidth_sampler.go	100%
congestion/bandwidth_sampler_test.go	100%
congestion/bbr_sender.go	100%
congestion/bbr_sender_test.go	30%

[aggresss]

In Wi-Fi 433Mbps network environment, use BBR congestion algorithm can up to 13MiB/s but CUBIC can only up to 6MiB/s

perf % go run cmd/main.go --upload-bytes=1\000\000\000 --download-bytes=1\000\000\000 --server-address=xxx
2023/08/30 14:23:09 uploaded 953.7 MiB: 155.20s (6.1 MiB/s)
2023/08/30 14:23:09 downloaded 953.7 MiB: 68.04s (14.0 MiB/s)
{"latency":223.251678209}


perf % go run cmd/main.go --upload-bytes=1\000\000\000 --download-bytes=1\000\000\000 --server-address=xxx --use-bbr
2023/08/30 14:25:55 uploaded 953.7 MiB: 72.91s (13.1 MiB/s)
2023/08/30 14:25:55 downloaded 953.7 MiB: 69.68s (13.7 MiB/s)
{"latency":142.597904583}

[zengxiaobai]

In Wi-Fi 433Mbps network environment, use BBR congestion algorithm can up to 13MiB/s but CUBIC can only up to 6MiB/s

perf % go run cmd/main.go --upload-bytes=1\000\000\000 --download-bytes=1\000\000\000 --server-address=xxx
2023/08/30 14:23:09 uploaded 953.7 MiB: 155.20s (6.1 MiB/s)
2023/08/30 14:23:09 downloaded 953.7 MiB: 68.04s (14.0 MiB/s)
{"latency":223.251678209}


perf % go run cmd/main.go --upload-bytes=1\000\000\000 --download-bytes=1\000\000\000 --server-address=xxx --use-bbr
2023/08/30 14:25:55 uploaded 953.7 MiB: 72.91s (13.1 MiB/s)
2023/08/30 14:25:55 downloaded 953.7 MiB: 69.68s (13.7 MiB/s)
{"latency":142.597904583}

Is this BBRV2 or BBRV3?

[tobyxdd]

@aggresss Thanks for your amazing work. However I believe there's a serious bug when using this over localhost connections (at least on Windows):

Steps to reproduce the problem:.

1. Create a large random file (say 100MB) in a directory.
2. Go to the example directory and run the quic-go example server with the command:

   go run . -b -www C:\dir\to\random\
   

3. Change to the example\client directory:

   go run . -q -insecure https://127.0.0.1:6121/random_file
   

Observations:.

• On both of my Windows PCs, the connection hangs indefinitely, blocking the transfer of the file.
• CPU usage also spikes to very high load while the connection is hanging.

I've tried debugging, but to no avail. I suspect this problem is due to the low latency or some other properties of localhost that make the code run into some edge cases. (This bug is NOT seen with the default cubic congestion control)

I'm the developer of Hysteria, a QUIC-based proxy project, and I'm very interested in integrating your BBR implementation. If you are interested, we can work together to further debug this.

[tobyxdd]

Alloc & CPU pprof files of the server (during connection hanging & CPU 100%) are attached.

Showing nodes accounting for 7702.53MB, 99.90% of 7710.56MB total
Dropped 20 nodes (cum <= 38.55MB)
      flat  flat%   sum%        cum   cum%
 7702.03MB 99.89% 99.89%  7702.03MB 99.89%  github.com/quic-go/quic-go.init.0.func1
    0.50MB 0.0065% 99.90%  7708.54MB   100%  github.com/quic-go/quic-go.(*connection).run
         0     0% 99.90%  7708.04MB   100%  github.com/quic-go/quic-go.(*connection).maybeSendAckOnlyPacket
         0     0% 99.90%  7708.04MB   100%  github.com/quic-go/quic-go.(*connection).triggerSending
         0     0% 99.90%  7708.04MB   100%  github.com/quic-go/quic-go.(*packetPacker).PackAckOnlyPacket
         0     0% 99.90%  7708.04MB   100%  github.com/quic-go/quic-go.getPacketBuffer (inline)
         0     0% 99.90%  7708.04MB   100%  sync.(*Pool).Get

This triggerSending -> maybeSendAckOnlyPacket code path allocated nearly 8GB of memory in 15 seconds.

Duration: 15.13s, Total samples = 15.46s (102.20%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 7990ms, 51.68% of 15460ms total
Dropped 214 nodes (cum <= 77.30ms)
Showing top 10 nodes out of 142
      flat  flat%   sum%        cum   cum%
    1520ms  9.83%  9.83%     1520ms  9.83%  runtime.stdcall2
    1400ms  9.06% 18.89%     1400ms  9.06%  runtime.stdcall1
    1400ms  9.06% 27.94%     1400ms  9.06%  runtime.stdcall4
     680ms  4.40% 32.34%      680ms  4.40%  runtime.unlock2
     620ms  4.01% 36.35%      620ms  4.01%  runtime.stdcall6
     610ms  3.95% 40.30%     1350ms  8.73%  runtime.selectgo
     580ms  3.75% 44.05%     2030ms 13.13%  runtime.lock2
     450ms  2.91% 46.96%      450ms  2.91%  runtime._ExternalCode
     400ms  2.59% 49.55%      400ms  2.59%  runtime.procyield
     330ms  2.13% 51.68%      330ms  2.13%  runtime.memclrNoHeapPointers

pprof-alloc.pb.gz
pprof-cpu.pb.gz

@marten-seemann EDIT: please ignore my previous message saying cubic has the same bug - it doesn't. During the testing process, I actually ran into a Go compiler optimization bug around io.Copy on arm64 and mistook it for the same bug here 🤦 , but that's a story for another day. I will look into that as well and report back to the Go developers.