fix: Switch to futures-0.3 channels

[ktff]

Closes #6043

Replaces tokio-0.2 channels introduced in #5868 with futures-0.3 channels.

Todo

• Run test-harness benches. Currently I'm failing to do so Your query returned no results error vector-test-harness#67
• Benches that guided this were done on a local computer, let's see what will the benches show here.
• Investigate if are indeed hitting a race in the futures channels when shutting down Race condition in futures::sync::mpsc rust-lang/futures-rs#909, fortunately there is PR that fixes it Port "fix race with dropping mpsc::Receiver" to 0.3 rust-lang/futures-rs#2304. timely_shutdown_lua_timer is failing consistently. (EDIT: nope, it was just a regular bug)

OpenQuestions

• Should we also replace, or at least try to replace guided by benchmarks, other usages of tokio channels with future3 channels? (EDIT: Nope, better to leave that for a future issue)

[lukesteensen]

Should we also replace, or at least try to replace guided by benchmarks, other usages of tokio channels with future3 channels?

I think this can be driven by benchmarks and profiling. I would not go ahead and spend the time to change everything everywhere without good evidence that performance improves meaningfully.

[lukesteensen]

So far this looks reasonable to me!

[ktff]

I would not go ahead and spend the time to change everything everywhere without good evidence that performance improves meaningfully.

I agree, then we'll postpone that to a different issue where it will be easier to bench the difference.

[ktff]

Fixes a bug where if no events are sent before shutting down this, then into_future will poll this and get None which it will pass with the rest of the stream which will be polled again later therefor breaking the stream contract of not calling it after first None.

[jszwedko]

I was able to run the test harness against these changes and the latest nightly.

Specifically for the real world performance test mentioned in #6043 it looks like this is an improvement:

--------------------------------------------------------------------------------
Test Comparison
Test name: real_world_1_performance
Test configuration: default
Subject: vector
Versions: dev-56a30078b6f5c29f61650d9733c0873f35c749a2 nightly/2021-02-01
--------------------------------------------------------------------------------
Metric          | dev-56a30078b6f5c29f61650d9... | nightly/2021-02-01
----------------|--------------------------------|-------------------
Test count      | 1                              | 2                 
Duration (avg)  | 61s                            | 61.5s             
Duration (max)  | 61s                            | 62s               
CPU sys (max)   | 1.5 W                          | 1.8 (+16%)        
CPU usr (max)   | 100 (+0%)                      | 99.5 W            
Load 1m (avg)   | 1.8 (+19%)                     | 1.5 W             
Mem used (max)  | 325.3 MiB (+3%)                | 315.3 MiB W       
Disk read (avg) | 434.6 kib/s (-2%)              | 446.4 kib/s W     
Disk read (sum) | 25.9 MiB W                     | 26.9 MiB (+3%)    
Disk writ (sum) | 277.6 MiB (+554%)              | 42.4 MiB W        
Net recv (avg)  | 6.7 MiB/s W                    | 6.3 MiB/s (-5%)   
Net recv (sum)  | 407.6 MiB W                    | 389.7 MiB (-4%)   
Net send (sum)  | 261.5 MiB                      | 248.1 MiB         
TCP estab (avg) | 603                            | 597               
TCP syn (avg)   | 0                              | 0                 
TCP close (avg) | 0                              | 0                 
--------------------------------------------------------------------------------
W = winner
vector = dev-56a30078b6f5c29f61650d9733c0873f35c749a2
vector = nightly/2021-02-01

However I do see some less favorable results for other configurations, for example:

--------------------------------------------------------------------------------
Test Comparison
Test name: real_world_1_performance
Test configuration: high_concurrency
Subject: vector
Versions: dev-56a30078b6f5c29f61650d9733c0873f35c749a2 nightly/2021-02-01
--------------------------------------------------------------------------------
Metric          | dev-56a30078b6f5c29f61650d9... | nightly/2021-02-01
----------------|--------------------------------|-------------------
Test count      | 5                              | 2                 
Duration (avg)  | 63.2s                          | 62s               
Duration (max)  | 66s                            | 62s               
CPU sys (max)   | 2.4 W                          | 2.6 (+5%)         
CPU usr (max)   | 28.5 W                         | 32.8 (+15%)       
Load 1m (avg)   | 4.2 W                          | 4.7 (+13%)        
Mem used (max)  | 687.8 MiB (+5%)                | 654.2 MiB W       
Disk read (avg) | 879.7 kib/s W                  | 681.2 kib/s (-22%)
Disk read (sum) | 54 MiB (+31%)                  | 41.2 MiB W        
Disk writ (sum) | 114.2 MiB (+29%)               | 88.1 MiB W        
Net recv (avg)  | 13.9 MiB/s (-22%)              | 18 MiB/s W        
Net recv (sum)  | 876.4 MiB (-21%)               | 1.1 gib W         
Net send (sum)  | 566.4 MiB                      | 722.1 MiB         
TCP estab (avg) | 607.4                          | 602.5             
TCP syn (avg)   | 0                              | 8.5               
TCP close (avg) | 0                              | 0                 
--------------------------------------------------------------------------------
W = winner
vector = dev-56a30078b6f5c29f61650d9733c0873f35c749a2
vector = nightly/2021-02-01

My confidence in the test harness is lower than it has been though, for this magnitude of difference. I think #6274 will help here.

Full test harness results: https://gist.github.com/jszwedko/9f4236899a0df1450d698f08d86a20bb

I'll try running the criterion benches.

[jszwedko]

I keep the criterion ones are already in CI 😄

For the topology benches I see:

2021-01-29T18:03:48.8351489Z pipe/pipe_simple                                                                                     38.998 ms  -3.2001%     24.455 MiB/s    +3.3059%           0.00  none
2021-01-29T18:03:48.8352485Z pipe/pipe_small_lines                                                                                31.661 ms  -1.6706%     308.45 KiB/s    +1.6989%           0.00  none
2021-01-29T18:03:48.8353287Z pipe/pipe_big_lines                                                                                  156.54 ms  -3.6279%     121.85 MiB/s    +3.7645%           0.00  none
2021-01-29T18:03:48.8354086Z pipe/pipe_multiple_writers                                                                           41.732 ms  -4.7469%     2.2852 MiB/s    +4.9835%           0.00  none
2021-01-29T18:03:48.8354975Z interconnected/interconnected                                                                        64.958 ms  -8.4663%     29.363 MiB/s    +9.2494%           0.02  none
2021-01-29T18:03:48.8355858Z transforms/transforms                                                                                77.984 ms  -1.4718%     13.452 MiB/s    +1.4938%           0.00  none
2021-01-29T18:03:48.8356683Z complex/complex                                                                                      3.0303 s   -5.6531%     unknown         unknown            0.00  none

Which does show some improvement, though not more than our noise threshold for those tests (20%). The fact that the improvement is consistent across all of them does give me some confidence though. Given that, I'm fine with this change going in. Hopefully #6274 will illuminate more as well as #6166.

[ktff]

@jszwedko

The fact that the improvement is consistent across all of them does give me some confidence though.

Indeed, in criterion it's more or less consistent, even all of 15 topology/tcp_socket benches are showing improvement, but the test harness is somewhat inconclusive, nightly seams better overall, but dev shows a persistent 2-3% improvement in net recv. Let's wait for #6274 to see what it will show.

Looking at this results, my main worry is, that since futures channels relies more on allocations, that it's faster in criterion benches since it doesn't have to clean up after itself/has more free clean memory/etc during the bench, but that's not true during test harness so it's slower or on par there.

[jszwedko]

I merged in master so what the new topology bench looks like.

[ktff]

Now we are talking

2021-02-04T22:08:55.7594172Z Benchmarking real_world_1/topology: Analyzing
2021-02-04T22:08:57.1529661Z real_world_1/topology   time:   [3.4942 s 3.4978 s 3.5015 s]
2021-02-04T22:08:57.1530289Z                         thrpt:  [28.559 Kelem/s 28.590 Kelem/s 28.619 Kelem/s]
2021-02-04T22:08:57.1530983Z                  change:
2021-02-04T22:08:57.1532055Z                         time:   [-9.8741% -9.7432% -9.6084%] (p = 0.00 < 0.05)
2021-02-04T22:08:57.1532589Z                         thrpt:  [+10.630% +10.795% +10.956%]

The new benchmark shows consistent 10% improvement, so I'll run this one more time for good measure since it's a new bench and then we can go ahead with this.

cc @jszwedko

[jszwedko]

Nice! 🎉