[Tracking Issue] transports/quic: Add QUIC Transport #2883

elenaf9 · 2022-09-09T20:34:38Z

Summary

Implement the QUIC transport for rust-libp2p.

Done criteria

Quic can be used as a transport in the Swarm for communication between nodes that can directly reach each other
TLS handshake is compliant with the libp2p/specs TLS Handshake and unit tested
Transport is unit & integration tested
Benchmarked with mxinden/libp2p-perf
Tested again go-libp2p with Testground
Public API is documented

Status Quo

Alpha implementation based on quinn crate. See #3454.

Previous attempts / History

Implement QUIC #211
- Initial Issue
[WIP] Add libp2p-quic #563
- First implementation using picoquic
- Not using quinn because back then most quinn types were !Send
[WIP] Add experimental QUIC support. #862
- Continuation of [WIP] Add libp2p-quic #563
QUIC support #1334
- Use quinn-proto
- Continuation in branch tomaka/quiccc-again
Add libp2p-quic. #2144
- Inspired by QUIC support #1334 but does not share code
- Related repo: https://github.com/ipfs-rust/libp2p-quic
- published on crates.io as libp2p-quic
- Support noise in addition to tls
- Closed because quic+noise not in libp2p specs
Libp2p quic second attempt #2159
- Continuation of Add libp2p-quic. #2144
- Handles ecn codes and gso (throughput)
- "Support for plugging in your own crypto"
transports/quic: Add implementation based on quinn-proto #2289 [Most recent PR]
- Initially a continuation of Libp2p quic second attempt #2159
- Later code was replaced with old code from tomaka/quiccc-again branch
Quic attempt 4 #2801 Alternative to transports/quic: Add implementation based on quinn-proto #2289
- Uses quinn instead of quinn-proto

Tasks

Long-term (not required for first implementation):

Properly implement Transport::dial_as_listener to support hole-punching feat(quic): implement hole punching #3964
Handle address change if a connection migrates
Handle / use ECN bits (Happening now with the move to quinn refactor(quic): rewrite quic using quinn #3454)
Fix local_address when processing an incoming UDP datagram for a new connection on a socket that is bound to a wildcard address.
Similarly, select correct socket when listening on wildcard address fix(quic): improve listener socket selection when dialing #4259
When first starting to listen, enable server-capabilities on existing client-only endpoint instead of creating a new one
Evaluate alternatives to the quinn_proto quic implementation
- quinn refactor(quic): rewrite quic using quinn #3454
- s2n-quic (AWS)
- quiche (Cloudflare)
Refactor tls-config handling to support PeerId-validation on TLS level: Add muxer compliance test suite and refactor tests to not depend on libp2p-swarm kpp/rust-libp2p#27 (comment) Consider verifying expected PeerId as part of auth upgrades #2946

The text was updated successfully, but these errors were encountered:

mxinden · 2022-09-11T07:12:29Z

Thanks for creating this @elenaf9!

Whenever you make a significant change to the original issue description, would you mind posting a comment like #2052 (comment)? That way, everyone subscribed to this issue gets a notification.

elenaf9 · 2022-09-16T16:13:40Z

Changelog

Handle address change if a connection migrates

Moved to Long-term. For now we can simply disable connection migration.

elenaf9 · 2022-09-22T09:02:39Z

Changelog
QuicMuxer refactoring (elenaf9#6) was replaced and merged into #2289 with kpp#23:

Updated the status-quo description
Marked corresponding task as done.

elenaf9 · 2022-09-27T11:56:25Z

Changelog

Evaluate whether we should switch to use UdpSocket from quinn_udp instead of the rust std implementation

For the first implementation we will stick with UdpSocket from rust std. Integrating quinn_udp::UdpSocket requires additional complexity, and we don't strictly need it, especially if we plan on long-term switching to quinn.

Handle / use ECN bits

Moved to Long-term since the UdpSocket from std does not support this.

Fix local_address when processing an UDP datagram for a new incoming connection on a socket that is bound to a wildcard address

Moved to Long-term. To fix this, we need to obtain the destination-ip of the first udp packet on the connection, which we can not access through the normal UdpSocket API. quinn_udp supports it for linux only. Long-term we can contribute support for other platforms as well (related issue: Provide local and remote socket addresses from a Connecting quinn-rs/quinn#508). For now the local_address will have an unspecified IP in this case.

marten-seemann · 2022-10-14T12:37:35Z

Should Stateless Reset handling / generation be added to the Long Term list?

elenaf9 · 2022-10-23T23:52:45Z

Changelog:

Finish remaining TODOs / FIXMEs in Libp2p quic third attempt transports/quic: Add implementation based on quinn-proto #2289
Enhance docs
Review how backpressure is enforced throughout the whole implementation
-> Described in the docs on [endpoint::Driver] in transports/quic: Add implementation based on quinn-proto #2289

thomaseizinger · 2022-11-10T23:37:09Z

Changelog:

Added link to kpp#27 (comment) so we don't forget to refactor the TLS-config handling.

elenaf9 · 2022-11-14T19:01:06Z

Changelog:

First QUIC implementation based on quinn proto is merged with transports/quic: Add implementation based on quinn-proto #2289 🎉. Updated Status Quo description.
Add new task for support the quic codepoint / draft-29. transports/quic: Add implementation based on quinn-proto #2289 only added support for quic-v1. See Proposal: add a new codepoint for QUIC v1 (RFC 9000), and for future incompatible versions multiformats/multiaddr#145.

geotro · 2022-11-17T05:23:07Z

I hope this is the right place to ask.

Is quic supported on relay-v2, and if not, what are the future plans regarding this?

elenaf9 · 2022-11-17T08:30:55Z

@geotro yes it is supported. You can combine them in the same manner as it is done here with tcp and quic:

rust-libp2p/transports/quic/tests/smoke.rs

Lines 118 to 136 in a714864

    
           let quic_transport = quic::async_std::Transport::new(config); 
        
           let tcp_transport = tcp::async_io::Transport::new(tcp::Config::default()) 
        
               .upgrade(upgrade::Version::V1) 
        
               .authenticate( 
        
                   noise::NoiseConfig::xx( 
        
                       noise::Keypair::<noise::X25519Spec>::new() 
        
                           .into_authentic(&keypair) 
        
                           .unwrap(), 
        
                   ) 
        
                   .into_authenticated(), 
        
               ) 
        
               .multiplex(yamux::YamuxConfig::default()); 
        
           let transport = OrTransport::new(quic_transport, tcp_transport) 
        
               .map(|either_output, _| match either_output { 
        
                   EitherOutput::First((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)), 
        
                   EitherOutput::Second((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)), 
        
               }) 
        
               .boxed();

and either replace the tcp_transport with your relay transport, or if you'd like to support tcp as well you have to combine (OrTransport) tcp and relay and then do the upgrade.

Does this help?

elenaf9 · 2022-11-17T09:27:47Z

Changelog

Add link to quic: Add support for quic codepoint , interpreted as QUIC version draft-29 #3133

geotro · 2022-11-19T05:59:28Z

@elenaf9 it helps a bit, but I was hoping that once QUIC is implemented to be able to do UDP, as well as DCUTR.

Why are you referring to TCP in the context of QUIC?

I understand DCUTR with QUIC is not yet supported so I'll have to wait until that works before I retry it.

elenaf9 · 2022-11-21T11:33:06Z

Why are you referring to TCP in the context of QUIC?

I was referring to TCP to point out how you could support both TCP and QUIC together with the relay-v2 protocol. But TCP is not required, you may as well just combine QUIC and relay-v2.

geotro · 2022-11-21T12:43:09Z

Ah OK, thank you for clarifying this!

My use case requires DCUTR so I'm going to have to wait until this is available together with QUIC before I can begin testing/using it. Very much looking forward to this as I hope it's going to greatly improve the success rates of establishing a direct connection!

Notes: - This is only compatible with v1 so likelihood of connecting to go-libp2p and go-ipfs/kubo peers via quic will unlikely to work until draft-29 is supported in rust-libp2p (see libp2p/rust-libp2p#3133) - DCuTR will not work with quic-v1 at this time so unless port mapping/UPNP is used (coming later), we wont be able to utilize quic-v1 with hole punching, but should work fine with relay (see libp2p/rust-libp2p#2883) Closes #10

Frando · 2022-12-19T15:35:22Z

Is there an issue or note somewhere what is missing for a working setup with QUIC + DCUTR + relay v2? Looking what pieces are missing there and would love any pointers.

mxinden · 2022-12-29T20:10:07Z

Is there an issue or note somewhere what is missing for a working setup with QUIC + DCUTR + relay v2?

As far as I remember, the only thing missing is the implementation of <quic::GenTransport as Transport>::dial_as_listener.

rust-libp2p/transports/quic/src/transport.rs

Lines 182 to 191 in 56b3b8f

    
           fn dial_as_listener( 
        
               &mut self, 
        
               addr: Multiaddr, 
        
           ) -> Result<Self::Dial, TransportError<Self::Error>> { 
        
               // TODO: As the listener of a QUIC hole punch, we need to send a random UDP packet to the 
        
               // `addr`. See DCUtR specification below. 
        
               // 
        
               // https://github.com/libp2p/specs/blob/master/relay/DCUtR.md#the-protocol 
        
               Err(TransportError::MultiaddrNotSupported(addr)) 
        
           }

Note that https://github.com/libp2p/punchr/blob/main/rust-client/ is already using QUIC for hole punching, though only on the Transport::dial and not on the Transport::dial_as_listener side.

@Frando contributions here are welcome.

mxinden · 2023-05-15T04:56:45Z

Posting some in-progress research on the various Rust QUIC implementations out there.

Verdict

Thus far my preference is still quinn via #3454.

We know at least quinn-proto works well for us (see libp2p-quic alpha release).
It provides all the features we need (see table below).
From what I can tell it has the most promising in-progress WebTransport implementation, see WebTransport column below.
Thus far, unable to spot any performance issues via feat(perf): add (provision, build, run) tooling test-plans#163, see https://observablehq.com/@mxinden-workspace/libp2p-perf using quinn.

With #3454, quinn is no longer exposed on the public API surface. Thus we can easily switch out the underlying QUIC implementation in the future in case we ever want to move off of quinn.

Next step: Report our performance measurements to quinn maintainers to push quinn-rs/quinn#1219 forward.

Comparison

	integration effort	comments	WebTransport	Pacing	Managed UDP socket	GSO GRO	random bytes for hole-punching	TLS	license
quinn	low, see PR 3454	Uses unbounded channels	Work-in-progress	Pacing implementation	Yes, through `quinn` crate	Added with GSO GRO	Yes. One can gain access to the socket via `Endpoint::new`	rustls and ring	Apache and MIT
quiche	unknown	Large cooperation focused on performance (Cloudflare)	Work-in-progress out of tree community owned	Pacing implementation	No, see example on manual management	Doesn't manage socket, thus only example	Yes, since UDP socket isn't managed	boring ssl	BSD 2-Clause
s2n	unknown	Large cooperation focused on performance (AWS) \n use unbounded channel to accept new connections. Needs to be investigated whether that is a problem.	Can't find any	yes	Yes, see client and server	yes	Not sure how to access the socket on a `Server`.	s2n-tls	Apache
msquic	unknown	API for Rust, published on crates.io
neqo	unknown	not published on crates.io

//CC @marten-seemann

marten-seemann · 2023-05-18T14:53:11Z

Thanks for compiling this list @mxinden! This looks pretty comprehensive.

Some thoughts:

Not sure if GSO and pacing warrant their own column, they’re just part of a long list of things a QUIC implementation can (optionally) do to speed things up. Others would be DPLPMTUD and ECN. They might serve as a proxy variable to see how advanced a QUIC stack is though.
I expect WebTransport to be supported by more implementations once the draft makes more progress at the IETF. Not a lot of implementations are willing to pay the cost of shooting at a moving target, unless they have a very clear use case for the protocol (as quic-go obviously has, and Chrome’s implementation as well). This is even more so true after the adoption of the RELIABLE_RESET_STREAM draft at the last IETF meeting.
Choosing Quinn (for now) doesn’t seem like a terrible choice, especially since the work is mostly driven by the community. Making the package API agnostic to the underlying QUIC implementation seems like a wise choice, enabling more experimentation in the future.

kpp · 2023-05-22T18:46:35Z

Column random bytes for hole-punching
quinn - Yes. One can gain access to the socket via Endpoint::new

The way it is implemented in #3964 is through channels sending quinn_proto::Transmit. However the go impl uses multiple sockets with SO_REUSEPORT. So in quinn wrapper we can go both ways not sure which one is better.

Implement `Transport::dial_as_listener` for QUIC as specified by the [DCUtR spec](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md). To facilitate hole punching in QUIC, one side needs to send random UDP packets to establish a mapping in the routing table of the NAT device. If successful, our listener will emit a new inbound connection. This connection needs to then be sent to the dialing task. We achieve this by storing a `HashMap` of hole punch attempts indexed by the remote's `SocketAddr`. A matching incoming connection is then sent via a oneshot channel to the dialing task which continues with upgrading the connection. Related #2883. Pull-Request: #3964.

mxinden · 2023-07-31T20:01:28Z

Status Update

#3454 merged and thus our libp2p-quic implementation depends on upstream quinn directly. Thank you @kpp for the work. In case no major blockers are discovered in libp2p-quic v0.9.0-alpha we can release it as v0.9.0 soonish.

thomaseizinger · 2023-07-31T20:14:20Z

Status Update

#3454 merged and thus our libp2p-quic implementation depends on upstream quinn directly. Thank you @kpp for the work. In case no major blockers are discovered in libp2p-quic v0.9.0-alpha we can release it as v0.9.0 soonish.

It appears that the above description of this issue is rather outdated, do you mind updating it?

mxinden · 2023-08-24T17:39:48Z

Status Update

We just released libp2p v0.52.3 which contains a stable release of libp2p-quic v0.9.2. That is the first stable libp2p-quic release in rust-libp2p and thus a big step for the project 🎉

mxinden · 2023-09-04T18:59:40Z

I am closing here as the major milestone, adding QUIC to rust-libp2p, is achieved. Smaller follow-up improvements are tracked in individual issues.

Thank you to the many many many people involved in this effort. ❤️

mxinden mentioned this issue Sep 16, 2022

Add content for QUIC libp2p/docs#191

Merged

elenaf9 mentioned this issue Sep 22, 2022

transports/quic: Refactor Muxer and Connection kpp/rust-libp2p#23

Merged

xpepermint mentioned this issue Sep 27, 2022

protocols/stream: Add a streaming-response protocol to rust-libp2p #2657

Closed

elenaf9 mentioned this issue Sep 28, 2022

transports/quic: Add implementation based on quinn-proto #2289

Merged

7 tasks

mxinden mentioned this issue Oct 5, 2022

Implement QUIC #211

Closed

mxinden added difficulty:hard in progress labels Oct 6, 2022

mxinden changed the title ~~[Tracking Issue] QUIC Transport~~ [Tracking Issue] transports/quic: Add QUIC Transport Oct 6, 2022

thomaseizinger mentioned this issue Nov 18, 2022

examples: Add example combining dns + tcp + quic + relay #3143

Closed

elenaf9 mentioned this issue Nov 21, 2022

rust-client: Add QUIC transport libp2p/punchr#41

Closed

goodboy mentioned this issue Mar 24, 2023

QUIC transport research & planning goodboy/tractor#353

Open

arpankapoor mentioned this issue May 22, 2023

feat(quic): implement hole punching #3964

Merged

4 tasks

mxinden mentioned this issue Jun 19, 2023

Consider dropping multistream-select sim open libp2p/go-libp2p#2330

Closed

thomaseizinger removed the in progress label Jul 19, 2023

mxinden mentioned this issue Aug 31, 2023

quic: don't support draft versions #3395

Open

mxinden closed this as completed Sep 4, 2023

kpp mentioned this issue Sep 5, 2023

Tracking issue for QUIC support paritytech/polkadot-sdk#536

Open

10 tasks

AgeManning mentioned this issue Apr 7, 2024

p2p-interface.md: Add quic ENR entry. ethereum/consensus-specs#3644

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Tracking Issue] transports/quic: Add QUIC Transport #2883

[Tracking Issue] transports/quic: Add QUIC Transport #2883

elenaf9 commented Sep 9, 2022 •

edited by mxinden

mxinden commented Sep 11, 2022

elenaf9 commented Sep 16, 2022

elenaf9 commented Sep 22, 2022

elenaf9 commented Sep 27, 2022 •

edited

marten-seemann commented Oct 14, 2022

elenaf9 commented Oct 23, 2022

thomaseizinger commented Nov 10, 2022

elenaf9 commented Nov 14, 2022

geotro commented Nov 17, 2022

elenaf9 commented Nov 17, 2022

elenaf9 commented Nov 17, 2022

geotro commented Nov 19, 2022 •

edited

elenaf9 commented Nov 21, 2022

geotro commented Nov 21, 2022

Frando commented Dec 19, 2022

mxinden commented Dec 29, 2022 •

edited

mxinden commented May 15, 2023

marten-seemann commented May 18, 2023

kpp commented May 22, 2023

mxinden commented Jul 31, 2023

thomaseizinger commented Jul 31, 2023 •

edited

mxinden commented Aug 24, 2023

mxinden commented Sep 4, 2023

[Tracking Issue] transports/quic: Add QUIC Transport #2883

[Tracking Issue] transports/quic: Add QUIC Transport #2883

Comments

elenaf9 commented Sep 9, 2022 • edited by mxinden

Summary

Done criteria

Status Quo

Tasks

mxinden commented Sep 11, 2022

elenaf9 commented Sep 16, 2022

elenaf9 commented Sep 22, 2022

elenaf9 commented Sep 27, 2022 • edited

marten-seemann commented Oct 14, 2022

elenaf9 commented Oct 23, 2022

thomaseizinger commented Nov 10, 2022

elenaf9 commented Nov 14, 2022

geotro commented Nov 17, 2022

elenaf9 commented Nov 17, 2022

elenaf9 commented Nov 17, 2022

geotro commented Nov 19, 2022 • edited

elenaf9 commented Nov 21, 2022

geotro commented Nov 21, 2022

Frando commented Dec 19, 2022

mxinden commented Dec 29, 2022 • edited

mxinden commented May 15, 2023

Verdict

Comparison

marten-seemann commented May 18, 2023

kpp commented May 22, 2023

mxinden commented Jul 31, 2023

thomaseizinger commented Jul 31, 2023 • edited

mxinden commented Aug 24, 2023

mxinden commented Sep 4, 2023

elenaf9 commented Sep 9, 2022 •

edited by mxinden

elenaf9 commented Sep 27, 2022 •

edited

geotro commented Nov 19, 2022 •

edited

mxinden commented Dec 29, 2022 •

edited

thomaseizinger commented Jul 31, 2023 •

edited