Create QUIC library that can be exposed to JS and uses the Node dgram module
#1
Comments
CMCDragonkai
changed the title
dgram module or libuv UDP socket
|
Ok I have managed to get the ability for quiche to send a packet. Actually I haven't sent the packet to the UDP socket, but I can see how quiche attempts to create an initial packet for me plumb into the UDP socket. This involved several challenges. The first is that This buffer can actually be taken from the outside back in JS land. This might be a better idea, as it minimises the amount of copy pasting going on. Alternatively we can create the array in Rust, then return as a Buffer. However the Therefore the buffer is pre-allocated to be This means later the array needs to be sliced according the to the write size and then returned as a buffer. This means we generally 2 styles of calling the Rust functions. We can call them the way rust expected to be called, by passing in a buffer as a parameter and returning the write length as well. It turns out that The other way is more idiomatic JS, which would mean that the function should return the buffer to be sent that is properly sliced up. I think we should preserve the rust style, and then do any JS-specific idiomatic abstractions on the JS side. At the same time, the With rust style calls, the done is returned as an exception which is kind of strange, but also doable. But this is very much different from how JS should be used, and it means we also have to catch non-done exceptions. Another thing to realise is that when you have situations where you have a buffer of some fixed size, and a then write length indicating how much of that buffer was used. This will always eventually lead to a slicing operation. When doing any slicing, the function's stack will end up with a pointer (reference) to the slice, it won't do any copying. Rust does not have dynamically allocated arrays that sit on the stack, so you have to use vectors instead. Vectors are heap-allocated arrays that can be dynamically changed. String is also heap allocated string relative to So if we wanted to slice out from the array and also return it as a Finally another problem is sending structs back to JS as objects. This requires hte usage of The send info contains information that is necessary to know how to send the packet data to the UDP socket. It however ends up containing Alternatively I created my own This led to a few other issues:
So with this all in mind, I ended up creating another custom struct just so that so I can get the value out of This reveals a structure that always contains the initial packet. This is probably the packet that leads to connection negotiation. If you call it again right now, you get a JS error with Now we can just plumb this into the dgram module to send. |
|
When should the https://docs.rs/quiche/0.16.0/quiche/struct.Connection.html#method.send And it should be looped in fact, until it is done. The conditions are:
|
|
So I can imagine attaching callback to things like We could call this "socketFlushing", it will loop until done. The socket flushing would be triggered every time any of the above things happen... which in turn will flush all the packets to the socket.... will it wait for the socket to have actually sent it by waiting on the callback? I'm not sure if that's necessary... I guess it depends. On the QUIC stream level, it should be considered independent, you don't wait for the socket to be flushed before returning to the user. But one could await a promise will resolve when they really want to have the entire socket flushed. |
|
Ok I can see that there is no support for tuples at all here: https://github.com/napi-rs/napi-rs/tree/main/examples/napi. Also not for tuple-structs which I use in a newtype style pattern. The only thing closest is to create an array like so: On the generated TS it looks like a I reckon it would be dynamically deconstructable as an array. Pretty useful here, to avoid having to define one-off structs. |
|
I've settled on just returning an |
|
The streams in quiche doesn't have its own data type. Instead they are identified by a stream ID. The connection struct provides the methods https://docs.rs/quiche/0.16.0/quiche/struct.Connection.html#method.stream_recv and you have to keep track of the stream IDs somehow. So we would expect that upon creating an RPC call, it would call into the network to create a new stream for a connection. It seems creating a new stream is basically performing this call: https://docs.rs/quiche/0.16.0/quiche/struct.Connection.html#method.stream_send You select a stream ID like 0, and you just keep incrementing that number to ensure you have unique stream IDs. The client-side just increments the number, the server side just receives the streams. Note that once we have a stream, the stream is fully duplex (not half-duplex), in-order and reliable. |
|
The However in the case of the server side, you can't create a new connection until you've taken a specific packet from the UDP socket and parsed it first. You have to check the internal packet information to decide whether to accept a new connection, or to simply plumb it in as if you received the data for a given stream. So the order of operations:
This does mean we maintain |
|
I created a function to randomly generate the connection ID. It's a random 20 bytes. Instead of using Rust's ring library to get the random data. I'm just calling into a passed in JS function to acquire the random data. This unifies the random data usage directly from the Node runtime (and reduces how much rust code we are using here). /// Creates random connection ID
///
/// Relies on the JS runtime to provide the randomness system
#[napi]
pub fn create_connection_id<T: Fn(Buffer) -> Result<()>>(
get_random_values: T
) -> Result<External<quiche::ConnectionId<'static>>> {
let scid = [0; quiche::MAX_CONN_ID_LEN].to_vec();
let scid = Buffer::from(scid);
get_random_values(scid.clone()).or_else(
|err| Err(Error::from_reason(err.to_string()))
)?;
let scid = quiche::ConnectionId::from_vec(scid.to_vec());
eprintln!("New connection with scid {:?}", scid);
return Ok(External::new(scid));
}However I realised that since it's just a 20 byte buffer. Instead of passing an We can just take a Meaning the connection ID is just made in JS side, and then just passed in. On the rust side, they can use This basically means we generally try to keep as much data on the JS side, and manage minimal amounts of memory on the Rust side. |
|
The |
|
A better alternative to just But I can't seem to create good constructor for it that would take a callback. Problem in napi-rs/napi-rs#1374. |
|
On the connecting side, there's an option for the server name. If QUIC uses the standard TLS verification, we could imagine that this will not work for us. I'm not entirely sure. We need to provide our custom TLS verification logic. I haven't found how to do this yet with QUIC but it should be possible. Found it: cloudflare/quiche#326 (comment) It is in fact possible. The standard certificate verification must be disabled, and you have to fetch the certificate itself to verify. |
|
The stream methods are mostly done on the rust side. I've found that the largest type I can make use of is So I've converted it to those. I'm still using I've also found how to write the So basically whenever napi-rs can't automatically derive the marshalling code, we have to implement them like I think the napi macros also expose ways of writing out exactly what the generated types should be. I'm still not entirely sure we want to do use the generated types from |
|
On the JS side, I'm hoping to manage most of the state there. That means things like a map of connection IDs to connection objects, and maps of stream IDs to stream data. That way the Rust part stays as pure as possible. We would only have 1 UDP socket for all of this. So that would simplify the the network management. |
|
It turns out in order to send dgrams, you need to have a connection object created. However if we aren't calling This means, we can use One thing I'm not sure about is how the TLS interacts with the dgram send. For hole punching packets we don't care encrypting the data that much, but if we can encrypt the hole punching packets that would nice. However this would at the very least require some handshake for the TLS to work... and if we are hole punching then we cannot have already done the TLS handshake. |
|
I've started modularising the rust code for consumption on the TS side. Some things I've found out. For enums, there are 2 ways to expose third party enums to the TS side. The first solution is to something like this: use napi::bindgen_prelude::{
FromNapiValue,
sys
};
#[napi]
pub struct CongestionControlAlgorithm(quiche::CongestionControlAlgorithm);
impl FromNapiValue for CongestionControlAlgorithm {
unsafe fn from_napi_value(env: sys::napi_env, value: sys::napi_value) -> Result<Self> {
let value = i64::from_napi_value(env, value)?;
match value {
0 => Ok(CongestionControlAlgorithm(quiche::CongestionControlAlgorithm::Reno)),
1 => Ok(CongestionControlAlgorithm(quiche::CongestionControlAlgorithm::CUBIC)),
2 => Ok(CongestionControlAlgorithm(quiche::CongestionControlAlgorithm::BBR)),
_ => Err(Error::new(
Status::InvalidArg,
"Invalid congestion control algorithm value".to_string(),
)),
}
}
}The basic idea is the new type pattern, but then we have to implement the The problem here is that the generated code results in On the TS side, there's no indication of how you're supposed to construct this object. We would most likely have to create a constructor or factory method implementation that enables the ability to construct However this is kind of verbose for what we want. Another alternative is to use the https://napi.rs/docs/concepts/types-overwrite on the method that uses #[napi(ts_arg_type = "0 | 1 | 2")] algo: CongestionControlAlgorithm,However I found another way. Redefine the macro: /// Equivalent to quiche::CongestionControlAlgorithm
#[napi]
pub enum CongestionControlAlgorithm {
Reno = 0,
CUBIC = 1,
BBR = 2,
}Then this is converted to TS like: /** Equivalent to quiche::CongestionControlAlgorithm */
export const enum CongestionControlAlgorithm {
Reno = 0,
CUBIC = 1,
BBR = 2
}Then we use it, we use the #[napi]
pub fn set_cc_algorithm(&mut self, algo: CongestionControlAlgorithm) -> () {
return self.0.set_cc_algorithm(match algo {
CongestionControlAlgorithm::Reno => quiche::CongestionControlAlgorithm::Reno,
CongestionControlAlgorithm::CUBIC => quiche::CongestionControlAlgorithm::CUBIC,
CongestionControlAlgorithm::BBR => quiche::CongestionControlAlgorithm::BBR,
});
}This application of the Maybe we can provide a |
|
By doing this: We get bidirectional transformation of the enums and we can use |
|
The pacing #[cfg(not(any(target_os = "macos", target_os = "ios", target_os = "windows")))]
fn std_time_to_c(time: &std::time::Instant, out: &mut timespec) {
unsafe {
ptr::copy_nonoverlapping(time as *const _ as *const timespec, out, 1)
}
}
#[cfg(any(target_os = "macos", target_os = "ios", target_os = "windows"))]
fn std_time_to_c(_time: &std::time::Instant, out: &mut timespec) {
// TODO: implement Instant conversion for systems that don't use timespec.
out.tv_sec = 0;
out.tv_nsec = 0;
}A time spec is basically a timestamp with seconds component and a nano seconds component. There isn't any portable rust functions right now that convert a On the other hand it is intended to be an opaque type, so that's why I have it as an One idea is to just do what the However there's no information how to do this for the macos, ios and windows case. So it's kind of useless atm. |
|
I think pacing isn't necessary since it isn't used in the examples, so we can leave it out for now. But the generated types won't be usable as it is referring to I'm not entirely sure how generated types are supposed to work with I'm likely end up writing our own TS file instead of relying on the generated code, but the generated code is useful for debugging. |
|
Ok so I'm starting to experiment with the rust library by calling it from the JS. One thing I'm unclear about is that this library being In this library do we also create the socket and thus expose a socket runtime, such as creating a It seems so, because We know that in both cases, they can use the same socket address, thus using 1 socket address to act as both client and server. If we do this, we might just create a So if we just expose the QUICHE design, then the actual socket management will be in PK. |
|
Ok so then only in the tests would we be making use of the sockets. |
|
Ok as I have been creating the QUIC server, I've found that there are many interactions with the socket during the connection establishment. It makes sense that this library would handle this complexity too. Things like address validation, stateless retries and other things all happen during connection establishment. |
|
Ok I'm at the point where it's time to deal with streams. So therefore this library does have to do:
So there's quite a bit of work still necessary on the JS side. |
|
The ALPNs may not be relevant unless we are working with HTTP3. It also seems we could just refer to our own special strings, since our application level protocol isn't HTTP3. |
|
We could do something like |
|
In the In JS we have to do things by registering handlers against event emitters, we cannot block the main thread. The above can be translated into several possible events.
For the first event, this is easy to do. The UDP dgram socket is already an event emitter, we simply register |
|
The above structure also implies that upon timeout, and readable socket events, that we also start to send packets out. The However in JS again, this doesn't seem idiomatic. Instead we should have an event emitter that emits events upon any Upon doing so, we would want to then perform the Note how it iterates over all connections in an attempt to find if there's any data to send. It would only make sense to send data from connections that have stream sends. Unless... it's possible that that connections have data to be sent even if no stream has sent anything. For example things like keepalive packets. I'm not sure. But in that case, shouldn't that be done on the basis of timeouts? All of this requires some prototyping. |
|
I think the tls utilities cna be rolled into the We primarily need a way to just produce pem files from the JWK files. |
|
I created a benchmark for sending data through a stream. It has a problem however, when running the benchmark we get an error relating to Just to save time as a stop-gap measure I made a quick I also made some changes to the CI yaml file to have |
|
We are using |
|
So definitely it should work. |
|
BTW, we use swc regularly in via |
|
I'm doing my test review fixes in #26. This will be merged into staging. In the meantime @tegefaulkes you should figure out how to get the benches working and finish off the CI/CD. I don't think |
|
When you finish the benches and CI/CD work push changes to staging so I can rebase with #26. |
|
The This seems to be a platform specific problem? I don't see this happening in the other jobs. Testing and fixing this through the CI is going to be slow, the job takes 20 min to complete so the feedback is really slow. I can test on the local mac machine, but I'll need to configure the environment to build the library. It's not quite 1:1 process with the ci mac platform. It will take a little bit of working out. Depending on the priority or what's required at this point, I can just disable the benches and the bench artefact requirements. Maybe the artefacts has an option to allow no files to be uploaded without error? If the goal is just to reach a published pre-release version then the quickest way is to prevent the benches from failing the job. |
|
I've already used the local Mac and built the library successfully. I don't think it's that difficult. Try running the benches on the local Mac and find out why it's not working. |
|
Your tests work fine which means it does connect. If your benches don't work... Try doing it on the local Mac without swc enabled. We are switching the latest Mac soon on May 31st anyway. So whatever happens on our local Mac should be 1:1 to the CI/CD mac. |
|
Fixed, We were binding and trying to connect to the mapped ipv4 address. This was causing a problem on the mac. We also fixed a side bug where the benchmark summary wasn't being awaited resulting in concurrent behaviour. I've pushed the fix, hopefully the CI will finally pass. |
|
IPv4 mapped IPv6 addresses should only be used when you are a dual stack client attempting to connect to an IPv4 server. It should not be used on the host-side to avoid confusion. |
|
I noticed that I think this should be part of our |
|
I think Also I believe these options are by convention called the |
|
It appears the default CA stuff is already configured with boring ssl. I think it uses whatever is already configured in the OS. I haven't confirmed this yet though since I haven't tried using the In the mean time, I'll convert the |
|
It should look like this: We shouldn't be using Anyway this would go into the current |
|
Node by default's TLS system uses its own bundled Mozilla certificates. A special flag With boringssl it might be behaving similar to as if I was using We should make sure we align our behaviour between websockets and quic here to ensure that we are always defaulting to the OS's openssl CA store. Node's tls module defaults to its own bundled CA store: https://stackoverflow.com/questions/14619576/where-is-the-default-ca-certs-used-in-nodejs. This means that there's no actual need to have a file version of this. We can just use the |
|
CI is passing now, I've created a pre-release version so that should be published once the CI completes again. |
|
CI failed with pre-release on mac and linux, same problem for both. Failed after checking if the cargo.toml version and the package.json version matches. Since this is pre-release the package version is @CMCDragonkai mentioned that the pre-release should update the cargo version to match automatically? I'm not sure it's doing that yet. Do we need to update the cargo version manually? Can we set it to the pre-release version number? |
|
The cargo version has to be set manually. There's no automatic version update see the postversion command... On the other hand maybe we can update the cargo version... But I don't know if cargo understands those prerelease version tags. Check if that even exists in Cargo. If they do exist then we could make postversion do an automatic replacement. If not, then we may need to discard prerelease tags entirely for this project. |
|
In the future note that NAPI-rs seems to have a problem with callback related code, and we were dealing with this in our custom TLS verification. Watch these 2 threads as they are not answered:
Additionally there are some issues with using callbacks in Rust... especially relating to the context. @tegefaulkes can you write some notes down here about what you learned about callbacks so we have something to refer back to. |
|
Sure I can write some up. Do you want them here? |
|
Yes for now. |
|
I've explained the kinds of callbacks that can be done in this comment #30 (comment). Really there are 3 methods as method 3 in the comment seems to be a very old way of doing it. Method 1 is the simplest as it handles all of the type conversions between rust and node automatically. Args and results are just used as the rust types and converted to node types. This function is synchronous and as far as I know must only be called within the context of that native function. I may be possible to store the callback and use it later by creating a reference to it to prevent garbage collection but I'm not sure. Method 2 is similar to 1 but with more boilerplate. Args and results are Both methods 1 and 2 can be asynchronous since tasks can be converted to a Both method 1 and 2 can only be called within the main thread. References and Method 4 is similar to Method 2 but the This brings us to our use case. The General summary.
|
CMCDragonkai
added
the
r&d:polykey:core activity 4
|
Upstream has labelled this as a bug now. And we also aren't using callbacks for now since we figured out an alternative for custom TLS. |
Specification
We need QUIC in order to simplify our networking stack in PK.
QUIC is a superior UDP layer that can make use of any UDP socket, and create multiplexed reliable streams. It is also capable of hole punching either just by attempting to send ping frames on the stream, or through the unreliable datagrams.
Our goals is to make use of a QUIC library, something that is compilable to desktop and mobile operating systems, expose its functionality to JS, but have the JS runtime manage the actual sockets.
On NodeJS, it can already manage the underlying UDP sockets, and by relying on NodeJS, it will also ensure that these sockets will mix well with the concurrency/parallelism used by the rest of the NodeJS system due to libuv and thus avoid creating a second IO system running in parallel.
On Mobile runtimes, they may not have a dgram module readily available. In such cases, having an IO runtime to supply the UDP sockets may be required. But it is likely there are already existing libraries that provide this like https://github.com/tradle/react-native-udp.
The underlying QUIC library there is expected to be agnostic to the socket runtime. It will give you data that you need to the UDP socket, and it will take data that comes of the UDP socket.
However it does need 2 major duties:
Again if we want to stay cross platform, we would not want to bind into Node.js's openssl crypto. It would require instead that the library can take a callback of crypto routines to use. However I've found that this is generally not the case with most existing QUIC libraries. But let's see how we go with this.
Additional context
QUIC and NAPI-RS
Sub issues
TLScert arbitraries for testing #8QUICStreams#10Tasks
dgrammodule #1 (comment)Extract out the TLS configuration so that it can be set via in-memory PEM variable and key variable. - 2 day- see Enable TLS Configuration with in-memory PEM strings #2Decide whether application protocols are necessary here, or abstract the quiche- see Check if application protocols are required #13Configso the user can decide this (especially since this is not a HTTP3 library). - 0.5 daynull. Right now when a quiche client connects to the server, even after closing, the server side is keeping the connection alive. - 1 dayQUICConnectionandQUICStreamandQUICServerandQUICSocket. This will allow users to hook into the destruction of the object, and perhaps remove their event listeners. These events must be post-facto events. - 0.5 day[ ] Test the- see Fixed size buffer for QUICStream instead of object stream #5.QUICStreamand change to BYOB style, so that way there can be a byte buffer for it. Testing code should be able generator functions similar to our RPC handlers. - 1 dayQUICClientwith the shared socketQUICSocket. - 3 dayTest the multiplexing/demultipexing of the UDP socket with multiple- See Test connection multiplexing across multiple servers and clients #14QUICClientand a singleQUICServer. - 1 day[ ] Test the error handling of the QUIC stream, and life cycle and destruction routines. - 1 day- Create tests forQUICStreams#10Benchmark this system, by sending lots of data through. - 1 day- See Create benchmarks for sending data #15Propagate the- See Propagate connection info along side stream creation #16rinfofrom the UDP datagram into theconn.recv()so that the streams (either during construction or otherwise) can have itsrinfoupdated. Perhaps we can just "set" therinfoproperties of the connection every time we do aconn.recv(). Or... we just mutate theconnparameters every time we receive a UDP packet.Ensure that when a user asks- See Propagate connection info along side stream creation #16stream.connectionthey can acquire the remote information and also all the remote peer's certificate chain.[ ] Integrate dgram sending and recving for hole punching logic.- see Allow Bidirectional Hole Punching from QUICClient and QUICServer #4[ ] Integrate the- see Packaging up js-quic for Linux, Windows, MacOS #7napiprogram into thescripts/prebuild.jsso we can actually build the package in a similar to other native packages we are using likejs-dbThe text was updated successfully, but these errors were encountered: