Custom buffer management

[alexcrichton]

In the server I've been working on the load profile of it currently has thousands/millions of connected websockets, but they're all idle. One particular constraint has been memory so far, and we're thinking that a possible optimization would be to omit the InputBuffer for idle connections. In theory with an async server there's no need to actually allocate a buffer for input until the socket is itself readable.

I was curious if y'all had thought about this before? I think this could possible get built externally with just Frame and other low-level bits, but ideally we could reuse the WebSocket logic itself without having to reimplement various portions.

[agalakhov]

I wanted to avoid extra allocations here. In our case of all active connections on the server allocating would be too expensive. Also, having the data preallocated lowers the risk of being killed by the OOM killer. So my choice is to make it configurable. In fact, there are exactly two configuration values: the maximum buffer size and the minimum allocation amount (that's constantly kept allocated). Zero amount means no preallocation then.

[alexcrichton]

@agalakhov that makes sense to me! I wonder if perhaps something like this could be done?

pub trait WebsocketIo: Read  + Write {
    fn is_readable(&self) -> bool;
    fn max_read_buffer_size(&self) -> usize;
    fn min_read_buffer_size(&self) -> usize;
    fn max_write_buffer_size(&self) -> usize;
    fn min_write_buffer_size(&self) -> usize;
}

Most internal functions could be generic over WebsocketIo where top-level functions might look like:

pub fn accept<S: Read + Write>(stream: S, callback: Option<Callback>)
    -> Result<WebSocket<S>, HandshakeError<ServerHandshake<S>>>
{
    accept_io(DefaultIo(stream), callback)
}

pub fn accept_io<S: WebsocketIo>(stream: S, callback: Option<Callback>)
    -> Result<WebSocket<S>, HandshakeError<ServerHandshake<S>>>
{
    // implementation ...
}

where basically all Read + Write objects still get reasonable defaults, but there's a vector to insert a custom trait implementation as well?

[agalakhov]

Good idea, makes sense! Thanks!