Add an example that documents how to build an "escape hatch" NetworkBehaviour that just hands out streams

[thomaseizinger]

Simply getting access to a stream without having to implement NetworkBehaviour has been discussed a lot. Mostly in #2657 and other linked issues.

This issue documents an example that we would like to add to the repository which users can copy and modify to their needs. Perhaps we can eventually grow something useful out of that :)

The example should:

• Have a binary and a library component
• Binary listens on a random port and prints listen address
• Binary accepts an address to dial and connects to it
• Binary opens a stream, send 32 random bytes, waits for the echo and closes again

The above is essentially the ping protocol but implemented "outside" of rust-libp2p, i.e. not as a module. This should serve as a good-enough starting point for users that want to implement something custom.

The library component of the example should contain a NetworkBehaviour with roughly the following API:

/// A behaviour that manages requests to open new streams which are directly handed to the user.
pub struct Behaviour { }

/// A control acts as a "remote" that allows users to request streams without interacting with the `Swarm` directly.
#[derive(Clone)]
pub struct Control { }

impl Behaviour {
	pub fn new<F: Future<Output = io::Result<()>>>(protocol: StreamProtocol, handler: impl Fn(Stream) -> F) -> (Self, Control);
}

impl Control {
	pub async fn open_stream(&self, peer: PeerId) -> Result<Stream, Error>;
}

pub enum Error {
	NotConnected(PeerId),
	UnsupportedProtocol,
	Io(io::Error)
}

pub enum Event {
	InboundStreamFailed(io::Error),
}

A couple of design notes:

• Only a single protocol is supported, if users need fallback to others, they can modify it.
• Not sharing of state between handler invocations is supported. We use an Fn because it is simple. If state needs to be shared across inbound streams, it is better to just implement a ConnectionHandler as it will come out much cleaner.
• The Control object allows creation of streams without interacting with Swarm. The assumption is that most users that want this kind of functionality don't want to interact with the Swarm through an event loop.
• This behaviour does not support establishing connections. If users want to create or manage connections, they can easily extend the Control object with more messages.

[p0mvn]

I would love to give this a shot. Can I get assigned to this if possible?

[thomaseizinger]

Side-note: Such an oppinonated behaviour could also be useful for low-level testing of protocols because we can plug the real implementation in on one end and use this behaviour on the other end where we get direct access to the stream.

[p0mvn]

I'm sorry, I haven't been able to put in the hours into investigating this, and I don't foresee being able to in the next few weeks. Unasigning myself for now.

If still open, I will pick this up later

[thomaseizinger]

I'm sorry, I haven't been able to put in the hours into investigating this, and I don't foresee being able to in the next few weeks. Unasigning myself for now.

If still open, I will pick this up later

No worries at all, thanks for being proactive :)

[eserilev]

id like to pick this up

[thomaseizinger]

That would be great actually! Coincidentally, I just bumped this to the top of my priority list this morning and was about to assign myself 😄

I think it is something that will really benefit our users.

Since I wrote the above description, I slightly changed my mind on what the API should be after talking to @mxinden. Instead of taking a handler function, I think we should return a struct that is called IncomingStreams:

impl IncomingStreams {
	pub async fn next(&mut self) -> (PeerId, Stream);
	pub fn poll_next(&mut self) -> Poll<(PeerId, Stream)>;
}

impl Behaviour {
	pub fn new(protocol: StreamProtocol) -> (Self, Control, IncomingStreams);
}

The advantage of the above API is that it supports backpressure and if users want, they can always implement their own loop where they take a handler and call it with every stream.

With this in mind, the goal of the example should be:

• To serve as a "copy-paste"-able example for users that "just want access to a stream". There are many ways in how this can be implemented and what is "good" will be very specific to a user's needs. Thus, it is kind of hard to offer this as a proper library component, hence we provide it as an example for a starting point that users can fork and modify.
• To document how to implement your own NetworkBehaviour.

Let me know if this makes sense and whether you need any more guidance :)

[thomaseizinger]

This is now released!

https://github.com/libp2p/rust-libp2p/releases/tag/libp2p-stream-v0.1.0-alpha