protocols/stream: Add a streaming-response protocol to rust-libp2p

[b5]

Description

Add support for streaming multiple responses, instead of a 1-1 request-response model in rust-libp2p.

Motivation

• This has been explored in protocols/streaming-response #1947
• we've taken a stab at an implementation for Iroh: n0-computer/iroh@b55e00e
• our use case was to use libp2p in place of a gRPC style RPC/IPC framework
• we had to abandon this effort due to time constraints
• @carsonfarmer is interested in similar functionality
• substrate has a similar generic notifications protocol. It isn't very usable by third parties, but they clearly have a usecase for it. This use case isn't the same as ours, but may be worth considering

Requirements

• would be great to gather other's use cases before fleshing out strict requirements

Open questions

Would this constitute a change to the libp2p spec? Interop story across implementations is an open question. Our use case only requires rust support this feature.

Are you planning to do it yourself in a pull request?

Maybe. We've taken a stab at it, and may be able to help in a joint effort starting sometime after July 20th. I'd be happy to help with planning & coordination in the meantime.

[carsonfarmer]

Thanks for kicking this off @b5. I'm also willing to help with planning, and potentially contribute some code and testing.

[rkuhn]

We (Actyx) have used our own implementation based on OneShot requests in our control plane (e.g. the admin accessing the node for settings and debugging). This turned out to be a performance bottleneck for high rate data transfers (e.g. dump/restore of a node’s whole data), which led me to try my hand at a protocol based on persistent substreams. Results for our use-case are very promising, but it is not yet battle-hardened (other features of higher priority intervened). You can find the current code in this gist.

The API is as simple as I could make it:

• request() takes a request message and a channel for transmitting the responses
• the behaviour emits requests paired with a channel for sending the responses

What do you think?

[mxinden]

• request() takes a request message and a channel for transmitting the responses

  • the behaviour emits requests paired with a channel for sending the responses

What do you think?

That sounds reasonable to me. Something consider: The API on request should likely have a mechanism to enforce backpressure, i.e. slow down the producer of requests when the consumer of requests is overloaded.

[rkuhn]

The shape of this mechanism will probably depend on the underlying implementation: currently, there is only a fire-and-forget NotifyHandler that we can use, which breaks the backpressure chain. Backpressure for request→stream calls is also non-trivial to design, since you cannot just limit the number of still open requests — they may last indefinitely while not consuming relevant resources most of the time.

So the question is: where and when can the client side determine that backpressure needs to be exerted for a given request? As a caller, I’d expect some queueing between myself and the NIC, possibly with synchronous error while buffers are full — like for a bounded queue. Implementing this based on the current libp2p interfaces is extremely difficult, though. One common theme is that messages are shovelled from one queue into another in many places without a notion of finite capacity.

Should this error be asynchronous, then? This would make it harder to use the API correctly, and it would still allow an unbounded number of requests to be dumped into the behaviour with the expectation of getting some kind of reaction later. Which would point towards there being synchronous as well as asynchronous error cases — not nice. With such a scheme the caller will also need a means to register for a notification once capacity becomes available.

---

I agree that it is desirable to solve these issues, but I also have some experience with how difficult it is to do so, cf. Reactive Streams. Such a change is pervasive in a programming language ecosystem since it needs to be supported by several libraries in order to become useful. It also requires code to be written differently, so that e.g. requests can be pulled out of a source at an adequate rate — this doesn’t mesh well with imperative procedural style of the kind “lemme find X in the DHT, then publish messages A, B, C and then send request R”.

Perhaps I’m overthinking this and the answer is “just return BoxFuture<Result<...>> and be happy.”

[thomaseizinger]

I had an idea that could perhaps solve this issue. I am calling it libp2p-bare-stream.

Here is the tl;dr:

1. BareStreamBehaviour is constructed with an arbitrary protocol name (i.e. user-configured)
2. BareStreamBehaviour has a single, public function to open a new bare stream:

impl BareStreamBehaviour {
	pub fn new_outbound_stream(&mut self) -> BareStreamId;
}

3. BareStreamEvent reports successfully opened outbound and inbound streams:

enum BareStreamEvent {
	NewOutboundStream {
		stream: NegotiatedSubstream,
		id: BareStreamId
	},
	NewInboundStream {
		stream: NegotiatedSubstream
	}
}

This would replace streaming-response and perhaps also request-response.

Expanding on this:

The current abstractions within libp2p-swarm are designed around the ideas of actors which communicate via message passing. This works well as long as the logic you want to implement can be contained within an abstraction layer, i.e. by implementing NetworkBehaviour.

request-response already provides an escape hatch from this model where the actor (the NetworkBehaviour) is no longer aware of what is actually going on. Yet, it still constrains the implementation to follow a 1-to-1 messaging model. As we can see from requests like streaming-response, this is a limitation for several usecases.

Implementing a request-response protocol is trivial once you have something that implements AsyncRead and AsyncWrite, in other words: a substream. Crates like asynchronuous-codec make it trivial to communicate over such a stream in a type-safe way.

My proposal would be to introduce the BareStreamBehaviour as the ultimate escape hatch. In order to not have too many ways of doing the same thing, request-response could be removed. I expect that implementing a request-response protocol requires slightly more code if it is based on BareStreamBehaviour compared to request-response but the added flexibility and removed complexity seems worth it.

In summary: For usecases where the protocol at hand can be implemented in a contained way, the recommendation to people would be to implement NetworkBehaviour. For literally anything else, BareStreamBehaviour would just hand them substreams and they can do whatever they'd like.

Thoughts?

[mxinden]

In summary: For usecases where the protocol at hand can be implemented in a contained way, the recommendation to people would be to implement NetworkBehaviour. For literally anything else, BareStreamBehaviour would just hand them substreams and they can do whatever they'd like.

Agreed with this mindset. I am in favor of offering "the ultimate escape hatch".

impl BareStreamBehaviour {
	pub fn new_outbound_stream(&mut self) -> BareStreamId;
}

I don't think this is sophisticated enough.

• We would need to specify the peer to open an outbound stream to.
• In case the local node is not connected, would BareStreamBehaviour drive the connection establishment? Should this be configurable per stream?
• BareStreamBehaviour would require some mechanism to enforce back-pressure, see @rkuhn's post above.

Something along the lines of:

impl BareStreamBehaviour {
	pub fn new_outbound_stream(&mut self, remote_peer: PeerId, cx: &mut Context) -> Poll<BareStreamId>;
}
`

Where new_outbound_stream returns Poll::Pending when the maximum number of negotiating streams to the provided peer is at some defined maximum.

[rkuhn]

Yes, this simplification is much better, great idea @thomaseizinger!

@mxinden Returning Poll is a bit unclear to me, since Poll::Pending would have to mean that the stream is “in the works”, but we have no way of polling this particular outbound stream again to see whether it is now established. Storing a request for this purpose would imply an unbounded buffer on the local swarm, which I’d also like to avoid.

The following would be nice because it pushes the state storage onto the caller, but unfortunately it violates the aliasing rules:

pub async fn new_outbound_stream(&mut self, remote_peer: PeerId) -> BareStreamId

(The issue is that the no other method can be called on the behaviour while awaiting the result.)

So perhaps the best we can do is to keep the knowledge of queue sizes available in the behaviour so that we can return an immediate result:

pub fn new_outbound_stream(&mut self, remote_peer: PeerId) -> Result<BareStreamId, Backpressure>

Then the caller will need to wait for a SubstreamClosed event before it can try again. And if we add smarter limits at some point, we can add a PeerNoLongerBackpressured event.

[rkuhn]

Oh, one more thing: it would be really great if the BareStreamBehaviour accepted a list of protocol names to allow the underlying negotiation facilities to be used for higher-level purposes. The negotiated stream should then come bundled with the negotiated protocol.

Not this sounds like a really cool improvement over TCP sockets to me!

[thomaseizinger]

Oh, one more thing: it would be really great if the BareStreamBehaviour accepted a list of protocol names to allow the underlying negotiation facilities to be used for higher-level purposes. The negotiated stream should then come bundled with the negotiated protocol.

Not this sounds like a really cool improvement over TCP sockets to me!

I've had a similar thought yes but left it out from the initial proposal so we can focus on the core idea :)

Ideally, we make these protocols type-safe as well (an enum or something) instead of strings.

[thomaseizinger]

• In case the local node is not connected, would BareStreamBehaviour drive the connection establishment? Should this be configurable per stream?

I am tempted to say no. The method could fail immediately if there is no active connection to the peer for example.

In my experience, the policy on which peers a node should be connected to is extremely application-specific. We are talking about usecases here which don't fit into the typical p2p setting (hence the bare-stream escape hatch), hence the chances of an atypical connection-policy is high.

I'd put out a recommendation to users to implement their own NetworkBehaviour which encapsulates their connection policy, for example, always maintaining a connection to certain nodes.

[mxinden]

@mxinden Returning Poll is a bit unclear to me, since Poll::Pending would have to mean that the stream is “in the works”, but we have no way of polling this particular outbound stream again to see whether it is now established. Storing a request for this purpose would imply an unbounded buffer on the local swarm, which I’d also like to avoid.

I am not sure I follow. If Poll::Pending is returned, no work has been done, thus the user is free to try again any time. The actual stream is returned later on as a SwarmEvent::Behaviour.

The following would be nice because it pushes the state storage onto the caller, but unfortunately it violates the aliasing rules:

pub async fn new_outbound_stream(&mut self, remote_peer: PeerId) -> BareStreamId

(The issue is that the no other method can be called on the behaviour while awaiting the result.)

Agreed. I don't think an async fn is an option here.

So perhaps the best we can do is to keep the knowledge of queue sizes available in the behaviour so that we can return an immediate result:

pub fn new_outbound_stream(&mut self, remote_peer: PeerId) -> Result<BareStreamId, Backpressure>

Then the caller will need to wait for a SubstreamClosed event before it can try again. And if we add smarter limits at some point, we can add a PeerNoLongerBackpressured event.

Note that I am proposing to enforce backpressure on negotiating (to be established) substreams. Once negotiated the life-cycle management of a substream is up to the user.

I think the user should learn when to poll again through the standard Context mechanism.

Also in regards to the return type, is this not the same to Poll<BareStreamId>?

Not this sounds like a really cool improvement over TCP sockets to me!

The more I think about it, the more I like the framing. TCP with multiplexing, security, hole punching, discovery, ...

• In case the local node is not connected, would BareStreamBehaviour drive the connection establishment? Should this be configurable per stream?

I am tempted to say no. The method could fail immediately if there is no active connection to the peer for example.

Fine not including this at all, or at least not in an initial design.

[thomaseizinger]

So perhaps the best we can do is to keep the knowledge of queue sizes available in the behaviour so that we can return an immediate result:

pub fn new_outbound_stream(&mut self, remote_peer: PeerId) -> Result<BareStreamId, Backpressure>

Then the caller will need to wait for a SubstreamClosed event before it can try again. And if we add smarter limits at some point, we can add a PeerNoLongerBackpressured event.

Note that I am proposing to enforce backpressure on negotiating (to be established) substreams. Once negotiated the life-cycle management of a substream is up to the user.

I think the user should learn when to poll again through the standard Context mechanism.

Also in regards to the return type, is this not the same to Poll<BareStreamId>?

How would this go together in practise?

If I see a function that returns Poll, my first thought would be to use future::poll_fn and await the future. Where would I otherwise be getting a Context from?

If I use future::poll_fn however, I cannot poll the Swarm/Behaviour at the same time, thus nothing will make progress.

[rkuhn]

@mxinden

I am not sure I follow. If Poll::Pending is returned, no work has been done, thus the user is free to try again any time.

This violates the Poll contract: returning Poll::Pending implies that the supplied Context will eventually be woken up when the work is finished or progress can be made. But since no work has even been started, “pending” just means “I didn’t do anything”.

[thomaseizinger]

We could implement the Behaviour to simply wait internally for a "slot" to establish a new substream. i.e. return an ID immediately but the event with the substream would only be emitted with respect to backpressure limits.

We would also need a timeout then I think!

[mxinden]

I am not sure I follow. If Poll::Pending is returned, no work has been done, thus the user is free to try again any time.

This violates the Poll contract: returning Poll::Pending implies that the supplied Context will eventually be woken up when the work is finished or progress can be made. But since no work has even been started, “pending” just means “I didn’t do anything”.

The Waker of the Context would be cached. Once the maximum number of negotiating streams for a given connection to a remote peer decreases by one, the waker is wakened and thus the user informed.

[rkuhn]

I wrote up my thoughts on a peer-to-peer internet, which would be my motivation for wanting the currently discussed feature — and I obviously wouldn’t consider it a “ultimate escape hatch” but instead the true purpose of libp2p :-) But let’s take one step at a time.

Considering unreliable datagrams: I’d still want the PeerId/ProtocolName addressing and negotiation, but instead of AsyncRead/Write the result would need to support sending and receiving datagrams. The latter should not be swarm events (since live video streaming is one of the intended use-cases), so a NegotiatedDatagramChannel might be needed. This is very much future work since such a thing only becomes useful once we have a datagram transport we can put this onto, which poses further questions regarding sharing a cryptographic context with some (signaling) streams.

---

Regarding the waker: this tool feels more and more like a really blunt instrument to me, since wake-ups cannot be traced back to their cause for efficient processing. Future/Poll/Waker is a nice package for a task that makes linear progress, but not a nice tool for running a hierarchical NetworkBehaviour. Hence I would prefer an emitted event for notifying the calling code of the end of a congestion condition.

[thomaseizinger]

Not this sounds like a really cool improvement over TCP sockets to me!

The more I think about it, the more I like the framing. TCP with multiplexing, security, hole punching, discovery, ...

Sorry for OT but this pun ("framing") is too good to be left unaddressed 😁

[chayleaf]

to throw a use case out there, I wanted to implement a simple p2p proxy, but couldn't find an obvious way to do it using libp2p. Note that this goes beyond "single request+streaming response" and instead needs a bidirectional stream interface.

[thomaseizinger]

to throw a use case out there, I wanted to implement a simple p2p proxy, but couldn't find an obvious way to do it using libp2p. Note that this goes beyond "single request+streaming response" and instead needs a bidirectional stream interface.

I think the idea from above would solve this?

[xpepermint]

Any progress on this? I have to download video files from the closest nodes and streams are much needed.

[thomaseizinger]

You already have streams as of today, you just need to implement your own ConnectionHandler and NetworkBehaviour.

I've somewhat changed my mind on this idea since I wrote it. The problem is that once we had out bare streams from NetworkBehaviours, we have no control over where they run (i.e. which task polls them), for how long they live etc. This makes it for example hard to correctly perform back-pressure, i.e. how many streams should we accept from a node that is faster than us before we stop? A BareStreamBehaviour has no idea how many streams are still alive and what they are used for so it cannot help you with that.

I think in the long-run, we will be better off if we make it easier for users to implement NetworkBehaviours and ConnectionHandlers. There are multiple efforts underway in regards to this:

• swarm/behaviour: Inject events via single method #2832
• misc/multistream-select: Treat protocol as String and not [u8] #2831
• Simplify ConnectionHandler trait by removing as many associated types as possible #2863
• swarm/handler: Rename InEvent and OutEvent to FromBehaviour and ToBehaviour #2854
• On a more dynamic approach for writing network behaviours #2938

My suggestion would be: Bite the bullet and work through the ConnectionHandler and NetworkBehaviour abstraction once so you are comfortable in writing them yourself for your usecase. Especially when you are doing IO-intensive stuff like streaming video files, you very likely want to take advantage of the fact that each connection will run on its own task¹ which will unblock the main event loop that the swarm and its behaviours run on. That is important if you want to have low latency.

Feel free to open a new issues about specific problems that you have whilst doing so. There is a fair bit of documentation already but I also think that is can always be improved :)

Footnotes

1. A multi-threaded scheduler will also parallelize those across threads. ↩

[xpepermint]

@thomaseizinger thanks for the reply. I don't know libp2p in detals just yet, but here are some thoughts.

In one form or another, streams should IMO be implemented like general HTTP2 or QUIC streams where major "concerns" are resolved by flow-control. A stream would probably be a transparent "channel" on top of a lower-level protocol.

Since QUIC is coming to Rust I guess relatively soon #2883 (comment), #2801 (comment) I wonder, how will Handler/Behaviour work in terms of multiple streams where each stream should run concurently if not in parallel thread. This feels like the whole stream logic should be handled by the transport and not actually the protocol.

Finally, writting your own ConnectionHandler and NetworkBehaviour seems increadably complex task at the moment and I don't feel the "lightness" in terms of performance. It should be at the level of simple AsyncRead and AsyncWrite traits or smth.

Anyhow, I guess I'll have to dig deeper into it and see what's possible.

[thomaseizinger]

Thank you for your thoughts!

A transport like QUIC is indeed somewhat similar to what rust-libp2p does in that it provides an authenticated & multiplexed connection to another peer.

We do several things on top of it though:

• P2P authentication: Both peers are assured they are connecting to the peer they think they are.
• Protocol selection: Libp2p is structured in terms of various protocols that run concurrently. With plain QUIC, you get streams but you will need to hand-roll some kind of negotiation mechanism, what either party wants to use which stream for.

The split into NetworkBehaviour and ConnectionHandler allows you to run your code on any transport layer, including a MemoryTransport. This is incredibly useful for unit-testing your networking logic.

how will Handler/Behaviour work in terms of multiple streams where each stream should run concurently if not in parallel thread. This feels like the whole stream logic should be handled by the transport and not actually the protocol.

From an application PoV, nothing will change and all NetworkBehaviour code will instantly be compatible with QUIC, thanks to the abstraction layer in between.

Finally, writting your own ConnectionHandler and NetworkBehaviour seems increadably complex task at the moment and I don't feel the "lightness" in terms of performance. It should be at the level of simple AsyncRead and AsyncWrite traits or smth.

We are working on making it easier :)

Note that, the individual streams that are given to a ConnectionHandler already implement AsyncRead and AsyncWrite. The reason for the indirection through NetworkBehaviour and ConnectionHandler is because:

• Per protocol (like identify) you can connect to N peers. NetworkBehaviour manages the state of one protocol across all peers.
• Per peer connection and protocol, you can have M substreams (the AsyncRead and AsyncWrite thingys). A ConnectionHandler manages those streams.

I totally understand that these abstractions can feel overwhelming. I've been there myself and was utterly confused initially :)
My best recommendation is to think it through from first-principles (i.e. how many 1-N relationships are there) and what you would build yourself and it should make a whole lot more sense!

[thomaseizinger]

I've been building something in #2852.

There might still be a few rough edges and the documentation needs more work but I'd welcome feedback on the abstraction introduced there.

It introduces an abstraction for declaring ConncetionHandlers very easily. Those are in my view the biggest chunk of boilerplate when it comes to implementing custom NetworkBehaviours. Please try it out and let me know if the general idea works for you.

I already implemented libp2p-ping in that branch on top of the new abstraction. There is also a test that showcases a custom /hello protocol with some shared state.

[Frando]

While exploring some ideas and getting to know libp2p internals better, I ended up implementing a NetworkBehaviour that simply exposes the raw AsyncRead + AsyncWrite substreams to higher-level code. Similar to the idea explorered in this comment by thomaseizinger.
I just made the repo public if others are interested. It works, but likely has corner cases not yet covered. It's here: libp2p-bistream.

[thomaseizinger]

While exploring some ideas and getting to know libp2p internals better, I ended up implementing a NetworkBehaviour that simply exposes the raw AsyncRead + AsyncWrite substreams to higher-level code. Similar to the idea explorered in this comment by thomaseizinger. I just made the repo public if others are interested. It works, but likely has corner cases not yet covered. It's here: libp2p-bistream.

Thank you for building this! I would like to note that whilst such a design looks appealing (after all, I was an advocate myself), it also comes with downsides. Most importantly, any protocol built on top of it will likely not benefit from any work we are doing in libp2p-swarm to e.g. provide back-pressure between all components.

I see this as a workaround for us not yet providing a convenient enough interface to implement a ConnectionHandler. Ideally, all protocols just use the ConnectionHandler and NetworkBehaviour abstraction. I do however understand that they are still too complicated and cumbersome to implement for users who just want to send some messages from A to B and have no other logic associated with their protocols.

If I can ask, what do the protocols look like that you are building @Frando?

[Frando]

If I can ask, what do the protocols look like that you are building @Frando?

One thing I'm looking into at the moment is a simple tar stream between two peers. Usecase is account transfer for Delta Chat - there's a proof of concept using a full Iroh node plus bitswap, but this is quite heavy for just sending a single (streaming) tar file between two peers, thus I was looking into how a very simple "just a tar stream" solution could look like. For this, I looked into how to access the muxer streams directly, and in the process wrote the PoC crate linked above.

Most importantly, any protocol built on top of it will likely not benefit from any work we are doing in libp2p-swarm to e.g. provide back-pressure between all components.

Interesting. I was under the impression that backpressure would be handled on the AsyncRead + AsyncWrite streams directly. If not, where would backpressure be applied that wouldn't be usable when exposing the Negotiated<SubstreamBox> streams to the surface (as I do in libp2p-bistream)?

[thomaseizinger]

If I can ask, what do the protocols look like that you are building @Frando?

One thing I'm looking into at the moment is a simple tar stream between two peers. Usecase is account transfer for Delta Chat - there's a proof of concept using a full Iroh node plus bitswap, but this is quite heavy for just sending a single (streaming) tar file between two peers, thus I was looking into how a very simple "just a tar stream" solution could look like. For this, I looked into how to access the muxer streams directly, and in the process wrote the PoC crate linked above.

Right. You could use libp2p-request-response for that. Perhaps the OneShotHandler abstraction is also useful for that.

Most importantly, any protocol built on top of it will likely not benefit from any work we are doing in libp2p-swarm to e.g. provide back-pressure between all components.

Interesting. I was under the impression that backpressure would be handled on the AsyncRead + AsyncWrite streams directly. If not, where would backpressure be applied that wouldn't be usable when exposing the Negotiated<SubstreamBox> streams to the surface (as I do in libp2p-bistream)?

It is a bit more subtle than that (see #3078 for details). For example, in the future we don't just want backpressure on a stream level but also on the number of streams that a remote can open. If you are just handing every stream "out" of the behaviour, you'll have to rebuild whatever mechanism we come up with :)

That is just one example. Swarm as a whole is a kind of runtime that will see improvements over time.

Don't get me wrong, I totally understand why the escape-hatch is needed currently. We are just not working on it because I think there is more value in making our abstractions better rather than not using them!

[thomaseizinger]

I think looking at it holistically, we have the following choices when it comes to making people's life easier for writing their own protocols:

1. Re-think the abstractions of NetworkBehaviour and ConnectionHandler
2. Simplify the above interfaces so they are easier to implement
3. Provide utility types to take some boilerplate away when implementing them
4. Provide purpose-driven implementations of these interfaces for specific usecases

---

The introduction of libp2p-request-response falls into category (4). So did the experiment with the FromFn handler.

The work around decoupling the inbound/outbound upgrade falls into (2).

Things like prost-codec fall into (3).

(2) and (3) are useful but are not going to cut it long-term. Having to implement your own ConnectionHandler for each protocol is simply too much work.

On the other hand, libp2p-request-response is not flexible enough as we can see by this very GitHub issue existing.

What if instead of an entire NetworkBehaviour, we just provide more ConnectionHandler implementations? ConnectionHandlers are by design stateful which makes them more difficult to implement (driving streams, computing keep-alive, etc).

• We could make one that is focused on request-response protocols where the user only needs to provide an Encoder and Decoder implementation.
• Another one could be similar, i.e. based on a codec but with a series of responses, similar to HTTP's server-sent events.

Implementing a NetworkBehaviour is very easy compared to a handler so having more ready-to-use handlers should make people's life easier.

@mxinden A request-response handler could also be a good way of creating a convention among protocols how they should interact.

[dvasanth]

to throw a use case out there, I wanted to implement a simple p2p proxy, but couldn't find an obvious way to do it using libp2p. Note that this goes beyond "single request+streaming response" and instead needs a bidirectional stream interface.

I also had similar requirements to build a port forwarding application over libp2p rust. Build a proxy server that tunnels/relay all internet traffic from one peer to another one. Able to tweak the Gossipsub topics to multiplex multiple socket connection data over a topic & demultiplex it at other side. Here is the link to working code:
https://github.com/dvasanth/portforwarding-over-libp2p-rust/
It would be good to build high level methods over Gossipsub to send messages to a particular peer, build stream connection & broadcast to all. Something like below:
send_to_peer(peerid, data) -- message received in proper order
send_to_all(data)
create_stream(peerid),
Methods to sync global state across peers AddObject(index, object) GetObject(index) that can be useful for syncing application session information's.

[thomaseizinger]

Build a proxy server that tunnels/relay all internet traffic from one peer to another one

This is likely a bad idea because you are layering flow-control algorithms on top of each other.

---

In regards to this particular feature, there have been many proposals and none of them quite made it into master simply because there are so many different needs to consider. @mxinden and I discussed this recently and we agreed on a compromise: Have an example that showcases how users can build a simple "escape-hatch" NetworkBehaviour that just hands out streams.

This would fill two needs:

1. Users can copy it if they want such a functionality
2. It somewhat documents, how to implement your own NetworkBehaviour

Contributions welcome!

[thomaseizinger]

I am going to close this in favor of #4457 now. Thank you all for the discussion and input!

[thomaseizinger]

#4457 is about to be fixed by #5027 which adds a generic libp2p-stream protocol. I'll release this immediately after. It is in version 0.1.0-alpha for now, meaning you need to depend on it separately from libp2p. Feedback on the API would be much appreciated!