core: Wrap listeners with type that can pipe

[mholt]

Once a server has a listener, the only way for it to act on connections is to call Accept(). Unfortunately, the implementations of Accept() are opaque and hard-coded; that is, a server looping on a TCP listener can only accept real TCP connections. It's not really possible (AFAIK) to give that listener a virtual "connection" in user space, we have to open a real TCP socket through the kernel and proxy all the bytes.

This change allows servers to accept connections whether they be real or virtual. It exports a Pipe() method that allows other code to give a server a net.Conn. This means the server can still use real TCP connections, but also allows us to add our own connections if we have something we want it to do or act on.

For example, the layer4 module could hand off a connection to the HTTP app without having to proxy. Very efficient!

This is an early-stages experiment and still WIP. Haven't even tested it yet.

(Currently, this functionality does not apply to PacketConn or quic.EarlyListener types.)

[WeidiDeng]

After digging through caddy code and caddy-l4, I think this problem can be solved by http App export a method to allow other modules to register listening addresses to offload connection handling and return a channel that pipes net.Conn from modules to net.Listener that can be used by http.Server.

@mholt If you like, I'll work on a new branch to test this, and make modifications in caddy-l4 to showcase it.

[mholt]

@WeidiDeng You know, I was just thinking about this the other day, although my approach would be a little different.

Do you think it'd be much simpler if we could have the HTTP app be a Layer4 handler, for example? That way it could be embedded directly within layer4?

I don't know which is better or makes more intuitive sense.

[WeidiDeng]

@mholt maybe that be integretated in the next major version of caddy. Because of how golang http works, if l4 doesn't return *tls.Conn, automatic http2 will not work. And because how complex l4 can be, Caddyfile integration is still not finished.

I think for now caddy version 2, just export a method from http app to allow another module to handle connection accept logic (which will also bypass listener wrapper to make things simple). I think listener wrapper could also be moved to l4.

Back when l4 is still sponsor only, I thought about implementing a traffic multiplexer but that requires another listen port for http app and it breaks http2 and strict_sni.

If 4929 is merged, h2 handling and strict_sni will not be affected.