KTOR-4766 - Add basic support to use UnixSockets with CIO

[hfhbd]

Subsystem
CIO Client and Server

Motivation
Add basic support for unix sockets with a custom CIO client and server. Native (and better) support would require breaking changes.

Solution
Extract resolving the socketAddress into new functions without breaking changes. A user can now implement unix socket support using a custom cio client/server with minimal code.

[e5l]

Hey @hfhbd, sorry for the delay. Could you rebase it on the main branch? I will take a look

[hfhbd]

You do accept breaking changes now, don't you?

[e5l]

Yep, we do :)

[JakeWharton]

This argument can be eliminated. Right now it's only for:

    serverJob.invokeOnCompletion {
        selector.close()
    }

but serverJob is available in the returned HttpServer instance as rootServerJob.

So the caller can do

val selector = ...
val server = httpServer(...)
server.rootServerJob.invokeOnCancelation {
  selector.cancel()
}

[JakeWharton]

I would absolutely love the new httpServer factory function to be made available so that I can use unix domain sockets in the abstract namespace (i.e., ones which do not appear on the filesystem).

Can we merge that separately from the CIO application stuff?

I would love to discuss propagating support for unix domain sockets into the public API, but also a non-internal API for wrapping arbitrary Socket sources.

[e5l]

Hey @JakeWharton, thanks for the feedback!

Sure, it would be great! We already have some basic Unix socket API in the ktor-network module and would be happy to have some feedback and cases we're missing.

Could you also tell me what you mean by separating the HTTP server from the CIO application? At the moment, the CIO is responsible for making HTTP over sockets.

[JakeWharton]

The built-in unix domain socket support relies on UnixDomainSocketAddress API which is specific to the JVM (on Java 16+), not available on Android, and does not support usage of the abstract namespace (i.e., one that does not have a path) or unnamed.

Immediately, I am trying to wrap Android's LocalServerSocket to host a server on a unix domain socket in the abstract namespace. With the proposed changes in this PR, the () -> Socket factory lambda would should be sufficient to let me accomplish this. At least, to get an HttpServer. I'm not actually sure if that's enough to get everything working, as the concept of addresses is pervasive throughout the API.

In general, I would like to see setup of the transport sufficiently decoupled from the actual HTTP server so that I can set up whatever mechanism I want. A litmus test for this being successful would be using a file watcher on a folder that has requests/ and responses/ and responding to new text files in requests/ as HTTP requests by writing the associated HTTP response as a same-named file in responses/. Is this a completely contrived and useless mechanism in practice? Absolutely. But it would demonstrate the right level of decoupling of the layers such that I don't have to worry about the use of transport setups as yet unknown to ktor being a blocker.

Could you also tell me what you mean by separating the HTTP server from the CIO application? At the moment, the CIO is responsible for making HTTP over sockets.

Sorry I was a little imprecise here. I was referring to the CIO value usually supplied to embeddedServer, or the associated setup of a CIOApplicationEngine which seems firmly tied to TCP or UDP and addresses as a transport mechanism. There is no non-internal API that would let me create a custom transport on top of the CIO HTTP engine. In the end of all this I would like to be able to do embeddedServer(CIOLocalServerSocket(name)) { .. } where CIOLocalServerSocket is a type I write to adapt Android's LocalServerSocket and LocalSocket APIs to ktor-network's ServerSocket and Socket APIs.

[JakeWharton]

I'll also add that the client should ideally support similar levels of layering if it doesn't already. What use an HTTP server listening on an UDS in the abstract namespace if I can't also send it requests from the client using that mechanism as its transport?

[e5l]

@JakeWharton, thanks for the clarification!

Got it! It looks possible. Let me play with it and check what API needs to be public and how we can expose it. It seems it should be all done on the ktor-network level (both for client and server).

Btw if you already have an idea of how to do this, please let me know

[JakeWharton]

Prior to this my usage of ktor server has never exceeded the most basic setup of one or two routes. I'm just now starting to investigate more wild usages so still trying to learn the structure of the internals.