Support Binding to Multiple Ports

[bartboy011]

Hello there! I'd like to use Uvicorn for an application I'm working on, however, my use-case requires the ability to bind the application to multiple ports. Looking through the source code, this doesn't seem possible, but I'm curious if a workaround has been discussed before, and/or if this is on the roadmap?

[tomchristie]

Not currently planned. Out of interest, why is binding to multiple ports necessary/useful here?

[tomchristie]

Actually now I think about it, that’s not quite true, you can do it using Gunicorn with the Uvicorn worker.

[bartboy011]

@tomchristie thank you! We'll look into that.

The use case is that the app will be deployed in a kubernetes cluster - the main port uses a TLS connection, but the kubelet doesn't have access to the TLS certs so it can't connect to the primary port to conduct a health check. So, effectively the use-case for a second port binding is to allow for one port to be for web traffic while the other is used for health checks by Kubernetes.

[jeet-parekh]

@tomchristie, adding my use case here (I had to delete my previous comment because I forgot to redact some information).

I am trying to add SSL and HTTPS redirect to a plotly dash app without having to add nginx to the stack. Dash uses flask internally. This is what I could do with hypercorn.

from hypercorn.middleware import AsyncioWSGIMiddleware, HTTPToHTTPSRedirectMiddleware
app_server = HTTPToHTTPSRedirectMiddleware(AsyncioWSGIMiddleware(dash_app.server), "127.0.0.1:443")

$ hypercorn --certfile mycertfile --keyfile mykeyfile --bind 0.0.0.0:443 --insecure-bind 0.0.0.0:80 main:app_server

I attempted to do the same with fastapi+uvicorn.

from fastapi import FastAPI
from fastapi.middleware.httpsredirect import HTTPSRedirectMiddleware
from fastapi.middleware.wsgi import WSGIMiddleware
app_server = FastAPI()
app_server.mount("/", WSGIMiddleware(dash_app.server))
app_server.add_middleware(HTTPSRedirectMiddleware)

$ uvicorn --ssl-keyfile mykeyfile --ssl-certfile mycertfile --port 443 main:app_server

This won't accept connections on port 80. I haven't been able to find a way to implement my use case with uvicorn. If there is a way to do it, please let me know.

[FlorianLudwig]

Adding another use case:

Having a service for which one port is exposed to public internet while another to an internal network.

Something like:

gunicorn -k uvicorn.workers.UvicornWorker -w 1 --bind ip1:port1 --bind ip2:port2 --bind ip3:port3

works.

[NargiT]

Another use case, real traffic goes through default 80 whereas prometheus metrics to 8088.

[bartboy011]

FYI - I've ended up following this guide: https://fastapi.tiangolo.com/deployment/manually/#hypercorn-with-trio and am no longer using Uvicorn. Hypercorn supports this use case perfectly.

[MatthewScholefield]

This would be useful to have with uvicorn directly (in my case to expose prometheus metrics internally within a kubernetes cluster). In my case my app is already customized to uvicorn, but I might look into hypercorn.

[NargiT]

This would be useful to have with uvicorn directly (in my case to expose prometheus metrics internally within a kubernetes cluster). In my case my app is already customized to uvicorn, but I might look into hypercorn.

We did some testing internally and for our case performance were low with hypercorn compare to uvicorn. Make sur to do some benckmark before to switch @MatthewScholefield

[Atheuz]

Ran into this also. Had to switch to hypercorn and use --insecure-bind and --bind. My use case was exposing prometheus metrics internally within a kubernetes cluster, which had to be on HTTP, so I couldn't use the main HTTPS binding, HTTPS was otherwise required. I hope this feature can be added to uvicorn.

[MatthewScholefield]

@Atheuz As I wrote above, that's the same as my use case, but I just wanted to add that in my case I found out it's actually a littler cleaner to just start a metrics server independent of fast api just for prometheus metrocs hosting. Then you don't have to worry about concealing the metrics endpoint from public traffic. You can actually do this out of the box with fastapi-prometheus-instrumentor (if you happen to be using this) by just not calling .expose() and instead doing start_http_server()(from prometheus_client) in a fastapi startup handler (and if you want to make it correct, also close it in the shutdown handler).

[Atheuz]

@MatthewScholefield I'm not concerned about the metrics endpoint being public. I wasn't using fastapi-prometheus-instrumentor (instead using starlette-exporter), and I was having issues with using start_http_server because it didn't capture http request metrics or much of anything other than custom metrics.

I'll keep experimenting to see if I can get uvicorn to work, but hypercorn seems to do what I need for now.

[NargiT]

@MatthewScholefield

@Atheuz As I wrote above, that's the same as my use case, but I just wanted to add that in my case I found out it's actually a littler cleaner to just start a metrics server independent of fast api just for prometheus metrocs hosting. Then you don't have to worry about concealing the metrics endpoint from public traffic. You can actually do this out of the box with fastapi-prometheus-instrumentor (if you happen to be using this) by just not calling .expose() and instead doing start_http_server()(from prometheus_client) in a fastapi startup handler (and if you want to make it correct, also close it in the shutdown handler).

Is this present in the doc ? As most of the people we are using starlette-exporter but ready to switch. I like this idea because it allow to split network between public and technical aspect of the app.

[Atheuz]

@tomchristie I still want this and I'm wondering how big of a task it is to add additional options to the command line, such as:

--insecure-host=0.0.0.0
--insecure-port=8000

for serving HTTP even if HTTPS is enabled on the main command line options (--host and --port with --ssl-keyfile and --ssl-certfile).

That plus the changes required to uvicorn itself to handle serving HTTPS on the primary host/port and HTTP on the secondary host/port.

That and would you accept a PR for this if I working on implementing it?

[tomchristie]

I don't think we'd likely want to add those extra command-line flags.
Too much of a niche benefit vs the complexity that it adds.

Might be worth looking at if you're able to do this with gunicorn w/ uvicorn workers or not. Otherwise, starting two different processes (one of HTTP and one for HTTPS) is probably an okay approach I guess.

[NargiT]

One of the issue using gunicorn is the lack of integration between gunicorn parameters and uviwork workers.
Each projects must create it's own worker from uvicorn.workers.UvicornWorker and this becomes a nightmare in an entreprise.

Moreover gunicorn with uvicorn seems to have some serious issue inside kubernetes: #1226.

[Atheuz]

@tomchristie having two different processes would not be acceptable for all use cases. As an example: Prometheus metrics. If you have two separate processes, then stats would be counted separately for HTTP and HTTPS. If you're just using HTTP to serve scrapes from the Prometheus agent, then the only real contribution would be those scrapes and and not any real traffic, basically rendering it pointless. (Unless you do some screwing around where you make them somehow share metrics, like writing them to a file and giving that to the Prom agent).

Unsure about Gunicorn with a Uvicorn worker.

The solution that @MatthewScholefield mentioned where you have a separate HTTP server in the same process for only metrics would probably be OK though, but this is only OK in the use case for Prometheus. There are probably other use cases where it would be much nicer to just have the server expose on both HTTP and HTTPS.

Another alternative is to use Hypercorn because you can specify --insecure-bind there, but Uvicorn seems faster and more broadly used.

But OK, thank you for responding and giving your input on this, I wasn't sure if this was ever going to progress and at least having some certainty that there probably won't be anything done about this, helps to decide future steps.

[tomchristie]

Moreover gunicorn with uvicorn seems to have some serious issue inside kubernetes: #1226.

It would be great to see someone properly take on an ASGI worker built-in to Gunicorn, rather than Uvicorn's somewhat spotty support.

Another alternative is to use Hypercorn because you can specify --insecure-bind there

Yup - hypercorn is really nicely put together, and I prefer some of the design decisions made there vs. uvicorn.

Going to close this off for now, since I think uvicorn ought to be in feature-pause right now, and just focus on making sure it's doing a decent job with it's current feature set.