[Feature] Service Discovery using Consul

[thibaultmeyer]

Hello,

It could be very nice if Caddy can be dynamically reconfigured by Consul (consul.io).

In a micro-services environment (or not) it allow caddy to get list of available proxy address and path without manual reload.

Nginx have this kind of feature, but only on the "Plus" version. I think it was a killer feature for Caddy.

https://www.nginx.com/blog/service-discovery-with-nginx-plus-and-consul/
https://www.nginx.com/blog/service-discovery-nginx-plus-srv-records-consul-dns/

Thanks

[francislavoie]

In the short-term, check out Traefik. That's what it specializes in. Maybe a plugin would be more appropriate for this feature?

[thibaultmeyer]

Yes i think a plugin could be a good solution. But is plugin can create in-memory caddyfile configuration ? Dont Know if a hook Exist...

[mholt]

I think the rewrite of the proxy will make this possible. (See linked issue.)

[josegonzalez]

Semi-related: https://github.com/brimstone/caddy-consul

[mholt]

@josegonzalez That's cool, what does it do exactly? (ELI5-and-have-never-used-consul)

[josegonzalez]

Seems like a caddy plugin (not that I've used any) that will:

• watch for changes in the consul catalog of services
• build a caddyfile config
• reload caddy

[krishamoud]

Hi everyone, I forked https://github.com/brimstone/caddy-consul for my own use.

https://github.com/krishamoud/caddy-consul is available for anyone who wants to give it a shot. I tried to make it as generic as possible so you can add your own template and the caddyfile will look how you want it to.

[mholt]

Cool. I think this is now possible by reverse proxying to srv:// endpoints. #1915

[josegonzalez]

Note: For those more familiar with consul, a slightly faster way would be to watch for consul service changes on the local client via stale reads (non-stale reads will go direct to the consul cluster leader). Consul can then invalidate records for services you are watching for before you actually invalidate DNS.

[mholt]

@josegonzalez How trivial is that change to implement, do you think? Is it something that could be a good PR?

[tcurdt]

I am a little confused on the status of this. The issues is closed but I cannot seem to find any docs on a consul integration. Is the following traefik config possible with caddy or not?

        image = "traefik:v2.4"
        args = [
          # consul
          "--providers.consulCatalog=true",
          "--providers.consulCatalog.prefix=traefik",
          "--providers.consulCatalog.exposedByDefault=false",
          # ports
          "--entrypoints.web.address=:80",
          "--entrypoints.websecure.address=:443",
          "--entrypoints.websecure.http.tls=true",
          # redirects
          "--entrypoints.web.http.redirections.entrypoint.to=websecure",
          "--entrypoints.web.http.redirections.entrypoint.scheme=https",
          # certificats
          "--entrypoints.websecure.http.tls.certResolver=le",
          "--certificatesresolvers.le.acme.email=ops@foo.com",
          "--certificatesresolvers.le.acme.storage=/acme.json",
          "--certificatesresolvers.le.acme.tlschallenge=true",

        "traefik.enable=true",
        "traefik.http.routers.backend.tls=true",
        "traefik.http.routers.backend.entrypoints=websecure",
        "traefik.http.services.backend.loadbalancer.server.port=${NOMAD_PORT_nodejs}",
        "traefik.http.routers.backend.rule=Host(`${var.domain}`) && PathPrefix(`/api`)",

[francislavoie]

There's no direct consul integration, but you can point Caddy's proxy to its srv:// endpoint to use service discovery to select your upstream.

[josegonzalez]

@mholt forgot to respond but the big thing with my method is you would need a method to store global state (so that you can store the lookup of services on your end). Not sure how I'd go about that with caddy, but definitely could be a good optimization (maybe a scheme of consul://?).

[francislavoie]

@josegonzalez we're working on a rewrite of upstream management which will bring major improvements to SRV upstreams. I think this will help significantly with Consul users as well: #4470 Please take a look at that PR and give it a try it you can, we're collecting feedback!