Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples to add to the docs #324

Open
mholt opened this issue Jun 4, 2023 · 23 comments
Open

Examples to add to the docs #324

mholt opened this issue Jun 4, 2023 · 23 comments
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers help wanted Extra attention is needed

Comments

@mholt
Copy link
Member

mholt commented Jun 4, 2023

We are planning to add a large repository of examples to the docs.

However, the breadth of use cases is nearly infinite and there are only a few of us maintainers, so we need your help by posting your configs for your use cases here.

Please post your examples here. Doesn't matter how niche; let us decide that. If you think it's useful, or something you wish you had when you were starting, share it!

We are accepting both JSON and Caddyfile configurations. Note that we may revise examples when posting them to our site. Feel free to reply here with your examples to contribute.

Instructions

  • Describe the use case (could be a backend app name, a specific config task, an integration, something to do with auto HTTPS/TLS, a clever solution, etc.)
  • Include at least the minimal config needed. You may include variations of your example, or additional configuration that may be "recommended," but please annotate them as optional and explain what they do in comments.
  • Comments are useful, but some things are obvious and don't need comments.
  • Link to any source if you got the example from a web page (e.g. if a third-party app shows how to integrate with Caddy on their docs, please link to that page).

I would love to see 100+ examples posted here from all of you.

Thank you!

Here's an example 😉

WordPress

example.com

root * /var/www/wordpress
php_fastcgi unix//run/php/php-version-fpm.sock
file_server

# block access to sensitive paths (optional, recommended)
@blocked path /xmlrpc.php *.sql /wp-content/uploads/*.php
rewrite @blocked /index.php

# enable compression (optional)
encode gzip
@mholt mholt added documentation Improvements or additions to documentation good first issue Good for newcomers help wanted Extra attention is needed labels Jun 4, 2023
@mholt mholt pinned this issue Jun 5, 2023
@gedw99
Copy link

gedw99 commented Jun 25, 2023

@mholt happy to help. working / maintained examples are really needed.

How about making a list of types of examples that you feel are appropriate, so then people can co-ordinate around that ?

@mholt
Copy link
Member Author

mholt commented Jun 25, 2023

Basically, just post your Caddy configs you're already using and we'll take it from there. It would be nice if you comment parts that aren't obvious, or which are optional, that could be helpful.

@karimfromjordan
Copy link

karimfromjordan commented Jun 29, 2023

I have a request for a config example. My general reverse proxy config looks like this:

example.com {
  reverse_proxy :5001
  encode gzip
}

I used to have a directive in there to serve a simple maintenance.html page whenever the app on port 5001 was offline — because I was deploying an update for example. But the way to do this in Caddy changed and I can't figure out how to do this today.

@francislavoie
Copy link
Member

@karimfromjordan you're looking for https://caddyserver.com/docs/caddyfile/directives/handle_errors. The reverse_proxy will trigger an error if the upstream can't be connected to, and you can handle it with handle_errors to do something else instead.

@robgordon89
Copy link

We make heavy use of snippets and ENV's for our TLS on-demand setup

{
    order rate_limit before basicauth

    on_demand_tls {
        ask      {$CADDY_ASK_URL}
        interval {$CADDY_ASK_INTERVAL}
        burst    {$CADDY_ASK_BURST}
    }
    servers {
        metrics
        trusted_proxies static 0.0.0.0/0
        client_ip_headers CF-Connecting-IP X-Forwarded-For
    }
}

(common) {
    rate_limit {
        distributed
        zone ip_rate {
            key    {client_ip}
            events {$CADDY_RATE_EVENTS:10}
            window {$CADDY_RATE_WINDOW:1s}
        }
    }

    @online {
        expression "{env.CADDY_MAINTENANCE_ENABLED} == \"false\""
    }

    @offline {
        expression "{env.CADDY_MAINTENANCE_ENABLED} == \"true\""
    }

    @whitelist {
        path {$CADDY_PATH_WHITELIST:*}
    }
}

(tls_ondemand) {
    tls {
        on_demand
    }
}

(tls_cloudflare) {
    tls {
        dns cloudflare {$CLOUDFLARE_API_TOKEN}
    }
}

(online_handle) {
    handle @online {
        reverse_proxy @whitelist {$CADDY_BACKEND_HOST}:{$CADDY_BACKEND_PORT}
    }
}

(offline_handle) {
    handle @offline {
        reverse_proxy {$CADDY_MAINTENANCE_HOST:https://maintenance.com} {
            header_up Host {$CADDY_MAINTENANCE_HEADER_HOST:maintenance.com}
        }
    }
}

(fallback_handle) {
    handle {
        respond "" 404
    }
}

:8080 {
    respond /healthz "200 ok"
}

import extra/*

:443 {
    import common
    import tls_ondemand

    import online_handle
    import offline_handle

    import fallback_handle
}

test.com {
    import common
    import tls_cloudflare

    import online_handle
    import offline_handle

    import fallback_handle
}

@MattsBos
Copy link

This is what I have running at the moment. I have 2 caddy servers to achieve high availability. Their only function is to get SSL certificates and do reverse proxying.
Both Caddy servers run in docker and share the same data folder. They both start with a basic config.json telling Caddy to load the actual config using HTTP.

Initial config present on both Caddy servers:

{
    "admin": {
        "config": {
            "load_delay": "30s",
            "load": {
                "module": "http",
                "url": "http://10.10.2.157:30050/config.json"
	    }
        }
    }
}

Config that is centrally hosted and gets loaded through HTTP:

{
    "admin": {
        "config": {
            "load_delay": "30s",
            "load": {
                "module": "http",
                "url": "http://10.10.2.157:30050/config.json"
	    }
        }
    },
    "apps": {
        "http": {
            "servers": {
                "reverse-proxy": {
                    "listen": [":443"],
                    "routes": [
                        {
                            "match": [
                                {
                                    "host": ["subdomain1.example.com"]
                                }
                            ],
                            "handle": [
                                {
                                    "handler": "reverse_proxy",
                                    "upstreams": [
                                        {
                                            "dial": "tcp/10.10.2.157:32001"
                                        }
                                    ]
                                }
                            ]
                        },
                        {
                            "match": [
                                {
                                    "host": ["subdomain2.example.com"]
                                }
                            ],
                            "handle": [
                                {
                                    "handler": "reverse_proxy",
                                    "upstreams": [
                                        {
                                            "dial": "tcp/10.10.2.157:32100"
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            }
        }
    }
}

@mustafa0x
Copy link

mustafa0x commented Jul 10, 2023

Related: https://github.com/search?q=%28path%3A**%2FCaddyfile+OR+path%3A**%2F*.caddy%29&type=code

Would be nice to scrape all of these (~7k apparently, probably a fraction of that after dedepulicating), then put a simple but blazing fast search on top of them (could even be client side only). The use case: often you have an idea or problem you need to solve, and the fastest way to validate that and see how it's possible is to search a corpus of config others wrote. Also, often the best way to understand a directive or config parameter is to see the different ways it's used in the real world.

(tangent: I tried to add the created:>2020 stars:>100 qualifiers to the search but GH didn't show anything then.)

@gedw99
Copy link

gedw99 commented Jul 10, 2023

feels like we need a high level categorisation from easy to hard to classify the config examples.

and then a examples repo where the examples are exercised. The endpoints just need to respond with fake data - so just a simple main.go for each endpoint.

Also would suggest using overmind to run each example. Its just a procFile for each example folder.
https://github.com/DarthSim/overmind

In the end, this would make it so much easier to use caddy and have someone to go to see when things screw up in your config.

@randomairborne
Copy link

I would suggest

example.com {
  handle_path /foo {
    reverse_proxy localhost:1234
  }
}

as an example for consolidating microservices under domains or similar.

@mholt
Copy link
Member Author

mholt commented Jul 10, 2023

@mustafa0x Thanks for that search, I dunno how useful a pile of 7K of them will be, but maybe I can scan those and look for some common patterns that stick out to me!

@gedw99 Evaluating examples is a whole 'nother dealio; maybe we can have a Caddy sandbox/playground, but the new site won't initially have one because of priorities.

@randomairborne Thanks! I will have to make a note with that example that most backend apps don't like being proxied to in a "subfolder" unless they're explicitly configured to do so.

@dbaynard
Copy link
Contributor

dbaynard commented Jul 10, 2023

I'm happy to see this issue is getting some love. It's one thing having helpful users and developers in the community site, but standalone resources and examples would have been brilliant for me.

I have a mix of elegant and horrible examples… so you can have one of each, for now.

CORS

This is based on Setup CORS in Caddy 2 and Caddy Server, CORS, and Preflight Requests - AUXNET but changes (fixes?) a few things.

  1. Adds Vary: Origin to prevent caching.
  2. Adds defer to prevent duplicate headers.
  3. Adds {args.0} to the names to enable multiple cors snippets.
  4. Customizes the Access-Control-Expose-Headers.
  5. Allows credentials to be sent with cross origin requests.
  6. [edit] Can be imported multiple times to support multiple Origins (more precise matching on pre-flight).

I'm not 💯 it's doing all the right things; check for your own use case.

(cors) {
  @cors_preflight{args.0} {
    method OPTIONS
    header Origin {args.0}
  }
  @cors{args.0} header Origin {args.0}

  handle @cors_preflight{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Allow-Headers "Authorization, Cache-Control, Content-Type"
      Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE"
      Access-Control-Max-Age "3600"
      Vary Origin
      defer
    }
    respond "" 204
  }

  handle @cors{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Expose-Headers "{args.1}"
      Vary Origin
      defer
    }
  }
}

Use as follows:

import cors https://{$HOST} "ETag"

Rewriting query parameters

This is a bit of a bruiser… and I think there's a race condition, as the iteration ordering of go maps is not defined (but is usually alphabetical…) — if I can reproduce it, I'll raise an issue.

This snippet pulls out a query parameter that is set to true, and builds a list of ,-separated parameters instead.

(query_select) {
  @query_{args.0} query {args.0}=true
  vars @query_{args.0} {
    select "{vars.select}{vars.sep}{args.0}"
    sep ","
  }
}

Use as follows.

https://{$HOST} {
  import query_select first
  import query_select second
 
  rewrite * /?new_query={vars.select} 
  …
}

This then converts https://{$HOST}/?first=true&second=true to https://{$HOST}/?new_query=first,second for subsequent use.

@randomairborne
Copy link

@mholt this one is really specific to things like APIs that don't directly interact with the user, imo

@mholt
Copy link
Member Author

mholt commented Jul 11, 2023

@dbaynard Thanks, there's some good content there.

@randomairborne Which one, exactly?

@randomairborne
Copy link

@mholt #324 (comment) is specifically for a lot of API servers, and will utterly break a lot of more traditional templated websites.

@mholt
Copy link
Member Author

mholt commented Jul 11, 2023

Yes, true -- as I said above, most websites don't like that kind of proxying, so we'd have to make a note of it. I like the distinction between templated sites and "API servers". Makes a lot of sense.

@dbaynard
Copy link
Contributor

CORS

This is based on Setup CORS in Caddy 2 and Caddy Server, CORS, and Preflight Requests - AUXNET but changes (fixes?) a few things.

  1. Adds Vary: Origin to prevent caching.
  2. Adds defer to prevent duplicate headers.
  3. Adds {args.0} to the names to enable multiple cors snippets.
  4. Customizes the Access-Control-Expose-Headers.
  5. Allows credentials to be sent with cross origin requests.
  6. [edit] Can be imported multiple times to support multiple Origins (more precise matching on pre-flight).

I'm not 💯 it's doing all the right things; check for your own use case.

I found and fixed an issue, with this (the issue is in the sources I referenced). When imported multiple times, only the origin from the first was allowed. Now only the pre-flight with the correct origin matches.

(cors) {
  @cors_preflight{args.0} {
    method OPTIONS
    header Origin {args.0}
  }
  @cors{args.0} header Origin {args.0}

  handle @cors_preflight{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Allow-Headers "Authorization, Cache-Control, Content-Type"
      Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE"
      Access-Control-Max-Age "3600"
      Vary Origin
      defer
    }
    respond "" 204
  }

  handle @cors{args.0} {
    header {
      Access-Control-Allow-Origin "{args.0}"
      Access-Control-Allow-Credentials true
      Access-Control-Expose-Headers "{args.1}"
      Vary Origin
      defer
    }
  }
}

@steffenbusch
Copy link
Contributor

@robgordon89

We make heavy use of snippets and ENV's for our TLS on-demand setup

How are you passing the ENV to Caddy (--envfile?) and how do you do reloads when you want to change for example CADDY_MAINTENANCE_ENABLED from false to true to take effect?

I definitely like the approach with environment variables, but caddy reload does not accept the --envfile flag.

@mholt
Copy link
Member Author

mholt commented Jul 31, 2023

but caddy reload does not accept the --envfile flag.

We could probably change that. I don't think it would be able to reset the environment but we could at least update any prior/existing values and add new ones.

@steffenbusch
Copy link
Contributor

We could probably change that. I don't think it would be able to reset the environment but we could at least update any prior/existing values and add new ones.

Hi Matt, I think that would be very helpful to have --envfile in the reload command available. Thank you for considering this.

I'm still starting with Caddy Server configuration and still doing lots of tests and trials regarding my configuration. I have to adopt a lot of logic and features from Caucho Resin to Caddy. An ability to have Caddy reload the Caddyfile configuration with consideration of values from envfile would be great.
Honestly, I have not yet understood (from the docs) what's the difference between using {$CADDY_MAINTENANCE_ENABLED} and {env.CADDY_MAINTENANCE_ENABLED} in Caddyfile when it comes to environment variables.

@francislavoie
Copy link
Member

Getting off topic here, but I don't see how reload --envfile would work. A reload involves pushing the new config via Caddy's API, and I don't see how we can also push the envfile contents at the same time, unless we encode it as HTTP headers or something which is strange.

Regarding the difference between {$VAR} and {env.VAR}, the former is a textual replacement in the Caddyfile before it is parsed (so it can be placed anywhere and expand into many config tokens) and the latter is replaced at runtime, and can only be used in config locations where the replacer is run (in practice this is most places, but with many limitations). For example: for secrets, the latter is preferred because it will hide it from the autosave.json file since it's a static value, but for a domain name (site address) you need to use the former because it needs to be known up-front and can't be replaced every time it's needed at runtime.

I'll try to clean up the explanation on https://caddyserver.com/docs/caddyfile/concepts#environment-variables but it's essentially saying the same thing.

@steffenbusch
Copy link
Contributor

The second paragraph is an awesome explanation. That would be very beneficial for the documentation.

@dbaynard
Copy link
Contributor

dbaynard commented Aug 1, 2023

At a meta level, I'm happy to see you (maintainers) going through this process. I came to caddy from tailscale, and there are many things I like about it, but it has been difficult to figure out how to use its functionality, in particular for what must be common use cases. So I'm happy to help contribute, too.

@dbaynard
Copy link
Contributor

dbaynard commented Aug 9, 2023

I've hit a situation that the CORS block above doesn't handle: if the Origin should be a regex. There's a straightforward modification to handle this situation, and I've created a variable to have the actual matched origin.

However, in doing so, I found bugs in my configuration that took me a long time to resolve. As part of it, I learned how to interpret the actual caddy json structure. Turns out I had to wrap the import cors lines together in a route block, and everything else in another route block, as the two are not mutually exclusive (and also the cors matchers have to run first to extract the actual origin, where all I know is the regex).

I did know about Similar directives — handle (Caddyfile directive) — Caddy Documentation but didn't clock that the reason caddy was returning a 200 where I had intended a 418 is because the CORS handlers and the actual routing logic were mutually exclusive.

The differences between handle and route seem pretty crucial to understanding the control flow through caddy, and so I think it would be helpful to show an example like this, to give it more prominence. I know they are introduced in Directives — Caddyfile Concepts — Caddy Documentation; I think it's easy to miss.

(I'm not particularly obsessed with CORS configuration, it's just that's where my attention is needed…)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

10 participants