[Support]: Multi-Domain Forward-Auth with Traefik/k3s

[IngwiePhoenix]

OAuth2-Proxy Version

7.6.0

Provider

keycloak-oidc

Current Behaviour of your Problem

Hello there!

I am trying to use Keycloak + Oauth2 proxy to protect some resources that do not - or can not - have authentication on their own. The larger idea for this is to secure several endpoints and use Keycloak to determine who can and who can not access my stuff. For instance, I have a Discord provider installed that will allow users in certain servers to authenticate - and now, I need to plug the other end of the chain, making my apps return to the oauth2-proxy for doing the login.

All my apps live on *.birb.it - the root domain itself is tied to another service.

Right now, I have gotten to the point where I can see the oauth2-proxy login screen - but ONLY when I am NOT visiting /. In this here example, I used a reverse-proxy configuration to point to my router's management UI. It uses all kinds of odd things - so if I can make that one work, everything else is a given. Also, it's broken:

I am pretty sure that there is but a minor oopsie that I am overlooking... Maybe you can help me spot it?

Thank you! Kind regards,
Ingwie

Configuration details or additional information

apiVersion: v1
kind: Namespace
metadata:
  name: auth-proxy
---
apiVersion: v1
kind: Secret
metadata:
  name: oidc-creds
  namespace: auth-proxy
type: Opaque
stringData:
  OAUTH2_PROXY_PROVIDER: keycloak
  OAUTH2_PROXY_CLIENT_ID: <snip>
  OAUTH2_PROXY_CLIENT_SECRET: <snip>
  OAUTH2_PROXY_OIDC_ISSUER_URL: https://keycloak.birb.it/realms/master/.well-known/openid-configuration
  OAUTH2_PROXY_COOKIE_SECRET: <snip>
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: op2-cm
  namespace: auth-proxy
data:
  OAUTH2_PROXY_COOKIE_DOMAIN: .birb.it
  OAUTH2_PROXY_EMAIL_DOMAINS: "*"
  OAUTH2_PROXY_HTTP_ADDRESS: "0.0.0.0:8080"
  OAUTH2_PROXY_REVERSE_PROXY: "true"
  OAUTH2_PROXY_SET_XAUTHREQUEST: "true"
  OAUTH2_PROXY_SILENCE_PING_LOGGING: "true"
  #OAUTH2_PROXY_SKIP_PROVIDER_BUTTON: "true"
  #OAUTH2_PROXY_TRUSTED_IPS: "192.168.1.0/24,100.64.0.0/24"
  OAUTH2_PROXY_WHITELIST_DOMAINS: "keycloak.birb.it"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: oauth2-proxy-app
  namespace: auth-proxy
  labels:
    app: oauth2-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: oauth2-proxy
  template:
    metadata:
      labels:
        app: oauth2-proxy
    spec:
      containers:
        - name: app
          image: quay.io/oauth2-proxy/oauth2-proxy:v7.6.0
          envFrom:
            - secretRef:
                name: oidc-creds
            - configMapRef:
                name: op2-cm
          ports:
            - containerPort: 8080
              name: http
              protocol: TCP
            - containerPort: 8443
              name: https
              protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: oauth2-proxy-svc
  namespace: auth-proxy
  labels:
    app: oauth2-proxy
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 8080
      targetPort: 8080
    - name: https
      port: 8443
      targetPort: 8443
  selector:
    app: oauth2-proxy
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: oauth2-proxy-tr
  namespace: auth-proxy
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`op.birb.it`)
      kind: Rule
      services:
        - name: oauth2-proxy-svc
          port: http
          passHostHeader: true
          scheme: http
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: auth-trm
  namespace: auth-proxy
spec:
  forwardAuth:
    address: https://op.birb.it/oauth2/auth
    trustForwardHeader: true
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: auth-redir-trm
  namespace: auth-proxy
spec:
  redirectRegex:
    regex: /oauth2/(.*)
    replacement: https://op.birb.it/oauth2/${1}
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: auth-login-trm
  namespace: auth-proxy
spec:
  errors:
    status:
      - "401-403"
    service:
      name: oauth2-proxy-svc
      port: http
    query: "/oauth2/sign_in?rd={url}"

And this is the IngressRoute in the "router endpoint" itself

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: proxy-tr
  namespace: proxy-router
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`router.birb.it`)
      kind: Rule
      priority: 1
      services:
        - name: proxy-svc
          port: webui
    - match: Host(`router.birb.it`) && ClientIP(`192.168.1.0/24`)
      kind: Rule
      priority: 10
      middlewares:
        - name: auth-redir-trm
          namespace: auth-proxy
        - name: auth-login-trm
          namespace: auth-proxy
        - name: auth-trm
          namespace: auth-proxy
      services:
        - name: proxy-svc
          port: webui

(Priorities are swapped so they do apply on my local network - this will be rewersed later, so I can selectively protect or not-protect things. Alternatively, chaining the various auth-middlewares into the traefik endpoints config to apply them everywhere might be a good alternative - not sure which way I will go. But, this endpoint's config is temporary - this will actually go away entirely down the line, as there is no real need to expose it through k3s. Great for tests though!)

Steps To Reproduce

1. Install k3s
2. Implement a throw-away test endpoint (like whoami)
3. Implement a Keycloak deployment or another OIDC provider
4. Edit and apply the above configuration snippet
5. Use the three middlewheres in an IngressRoute to see the result.