Kubeapps in k8s with OAuth2 proxy: fail to login in (blank page)

[Nibelheims]

Hello,

First of all thank you for the Kubeapps tool.
I have used it for a POC in a kubernetes cluster and it worked well. That first instance relied on a token to work.

I now want to move from that kind of devel setup to something more production ready. I now work with a k8s cluster which uses keycloak and OAuth2 proxy. SSO works well to log into services or apps such as a dummy dokuwiki for testing purposes. kubelogin also works well.

I have tried to follow the following docs:

• https://kubeapps.com/docs/access-control/
• https://kubeapps.com/docs/using-an-OIDC-provider/
• https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider/#keycloak-oidc-auth-provider

I ended up with the following:

• a keycloak client named k8s with a groups claim
• k8s api server configured to use OIDC (with the groups-claim, issuer URL for authenth, and a username-claim)
• kubeapps 2.4.2 deployed with the following values:

        authProxy:
          enabled: true
          external: true
          skipKubeappsLoginPage: false
          provider: oidc
          clientID: k8s
          scope: "groups"
          oauthLoginURI: https://auth.<DOMAIN>/oauth2/start?rd=https://kubeapps.<DOMAIN>/oauth2/callback
          additionalFlags: [--oidc-issuer-url=https://iam.<DOMAIN>/auth/realms/k8s]
        postgresql:
          postgresqlPassword: xxxx
          persistence:
            enabled: true

• ingress rule to kubeapps (w/o OAuth2 annotations, this is a "direct" route to kubeapps)
• a keycloak group with proper (hope so) RBAC config on k8s side (no specific assigned role on keycloak's side) ; RBAC are defined with the --group flag, and the name of that group
• a keycloak user which belongs to that group

When I try to log into kubeapps, the following happens:

1. I first clean every cookie in my browser, just in case
2. I open the kubeapps' URL in my browser
3. Kubeapps' login page shows up, with a button "OIDC login"
4. I click that button
5. A keycloak auth form shows up, I fill in the credentials of the dedicated user
6. The credentials are accepted, and I get redirected to the kubeapps/oauth2/callback page, which is blank, nothing happens

I'm stuck there currently. Do you have any idea of what could be wrong? That kind of environment can become complex, I hope I have provided enough information.

Thanks and best regards

[absoludity]

I don't have much experience with KeyCloak, but one thought based on your details (thanks for providing such clear details :) ):

6. The credentials are accepted, and I get redirected to the kubeapps/oauth2/callback page, which is blank, nothing happens

I was surprised that you're being redirected back to this URL, given that it is the URL used by Kubeapps' own oauth2-proxy when being used. But you're explicitly not using that, rather using your own oauth2-proxy. I then saw in your config above that you've explicitly requested this redirect with:

          oauthLoginURI: https://auth.<DOMAIN>/oauth2/start?rd=https://kubeapps.<DOMAIN>/oauth2/callback

Assuming that auth.<DOMAIN>/oauth2/start is your external oauth2-proxy, I don't know why you're including the /oauth2/callback path in the redirect. In our normal setup we have oauth2-proxy sitting in front of our Kubeapps nginx, so the redirect is added automatically when a cookie is not present, but in your case, I guess you need to add the ?rd= explicitly as you've done... I just think it should be without the /oauth2/start path which won't exist (be handled) by Kubeapps at all in your setup. What happens if you remove the /oauth2/start from your ?rd=... redirect?

I'm stuck there currently. Do you have any idea of what could be wrong? That kind of environment can become complex, I hope I have provided enough information.

It can indeed become complex. Thanks again for the details - in this case, it may be an incorrect redirect that you're explicitly asking for. Let's see.

[absoludity]

Hi @Nibelheims .

I have tried removing that, and/or the oauth2/start path before the ?rd=, but it sill does not work. The oauth2/callback, when used, is a blank page, nothing happens. When unused, I either get redirected to a bad page (https://auth., error 404) or back to the kubeapps login page (with the "LOGIN VIA OIDC PROVIDER" button), depending on the configuration I am trying.

Just to be clear, I meant removing the explicit redirect you had in the URL (?rd=) back to a non-existent page (you had the url which is correct if you're using kubeapps' own oauth2-proxy, but in your case, you're using an external instance of oauth2-proxy). I was suggesting removing that explicit redirect because your keycloak configuration should already be redirecting you to the correct oauth2-proxy /oauth2/callback of your external instance (as per the docs you linked to, for oauth provider config with keycloak).

It sounds as though that value is not configured properly, but I'm not sure (and from what you've said, you've done similar configurations for other services, and they work).

Sorry, I've not had a chance to try out with KeyCloak. I know that @dlaloue-vmware spent some time setting up Kubeapps with KeyCloak. Perhaps he knows more?

[dlaloue-vmware]

Hi,
last time i worked on integration with keycloak was about a year ago, but i do remember struggling with the ?rd issue as well.
from what i remember, i think i had 2 issues with it: one was that at the time of deploying kubeapps with the ?rd i did not have the IP/hostname of the load balancer yet, so i would have been forced to update kubeapps again (which does not seem to be an issue here); but most importantly i think ?rd was not supported by the auth proxy (or something like that) and so we could not use ?rd.

The redirect URL must be configured in your keycloak instance.

We have a complete step-by-step installation document to configure keycloak with kubeapps, but it has not been converted into a readme file and checked in into kubeapps.
I will need a little bit of time to find time to work on it. In the meantime, i could send you a PDF version of it maybe if that can help.

[absoludity]

In case it helps you @Nibelheims , Dimitri is updating the docs with more info. It should land in our main branch soon, but you can read the updated keycloak integration docs in his personal branch.

[Nibelheims]

Hello,

I thank both of you for your support, I find this very nice of you to help people with Kubeapps. Kudos to you 👍
Sorry for not having replied before, I cannot work on this everyday.

I happen to have almost succeeded in the meantime. I gave up on relying on an external OAuth2 proxy and finally managed to make Kubeapps' own instance of OAuth2 connect to Keycloak.
Thank you for the document you have linked, it looks a lot like what I ended up with. I confirm I had to explicitly add the kubeapps redirect URL to the set of the valid ones in the keycloak client I use.

The RBAC configuration is also ok, I can now deploy and remove apps :)

I still have some issues though:

• kubeapps failed to get the refresh token, but I did not enable the Direct Access Grants Enabled switch ==> I am going to test that
• I also configured kubeapps with --ssl-insecure-skip-verify, which is bad, but I could locate the secret containing keycloak's CA certificate, I'm going to try exposing it as a file (see k8s' doc about this) and using the OAUTH2_PROXY_TLS_CERT_FILE varenv

Thanks and best regards

[Nibelheims]

Hi,

Just FYI, enabling Direct Access Grants allowed Kubeapps to properly get refresh tokens, now my session doesn't get killed anymore. Thanks for the doc you provided 👍