Mapping AWS IAM Identity Center managed roles to K8s permissions

[christophetd]

Hello,

I'm using AWS SSO to manage roles and permissions in several AWS accounts. Consequently, my roles are autoprovisioned in each AWS account by AWS SSO and their names look like AWSReservedSSO_SomeRole_e11aa594f712e1dc.

What would be the best way to manage this with aws-iam-authenticator? The current solution seems to force to specify the full role name, so the configuration looks like:

data:
  mapRoles: |
   groups:
        - system:masters
        rolearn: arn:aws:iam::xxx:role/AWSReservedSSO_SomeRole_e11aa594f712e1dc
        username: kubernetes-admin:{{SessionName}}

The issue is that the name of the role changes in each AWS account, so it makes things quite hard for automation. Do you have any guideline to suggest in this context?

AWS SSO is becoming more and more prevalent so I expect it should be a quite widely encountered issue

Thank you!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[christophetd]

/remove-lifecycle stale

[christophetd]

@nckturner (who seem to be the main repo contributor), do you have any insights on how this can be achieved in the current state? Otherwise, do you have some plans to support it? Thank you!

[callum-p]

I have separate dev/prod/etc accounts using AWS SSO. To achieve this I do something like this:

export ADMIN_ROLE=$(awscli iam list-roles --query 'Roles[?starts_with(RoleName, `AWSReservedSSO_KubernetesAdmin`) == `true`].RoleName' --output text)
export READ_ONLY_ROLE=$(awscli iam list-roles --query 'Roles[?starts_with(RoleName, `AWSReservedSSO_KubernetesReadOnly`) == `true`].RoleName' --output text)
export ACCOUNT_ID=$(awscli sts get-caller-identity --query Account --output text)

envsubst < specs/configmap.tpl.yml > specs/configmap.yml

Then the configmap:

---
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: kube-system
  name: aws-iam-authenticator
  labels:
    k8s-app: aws-iam-authenticator
data:
  config.yaml: |
    clusterID: $K8S_CLUSTER_NAME
    server:
      #  2) "{{SessionName}}" is the role session name.
      mapRoles:
      - roleARN: arn:aws:iam::${ACCOUNT_ID}:role/${ADMIN_ROLE}
        username: "{{SessionName}}"
        groups:
        - system:masters
      - roleARN: arn:aws:iam::${ACCOUNT_ID}:role/${READ_ONLY_ROLE}
        username: "{{SessionName}}"
        groups:
        - xxx:iam-read-only

To achieve it in the same account you'd just need to do some more shell work.

[christophetd]

This tackles initial provisioning, but since the name of AWS SSO managed roles is subject to change, this mapping can break at any point in time, doesn't it?

[callum-p]

This tackles initial provisioning, but since the name of AWS SSO managed roles is subject to change, this mapping can break at any point in time, doesn't it?

Does it? Maybe if you re-provision, I should test that in my env. Could always add a cronjob on k8s to poll AWS and update the configmap. Obviously not ideal but easy fixes.

[billinghamj]

I think this is a duplicate of #268, #153 and #98

[jeffgran]

@billinghamj I don't think this is a duplicate. This is related, but this is specifically about a strategy for using SSO roles (which have different names in different accounts, which means each account would have to have a duplicate entry in the configMap). Even if the paths-in-names bug were to be fixed, this would still be a separate question/issue (not quite a bug, just a non-ergonomic developer experience for those using SSO)

[gunzy83]

We have just hit this issue as well with a new SSO implementation. This will be fine with the CLI as I have a separate role we will assume there (from the SSO based role we assume via aws sso login) but for roles assumed in the console this is going to be extra work for us to support it in our automation.

Correct me if am wrong here but a heap of this problem would go away if this more closely followed what Hashicorp Vault does here in not naively matching strings in the ARNs and actually checking for a Unique Identifier, which in the case of assumed roles is the part before the colon in UserId returned from STS. This would require the addon to have knowledge of IAM in the account the role is in but I think that would only enhance security.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[christophetd]

/remove-lifecycle stale

[mimmus]

This tackles initial provisioning, but since the name of AWS SSO managed roles is subject to change,
this mapping can break at any point in time, doesn't it?

I can confirm that RANDOM_SUFFIX in AWSReservedSSO_AdministratorAccess_RANDOM-SUFFIX can change.
I don't know when or why but in our multi-account Organization, it changed and broke every EKS access.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[christophetd]

/remove-lifecycle stale

[dokuboyejo]

Indeed, a required feature for SSO enabled access

[christophetd]

Might be solved when #416 is merged, since using ArnLike would allow to support AWS SSO roles

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[christophetd]

/remove-lifecycle stale

[sftim]

One option - not simple - is to watch for changes to IAM roles (AWS Config, or EventBridge) and then use that update feed to drive updates to controller-managed IAMIdentityMappings.

I think this would be super fragile, but it is possible.

[gunzy83]

One option - not simply - is to watch for changes to IAM roles (AWS Config, or EventBridge) and then use that update feed to drive updates to controller-managed IAMIdentityMappings.

I think this would be super fragile, but it is possible.

It is definitely possible and I did this in at a previous company. If you are going for least privilege you will more than likely have a limited set of permission sets for EKS access anyways so once provisioned to the account the role names will remain static and your deployment automation can easily query for the role by pattern or tag and inject it into the configmap at deploy time.

I have actually abandoned the idea of using SSO for access to EKS because there is no way to audit who did what in the kubernetes audit log, it just appears as the username assigned to the role in the config map. With compliance requirements I will be looking at OIDC.

[zhelyan]

there is no way to audit who did what in the kubernetes audit log
try setting username: {{SessionName}} in aws-auth

[sftim]

/retitle Mapping AWS IAM Identity Center managed roles to K8s permissions

(AWS IAM Identity Center was previously known as AWS SSO)

[gunzy83]

try setting username: {{SessionName}} in aws-auth

Thanks. I read your comment and went searching for where this suggestion came from... I found a couple of Github issues that mention this variable and a single reference in this doc: https://aws.github.io/aws-eks-best-practices/security/docs/iam/. I guess I will have to read all of this as well to ensure I have not missed any undocumented options that have yet to make it into the official documentation.

This to me still feels like a hack (although it looks effective for AWS SSO, for now). I could be wrong, but the session name being set to the username from the IdP regardless of the role_session_name variable in ~/.aws/config seems like an undocumented change in behaviour for AWS SSO/IAM Id Center since the implementation I did in early 2021. My primary objection to this is that there is no guarantee for assumed roles that the session name will be set to something that is a username that can be mapped to a user unless you enforce it in IAM policy (that effectively forces the user to set it in their CLI config or elsewhere). The CLI left to its own devices will set the session name to something like botocore-session-1664592308 which is not that useful.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[christophetd]

/remove-lifecycle stale

[gabegorelick]

What's the status of this since #416 was merged? That's still marked as "alpha."

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[christophetd]

/remove-lifecycle stale

[justinas-b]

Hey! We have our EKS clusters spread across multiple AWS accounts and those random suffixes in AWS SSO roles are really painful for us. To overcome this, i have implemented https://github.com/justinas-b/aws-iam-authenticator-sso-wrapper utility until permissionSet names or wildcards will be supported natively.

If anyone would like to test and provide a feedback, i would really appreciate it. This tool basically monitors aws-auth configMap from it's own namespace where permissionset key is acceptable in addition to rolearn:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: aws-iam-authenticator-sso-wrapper
data:
  mapAccounts: |
    []
  mapRoles: |
    - "permissionset": "AdminRole"
      "username": "AdminRole:{{SessionName}}"
      "groups":
      - "system:masters"
    - "rolearn": "arn:aws:iam::123456789012:role/AWSReservedSSO_sre_0123456789abcdef
      "username": "SRE:{{SessionName}}"
      "groups":
      - "system:masters"
  mapUsers: |
    []

It translates that configMap to regular format and updates kube-system/aws-auth configMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapAccounts: |
    []
  mapRoles: |
    - "rolearn": "arn:aws:iam:: 123456789012:role/AWSReservedSSO_AdminRole_0123456789abcdef"
      "username": "AdminRole:{{SessionName}}"
      "groups":
      - "system:masters"
    - "rolearn": "arn:aws:iam::123456789012:role/AWSReservedSSO_sre_0123456789abcdef
      "username": "SRE:{{SessionName}}"
      "groups":
      - "system:masters"
  mapUsers: |
    []

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[christophetd]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[christophetd]

/remove-lifecycle stale