Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REQUEST: Need for AWS account for hosting CAPA generated AMIs #5010

Open
Ankitasw opened this issue Mar 23, 2023 · 19 comments · May be fixed by #6517
Open

REQUEST: Need for AWS account for hosting CAPA generated AMIs #5010

Ankitasw opened this issue Mar 23, 2023 · 19 comments · May be fixed by #6517
Labels
sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra.

Comments

@Ankitasw
Copy link
Contributor

Ankitasw commented Mar 23, 2023

We need to host all of the new CAPA AMIs going forward in a CNCF account such that it's maintained upstream.
Currently, the cost for running EC2 instances to generate AMIs is average of 5K USD in last 6 months. We also need small amount of budget for running a lambda function(costing around 20USD) and data transfer(costing around 36USD).

Would it be possible to provide a separate ACL to make this happen?

Refer slack thread here for more details.

@Ankitasw Ankitasw added the sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. label Mar 23, 2023
@Ankitasw
Copy link
Contributor Author

Ankitasw commented Mar 23, 2023

cc @richardcase @dims

@Ankitasw
Copy link
Contributor Author

Ankitasw commented Jun 6, 2023

Hey @dims 👋
Can we prioritize this?

@dims
Copy link
Member

dims commented Jun 6, 2023

+1 from me!

@ameukam
Copy link
Member

ameukam commented Jun 6, 2023

@Ankitasw who owns the current account ?
we can transfer it under the community funding if needed.

@Ankitasw
Copy link
Contributor Author

Ankitasw commented Jun 6, 2023

VMware owns the current account and hence we need a new one such that others also has access to the account

@hh
Copy link
Member

hh commented Jun 7, 2023

I'm not sure if it's the amount we are looking for, but I did notice we migrated "cncf-k8s-infra-aws-capa-ami" account from the CNCF AWS Org to Kubernetes AWS AWS Org a while back: #4626 (comment)

The password for arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734 is stored in the Kubernetes Community 1Password vault AWS CI Accounts.

export AWS_PROFILE=hh@kubernetes
aws organizations describe-account --account-id 819546954734
{
    "Account": {
        "Id": "819546954734",
        "Arn": "arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734",
        "Email": "cncf-k8s-infra-aws-capa-ami@lists.cncf.io",
        "Name": "cncf-k8s-infra-aws-capa-ami",
        "Status": "ACTIVE",
        "JoinedMethod": "INVITED",
        "JoinedTimestamp": "2023-02-07T01:11:07.090000+02:00"
    }
}

@hh
Copy link
Member

hh commented Jun 7, 2023

@dims / @ameukam Here is the 1Password Kubernetes community Vault and entry for the account. Share as you see fit. I'd like to find a way to manage ongoing access to 1Password Vault entries so passwords can be updated and shared more easily.

Check Kubernetes Vaults

op vault list
ID                            NAME
5qq5hxaboazxl5p4e5dr6gypqi    Private
atxptplouln57mbc5kvt3tq6jm    AWS CI accounts
x5q2rsmkygy56vwmrgsccuoi2a    Shared

Inspect “AWI CI accounts”

op vault get "AWS CI accounts"
ID:                   atxptplouln57mbc5kvt3tq6jm
Name:                 AWS CI accounts
Type:                 USER_CREATED
Attribute version:    1
Content version:      125
Items:                87
Created:              4 months ago
Updated:              1 month ago

Find capa-ami password item

op items list | grep capa-ami
vatlttczb3iebcmf7t5hlartwq    Awazon (cncf-k8s-infra-aws-capa-ami)                           AWS CI accounts            3 months ago

Retrieve capi-ami one password item

op items get vatlttczb3iebcmf7t5hlartwq | grep -v password:
ID:          vatlttczb3iebcmf7t5hlartwq
Title:       Awazon (cncf-k8s-infra-aws-capa-ami)
Vault:       AWS CI accounts (atxptplouln57mbc5kvt3tq6jm)
Created:     3 months ago
Updated:     3 months ago by Riaan
Favorite:    false
Tags:        aws,production,sig-k8s-infra
Version:     1
Category:    LOGIN
Fields:
  username:    cncf-k8s-infra-aws-capa-ami@lists.cncf.io
URLs:
  website:    https://signin.aws.amazon.com (primary)

Share this link (good for 7 days)

We could do this, but it’s only good for 7 days and then access to this is gone. I’d like to find a process to share this longer term.

op items get vatlttczb3iebcmf7t5hlartwq --share-link

@dims
Copy link
Member

dims commented Jun 7, 2023

@hh thank you I can confirm that i see it in my 1password

image

@dims
Copy link
Member

dims commented Jun 7, 2023

@Ankitasw please DM me, we'll work out what you need (looks like you want to install and run a lambda at least!)

@ameukam
Copy link
Member

ameukam commented Jul 20, 2023

@hh Can we instead create an dedicated user for @Ankitasw with AdministratorAccess on the account cncf-k8s-infra-aws-capa-ami ? Thanks!

@richardcase
Copy link
Contributor

@ameukam & @hh - Would it also be possible to add me as well? So we have 2 maintainers of CAPA on the account.

@ameukam
Copy link
Member

ameukam commented Jul 20, 2023

@ameukam & @hh - Would it also be possible to add me as well? So we have 2 maintainers of CAPA on the account.

+1 from me.

@hh
Copy link
Member

hh commented Jul 21, 2023

The AWS account cncf-k8s-infra-aws-capa-ami@lists.cncf.io / arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734 is probably where we want to create the iam roles. Probably similar to the way we create them at the top level Kubernetes AWS account.

However I’m not sure how we want to manage the terraform for AWS org member-accounts, and the resources (like IAM users) that are needed by them.

Should the CAPA team create their own terraform to create acccounts there? I can’t imagine scaling this if we have to manage all-the-k8s sub/member-acccounts with terraform in the same repo managed by the same team.

Suggestions welcome.

If we decide to go with top level accounts, and a good read might be Accessing member accounts in your organization, however I would still recommend we find a way to delegate or help setup a separate way to manage the AWS K8s organization member-account terraform.

Our top level kubernetes-aws-admins@lists.cncf.io account + org

aws organizations describe-organization
{
    "Organization": {
        "Id": "o-kz4vlkihvy",
        "Arn": "arn:aws:organizations::348685125169:organization/o-kz4vlkihvy",
        "FeatureSet": "ALL",
        "MasterAccountArn": "arn:aws:organizations::348685125169:account/o-kz4vlkihvy/348685125169",
        "MasterAccountId": "348685125169",
        "MasterAccountEmail": "kubernetes-aws-admins@lists.cncf.io",
        "AvailablePolicyTypes": [
            {
                "Type": "SERVICE_CONTROL_POLICY",
                "Status": "ENABLED"
            }
        ]
    }
}

List of current top level IAM Users

aws iam list-users --output=table --query 'Users[*].[UserName,Arn]'
---------------------------------------------------------------
|                          ListUsers                          |
+-------------+-----------------------------------------------+
|  arnaud     |  arn:aws:iam::348685125169:user/arnaud        |
|  bentheelder|  arn:aws:iam::348685125169:user/bentheelder   |
|  dims       |  arn:aws:iam::348685125169:user/dims          |
|  hh         |  arn:aws:iam::348685125169:user/hh            |
|  jeefy      |  arn:aws:iam::348685125169:user/jeefy         |
+-------------+-----------------------------------------------+

Inspect cncf-k8s-infra-aws-capa-ami@lists.cncf.io account

aws organizations describe-account --account-id 819546954734
{
    "Account": {
        "Id": "819546954734",
        "Arn": "arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734",
        "Email": "cncf-k8s-infra-aws-capa-ami@lists.cncf.io",
        "Name": "cncf-k8s-infra-aws-capa-ami",
        "Status": "ACTIVE",
        "JoinedMethod": "INVITED",
        "JoinedTimestamp": "2023-02-07T00:11:07.090000+01:00"
    }
}

@richardcase
Copy link
Contributor

richardcase commented Aug 17, 2023

Should the CAPA team create their own terraform to create acccounts there? I can’t imagine scaling this if we have to manage all-the-k8s sub/member-acccounts with terraform in the same repo managed by the same team.

@hh (and @Ankitasw ) It would be good if there was a way for the maintainers of CAPA to manage the access to the accounts they use (the one for AMIs in this case). It would be more scalable as a general principal.

I'd be happy maintaining terraform, or perhaps yaml for ACK for the CAPA specific account.

@richardcase
Copy link
Contributor

Circling back to this. We need to build & publish some new AMIs to the AWS account for CAPA. How should we manage access to the account for the CAPA maintainers?

@hh @dims - any thoughts on this?

@richardcase
Copy link
Contributor

The current list of maintainers of CAPA who will ideally need to be able to publish new AMIs are:

(source)

This changes over time so happy to contribute terraform or something else if needed.

@BobyMCbobs
Copy link
Member

I think that the best path is to create terraform in infra/aws/terraform/cncf-k8s-infra-aws-capa-ami, where the Terraform provider for AWS is either expected to be run in the account or that it is accessing through assume role with the OrganizationAccountAccessRole for full-permissions.
In that terraform, specifying the IAM users stated above.
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user
#5044

cc @hh

@richardcase
Copy link
Contributor

Thanks @BobyMCbobs .

I have started work on the terraform based on your suggestions and will have a PR for it soon.

@richardcase
Copy link
Contributor

Finally got around to updating the PR for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants