Configure and install out-of-tree gcp credential provider #111495

ndixita · 2022-07-28T07:52:16Z

What type of PR is this?

/kind feature
/sig node
/sig testing

What this PR does / why we need it:

This PR adds GCP-specific implementation for tests to validate external credential provider feature in order to promote the feature to GA. Changes include:

A new environment variable ENABLE_AUTH_PROVIDER_GCP.
The functionality to configure and install out-of-tree GCP auth provider binary built from https://github.com/kubernetes/cloud-provider-gcp/blob/master/cmd/auth-provider-gcp/BUILD#L11 and hosted on a public storage https://storage.googleapis.com/cloud-provider-gcp/auth-provider-gcp-linux_amd64.tar.gz. when environment variable ENABLE_AUTH_PROVIDER_GCP is set.

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

k8s-ci-robot · 2022-07-28T07:52:24Z

@ndixita: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2022-07-28T07:52:25Z

Hi @ndixita. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ndixita · 2022-07-28T07:52:33Z

/milestone v1.26

k8s-ci-robot · 2022-07-28T07:52:35Z

@ndixita: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.25

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2022-07-28T07:59:13Z

@ndixita: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ndixita · 2022-07-28T08:02:59Z

/cc @ruiwen-zhao

dims · 2022-07-28T12:18:37Z

/ok-to-test

ndixita · 2022-07-28T18:07:19Z

/retest

ndixita · 2022-07-28T21:21:11Z

/cc @bobbypage

bobbypage

Thanks for working on this! Left a few comments

cluster/gce/util.sh

cluster/gce/config-default.sh

cluster/gce/gci/configure.sh

bobbypage

Thanks for the updates, just a few small comments.

bobbypage · 2022-08-16T19:21:27Z

cluster/gce/gci/configure.sh

+  local auth_config_file="${KUBE_HOME}/cri_auth_config.yaml"
+  cat >> "${auth_config_file}" << EOF
+kind: CredentialProviderConfig
+apiVersion: kubelet.config.k8s.io/v1alpha1


should the api version be /v1beta1?

bobbypage · 2022-08-16T19:22:32Z

cluster/gce/gci/configure.sh

@@ -644,6 +683,16 @@ function install-kube-binary-config {
    log-wrap "RemountFlexVolume" remount-flexvolume-directory "${VOLUME_PLUGIN_DIR}"
  fi

+  # When ENABLE_AUTH_PROVIDER_GCP is set, following flags for out-of-tree credential provider for GCP
+  # are presented to kubelet:
+  # --feature-gates=DisableKubeletCloudCredentialProviders=true,KubeletCredentialProviders=true


This comment regarding feature gate being enabled automatically with ENABLE_AUTH_PROVIDER_GCP is no longer true (maybe note that it is required that those two feature gates should be enabled)

bobbypage · 2022-08-16T19:26:57Z

cluster/gce/gci/configure.sh

@@ -546,6 +548,43 @@ function install-containerd-ubuntu {
  sudo systemctl start containerd
 }

+function install-auth-provider-gcp {
+  local -r auth_provider_tar="auth-provider-gcp-${DEFAULT_AUTH_PROVIDER_GCP_VERSION}-${HOST_PLATFORM}_amd64.tar.gz"


Instead of hardcoding _amd64.tar.gz, I would use the use ${HOST_ARCH} (so we can expand it to other arch later if needed).

i.e. follow same naming scheme as NPD:

local -r npd_tar="node-problem-detector-${npd_version}-${HOST_PLATFORM}_${HOST_ARCH}.tar.gz"

https://github.com/kubernetes/kubernetes/blob/master/cluster/gce/gci/configure.sh#L290

dchen1107 · 2022-08-16T22:53:14Z

/lgtm
/approve

k8s-ci-robot · 2022-08-16T22:53:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dchen1107, ndixita

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cluster/gce/OWNERS~~ [dchen1107]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ndixita · 2022-08-23T23:17:28Z

/retest

bobbypage · 2022-08-24T02:31:02Z

Thanks for the updates!

/lgtm

liggitt · 2022-08-26T19:27:02Z

the alpha e2e job has been red for about 3 days, corresponding roughly with when this PR merged... is there any config in the alpha job that would cause problems with this configuration?

https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-cloud-provider/gcp/gcp-gce.yaml#L340-L384

liggitt · 2022-08-26T19:32:05Z

trying the alpha e2e job with #112075 reverting this change to see if that resolves the bring-up error

ruiwen-zhao · 2022-08-26T19:41:12Z

This PR is a no-op unless ENABLE_AUTH_PROVIDER_GCP is set. And I dont think any test job has set up ENABLE_AUTH_PROVIDER_GCP yet.

ruiwen-zhao · 2022-11-01T19:31:12Z

For tracking purposes, this PR is for kubernetes/enhancements#2133

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 28, 2022

k8s-ci-robot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Jul 28, 2022

k8s-ci-robot added area/provider/gcp Issues or PRs related to gcp provider sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jul 28, 2022

k8s-ci-robot requested review from marseel and roycaihw July 28, 2022 07:53

k8s-ci-robot requested a review from ruiwen-zhao July 28, 2022 08:03

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 28, 2022

k8s-ci-robot requested a review from bobbypage July 28, 2022 21:21

bobbypage reviewed Jul 29, 2022

View reviewed changes

ruiwen-zhao reviewed Aug 1, 2022

View reviewed changes

cluster/gce/gci/configure.sh Show resolved Hide resolved

liggitt mentioned this pull request Aug 2, 2022

Move the Kubelet Credential Provider feature to GA and Update the Credential Provider API to GA #111616

Merged

ndixita force-pushed the node-e2e-gcp-cred-provider branch from 1cce947 to e0b72d0 Compare August 15, 2022 22:21

bobbypage reviewed Aug 16, 2022

View reviewed changes

dchen1107 added this to the v1.26 milestone Aug 16, 2022

k8s-ci-robot assigned dchen1107 Aug 16, 2022

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 16, 2022

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 16, 2022

ruiwen-zhao mentioned this pull request Aug 22, 2022

[KEP:2133] E2E tests for external Kubelet Credential Provider #106248

Closed

Configure and install out-of-tree gcp credential provider

d1deca1

ndixita force-pushed the node-e2e-gcp-cred-provider branch from e0b72d0 to d1deca1 Compare August 23, 2022 01:00

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 23, 2022

k8s-ci-robot assigned bobbypage Aug 24, 2022

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 24, 2022

k8s-ci-robot merged commit 5fa65e9 into kubernetes:master Aug 24, 2022

andrewsykim mentioned this pull request Nov 1, 2022

Kubelet Credential Provider kubernetes/enhancements#2133

Closed

4 tasks

ndixita mentioned this pull request Apr 4, 2023

REQUEST: New membership for ndixita kubernetes/org#4142

Closed

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Configure and install out-of-tree gcp credential provider #111495

Configure and install out-of-tree gcp credential provider #111495

ndixita commented Jul 28, 2022 •

edited

k8s-ci-robot commented Jul 28, 2022

k8s-ci-robot commented Jul 28, 2022

ndixita commented Jul 28, 2022 •

edited

k8s-ci-robot commented Jul 28, 2022

k8s-ci-robot commented Jul 28, 2022

ndixita commented Jul 28, 2022

dims commented Jul 28, 2022

ndixita commented Jul 28, 2022

ndixita commented Jul 28, 2022

bobbypage left a comment

bobbypage left a comment

bobbypage Aug 16, 2022

bobbypage Aug 16, 2022

bobbypage Aug 16, 2022

dchen1107 commented Aug 16, 2022

k8s-ci-robot commented Aug 16, 2022

ndixita commented Aug 23, 2022

bobbypage commented Aug 24, 2022

liggitt commented Aug 26, 2022

liggitt commented Aug 26, 2022

ruiwen-zhao commented Aug 26, 2022

ruiwen-zhao commented Nov 1, 2022

Configure and install out-of-tree gcp credential provider #111495

Configure and install out-of-tree gcp credential provider #111495

Conversation

ndixita commented Jul 28, 2022 • edited

What type of PR is this?

What this PR does / why we need it:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

k8s-ci-robot commented Jul 28, 2022

k8s-ci-robot commented Jul 28, 2022

ndixita commented Jul 28, 2022 • edited

k8s-ci-robot commented Jul 28, 2022

k8s-ci-robot commented Jul 28, 2022

ndixita commented Jul 28, 2022

dims commented Jul 28, 2022

ndixita commented Jul 28, 2022

ndixita commented Jul 28, 2022

bobbypage left a comment

Choose a reason for hiding this comment

bobbypage left a comment

Choose a reason for hiding this comment

bobbypage Aug 16, 2022

Choose a reason for hiding this comment

bobbypage Aug 16, 2022

Choose a reason for hiding this comment

bobbypage Aug 16, 2022

Choose a reason for hiding this comment

dchen1107 commented Aug 16, 2022

k8s-ci-robot commented Aug 16, 2022

ndixita commented Aug 23, 2022

bobbypage commented Aug 24, 2022

liggitt commented Aug 26, 2022

liggitt commented Aug 26, 2022

ruiwen-zhao commented Aug 26, 2022

ruiwen-zhao commented Nov 1, 2022

ndixita commented Jul 28, 2022 •

edited

ndixita commented Jul 28, 2022 •

edited