Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FakeGCS] Add a new field emulatorEnabled in gcp secret & configure Gcloud client for GCS Emulator #697

Merged
merged 3 commits into from Mar 27, 2024

Conversation

anveshreddy18
Copy link
Contributor

@anveshreddy18 anveshreddy18 commented Jan 1, 2024

What this PR does / why we need it:

  • Add a new field emulatorEnabled in gcp secret
  • configure Google cloud client for GCS Emulator when emulatorEnabled is true

This is helpful to run etcdbr tests locally and on pipelines without actually using the real infra. This can also be used by Etcd-druid to use GCS Emulator fake-gcs-server for running e2e-tests.

For these, Google cloud client in backup-restore has to be configured to connect to the Emulator.

This PR enables that in either of the two ways

  1. GOOGLE_EMULATOR_ENABLED env var is set when running etcdbr as a process using etcdbrctl
  2. emulatorEnabled field in the gcp secret is set to true via helm chart

Which issue(s) this PR fixes:
Fixes #696

Special notes for your reviewer:

Release note:

Added support for using fake-gcs-server for all etcdbr functionalities. To enable: Either 
1) Set `GOOGLE_EMULATOR_ENABLED` environment variable when running `etcdbrctl` command OR 
2) Set `emulatorEnabled: true` in GCP backup secret when deploying via Helm chart.

@anveshreddy18 anveshreddy18 requested a review from a team as a code owner January 1, 2024 11:11
@gardener-robot gardener-robot added the needs/review Needs review label Jan 1, 2024
@anveshreddy18 anveshreddy18 self-assigned this Jan 1, 2024
@gardener-robot gardener-robot added the size/s Size of pull request is small (see gardener-robot robot/bots/size.py) label Jan 1, 2024
@anveshreddy18 anveshreddy18 added platform/gcp Google cloud platform/infrastructure area/backup Backup related and removed size/s Size of pull request is small (see gardener-robot robot/bots/size.py) labels Jan 1, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jan 1, 2024
@anveshreddy18 anveshreddy18 changed the title Add a new field enableGCSEmulator in gcp secret & configure Gcloud client for GCS Emulator FakeGCS: Add a new field enableGCSEmulator in gcp secret & configure Gcloud client for GCS Emulator Jan 4, 2024
@anveshreddy18 anveshreddy18 changed the title FakeGCS: Add a new field enableGCSEmulator in gcp secret & configure Gcloud client for GCS Emulator [FakeGCS] Add a new field enableGCSEmulator in gcp secret & configure Gcloud client for GCS Emulator Jan 4, 2024
@gardener-robot gardener-robot added the size/s Size of pull request is small (see gardener-robot robot/bots/size.py) label Jan 9, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jan 9, 2024
pkg/snapstore/gcs_snapstore.go Outdated Show resolved Hide resolved
pkg/snapstore/gcs_snapstore.go Outdated Show resolved Hide resolved
@gardener-robot gardener-robot added the needs/changes Needs (more) changes label Jan 11, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 11, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 11, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 16, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 16, 2024
@anveshreddy18 anveshreddy18 added kind/enhancement Enhancement, improvement, extension area/dev-productivity Developer productivity related (how to improve development) area/testing Testing related labels Jan 18, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Feb 5, 2024
@anveshreddy18 anveshreddy18 changed the title [FakeGCS] Add a new field enableGCSEmulator in gcp secret & configure Gcloud client for GCS Emulator [FakeGCS] Add a new field emulatorEnabled in gcp secret & configure Gcloud client for GCS Emulator Feb 6, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Feb 7, 2024
@shreyas-s-rao
Copy link
Collaborator

@anveshreddy18 please also modify the release note to talk about how to enable emulator for etcdbrctl command, rather than from the provider secret perspective. Something like

Added support for using (fake-gcs-server)[https://github.com/fsouza/fake-gcs-server] for taking backups. Please set environment variable `GOOGLE_EMULATOR_ENABLED` when running `etcdbrctl` command to enable this support. Alternatively, you may also set the field `emulatorEnabled: true` in the GCP backup secret when deploying etcd-backup-restore via the helm chart to enable this.

This mentions to users of both etcdbrctl process as well as the helm charts, as to how to enable this feature.

@gardener-robot
Copy link

@anveshreddy18 You need rebase this pull request with latest master branch. Please check.

@gardener-robot gardener-robot removed the reviewed/lgtm Has approval for merging label Mar 24, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Mar 24, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Mar 24, 2024
@gardener-robot gardener-robot added size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) and removed size/s Size of pull request is small (see gardener-robot robot/bots/size.py) labels Mar 24, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Mar 24, 2024
Copy link
Contributor

@seshachalam-yv seshachalam-yv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-robot gardener-robot added reviewed/lgtm Has approval for merging and removed needs/rebase Needs git rebase labels Mar 26, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Mar 26, 2024
Copy link
Member

@renormalize renormalize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for also reflecting the changes in Azure with regards to naming convention, @anveshreddy18.
Testing will be significantly easier for GCS and Azure; now that emulators are supported for the three major providers.

Remove the unnecessary commit messages from the squash commit before merging for a cleaner commit message and history.

Thanks!

@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Mar 27, 2024
@anveshreddy18 anveshreddy18 merged commit 17e447b into gardener:master Mar 27, 2024
9 checks passed
@gardener-robot gardener-robot added the status/closed Issue is closed (either delivered or triaged) label Mar 27, 2024
@anveshreddy18 anveshreddy18 deleted the e2e/fake-gcs-server branch March 27, 2024 13:33
@anveshreddy18 anveshreddy18 restored the e2e/fake-gcs-server branch March 27, 2024 13:33
@shreyas-s-rao shreyas-s-rao added this to the v0.29.0 milestone Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backup Backup related area/dev-productivity Developer productivity related (how to improve development) area/testing Testing related kind/enhancement Enhancement, improvement, extension needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) platform/gcp Google cloud platform/infrastructure reviewed/lgtm Has approval for merging size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) status/closed Issue is closed (either delivered or triaged)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] Add a new field enableGCSEmulator in gcp secret & configure Gcloud client for GCS Emulator
8 participants