Enable PR flow and dependabot and adapt secrets (#18)

* preparing pipelines to move to cryo-ci
- Moved secrets from lastpass to vault.cryo.team
- removed tags for workers on concourse
- used Cryo common keys for github and docker

[#178844953]

* updated pipelines to work in the cryo env
- corrected usages of github.ssh_key
- used the cryo toolsmiths token rather tahn persi

* fixed the update-integration-config path for drats

* using the freezing branch of persi-ci to run the tests

* using smb_cats secret from vault

* using git@github.com rather than https for repos fetched with the ssh key

* update generate drats integration config to use cf-deployment-env

- the underlying drats script was updated to use cf-deployment-env to get the enviornment name

* allow release info change in docker file build

Suite value caused due to the debian 'bullseye' release (https://www.debian.org/News/2021/20210814)

* tagging the smb-driver integration test to run on ubuntu

* stop using the forked git resource

- switching to dependabot

[#178844953]

* Updated the sync-pipelines script to use the freezing branch

* Use latest git-resource to bypass certificate issue (#2)

Getting packages from gopkg.in was throwing certificate issues.
This was caused by a combination of:
- Letsencrypt changing its root certificate authority
- git resource-type being pinned to a specific image by default

Old image for git resource doesn't include the new authority.
The problem can easily be solved by redefining the git resource
type to use latest official image.

In a future Concourse vers. this change shouldn't be necessary.

* stop-using-the-forked-git-resources

* Use Dependabot for bumping submodules (#6)

* Use Dependabot for bumping submodules

Use dependabot and a PR flow instead bumping submodules manually.
Let Dependabot open a PR. Then test. If tests pass then merge PR.
We will encode these tests in nfs-volume-release pipeline instead of here.

* Update nfs-driver.yml

* Adopt PR based flow (#7)

* Adopt PR-flow

* Update nfs-driver.yml

* Use Dependabot for bumping submodules (#8)

* Use Dependabot for bumping submodules

* Update smb-driver.yml

* Adopt PR based flow [smb] (#9)

* adopt-prflow-smb

* Update smb-driver.yml

* Enable PR flow for smb-volume-release repo (#10)

* Enable PR flow for smb-volume-release repo

* Bring back tests removed on PR #8

* Stop re-running every hour

Frequently exercising the pipelines is a good practice
The problem is how to effectively combine this practice with
a strict and safe PR based flow.

If we rerun every job every hour:
- Which PR should we test?
- Should we allow running against closed/merged PRs?
- What should the "merge-pr" in such scenario?

* Update ephemeral-diego-smb.yml

* Remove redundant job

The same tests are passed in job smb-volume-release-job-tests

* Enable PR flow for nfs-volume-service (#11)

* Enable PR flow for LTS nfs-volume-service (#12)

* Enable PR flow for LTS nfs-volume-service

* Fix merge-pr job

* Remove "passed" constraints for shipit-nfs job

After adopting a PR-flow, the shipit job no longer needs to run
immediately after all tests have passed but at a stable cadence.

We can assume that every commit in the LTS branch is the result
of a tested and merged PR and is therefore safe to release at any time.

* Use Dependabot for bumping submodules [mapfs] (#13)

Use dependabot and a PR flow instead bumping submodules manually.
Let Dependabot open a PR. Then test. If tests pass then merge PR.
We will encode these tests in mapfs-release pipeline instead of here.

* Adopt PR based flow [mapfs] (#14)

* Adopt PR based flow [mapfs]

* Fix merge-pr job

* Adoptpr mapfs release (#15)

* Clone ephemeral-diego pipeline for mapfs

* Remove releasing and merging jobs for nfs-volume-release repo

* Remove jobs for releasing mapfs from ephemeral-diego [nfs] pipeline

* Adopt PR flow using cryogenics/pr-queue-resource

* Add mapfs-release to drats and cats-nfs jobs

Without this change we can't use drats and cats-nfs in the "passed" condition for the merge-pr job

* adopt-prflow-shared-units (#16)

* adopt-prflow-shared-units

* Automatically test and merge PRs for goshims repo

* Auto test and merge PRs for existingvolumebroker

* Auto test and merge PRs for service-broker-store

* Auto test and merge PRs for volume-mount-options

* Fix resource definition for pull-requests

* Merge existingvolumebroker after unit-tests

* Update shared-units.yml

* Group related jobs to improve visualization

* Fix grouping wildcards

* Create new tags weekly, not for every PR (#17)

* Only tag existingvolumebroker weekly

* Fix passed constraint

* Add gate jobs to the auto-tagging group

* Add gate jobs to only create new tags at a weekly

This way, several PRs will fit into a single tag instead
of creating a new tag every time a new dependency is bumped.

* Fix ephemeral-diego-mapfs drats job

* Track master branch instead of freezing branch

Co-authored-by: Jatin Naik <jatinn@vmware.com>

docker-image-build.yml

@@ -13,61 +13,54 @@ resources:
     uri: https://github.com/cloudfoundry/persi-ci
     branch: master
 
-- name: fork-git-resource-docker-image
-  type: docker-image
-  source:
-    repository: cfpersi/fork-git-resource
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
-
 - name: mapfs-tests-docker-image
   type: docker-image
   source:
     repository: cfpersi/mapfs-tests
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
+    username: ((dockerhub.username))
+    password: ((dockerhub.password))
 
 - name: smith-cli-docker-image
   type: docker-image
   source:
     repository: cfpersi/smith-cli
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
+    username: ((dockerhub.username))
+    password: ((dockerhub.password))
 
 - name: nfs-driver-integration-tests-docker-image
   type: docker-image
   source:
     repository: cfpersi/nfs-integration-tests
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
+    username: ((dockerhub.username))
+    password: ((dockerhub.password))
 
 - name: bosh-release-test-docker-image
   type: docker-image
   source:
     repository: cfpersi/bosh-release-tests
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
+    username: ((dockerhub.username))
+    password: ((dockerhub.password))
 
 - name: nfs-unit-tests-docker-image
   type: docker-image
   source:
     repository: cfpersi/nfs-unit-tests
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
+    username: ((dockerhub.username))
+    password: ((dockerhub.password))
 
 - name: smb-unit-tests-docker-image
   type: docker-image
   source:
     repository: cfpersi/smb-unit-tests
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
+    username: ((dockerhub.username))
+    password: ((dockerhub.password))
 
 - name: nfs-cats-docker-image
   type: docker-image
   source:
     repository: cfpersi/nfs-cats
-    username: ((dockerhub_username))
-    password: ((dockerhub_password))
+    username: ((dockerhub.username))
+    password: ((dockerhub.password))
 
 - name: smb-volume-release
   type: git
@@ -87,24 +80,8 @@ resources:
     branch: master
     uri: https://github.com/cloudfoundry/mapfs-release.git
 
-- name: fork-git-resource
-  type: git
-  source:
-    uri: https://github.com/cloudfoundry/go-git-resource.git
-
 jobs:
 
-- name: build-fork-git-resource-docker-image
-  plan:
-    - get: nightly
-      trigger: true
-    - get: fork-git-resource
-      trigger: true
-    - put: fork-git-resource-docker-image
-      params:
-        build: fork-git-resource
-        dockerfile: fork-git-resource/dockerfiles/ubuntu/Dockerfile
-
 - name: build-nfs-driver-integration-tests-docker-image
   plan:
     - get: nightly
@@ -177,7 +154,7 @@ jobs:
           type: docker-image
           source: {repository: relintdockerhubpushbot/cf-deployment-concourse-tasks}
         params:
-          GITHUB_TOKEN: ((github-smith-repo-token))
+          GITHUB_TOKEN: ((github-smith-repo.access_token))
         outputs:
           - name: smith-cli-binary
         run:

dockerfiles/smith-cli/Dockerfile

@@ -8,4 +8,4 @@ RUN mv smith /usr/local/bin
 RUN echo "installing om"
 RUN wget -q -O - https://raw.githubusercontent.com/starkandwayne/homebrew-cf/master/public.key |  apt-key add -
 RUN echo "deb http://apt.starkandwayne.com stable main" |  tee /etc/apt/sources.list.d/starkandwayne.list
-RUN apt-get update && apt-get install om -y && apt-get install jq -y
+RUN apt-get --allow-releaseinfo-change-suite update && apt-get install om -y && apt-get install jq -y