Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(NODE-5464): OIDC machine and callback workflow #3912

Merged
merged 64 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
e41a69d
feat(NODE-5464): OIDC machine workflow
durran Nov 7, 2023
43016d7
test: migrate human callback tests part one
durran Apr 27, 2024
f2d6ec7
test: migrate human callback tests part two
durran Apr 28, 2024
9631718
test: migrate human callback tests part three
durran Apr 29, 2024
af1f424
fix: callback speculative auth
durran Apr 29, 2024
5745425
test: use util clients in prose tests
durran May 1, 2024
107180f
fix: addressing comments
durran May 2, 2024
eba741d
fix: next comment addressing
durran May 2, 2024
c944564
fix: next comment addressing
durran May 2, 2024
dbe8d7f
fix: next comment addressing tests and lint
durran May 2, 2024
d329f78
test: update command types
durran May 2, 2024
80332e8
fix: use test in azure endpoint
durran May 2, 2024
86d6c49
test: more comments
durran May 2, 2024
060338d
fix: lock all machine workflow token calls
durran May 5, 2024
d19587d
refactor: use get in common utils
durran May 5, 2024
d4211d5
test: remaining prose tests
durran May 5, 2024
6ace530
test: last prose test
durran May 5, 2024
8e74bc9
refactor: update workflow interface to void when needed
durran May 5, 2024
7e62668
fix: more comment addressing
durran May 6, 2024
00e259d
fix: use timeout as number
durran May 6, 2024
b67a92a
docs: update version comment
durran May 6, 2024
d37bdcf
Revert "fix: use timeout as number"
durran May 6, 2024
be8ba66
docs: document all the APIs
durran May 6, 2024
6514ab2
fix: forgot to pass username
durran May 6, 2024
0aacf51
docs: move comment
durran May 6, 2024
30d8601
refactor: locking and timeouts
durran May 6, 2024
e85d177
refactor: clear timeout in finally
durran May 6, 2024
9aa10c2
test: fix kms test
durran May 6, 2024
c99f750
test: fix fle unit test
durran May 6, 2024
fef5d55
test: fix unit tests
durran May 7, 2024
6dad21c
fix: type exports
durran May 7, 2024
5c7efde
test: fix tests
durran May 7, 2024
043620d
test: fix index test
durran May 7, 2024
c740bf2
test: fix gcp unified testS
durran May 7, 2024
89f2ecc
test: hold off on gcp unified for local env
durran May 7, 2024
1ae6cc1
test: fix fail points
durran May 8, 2024
f3098ab
test: use oidc uri in config
durran May 8, 2024
6b81a1d
Revert "test: use oidc uri in config"
durran May 10, 2024
b0fdfb7
test: use oidc config
durran May 10, 2024
dc16096
Revert "test: use oidc config"
durran May 12, 2024
94a9fb9
test: use oidc test env config
durran May 12, 2024
17f88c7
fix: lint
durran May 12, 2024
0fbca12
fix: comments
durran May 14, 2024
36f755c
test: fix unit test imports
durran May 14, 2024
158d9fe
test: fix stub
durran May 14, 2024
ae14727
fix: lint
durran May 14, 2024
00ef40a
test: add todo
durran May 15, 2024
fe2f415
fix: everything but debounce
durran May 27, 2024
d30ccd2
feat: add workflow executor
durran May 27, 2024
d4a9275
fix: use workflow executor
durran May 27, 2024
86e1323
test: no more custom uri
durran May 28, 2024
15a37d9
test: fix azure
durran May 28, 2024
6bb6d48
test: remove workflow executor
durran May 28, 2024
076db10
test: remove console logs
durran May 28, 2024
183f485
fix: locking
durran May 28, 2024
2e97a72
fix: move throttling to withLock
durran May 28, 2024
84547b0
feat: add connection cache
durran May 28, 2024
8cb6c06
test: update for DRIVERS-2915
durran May 29, 2024
7a97e6b
docs: doc token resource comma
durran May 29, 2024
5799f72
test: update connection string test
durran May 29, 2024
4913053
test: fix spec test json
durran May 29, 2024
961a1ee
fix: connection cache in access/refresh
durran May 29, 2024
b959f13
fix: dont permanently reject
durran May 29, 2024
e6ce764
feat: add timeout context
durran May 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
175 changes: 106 additions & 69 deletions .evergreen/config.in.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,58 +123,6 @@ functions:
env:
DRIVERS_TOOLS: ${DRIVERS_TOOLS}

"bootstrap oidc":
- command: ec2.assume_role
params:
role_arn: ${OIDC_AWS_ROLE_ARN}
- command: shell.exec
type: test
params:
working_dir: "src"
shell: bash
script: |
${PREPARE_SHELL}
cd "${DRIVERS_TOOLS}"/.evergreen/auth_oidc
# This is a bit confusing but the ec2.assume_role command before
# this task will overwrite these variables to a different value
# than we have set in our evergreen project config. As these are
# now specific to the OIDC ARN, we re-export for the python
# scripts.
export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
export AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}
export OIDC_TOKEN_DIR=/tmp/tokens
. ./activate-authoidcvenv.sh
python oidc_write_orchestration.py
python oidc_get_tokens.py
"setup oidc roles":
- command: subprocess.exec
params:
working_dir: src
binary: bash
args:
- .evergreen/setup-oidc-roles.sh
env:
DRIVERS_TOOLS: ${DRIVERS_TOOLS}

"run oidc tests aws":
- command: shell.exec
type: test
params:
working_dir: "src"
timeout_secs: 300
shell: bash
script: |
${PREPARE_SHELL}
OIDC_TOKEN_DIR="/tmp/tokens" \
AWS_WEB_IDENTITY_TOKEN_FILE="/tmp/tokens/test_user1" \
PROJECT_DIRECTORY="${PROJECT_DIRECTORY}" \
bash ${PROJECT_DIRECTORY}/.evergreen/run-oidc-tests.sh
"run tests":
- command: shell.exec
type: test
Expand Down Expand Up @@ -1260,23 +1208,75 @@ tasks:

- name: "oidc-auth-test-azure-latest"
commands:
- command: expansions.update
type: setup
params:
updates:
- { key: NPM_VERSION, value: "9" }
- func: "install dependencies"
- command: subprocess.exec
nbbeeken marked this conversation as resolved.
Show resolved Hide resolved
type: test
params:
working_dir: src
binary: bash
env:
DRIVERS_TOOLS: ${DRIVERS_TOOLS}
PROJECT_DIRECTORY: ${PROJECT_DIRECTORY}
AZUREOIDC_CLIENTID: ${testazureoidc_clientid}
PROVIDER_NAME: azure
ENVIRONMENT: azure
SCRIPT: run-oidc-prose-tests.sh
args:
- .evergreen/run-oidc-tests-azure.sh
- command: subprocess.exec
type: test
params:
working_dir: src
binary: bash
env:
DRIVERS_TOOLS: ${DRIVERS_TOOLS}
PROJECT_DIRECTORY: ${PROJECT_DIRECTORY}
ENVIRONMENT: azure
SCRIPT: run-oidc-unified-tests.sh
args:
- .evergreen/run-oidc-tests-azure.sh

- name: "oidc-auth-test-test-latest"
commands:
- func: "install dependencies"
- command: subprocess.exec
type: test
params:
working_dir: src
binary: bash
env:
DRIVERS_TOOLS: ${DRIVERS_TOOLS}
PROJECT_DIRECTORY: ${PROJECT_DIRECTORY}
ENVIRONMENT: test
SCRIPT: run-oidc-prose-tests.sh
args:
- .evergreen/run-oidc-tests-test.sh
- command: subprocess.exec
type: test
params:
working_dir: src
binary: bash
env:
DRIVERS_TOOLS: ${DRIVERS_TOOLS}
PROJECT_DIRECTORY: ${PROJECT_DIRECTORY}
ENVIRONMENT: test
SCRIPT: run-oidc-unified-tests.sh
args:
- .evergreen/run-oidc-tests-test.sh

- name: "oidc-auth-test-gcp-latest"
commands:
- func: "install dependencies"
- command: subprocess.exec
type: test
params:
working_dir: src
binary: bash
env:
DRIVERS_TOOLS: ${DRIVERS_TOOLS}
PROJECT_DIRECTORY: ${PROJECT_DIRECTORY}
ENVIRONMENT: gcp
SCRIPT: run-oidc-prose-tests.sh
args:
- .evergreen/run-oidc-tests-gcp.sh

- name: "test-aws-lambda-deployed"
commands:
Expand Down Expand Up @@ -1428,6 +1428,25 @@ task_groups:
tasks:
- test-azurekms-task

- name: testtestoidc_task_group
setup_group:
- func: fetch source
- command: ec2.assume_role
params:
role_arn: ${OIDC_AWS_ROLE_ARN}
- command: subprocess.exec
params:
binary: bash
include_expansions_in_env: ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_SESSION_TOKEN"]
env:
MONGODB_VERSION: "8.0"
args:
- ${DRIVERS_TOOLS}/.evergreen/auth_oidc/setup.sh
setup_group_can_fail_task: true
setup_group_timeout_secs: 1800
tasks:
- oidc-auth-test-test-latest

- name: testazureoidc_task_group
setup_group:
- func: fetch source
Expand All @@ -1437,25 +1456,43 @@ task_groups:
script: |-
set -o errexit
${PREPARE_SHELL}
export AZUREOIDC_CLIENTID="${testazureoidc_clientid}"
export AZUREOIDC_TENANTID="${testazureoic_tenantid}"
export AZUREOIDC_SECRET="${testazureoidc_secret}"
export AZUREOIDC_KEYVAULT=${testazureoidc_keyvault}
export AZUREOIDC_DRIVERS_TOOLS="$DRIVERS_TOOLS"
export AZUREOIDC_VMNAME_PREFIX="NODE_DRIVER"
$DRIVERS_TOOLS/.evergreen/auth_oidc/azure/create-and-setup-vm.sh
teardown_group:
$DRIVERS_TOOLS/.evergreen/auth_oidc/azure/setup.sh
teardown_task:
- command: shell.exec
params:
shell: bash
script: |-
${PREPARE_SHELL}
$DRIVERS_TOOLS/.evergreen/auth_oidc/azure/delete-vm.sh
$DRIVERS_TOOLS/.evergreen/auth_oidc/azure/teardown.sh
setup_group_can_fail_task: true
setup_group_timeout_secs: 1800
tasks:
- oidc-auth-test-azure-latest

- name: testgcpoidc_task_group
setup_group:
- func: fetch source
- command: shell.exec
params:
shell: bash
script: |-
set -o errexit
${PREPARE_SHELL}
export GCPOIDC_VMNAME_PREFIX="NODE_DRIVER"
$DRIVERS_TOOLS/.evergreen/auth_oidc/gcp/setup.sh
teardown_task:
- command: shell.exec
params:
shell: bash
script: |-
${PREPARE_SHELL}
$DRIVERS_TOOLS/.evergreen/auth_oidc/gcp/teardown.sh
setup_group_can_fail_task: true
setup_group_timeout_secs: 1800
tasks:
- oidc-auth-test-gcp-latest

- name: test_atlas_task_group
setup_group:
- func: fetch source
Expand All @@ -1471,7 +1508,7 @@ task_groups:
- command: expansions.update
params:
file: src/atlas-expansion.yml
teardown_group:
teardown_task:
- command: subprocess.exec
params:
working_dir: src
Expand Down Expand Up @@ -1499,7 +1536,7 @@ task_groups:
- command: expansions.update
params:
file: src/atlas-expansion.yml
teardown_group:
teardown_task:
- command: subprocess.exec
params:
working_dir: src
Expand Down