Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(ci): fix aws auth tests #2720

Merged
merged 4 commits into from Feb 2, 2021
Merged

chore(ci): fix aws auth tests #2720

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
3b04de6
chore(ci): fix aws auth tests
emadum Jan 27, 2021
9017a47
review feedback
emadum Jan 28, 2021
3da1e2c
cleanup
emadum Jan 28, 2021
ec398ac
point back to main drivers-evergreen-tools
emadum Jan 29, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 10 additions & 0 deletions .evergreen/config.yml
View file
Open in desktop
Expand Up @@ -243,6 +243,14 @@ functions:
"iam_auth_ec2_instance_profile" : "${iam_auth_ec2_instance_profile}"
}
EOF
upgrade boto3 aws library for python:
- command: shell.exec
type: test
params:
working_dir: src
silent: true
script: |
pip install --upgrade boto3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will upgrade boto3 for the entire system. This is bad practice because it can break applications that use the system python and even break future Evergreen tasks that run on this machine. Instead you should create and use a virtualenv, like this:

          # The aws_e2e_assume_role script requires python3 with boto3.
          virtualenv -p ${python3_binary} mongovenv
          if [ "Windows_NT" = "$OS" ]; then
            . mongovenv/Scripts/activate
          else
            . mongovenv/bin/activate
          fi
          pip install --upgrade boto3

Since this is a virtualenv you'll need to activate it before running each JS script, eg:

          # The mongovenv was created earlier.
          if [ "Windows_NT" = "$OS" ]; then
            . mongovenv/Scripts/activate
          else
            . mongovenv/bin/activate
          fi
          cd ${DRIVERS_TOOLS}/.evergreen/auth_aws
          ${MONGODB_BINARIES}/mongo aws_e2e_regular_aws.js

Ideally the drivers-evergreen-tools auth testing scripts would take care of these problems so that all drivers don't need to add this finicky boilerplate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, and thanks for fixing this bug! I was able to fix it in pymongo too: mongodb/mongo-python-driver#562

run aws auth test with regular aws credentials:
- command: shell.exec
type: test
Expand Down Expand Up @@ -1342,6 +1350,7 @@ tasks:
ORCHESTRATION_FILE: auth-aws.json
TOPOLOGY: server
- func: add aws auth variables to file
- func: upgrade boto3 aws library for python
- func: run aws auth test with regular aws credentials
- func: run aws auth test with assume role credentials
- func: run aws auth test with aws EC2 credentials
Expand All @@ -1358,6 +1367,7 @@ tasks:
ORCHESTRATION_FILE: auth-aws.json
TOPOLOGY: server
- func: add aws auth variables to file
- func: upgrade boto3 aws library for python
- func: run aws auth test with regular aws credentials
- func: run aws auth test with assume role credentials
- func: run aws auth test with aws EC2 credentials
Expand Down
9 changes: 9 additions & 0 deletions .evergreen/config.yml.in
View file
Open in desktop
Expand Up @@ -274,6 +274,15 @@ functions:
}
EOF

"upgrade boto3 aws library for python":
- command: shell.exec
type: test
params:
working_dir: "src"
silent: true
script: |
pip install --upgrade boto3

"run aws auth test with regular aws credentials":
- command: shell.exec
type: test
Expand Down
1 change: 1 addition & 0 deletions .evergreen/generate_evergreen_tasks.js
View file
Open in desktop
Expand Up @@ -303,6 +303,7 @@ AWS_AUTH_VERSIONS.forEach(VERSION => {
}
},
{ func: 'add aws auth variables to file' },
{ func: 'upgrade boto3 aws library for python' },
{ func: 'run aws auth test with regular aws credentials' },
{ func: 'run aws auth test with assume role credentials' },
{ func: 'run aws auth test with aws EC2 credentials' },
Expand Down
2 changes: 1 addition & 1 deletion .evergreen/run-mongodb-aws-ecs-test.sh
View file
Open in desktop
Expand Up @@ -16,5 +16,5 @@ export NVM_DIR="${PROJECT_DIRECTORY}/node-artifacts/nvm"
set -x

# run the tests
npm install aws4
npm install aws4
MONGODB_URI=$MONGODB_URI MONGODB_UNIFIED_TOPOLOGY=1 npx mocha test/functional/mongodb_aws.test.js