-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using use_credential_provider: aws with instance profiles gives HTTP error 400 #226
Comments
huh, k-- these kinds of issues are very hard for me to debug since I don't have ready access to the environment in question; I think the best bet here is to open a python shell/run a simple script that calls the relevant function in dbt-duckdb (which is defined here) and see if we can deduce where the error is coming from, e.g.:
|
...working on it, I've put together a simple Docker image to try out your approach, gotta get it running in AWS Batch to do the real deal |
Running on Batch prints out a dict with keys |
I tried the following inside a Fargate container:
...and ran a query like the one my DBT project gets the error for, but it works fine. Maybe elsewhere the adapter is doing something that interferes with this? I looked at e.g. |
Hrm-- maybe related to this? duckdb/duckdb#6563 |
My apologies, turns out my reproduction efforts failed to reproduce one of the elements of the original failure: the jobs with the errors are running in ECS cluster with EC2 nodes, but my earlier reproduction attempts ran in Fargate. I see this perhaps crucial difference:
And I can reproduce the HTTP 400 outside of AWS by not setting the s3_region. |
Ah, good to know-- and nice detective work! |
Thinking I should add some logging in that |
...and also that it's possible that this extension may run into some of the same issues: https://github.com/duckdblabs/duckdb_aws |
I've just confirmed a working workaround for the issue:
|
Actually I think we have a bug in the httpfs extension here. Its requests to the S3 endpoint are erroring with inscrutable errors in a scenario where other tools—most notably example boto3 the official AWS CLI—work fine. I wonder e.g. if it's sending an empty string for the region when it's supposed to either send none or send a valid one. |
When trying to use the
aws
target in the linked profile either from a ECS container or an EC2 instance that's known to have the correct permissions, we get nevertheless an HTTP 400 error:But if in the same EC2 instance I instead configure it this way, with credentials I get from
aws sts get-session-token
, it works:The text was updated successfully, but these errors were encountered: