Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Assert failing if you are using local configuration for testing (i.e: localstack locally) #494

Open
ffernandezcast opened this issue Apr 7, 2020 · 1 comment · May be fixed by #1264
Open

Assert failing if you are using local configuration for testing (i.e: localstack locally) #494

ffernandezcast opened this issue Apr 7, 2020 · 1 comment · May be fixed by #1264
Labels
enhancement New feature or request

Comments

@ffernandezcast
Copy link

ffernandezcast commented Apr 7, 2020 •
edited

I'm using localstack locally to test my terraform code:

This is my provider configuration

provider "aws" {
  access_key                  = "mock_access_key"
  region                      = "eu-west-2"
  s3_force_path_style         = true
  secret_key                  = "mock_secret_key"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    apigateway     = "http://localhost:4567"
    cloudformation = "http://localhost:4581"
    cloudwatch     = "http://localhost:4582"
    dynamodb       = "http://localhost:4569"
    es             = "http://localhost:4578"
    firehose       = "http://localhost:4573"
    iam            = "http://localhost:4593"
    kinesis        = "http://localhost:4568"
    lambda         = "http://localhost:4574"
    route53        = "http://localhost:4580"
    redshift       = "http://localhost:4577"
    s3             = "http://localhost:4572"
    secretsmanager = "http://localhost:4584"
    ses            = "http://localhost:4579"
    sns            = "http://localhost:4575"
    sqs            = "http://localhost:4576"
    ssm            = "http://localhost:4583"
    stepfunctions  = "http://localhost:4585"
    sts            = "http://localhost:4592"
  }
}

I.E: I'm using aws.AssertS3BucketExists method

But I constantly receiving an error about the credentials (In my case I have fake one and receive 403). This mean the aws import dependency is not using the terraform provider configuration and continue using the default aws cli config credentials.

I have checked the code and I realized the module https://github.com/gruntwork-io/terratest/blob/master/modules/aws/auth.go doesn't have the option to use local custom services endpoint
@ffernandezcast ffernandezcast mentioned this issue Apr 7, 2020
Open
@ffernandezcast ffernandezcast changed the title Assert failing if you are using localstack locally Assert failing if you are using local configuration for testing (i.e: localstack locally) Apr 7, 2020
@brikis98 brikis98 added enhancement New feature or request help wanted labels Apr 8, 2020
This was referenced Dec 3, 2022
Closed
Closed
sverch pushed a commit to sverch/terratest that referenced this issue Mar 18, 2023
Add support for a custom aws endpoint
b990105 
Fixes gruntwork-io#494.

This makes it possible to run terratest against a custom aws endpoint.
This allows it to be used woth [Moto's standalone server
mode](http://docs.getmoto.org/en/latest/docs/server_mode.html) for
example, to test AWS modules locally without needing an AWS account or
any access to AWS.

Unfortunately these tests don't pass as is, because they would require
setting up the moto server, and I'm not sure where that setup should be
added. They do pass if the moto server is running.
sverch pushed a commit to sverch/terratest that referenced this issue Mar 18, 2023
Add support for a custom aws endpoint
41bf74c 
Fixes gruntwork-io#494.

This makes it possible to run terratest against a custom aws endpoint.
This allows it to be used woth [Moto's standalone server
mode](http://docs.getmoto.org/en/latest/docs/server_mode.html) for
example, to test AWS modules locally without needing an AWS account or
any access to AWS.

Unfortunately these tests don't pass as is, because they would require
setting up the moto server, and I'm not sure where that setup should be
added. They do pass if the moto server is running.
@sverch sverch linked a pull request Mar 18, 2023 that will close this issue
Open
4 tasks
sverch pushed a commit to sverch/terratest that referenced this issue Apr 23, 2023
Add support for a custom aws endpoint
1b79e4b 
Fixes gruntwork-io#494.

This makes it possible to run terratest against a custom aws endpoint.
This allows it to be used woth [Moto's standalone server
mode](http://docs.getmoto.org/en/latest/docs/server_mode.html) for
example, to test AWS modules locally without needing an AWS account or
any access to AWS.

Unfortunately these tests don't pass as is, because they would require
setting up the moto server, and I'm not sure where that setup should be
added. They do pass if the moto server is running.
@jmoghisi
Copy link

jmoghisi commented Jul 9, 2023 •
edited

PR #1211 and #1264 use custom env variables to drive the configuration of the AWS client. Whilst these are great contributions, they are specific to a particular provider configuration. For example, if above HCL configuration used assume_role for auth or a http_proxy then neither PR would work since the AWS client configuration would deviate from the terraform module.

Instead should the provider HCL for the module under test be parsed using hclparse to derive the AWS client configuration? i.e. the list of endpoints to configure the AWS client with would be derived from the provider HCL configuration. It would be used to derive other client settings too such as Auth, HTTP Proxy, etc.

This approach may have some other benefits:

  • keep the provider HCL as the source of truth.
  • could be extended to support other providers such as GCP / not be AWS specific.
  • not require duplicating the configuration in env vars.
  • more extensible as it would not require adding more env vars for each customization (of each provider).
  • prevent tests deviating from provider HCL under test.

@denis256 / @brikis98 I'm happy to work on this and raise a PR if you agree this would be a better approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for a custom aws endpoint sverch/terratest
4 participants
@brikis98 @jmoghisi @infraredgirl @ffernandezcast