S3Client.builder() without credentials provider behaves differently than if set explicitly to DefaultCredentialsProvider.create()

[DocX]

Describe the bug

Using S3Client builder without credentials provider, e.g.:

        S3Client client = S3Client.builder()
                .region(Region.of("us-west-2"))
                .build();

instantiates internally DefaultCredentialsProvider with static ProfileFile that results in the credentials not being reloaded when using credentials file via AWS_SHARED_CREDENTIALS_FILE.

However passing own instance of DefaultCredentialsProvider:

        S3Client client = S3Client.builder()
                .region(Region.of("us-west-2"))
                .credentialsProvider(DefaultCredentialsProvider.create())
                .build();

works as expected, refreshing the credentials used when the credentials file is updated on disk.

Expected Behavior

The default builder should not set static ProfileFile in the default credentials provider. The two examples above should behave the same.

Current Behavior

The client builder without setting the credentials provider creates client that is not refreshing credentials from the credential file when the file updates.

Reproduction Steps

Create credentials mock file aws-sdk-bug/credentials to make the program failing until file is updated:

[default]
aws_access_key_id=test
aws_secret_access_key=test
aws_session_token=test

and set the environment variable to use the profile provider from the default provider chain:

# unset AWS_PROFILE
# unset AWS_ACCESS_KEY_ID
# unset AWS_SECRET_ACCESS_KEY
export AWS_SHARED_CREDENTIALS_FILE=aws-sdk-bug/credentials

run following java program:

public class MyClass {
    public static void main(String[] args) {
        S3Client client = S3Client.builder()
                .region(Region.of("us-west-2"))
                .build();

        ListObjectsV2Request list = ListObjectsV2Request.builder()
                .bucket("your-s3-bucket")
                .prefix("some-prefix/")
                .build();

        while(true) {
            System.out.println("Fetching list...");
            try {
                var objects = client.listObjectsV2(list);
                System.out.println("Objects count: " + objects.contents().size());
            } catch (S3Exception e) {
                System.err.println(e.awsErrorDetails().errorMessage());
            } catch (SdkClientException e) {
                System.err.println(e.getMessage());
            }
            System.out.println("");
            try {
                Thread.sleep(5000);
            } catch (InterruptedException e) {
                System.exit(0);
            }
        }
    }
}

while it is running, update the aws-sdk-bug/credentials file to contain valid credentials. The output should stop printing errors and start print the number of objects in the S3 bucket.

Possible Solution

The client builder code should use DefaultCredentialsProvider with keeping it's profileFile value null.

Additional Information/Context

No response

AWS Java SDK version used

2.25.25

JDK version used

openjdk 18.0.1 2022-04-19

Operating System and version

macOS 13.4.1

[bhoradc]

Hi @DocX,

I tried using both AWS_SHARED_CREDENTIALS_FILE and default ~/.aws/credentials but cannot reproduce the scenario you described above.

Using S3Client builder with or without credentials provider, the program doesn't read the updated (corrected) credentials saved during the execution.

Kindly let me know if I am missing anything here. Also, see if your mentioned scenario is related to this issue #1754.

Regards,
Chaitanya

[DocX]

Hello @bhoradc

Thank you for looking into this.

Using S3Client builder with or without credentials provider, the program doesn't read the updated (corrected) credentials saved during the execution.

Ok I've retested it again with the latest SDK version 2.25.25 and I cannot reproduce it anymore:

Content of credentials file: aws_access_key_id=test_test*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
---
Content of credentials file: aws_access_key_id        = ASIAYSPZOXOPWMC*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
---
Content of credentials file: aws_access_key_id        = ASIAYSPZOXOPWMC*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
---
Content of credentials file: aws_access_key_id=test_test*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
---

Perhaps the confusion was from me also testing it originally with the version we use in our actual service in production, which is 2.21.23. The issues is present there:

Content of credentials file: aws_access_key_id=test_test*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
---
Content of credentials file: aws_access_key_id=test_test*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
---
Content of credentials file: aws_access_key_id        = ASIAYSPZOXOPWMC*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
Objects count: 334
---
Content of credentials file: aws_access_key_id        = ASIAYSPZOXOPWMC*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
Objects count: 334
---
Content of credentials file: aws_access_key_id=test_test*****
Using client with default provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
Using client with DefaultCredentialsProvider.create() provider:
ERROR: The AWS Access Key Id you provided does not exist in our records.
---

Also, see if your mentioned scenario is related to this issue #1754.

Yes that seems like actual duplicate in the version 2.25.25. I am closing this then

Duplicate of #1754

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.