Temporary credential caching

[farski]

Describe the feature

It would be great if the Ruby SDK implemented temporary credential caching, similar to what boto3 or the AWS CLI do. With those libraries, when working with a command line tool that uses an AWS profile that requires assuming a role and MFA), the library will cache the temporary credentials for some period (e.g., in ~/.aws/boto3/cache).

Use Case

As per AWS best practices, we use many AWS accounts in our organization, and we access those accounts by assuming roles in each account from users in a management account. We also require MFA to be enabled to access our accounts.

Currently, as best I can tell, the Ruby SDK does not do any sort of credential caching. This means that if you build command line tools with it, and run those tools back-to-back, or even make multiple API calls through the SDK in a single script, the user will be prompted multiple times for their MFA code.

When working with the AWS CLI or tools based on boto3, this doesn't happen, because those libraries cache the temporary credentials, and subsequent calls can reuse those credentials.

Proposed Solution

The existing solution in boto3 is not particularly hard to replicate.

It basically makes a cache key from a hash of the parameters to the AssumeRole call, with some simple logic to deal with edge cases, and then subsequent calls to AssumeRole with matching parameters would find the cache, and use the credentials if they are not expired.

Other Information

If this does exist in some optional way in the SDK already, I haven't been able to find it. It definitely does not appear to be the default value, as I have never seen the Ruby SDK cache any credentials.

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

3.1.0

Environment details (OS name and version, etc.)

macOS 12.4

[rittneje]

The Ruby SDK will cache role credentials and reuse them until they are within 10 minutes of expiring. https://github.com/aws/aws-sdk-ruby/blob/version-3/gems/aws-sdk-core/lib/aws-sdk-core/refreshing_credentials.rb

It does not, however, persist the cache onto the file system, meaning credentials will not be cached across separate processes.

[mullermp]

Yes - the SDK caches in memory but not on the file system. CLI is a special case as it's not really an SDK. The Python SDK powers CLI so they are essentially the same. If we wrote out to disk, other SDKs would have to understand and respect it, and writing out to the file becomes very tricky (i.e. race conditions).

For updating token codes, have you looked at the before_refresh callback?

aws-sdk-ruby/gems/aws-sdk-core/lib/aws-sdk-core/assume_role_credentials.rb

Lines 31 to 40 in 485d4a7

	# @option options [Callable] before_refresh Proc called before
	# credentials are refreshed. Useful for updating tokens.
	# `before_refresh` is called when AWS credentials are
	# required and need to be refreshed. Tokens can be refreshed using
	# the following example:
	#
	# before_refresh = Proc.new do |assume_role_credentials| do
	# assume_role_credentials.assume_role_params['token_code'] = update_token
	# end
	#

[farski]

I don't know that there needs to be any expectation that other SDKs respect this SDK's cache. The Python SDK caches in a ~/.aws/boto3 directory, so I don't think it is expecting other SDKs to respect its cache.

[mullermp]

I see what you mean, so you'd want multiple Ruby applications (using the SDK) to have cached credentials? Does using the shared config file or using a gem-ified module shared across applications not help in this case?

[farski]

Currently I'm doing this (L36 to L119), which I'm sure could be cleaned up a bit, but it's still not ideal to have to implement it in each script. That functionality certainly could be moved into Gem. I opened this feature request because to me it seems like something that should be natively supported.

Also worth noting that the Python SDK and the CLI cache their credentials to ~/.aws/boto3/cache and ~/.aws/cli/cache, respectively, so there's precedent of various libraries not sharing a cache, and also at least some evidence that the Python SDK isn't doing the caching simply because it powers the CLI, or one would expect that when the SDK were being used in a non-CLI context, it wouldn't cache.

As a Ruby shop, we write a lot of AWS scripts, and often find ourselves choosing not to use Ruby, even though it would be preferred, because the CLI and Python are more conducive to developer workflows. Having this feature natively would check a big box in creating more parity.

[mullermp]

I think your feature request is valid and certainly something we should probably support - it needs some investigation. I want to raise this with CLI/Python team and get their motivations, and possibly look to see if other SDKs should be implementing this too.

[rittneje]

To offer an opposing view point, writing credentials to the file system is not something any SDK should do by default for security reasons. Plus this is only useful if you are attempting to coordinate between multiple processes. It makes sense for the AWS CLI where you will be continually invoking one-off commands, but not as much for a fully fledged application.

[farski]

@rittneje For sure I wouldn't want this to happen when, for example, the Ruby SDK were being used within a Rails app. I think an opt-in configuration option on the client, session, or Aws.shared_config would be reasonable.