AWS_EC2_METADATA_SERVICE_ENDPOINT port is ignored

[liamdawson]

Describe the bug

When I set a port as part of the AWS_EC2_METADATA_SERVICE_ENDPOINT environment variable, I expect the IMDS credential provider to attempt to get credentials using that hostname/port combo.

Expected Behavior

AWS_EC2_METADATA_SERVICE_ENDPOINT=http://127.0.0.1:9099, should result in fetching credentials from 127.0.0.1:9099 (not the default port, 80)

Current Behavior

Error retrieving instance profile credentials: Failed to open TCP connection to 127.0.0.1:80 (Connection refused - connect(2) for "127.0.0.1" port 80)

Reproduction Steps

Gemfile

source "https://rubygems.org"

gem 'base64'
gem 'nokogiri'

gem 'aws-sdk-ssm', `~> 1.166.0`

repro.rb

#!/usr/bin/env ruby

require "aws-sdk-ssm"

client = Aws::SSM::Client.new(region: "ap-southeast-2")

client.get_parameter(name: "test-param")

$ bundle install
$ env AWS_CONFIG_FILE=/does/not/exist AWS_SHARED_CREDENTIALS_FILE=/does/not/exist AWS_EC2_METADATA_SERVICE_ENDPOINT=http://127.0.0.1:9099 bundle exec ./repro.rb

Error retrieving instance profile credentials: Failed to open TCP connection to 127.0.0.1:80 (Connection refused - connect(2) for "127.0.0.1" port 80)
...

Possible Solution

Try using uri.port first in:

aws-sdk-ruby/gems/aws-sdk-core/lib/aws-sdk-core/ec2_metadata.rb

Line 186 in 64ad62e

http = Net::HTTP.new(uri.hostname || @endpoint, @port || uri.port)

Or, default @port to nil.

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-core 3.191.4

Environment details (Version of Ruby, OS environment)

ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [arm64-darwin23]

[mullermp]

The environment variable is checked when fetching credentials, it exists in InstanceProfileCredentials here: https://github.com/aws/aws-sdk-ruby/blob/version-3/gems/aws-sdk-core/lib/aws-sdk-core/instance_profile_credentials.rb#L114

From your output and testing, it is indeed getting the endpoint 127.0.0.1 however the port does appear to not be correct. I checked also with EC2Metadata and it is the same behavior. Both classes do support an ip_address/endpoint option and a port option.

I will investigate the designs for these and see if this is correct behavior or not.

May I ask your use case on why you are using a custom metadata endpoint?

[mullermp]

Checking other SDKs here, this does appear to be a bug. I've put out a PR to fix this and it should be released by EOD today. In the mean time, you can use the port option on InstanceProfileCredentials when creating clients.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[liamdawson]

Thanks for fixing that!

May I ask your use case on why you are using a custom metadata endpoint?

Some utilities (e.g. aws-vault) allow simulating local IMDS servers for credentials in development, but running it to match the default endpoint and port requires admin-level permissions. Being able to change the port is required for running it without net admin privileges (and I might want to use port 80 for other things locally anyway)