AWS Bedrock ReadTimeout NetworkingError TCPSocket closed

[datasciencedavid]

Describe the bug

API requests to AWS Bedrock timeout in 1-2 minutes when prompt is too large. Is there a way to increase the timeout? I canot find a way to change the timeout in the Ruby SDK docs.

Expected Behavior

API requests do not timeout unless the request takes an absurd amount of time. A few minutes is an expected wait time for requests with large prompts.

Current Behavior

This is the timeout error:
Crashed: [Seahorse::Client::NetworkingError] Net::ReadTimeout with #<TCPSocket:(closed)>: /usr/local/lib/ruby/2.7.0/net/protocol.rb:217:in `rbuf_fill' /usr/local/lib/ruby/2.7.0/net/protocol.rb:191:in `readuntil' /usr/local/lib/ruby/2.7.0/net/protocol.rb:201:in `readline' /usr/local/lib/ruby/2.7.0/net/http/response.rb:42:in `read_status_line' /usr/local/lib/ruby/2.7.0/net/http/response.rb:31:in `read_new' /usr/local/lib/ruby/2.7.0/net/http.rb:1528:in `block in transport_request' /usr/local/lib/ruby/2.7.0/net/http.rb:1519:in `catch' /usr/local/lib/ruby/2.7.0/net/http.rb:1519:in `transport_request' /usr/local/lib/ruby/2.7.0/net/http.rb:1492:in `request' /usr/local/bundle/gems/aws-sdk-core-3.184.0/lib/seahorse/client/net_http/connection_pool.rb:348:in `request'

Reproduction Steps

Make an invoke_model API request to AWS Bedrock (Claude version 2) with a prompt containing 25K+ tokens

Possible Solution

No response

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-bedrockruntime, aws-sdk-core

Environment details (Version of Ruby, OS environment)

Ruby 2.7

[datasciencedavid]

I was able to confirm that the crash is caused by the request timing out. It is not well-documented, but after hours of searching online I finally found a way to increase the timeout thanks to someone on StackOverflow:
Aws::BedrockRuntime::Client.new(http_read_timeout: 480)

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[mullermp]

This option should be documented on the Client.