Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RecursionError: Maximum recursion depth exceeded #4061

Open
MustaphaU opened this issue Mar 22, 2024 · 1 comment
Open

RecursionError: Maximum recursion depth exceeded #4061

MustaphaU opened this issue Mar 22, 2024 · 1 comment
Assignees
Labels
bug This issue is a confirmed bug. investigating This issue is being investigated and/or work is in progress to resolve the issue. p2 This is a standard priority issue s3

Comments

@MustaphaU
Copy link

Describe the bug

I need help with this recursion error maximum recursion depth exceeded from boto3. This occurs when I initialize an s3 client in my inference script to allow me read s3 objects. Your insights will be deeply appreciated! Similar issue was posted on stackoverflow 2 months ago here: https://stackoverflow.com/questions/77786275/aws-sagemaker-endpoint-maximum-recursion-depth-exceeded-error-when-calling-boto

Here is the relevant code block responsible for the error:

def get_video_bytes_from_s3(bucket_name, key):
    s3_client = boto3.client('s3')
    try:
        video_object = s3_client.get_object(Bucket= bucket_name, Key=key)
        video_bytes = video_object['Body'].read()
        return video_bytes
    except Exception as e:
        print(f"Failed to fetch video from S3: {e}")

Expected Behavior

The s3 client created to enable access to the s3 objects

Current Behavior

Here is the full error log:

Traceback (most recent call last):
  File "/sagemaker/python_service.py", line 423, in _handle_invocation_post
    res.body, res.content_type = handlers(data, context)
  File "/opt/ml/model/code/inference.py", line 156, in handler
    video_bytes = get_video_bytes_from_s3(key)
  File "/opt/ml/model/code/inference.py", line 16, in get_video_bytes_from_s3
    s3_client = boto3.client('s3')
  File "/usr/local/lib/python3.10/site-packages/boto3/__init__.py", line 92, in client
    return _get_default_session().client(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/boto3/session.py", line 299, in client
    return self._session.create_client(
  File "/usr/local/lib/python3.10/site-packages/botocore/session.py", line 997, in create_client
    client = client_creator.create_client(
  File "/usr/local/lib/python3.10/site-packages/botocore/client.py", line 159, in create_client
    client_args = self._get_client_args(
  File "/usr/local/lib/python3.10/site-packages/botocore/client.py", line 490, in _get_client_args
    return args_creator.get_client_args(
  File "/usr/local/lib/python3.10/site-packages/botocore/args.py", line 137, in get_client_args
    endpoint = endpoint_creator.create_endpoint(
  File "/usr/local/lib/python3.10/site-packages/botocore/endpoint.py", line 409, in create_endpoint
    http_session = http_session_cls(
  File "/usr/local/lib/python3.10/site-packages/botocore/httpsession.py", line 323, in __init__
    self._manager = PoolManager(**self._get_pool_manager_kwargs())
  File "/usr/local/lib/python3.10/site-packages/botocore/httpsession.py", line 341, in _get_pool_manager_kwargs
    'ssl_context': self._get_ssl_context(),
  File "/usr/local/lib/python3.10/site-packages/botocore/httpsession.py", line 350, in _get_ssl_context
    return create_urllib3_context()
  File "/usr/local/lib/python3.10/site-packages/botocore/httpsession.py", line 139, in create_urllib3_context
    context.options |= options
  File "/usr/local/lib/python3.10/ssl.py", line 620, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/local/lib/python3.10/ssl.py", line 620, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/usr/local/lib/python3.10/ssl.py", line 620, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 479 more times]

Reproduction Steps

simply initializing an s3 client within an inference script like so:

s3_client = boto3.client('s3')

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.34.55

Environment details (OS name and version, etc.)

Sagemaker endpoint for Tensorflow serving

@MustaphaU MustaphaU added bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged. labels Mar 22, 2024
@avishwanathan88
Copy link

avishwanathan88 commented Mar 30, 2024

I am facing this maximum recursion depth issue suddenly as well when trying to check if the object exists in the s3 bucket using

s3_client.head_object(Bucket=bucket_name, Key=key)

It used to work before but not sure if something changed suddenly.
the s3 client is created using

boto3.client(service_name='s3',
                                 use_ssl=False,
                                 region_name=region,
                                 endpoint_url=endpoint_url,
                                 aws_access_key_id=key_id,
                                 aws_secret_access_key=access_key,
                                 config=Config(
                                    s3={'addressing_style': 'path'},
                                    signature_version='s3v4'))

@RyanFitzSimmonsAK RyanFitzSimmonsAK self-assigned this May 7, 2024
@RyanFitzSimmonsAK RyanFitzSimmonsAK added investigating This issue is being investigated and/or work is in progress to resolve the issue. s3 p2 This is a standard priority issue and removed needs-triage This issue or PR still needs to be triaged. labels May 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a confirmed bug. investigating This issue is being investigated and/or work is in progress to resolve the issue. p2 This is a standard priority issue s3
Projects
None yet
Development

No branches or pull requests

3 participants