Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix incorrect timeout warnings in AWS Lambda and GCP integrations #854

Merged
merged 6 commits into from Oct 14, 2020
Merged

Fix incorrect timeout warnings in AWS Lambda and GCP integrations #854

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d808d64
Changes:
Oct 12, 2020
33eecb6
Changes:
Oct 12, 2020
1806fef
Changes:
Oct 13, 2020
7e841a5
Changes:
Oct 14, 2020
80d79aa
ref: reduce indentation
rhcarvalho Oct 14, 2020
59b32d4
fix: remove trailing whitespace
rhcarvalho Oct 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions sentry_sdk/integrations/aws_lambda.py
View file
Open in desktop
Expand Up @@ -116,6 +116,13 @@ def sentry_handler(event, context, *args, **kwargs):
)
hub.capture_event(event, hint=hint)
reraise(*exc_info)
finally:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@untitaker In this "finally" block, the code to stop the thread basically works in case of :-

  1. A successful run of lambda function.
  2. When an exception is there in the code.
    But, it doesn't stop the thread in case of timeout, which is an expected behavior.

By "stop the thread" I mean explicitly stopping a thread by sending a stop signal.

if (
integration.timeout_warning
and configured_time > TIMEOUT_WARNING_BUFFER
):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that timeout_thread is only conditionally defined. That's why you need to repeat this condition here.

It would be clearer to always define timeout_thread, initially setting it to None.
That way, here it would become:

if timeout_thread is not None:
    timeout_thread.stop()

.stop() sets a "stop event" to true, unblocking and terminating the thread if still running, and doing nothing otherwise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made this change in latest commit.

timeout_thread.stop_thread = True
timeout_thread.join()
rhcarvalho marked this conversation as resolved.
Show resolved Hide resolved

return sentry_handler # type: ignore

Expand Down
6 changes: 6 additions & 0 deletions sentry_sdk/integrations/gcp.py
View file
Open in desktop
Expand Up @@ -93,6 +93,12 @@ def sentry_func(functionhandler, event, *args, **kwargs):
hub.capture_event(event, hint=hint)
reraise(*exc_info)
finally:
if (
integration.timeout_warning
and configured_time > TIMEOUT_WARNING_BUFFER
):
timeout_thread.stop_thread = True
timeout_thread.join()
# Flush out the event queue
hub.flush()

Expand Down
31 changes: 21 additions & 10 deletions sentry_sdk/utils.py
View file
Open in desktop
Expand Up @@ -891,21 +891,32 @@ def __init__(self, waiting_time, configured_timeout):
threading.Thread.__init__(self)
self.waiting_time = waiting_time
self.configured_timeout = configured_timeout
self.stop_thread = False

def run(self):
# type: () -> None

time.sleep(self.waiting_time)
raise_exception = True
threading_is_running = True
while threading_is_running and self.waiting_time > 1:
time.sleep(1)
self.waiting_time = self.waiting_time - 1
if self.stop_thread:
raise_exception = False
threading_is_running = False

integer_configured_timeout = int(self.configured_timeout)
if raise_exception:
rhcarvalho marked this conversation as resolved.
Show resolved Hide resolved
time.sleep(self.waiting_time)

# Setting up the exact integer value of configured time(in seconds)
if integer_configured_timeout < self.configured_timeout:
integer_configured_timeout = integer_configured_timeout + 1
integer_configured_timeout = int(self.configured_timeout)

# Raising Exception after timeout duration is reached
raise ServerlessTimeoutWarning(
"WARNING : Function is expected to get timed out. Configured timeout duration = {} seconds.".format(
integer_configured_timeout
# Setting up the exact integer value of configured time(in seconds)
if integer_configured_timeout < self.configured_timeout:
integer_configured_timeout = integer_configured_timeout + 1

# Raising Exception after timeout duration is reached
raise ServerlessTimeoutWarning(
"WARNING : Function is expected to get timed out. Configured timeout duration = {} seconds.".format(
integer_configured_timeout
)
)
)