Fix bug where event triggered functions failed in debug mode #5211
+64
−41
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… trigger when running in debug mode. A classic case of deadlock! Internally, the Functions Emulator maintains a work queue which processes unit of work called "tasks" with configured parallelism. When Functions Emulator runs in debug mode, the work queue runs in `SEQUENTIAL` mode where tasks are processed one by one in FIFO order. When event functions are triggered via the multicast route (e.g. storage and auth triggers), Functions Emulator submits a task per function associated with the event to the work queue where each task makes call to the event function via its HTTP endpoint. Let's call this task an "invocation task". When the Function Emulator receives an HTTP request for function invocation, it submits a new task to the work queue to handle the http request. Let's call this task a "http task". So, a call to the multicast route begets zero or more "invocation task" (one per trigger). Each "invocation task" begets exactly one "http task". The code today is written such that an "invocation task" will wait for the corresponding "http task" to complete. In debug mode this causes the invocation task to wait forever - the "http task" is stuck behind its "invocation task" in the queue. It never has chance to be processed. This PR proposes that "invocation task" doesn't wait for the corresponding "http task" to complete, effectively breaking the deadlock. The result is that "invocation task" now fire-and-forget an "http task". AFAIK, this doesn't have any negative effect because "invocation task" didn't do anything with the result of the "http task" anyway. Fixes #5008, #5050
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A classic case of deadlock!
Internally, the Functions Emulator maintains a work queue which processes unit of work called "tasks" with configured parallelism.
When Functions Emulator runs in debug mode, the work queue runs in
SEQUENTIALmode where tasks are processed one by one in FIFO order.When event functions are triggered via the multicast route (e.g. storage and auth triggers), Functions Emulator submits a task per function associated with the event to the work queue where each task makes a call to the event function via its HTTP endpoint. Let's call this task an "invocation task".
When the Function Emulator receives an HTTP request for the function invocation, it submits a new task to the work queue to handle the http request. Let's call this task a "http task".
So, a call to the multicast route begets zero or more "invocation task" (one per trigger). Each "invocation task" begets exactly one "http task".
The code today is written such that an "invocation task" will wait for the corresponding "http task" to complete.
In debug mode this causes the invocation task to wait forever - the "http task" is stuck behind its "invocation task" in the queue. It never has chance to be processed.
This PR proposes that "invocation task" doesn't wait for the corresponding "http task" to complete, effectively breaking the deadlock.
The result is that "invocation task" now fire-and-forget an "http task". AFAIK, this doesn't have any negative effect because "invocation task" didn't do anything with the result of the "http task" anyway.
Fixes #5008, #5050