Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deadlock when mapped task with removed upstream is rerun #26518

Merged
merged 3 commits into from Sep 21, 2022

Commits on Sep 20, 2022

  1. Fix deadlock when mapped task with removed upstream is rerun

    When a dag with a mapped downstream tasks that depends on a mapped upstream tasks that have some mapped indexes
    removed is rerun, we run into a deadlock because the trigger rules evaluation is not accounting for removed
    task instances.
    
    The fix for the deadlocks was to account for the removed task instances where possible in the trigger rules
    
    In this fix, I added a case where if we set flag_upstream_failed, then for the removed task instance, the downstream of that task instance will be removed. That's if the upstream with index 3 is removed, then downstream
    with index 3 will also be removed if flag_upstream_failed is set to True.
    ephraimbuddy committed Sep 20, 2022
    Configuration menu
    Copy the full SHA
    8030bc1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    eed077a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2584bcb View commit details
    Browse the repository at this point in the history