Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take into account removed upstream tasks for non-mapped tasks. #33915

Closed

Conversation

avkirilishin
Copy link
Contributor

@avkirilishin avkirilishin commented Aug 30, 2023

closes: #33164

@ephraimbuddy Do you remember, why if ti.map_index > -1: was added?


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@uranusjr
Copy link
Member

The map_index > -1 condition ensures this only handles mapped task, since e91637f intends to only fix a deadlock that affects mapped tasks. So the question here is, did things work correctly prior to #26518?

@avkirilishin
Copy link
Contributor Author

The map_index > -1 condition ensures this only handles mapped task, since e91637f intends to only fix a deadlock that affects mapped tasks

So, in this PR, we extend the same behaviour to the usual tasks.

So the question here is, did things work correctly prior to #26518?

I don't think so because removed tasks were not taken into account at all, regardless of downstream task type. But I can check it later.

@hussein-awala
Copy link
Member

I suggested a similar change in a different PR. But yes, we have to check if we can reproduce the deadlock and check if it's fixed after this change before merging.

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Oct 16, 2023
@github-actions github-actions bot closed this Oct 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Stale PRs per the .github/workflows/stale.yml policy file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Removed mapped tasks block all_success trigger rule
3 participants