You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
we are trying to use the test method for a given dag in our integration tests. However, when one of the task fails, the code keeps retrying it indefinitely. I've created a small example for illustration:
import datetime
from airflow import DAG, AirflowException
from airflow.decorators import task
from airflow.operators.empty import EmptyOperator
from common.tagging import default_args
@task(task_id="test_task")
def test_task():
print("Failing Task")
raise AirflowException("Failing")
def create_dag():
with DAG(
"test1",
schedule=None,
start_date=datetime.datetime(2021, 1, 1),
tags=["test"],
catchup=False,
max_active_runs=1,
default_args=default_args
) as dag:
start = EmptyOperator(
task_id="start"
)
test = test_task()
end = EmptyOperator(
task_id="end"
)
start >> test >> end
return dag
def test_incremental_tagging_integration():
dag = create_dag()
print("Testing integration dag execution with updated data")
dag.test()
Is this expected? Is there a way to limit the number of retries in this case as when using the Airflow UI?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi all,
we are trying to use the test method for a given dag in our integration tests. However, when one of the task fails, the code keeps retrying it indefinitely. I've created a small example for illustration:
Is this expected? Is there a way to limit the number of retries in this case as when using the Airflow UI?
Beta Was this translation helpful? Give feedback.
All reactions