From 56d82fc3483f500d7a1da36019888849e8784c12 Mon Sep 17 00:00:00 2001 From: Howard Yoo <32691630+howardyoo@users.noreply.github.com> Date: Thu, 17 Feb 2022 14:01:58 -0600 Subject: [PATCH] added explaining concept of logical date in DAG run docs (#21433) (cherry picked from commit 752d53860e636ead2be7c3f2044b9b312ba86b95) --- docs/apache-airflow/concepts/dags.rst | 16 ++++++++++++++++ docs/apache-airflow/dag-run.rst | 2 ++ docs/apache-airflow/faq.rst | 7 +++++++ 3 files changed, 25 insertions(+) diff --git a/docs/apache-airflow/concepts/dags.rst b/docs/apache-airflow/concepts/dags.rst index 3edaf35d5759b..e339abeda65bc 100644 --- a/docs/apache-airflow/concepts/dags.rst +++ b/docs/apache-airflow/concepts/dags.rst @@ -157,6 +157,8 @@ The ``schedule_interval`` argument takes any value that is a valid `Crontab `. If ``schedule_interval`` is not enough to express the DAG's schedule, see :doc:`Timetables `. + For more information on ``logical date``, see :ref:`data-interval` and + :ref:`faq:what-does-execution-date-mean`. Every time you run a DAG, you are creating a new instance of that DAG which Airflow calls a :doc:`DAG Run `. DAG Runs can run in parallel for the @@ -177,6 +179,20 @@ In much the same way a DAG instantiates into a DAG Run every time it's run, Tasks specified inside a DAG are also instantiated into :ref:`Task Instances ` along with it. +A DAG run will have a start date when it starts, and end date when it ends. +This period describes the time when the DAG actually 'ran.' Aside from the DAG +run's start and end date, there is another date called *logical date* +(formally known as execution date), which describes the intended time a +DAG run is scheduled or triggered. The reason why this is called +*logical* is because of the abstract nature of it having multiple meanings, +depending on the context of the DAG run itself. + +For example, if a DAG run is manually triggered by the user, its logical date would be the +date and time of which the DAG run was triggered, and the value should be equal +to DAG run's start date. However, when the DAG is being automatically scheduled, with certain +schedule interval put in place, the logical date is going to indicate the time +at which it marks the start of the data interval, where the DAG run's start +date would then be the logical date + scheduled interval. DAG Assignment -------------- diff --git a/docs/apache-airflow/dag-run.rst b/docs/apache-airflow/dag-run.rst index 90bb404696106..62555b10ed3ab 100644 --- a/docs/apache-airflow/dag-run.rst +++ b/docs/apache-airflow/dag-run.rst @@ -84,6 +84,8 @@ scheduled one interval after ``start_date``. If ``schedule_interval`` is not enough to express your DAG's schedule, logical date, or data interval, see :doc:`/concepts/timetable`. + For more information on ``logical date``, see :ref:`concepts:dag-run` and + :ref:`faq:what-does-execution-date-mean` Re-run DAG '''''''''' diff --git a/docs/apache-airflow/faq.rst b/docs/apache-airflow/faq.rst index 857e685abbda2..7f72a0d162f5f 100644 --- a/docs/apache-airflow/faq.rst +++ b/docs/apache-airflow/faq.rst @@ -214,6 +214,8 @@ This allows for a backfill on tasks that have ``depends_on_past=True`` to actually start. If this were not the case, the backfill just would not start. +.. _faq:what-does-execution-date-mean: + What does ``execution_date`` mean? ---------------------------------- @@ -248,6 +250,11 @@ misunderstandings. Note that ``ds`` (the YYYY-MM-DD form of ``data_interval_start``) refers to *date* ***string***, not *date* ***start*** as may be confusing to some. +.. tip:: + + For more information on ``logical date``, see :ref:`data-interval` and + :ref:`concepts:dag-run`. + How to create DAGs dynamically? -------------------------------