Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added explaining concept of logical date in DAG run #21433

Merged
merged 13 commits into from Feb 17, 2022
16 changes: 16 additions & 0 deletions docs/apache-airflow/concepts/dags.rst
Expand Up @@ -157,6 +157,8 @@ The ``schedule_interval`` argument takes any value that is a valid `Crontab <htt
For more information on ``schedule_interval`` values, see :doc:`DAG Run </dag-run>`.

If ``schedule_interval`` is not enough to express the DAG's schedule, see :doc:`Timetables </howto/timetable>`.
For more information on ``logical date``, see :ref:`data-interval` and
:ref:`faq:what-does-execution-date-mean`.

Every time you run a DAG, you are creating a new instance of that DAG which
Airflow calls a :doc:`DAG Run </dag-run>`. DAG Runs can run in parallel for the
Expand All @@ -177,6 +179,20 @@ In much the same way a DAG instantiates into a DAG Run every time it's run,
Tasks specified inside a DAG are also instantiated into
:ref:`Task Instances <concepts:task-instances>` along with it.

A DAG run will have a start date when it starts, and end date when it ends.
This period describes the time when the DAG actually 'ran.' Aside from the DAG
run's start and end date, there is another date called *logical date*
(formally known as execution date), which describes the intended time a
DAG run is scheduled or triggered. The reason why this is called
*logical* is because of the abstract nature of it having multiple meanings,
depending on the context of the DAG run itself.

For example, if a DAG run is manually triggered by the user, its logical date would be the
date and time of which the DAG run was triggered, and the value should be equal
to DAG run's start date. However, when the DAG is being automatically scheduled, with certain
schedule interval put in place, the logical date is going to indicate the time
at which it marks the start of the data interval, where the DAG run's start
date would then be the logical date + scheduled interval.

DAG Assignment
--------------
Expand Down
2 changes: 2 additions & 0 deletions docs/apache-airflow/dag-run.rst
Expand Up @@ -84,6 +84,8 @@ scheduled one interval after ``start_date``.

If ``schedule_interval`` is not enough to express your DAG's schedule,
logical date, or data interval, see :doc:`/concepts/timetable`.
For more information on ``logical date``, see :ref:`concepts:dag-run` and
:ref:`faq:what-does-execution-date-mean`

Re-run DAG
''''''''''
Expand Down
7 changes: 7 additions & 0 deletions docs/apache-airflow/faq.rst
Expand Up @@ -243,6 +243,8 @@ This allows for a backfill on tasks that have ``depends_on_past=True`` to
actually start. If this were not the case, the backfill just would not start.


.. _faq:what-does-execution-date-mean:

What does ``execution_date`` mean?
----------------------------------

Expand Down Expand Up @@ -277,6 +279,11 @@ misunderstandings.
Note that ``ds`` (the YYYY-MM-DD form of ``data_interval_start``) refers to
*date* ***string***, not *date* ***start*** as may be confusing to some.

.. tip::

For more information on ``logical date``, see :ref:`data-interval` and
:ref:`concepts:dag-run`.


How to create DAGs dynamically?
-------------------------------
Expand Down