Avoid unintentional data loss when deleting DAGs

[SamWheating]

We encountered some data loss today due to a user deleting a DAG from the UI called project.load, which then deleted all of the history from other DAGs called project.load.bigquery and project.load.trino, which also caused them to run unexpectedly due to the resetting of run history.

Note - we don't use SubDAGs, we're just using . in the DAG ID as a separator for a hierarchical naming system.

As it turns out, deleting a DAG my_dag will delete all of the metadata for any DAG which starts with my_dag., as it is assumed that the latter are subdags of the former:

            cond = or_(model.dag_id == dag_id, model.dag_id.like(dag_id + ".%"))
            count += session.query(model).filter(cond).delete(synchronize_session='fetch')

This isn't always the case.

Anyways, this PR changes the delete_dag function so that it only deletes the intended DAG and DAGs starting with <dag_id>. which are also SubDAGs. I think that there may still be some other edge cases where DAGs can be unintentionally deleted, but this patches the most apparent case.

This can all be cleaned up even more once the deprecation of SubDAGs is complete (Airflow 3?)

[github-actions]

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

[potiuk]

Small NIT :)

[potiuk]

We encountered some data loss today due to a user deleting a DAG from the UI called project.load, which then deleted all of the history from other DAGs called project.load.bigquery and project.load.trino, which also caused them to run unexpectedly due to the resetting of run history.

Side-comment: I was almost sure that DAG_id cannot contain "." (precisely because of the subdag convention), But now I see this i not the case :). We excluded '.' for task group (for the reason task groups are also . separated) but not for the task ids:

KEY_REGEX = re.compile(r'^[\w.-]+$')
GROUP_KEY_REGEX = re.compile(r'^[\w-]+$')

Nice catch.

[kaxil]

        or_(
            DagModel.dag_id == dag_id,,
            DagModel.root_dag_id == dag_id

might have also worked @SamWheating