Currently, if a dag goes missing, the scheduler continues to queue the task instances
until the executor reports the tasks as failed and then the scheduler would now set the state properly.

This change ensures that tasks are not queued when the dag file goes missing. Instead of waiting on the
executor to fail the task without explicit reason, the task fails here with the reason why it failed. Thanks to
this, the Pool's queued slots will be freed for other tasks to be queued

fixup! Do not queue tasks when the DAG goes missing

add tests

fixup! fixup! Do not queue tasks when the DAG goes missing

Change implementation to check at regular interval instead of at every loop

fixup! Change implementation to check at regular interval instead of at every loop

fixup! fixup! Change implementation to check at regular interval instead of at every loop

fixup! fixup! fixup! Change implementation to check at regular interval instead of at every loop

change configuration name

apply review suggestions

fixup! apply review suggestions

add has_dag method to DagBag and improve missing dag fail method in scheduler

Update airflow/jobs/scheduler_job.py

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

Fix dag_bag.has_dag behaviour

fixup! Fix dag_bag.has_dag behaviour

Apply suggestions from code review

Co-authored-by: Kaxil Naik <kaxilnaik@gmail.com>

fixup! Apply suggestions from code review

fixup! fixup! Apply suggestions from code review

fixup! Do not queue tasks when the DAG goes missing

Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>