From 7b4c80aa38d98f863004974c8d95987766a49ad4 Mon Sep 17 00:00:00 2001 From: Kaxil Naik Date: Wed, 21 Jul 2021 21:16:26 +0100 Subject: [PATCH] Doc: Recommend using same configs on all Airflow components (#17146) Some users seem to be using different configs for different components. This doc makes it clear on what we recommend. Context: https://github.com/apache/airflow/pull/16754#issuecomment-884419057 >I'm new to Airflow and I haven't seen an explicit part within the docs that says "all configuration must be shared between airflow components". In fact, all configuration keys are prefixed with some naming scheme which led me to think that some are common, some are for webserver etc... I personally don't like to have common configuration because changing a single setting requires a change in all components (hence restart) but I get that this is intentional for Airflow. I think this can be better communicated throughout the docs. GitOrigin-RevId: 3a263ab5b0b8b8b796d96f060e6ccd4a0e379bf3 --- docs/apache-airflow/configurations-ref.rst | 5 +++++ docs/apache-airflow/howto/set-config.rst | 6 ++++++ 2 files changed, 11 insertions(+) diff --git a/docs/apache-airflow/configurations-ref.rst b/docs/apache-airflow/configurations-ref.rst index d9f76904825..63c5c96bfc1 100644 --- a/docs/apache-airflow/configurations-ref.rst +++ b/docs/apache-airflow/configurations-ref.rst @@ -22,6 +22,11 @@ Configuration Reference This page contains the list of all the available Airflow configurations that you can set in ``airflow.cfg`` file or using environment variables. +Use the same configuration across all the Airflow components. While each component +does not require all, some configurations need to be same otherwise they would not +work as expected. A good example for that is :ref:`secret_key` which +should be same on the Webserver and Worker to allow Webserver to fetch logs from Worker. + .. note:: For more information on setting the configuration, see :doc:`howto/set-config` diff --git a/docs/apache-airflow/howto/set-config.rst b/docs/apache-airflow/howto/set-config.rst index d36a96f741e..03cf0dea460 100644 --- a/docs/apache-airflow/howto/set-config.rst +++ b/docs/apache-airflow/howto/set-config.rst @@ -115,3 +115,9 @@ the example below. .. note:: See :doc:`../modules_management` for details on how Python and Airflow manage modules. + +.. note:: + Use the same configuration across all the Airflow components. While each component + does not require all, some configurations need to be same otherwise they would not + work as expected. A good example for that is :ref:`secret_key` which + should be same on the Webserver and Worker to allow Webserver to fetch logs from Worker.