diff --git a/docs/apache-airflow/security/secrets/index.rst b/docs/apache-airflow/security/secrets/index.rst index 96d576bc5a8b9..fb645a9c188cb 100644 --- a/docs/apache-airflow/security/secrets/index.rst +++ b/docs/apache-airflow/security/secrets/index.rst @@ -34,66 +34,4 @@ The following are particularly protected: Encryption at rest Using external Secret stores - -.. _security:mask-sensitive-values: - -Masking sensitive data ----------------------- - -Airflow will by default mask Connection passwords and sensitive Variables and keys from a Connection's -extra (JSON) field when they appear in Task logs, in the Variable and in the Rendered fields views of the UI. - -It does this by looking for the specific *value* appearing anywhere in your output. This means that if you -have a connection with a password of ``a``, then every instance of the letter a in your logs will be replaced -with ``***``. - -To disable masking you can setting :ref:`config:core__hide_sensitive_var_conn_fields` to false. - -The automatic masking is triggered by Connection or Variable access. This means that if you pass a sensitive -value via XCom or any other side-channel it will not be masked when printed in the downstream task. - -Sensitive field names -""""""""""""""""""""" - -When masking is enabled, Airflow will always mask the password field of every Connection that is accessed by a -task. - -It will also mask the value of a Variable, or the field of a Connection's extra JSON blob if the name contains -any words in ('password', 'secret', 'passwd', 'authorization', 'api_key', 'apikey', 'access_token'). This list -can also be extended: - -.. code-block:: ini - - [core] - sensitive_var_conn_names = comma,separated,sensitive,names - -Adding your own masks -""""""""""""""""""""" - -If you want to mask an additional secret that is already masked by one of the above methods, you can do it in -your DAG file or operator's ``execute`` function using the ``mask_secret`` function. For example: - -.. code-block:: python - - @task - def my_func(): - from airflow.utils.log.secrets_masker import mask_secret - - mask_secret("custom_value") - - ... - -or - -.. code-block:: python - - - class MyOperator(BaseOperator): - def execute(self, context): - from airflow.utils.log.secrets_masker import mask_secret - - mask_secret("custom_value") - - ... - -The mask must be set before any log/output is produced to have any effect. + mask-sensitive-values diff --git a/docs/apache-airflow/security/secrets/mask-sensitive-values.rst b/docs/apache-airflow/security/secrets/mask-sensitive-values.rst new file mode 100644 index 0000000000000..9f994380e04d3 --- /dev/null +++ b/docs/apache-airflow/security/secrets/mask-sensitive-values.rst @@ -0,0 +1,79 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. _security:mask-sensitive-values: + +Masking sensitive data +---------------------- + +Airflow will by default mask Connection passwords and sensitive Variables and keys from a Connection's +extra (JSON) field when they appear in Task logs, in the Variable and in the Rendered fields views of the UI. + +It does this by looking for the specific *value* appearing anywhere in your output. This means that if you +have a connection with a password of ``a``, then every instance of the letter a in your logs will be replaced +with ``***``. + +To disable masking you can set :ref:`config:core__hide_sensitive_var_conn_fields` to false. + +The automatic masking is triggered by Connection or Variable access. This means that if you pass a sensitive +value via XCom or any other side-channel it will not be masked when printed in the downstream task. + +Sensitive field names +""""""""""""""""""""" + +When masking is enabled, Airflow will always mask the password field of every Connection that is accessed by a +task. + +It will also mask the value of a Variable, or the field of a Connection's extra JSON blob if the name contains +any words in ('access_token', 'api_key', 'apikey','authorization', 'passphrase', 'passwd', +'password', 'private_key', 'secret', 'token'). This list can also be extended: + +.. code-block:: ini + + [core] + sensitive_var_conn_names = comma,separated,sensitive,names + +Adding your own masks +""""""""""""""""""""" + +If you want to mask an additional secret that is already masked by one of the above methods, you can do it in +your DAG file or operator's ``execute`` function using the ``mask_secret`` function. For example: + +.. code-block:: python + + @task + def my_func(): + from airflow.utils.log.secrets_masker import mask_secret + + mask_secret("custom_value") + + ... + +or + +.. code-block:: python + + + class MyOperator(BaseOperator): + def execute(self, context): + from airflow.utils.log.secrets_masker import mask_secret + + mask_secret("custom_value") + + ... + +The mask must be set before any log/output is produced to have any effect.