From b96e1724b4d6bb84ea9253664465d4c371529a1c Mon Sep 17 00:00:00 2001 From: Hiroshi Hatake Date: Wed, 13 Apr 2022 23:50:08 +0900 Subject: [PATCH] 1.0: Add document for YAML config format Signed-off-by: Hiroshi Hatake --- .gitbook.yaml | 1 + SUMMARY.md | 1 + configuration/config-file-yaml.md | 490 ++++++++++++++++++++++++++++++ 3 files changed, 492 insertions(+) create mode 100644 configuration/config-file-yaml.md diff --git a/.gitbook.yaml b/.gitbook.yaml index 369aca69..d11bd7fd 100644 --- a/.gitbook.yaml +++ b/.gitbook.yaml @@ -36,6 +36,7 @@ redirects: v1.0/articles/format-section: configuration/format-section.md v1.0/articles/parse-section: configuration/parse-section.md v1.0/articles/config-file: configuration/config-file.md + v1.0/articles/config-file-yaml: configuration/config-file-yaml.md v1.0/articles/storage-section: configuration/storage-section.md # Deployment diff --git a/SUMMARY.md b/SUMMARY.md index 6b9e5b32..64b11918 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -19,6 +19,7 @@ * [Post Installation Guide](installation/post-installation-guide.md) * [Configuration](configuration/README.md) * [Config File Syntax](configuration/config-file.md) + * [Config File Syntax (YAML)](configuration/config-file-yaml.md) * [Routing Examples](configuration/routing-examples.md) * [Config: Common Parameters](configuration/plugin-common-parameters.md) * [Config: Parse Section](configuration/parse-section.md) diff --git a/configuration/config-file-yaml.md b/configuration/config-file-yaml.md new file mode 100644 index 00000000..00d4ac86 --- /dev/null +++ b/configuration/config-file-yaml.md @@ -0,0 +1,490 @@ +# Config File Syntax (YAML) + +This article describes the basic concepts of Fluentd configuration file syntax for yaml format. + + +## Introduction: The Lifecycle of a Fluentd Event + +Here is a brief overview of the lifecycle of a Fluentd event to help you understand the rest of this page: + +The configuration file allows the user to control the input and output behavior of Fluentd by 1\) selecting input and output plugins; and, 2\) specifying the plugin parameters. The file is required for Fluentd to operate properly. + +See also: [Lifecycle of a Fluentd Event](../quickstart/life-of-a-fluentd-event.md) + +## YAML adoption + +Fluentd starts to support YAML configuration format but this is not 1-by-1 correspondence for Fluentd config file syntax. + +Normal Fluentd configuration syntax has the following the list of directives: + +1. **`source`** directives determine the input sources +2. **`match`** directives determine the output destinations +3. **`filter`** directives determine the event processing pipelines +4. **`system`** directives set system-wide configuration +5. **`label`** directives group the output and filter for internal routing +6. **`@include`** directives include other files + +In YAML configuration world, we reconstructed them for YAML format. + +In YAML syntax, Fluentd will handle the two top level objects: + +1. **`system`** The top level object that specifies system settings +2. **`config`** Another top level object that defines data pipeline + +Under `config` object, Fluentd will handle the following elements: + +1. **`source`** elements determine the input sources +2. **`match`** elements determine the output destinations +3. **`filter`** elements determine the event processing pipelines +4. **`label`** elements group the output and filter for internal routing + +### Special YAML elements + +1. **`!include`** defines including rules for other files +2. **`!fluent/s`** defines Fluentd string format that is equivalent for double quoted string +3. **`!fluent/json`** defines Fluentd JSON format that is used for Hash type object +4. **`$tag`** defines tag for output plugin +5. **`$label`** defines label routes for input plugin +6. **`$arg`** defines directive **arg**'s equivalent objects (e.g. \) +7. **`$name`** defines label directive **name** equivalent objects (e.g. \