README.OpenSearchGenID.md

Index

• Usage
• Configuration
  • hash_id_key
  • include_tag_in_seed
  • include_time_in_seed
  • use_record_as_seed
  • use_entire_record
  • record_keys
  • separator
  • hash_type
• Advanced Usage

Usage

In your Fluentd configuration, use @type opensearch_genid. Additional configuration is optional, default values would look like this:

<filter>
  @type opensearch_genid
  hash_id_key _hash
  include_tag_in_seed false
  include_time_in_seed false
  use_record_as_seed false
  use_entire_record false
  record_keys []
  separator _
  hash_type sha1
</filter>

Configuration

hash_id_key

hash_id_key _id

You can specify generated hash storing key.

include_tag_in_seed

include_tag_in_seed true

You can specify to use tag for hash generation seed.

include_time_in_seed

include_time_in_seed true

You can specify to use time for hash generation seed.

use_record_as_seed

use_record_as_seed true

You can specify to use record in events for hash generation seed. This parameter should be used with record_keys parameter in practice.

record_keys

record_keys request_id,pipeline_id

You can specify keys which are record in events for hash generation seed. This parameter should be used with use_record_as_seed parameter in practice.

use_entire_record

use_entire_record true

You can specify to use entire record in events for hash generation seed.

separator

separator |

You can specify separator charactor to creating seed for hash generation.

hash_type

hash_type sha1

You can specify hash algorithm. Support algorithms md5, sha1, sha256, sha512. Default: sha1

Advanced Usage

Opensearch GenID plugin can handle record contents differing with the following parameters:

<filter the.awesome.your.routing.tag>
  @type opensearch_genid
  use_entire_record true
  hash_type sha1
  hash_id_key _hash
  separator _
  inc_time_as_key true
  inc_tag_as_key true
</filter>

The above configuration can handle tag, time, and record differing and generate different base64 encoded hash per record.