Skip to content

ydb-platform/jaeger-ydb-store

Repository files navigation

YDB storage plugin for Jaeger

PkgGoDev GoDoc tests lint Go Report Card codecov

This is a storage backend using YDB for Jaeger

components

  • collector/query: these are docker images with jaeger-collector/jaeger-query and plugin built-in into image
  • watcher (see cmd/schema/watcher): this is schema-watcher that creates new tables for spans/indexes and removes old ones

docker images

  • cr.yandex/yc/jaeger-ydb-collector
  • cr.yandex/yc/jaeger-ydb-query
  • cr.yandex/yc/jaeger-ydb-watcher

how to run

cp docker-compose.example.yml docker-compose.yml
# edit docker-compose yml to set db dsn
docker-compose up -d

environment variables

Name Type Default Description
YDB_ADDRESS string db endpoint host:port to connect to
YDB_SECURE_CONNECTION string is secure connection enabled. One of ["enabled", "disabled"]
YDB_ANONYMOUS bool false anonymous auth
YDB_TOKEN string auth token for internal purposes
YDB_SA_META_AUTH bool false use metadata to authorize requests (documentation)
YDB_SA_ID string deprecated - use YDB_SA_KEY_JSON variable. service account id for Yandex.Cloud authorization (doc on service accounts: https://cloud.yandex.com/docs/iam/concepts/users/service-accounts)
YDB_SA_KEY_ID string deprecated - use YDB_SA_KEY_JSON variable. service account key id for Yandex.Cloud authorization
YDB_SA_PRIVATE_KEY_FILE string deprecated - use YDB_SA_KEY_JSON variable. to service account private key for Yandex.Cloud authorization
YDB_SA_KEY_JSON string service account key for Yandex.Cloud authorization (doc on service accounts: https://cloud.yandex.com/docs/iam/concepts/users/service-accounts) in JSON. This variable replaces YDB_SA_ID, YDB_SA_KEY_ID and YDB_SA_PRIVATE_KEY_FILE
YDB_PATH string database path
YDB_FOLDER string folder to store data in)
YDB_CONNECT_TIMEOUT duration 10s db connect timeout
YDB_WRITE_TIMEOUT duration write queries timeout
YDB_RETRY_ATTEMPT_TIMEOUT duration attempt to write queries timeout
YDB_READ_TIMEOUT duration 10s read queries timeout
YDB_READ_QUERY_PARALLEL integer 16 controls number of parallel read subqueries
YDB_READ_OP_LIMIT integer 5000 max operation names to fetch for service
YDB_READ_SVC_LIMIT integer 1000 max service names to fetch
YDB_POOL_SIZE integer 100 db session pool size
YDB_QUERY_CACHE_SIZE integer 50 db query cache size
YDB_WRITER_BUFFER_SIZE integer 1000 span buffer size for batch writer
YDB_WRITER_BATCH_SIZE integer 100 number of spans in batch write calls
YDB_WRITER_BATCH_WORKERS integer 10 number of workers processing batch writes
YDB_INDEXER_BUFFER_SIZE integer 1000 span buffer size for indexer
YDB_INDEXER_MAX_TRACES integer 100 maximum trace_id count in a single index record
YDB_INDEXER_MAX_TTL duration 5s maximum amount of time for indexer to batch trace_ids for index records
YDB_SCHEMA_NUM_PARTITIONS integer 10 number of partitioned tables per day. Changing it requires recreating full data set

Configuration options can be passed via config file. Use --grpc-storage-plugin.configuration-file to pass configuration to YDB Plugin. In case of watcher use --config for the same purpose.

schema watcher configuration

Name Type Default Description
WATCHER_AGE duration 24h delete partition tables older than this value
WATCHER_INTERVAL duration 5m check interval
YDB_FEATURE_SPLIT_BY_LOAD bool false enable table split by load feature
YDB_FEATURE_COMPRESSION bool false enable table compression feature, used for span storage

conference talks