Skip to content

Commit

Permalink
config_wrapper: Add support for named stores (#7946)
Browse files Browse the repository at this point in the history
Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>

**What this PR does / why we need it**:
This PR adds support for named stores which allows users to define
multiple object store configurations for a storage type. Named stores
can be referred to from other sections of the config similar to how
provider type is being used `i.e aws, gcs...`. Named store reference
should be of the format: `<store_type>.<key_name>`

Below is an example configuration with two named stores defined for aws
`store-1, store-2`. Schema config is referring to one of them as
`aws.store-1`
```
storage_config:
  aws:
    endpoint: s3://common-bucket
    region: us-east1
    access_key_id: abc123
    secret_access_key: def789
  named_stores:
    aws:
      store-1:
        endpoint: s3://foo-bucket
        region: us-west1
        access_key_id: 123abc
        secret_access_key: 789def
      store-2:
        endpoint: s3://bar-bucket
        region: us-west2
        access_key_id: 456def
        secret_access_key: 789abc
schema_config:
    configs:
        - from: "2020-07-31"
            index:
              period: 24h
              prefix: loki_prod_index_
            object_store: aws.store-1
            schema: v11
            store: boltdb-shipper
```
**Which issue(s) this PR fixes**:
Fixes #7276

**Special notes for your reviewer**:
Configuration defined in common storage config has no effect on the
named stores. Common storage config only overrides storageConfig if it's
not explicitly set - same behavior as before. This should be a
non-intrusive change.

Signed-off-by: Ashwanth Goli <iamashwanth@gmail.com>
  • Loading branch information
ashwanthgoli committed Jan 10, 2023
1 parent 6ba4b84 commit b3e038c
Show file tree
Hide file tree
Showing 9 changed files with 645 additions and 208 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
* [7964](https://github.com/grafana/loki/pull/7964) **slim-bean**: Add a `since` query parameter to allow querying based on relative time.
* [7989](https://github.com/grafana/loki/pull/7989) **liguozhong**: logql support `sort` and `sort_desc`.
* [7997](https://github.com/grafana/loki/pull/7997) **kavirajk**: fix(promtail): Fix cri tags extra new lines when joining partial lines
* [7946](https://github.com/grafana/loki/pull/7946) **ashwanthgoli** config: Add support for named stores
* [8027](https://github.com/grafana/loki/pull/8027) **kavirajk**: chore(promtail): Make `batchwait` and `batchsize` config explicit with yaml tags
* [7978](https://github.com/grafana/loki/pull/7978) **chaudum**: Shut down query frontend gracefully to allow inflight requests to complete.
* [8047](https://github.com/grafana/loki/pull/8047) **bboreham**: Dashboards: add k8s resource requests to CPU and memory panels.
Expand Down

0 comments on commit b3e038c

Please sign in to comment.