Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loki: Add gzip compression to resource calls #59059

Merged
merged 3 commits into from Nov 22, 2022

Conversation

svennergr
Copy link
Contributor

What is this feature?

Loki returns gzip compressed responses in certain situations. This PR enables this compression in the Loki datasource.

Why do we need this feature?

Improved performance if e.g. label values get large.

Special notes for your reviewer:

To get Loki into a setup where compression is enabled:

  1. Add the following lines to https://github.com/grafana/grafana/blob/main/devenv/docker/blocks/loki/loki-config.yaml:
frontend:
  compress_responses: true

This enables compression on responses which are larger than 1400 bytes.
2. Create a bunch of labels to get more than 1400 bytes by changing the https://github.com/grafana/grafana/blob/main/devenv/docker/blocks/loki/data/data.js file. E.g. change the main method to:

async function main() {
  for (let step = 0; step < 300000; step++) {

    const timestampMs = new Date().getTime();
    const item = getRandomLogItem(step + 1)
    await lokiSendLogLine(timestampMs, JSON.stringify(item), {[`test${step}`]: step, place:'moon', source: 'data', instance: 'server\\1', job: '"grafana/data"'});
    // lokiSendLogLine(timestampMs, logFmtLine(item), {place:'luna', source: 'data', instance: 'server\\2', job: '"grafana/data"'});
  }
}
  1. Run make devenv sources=loki
  2. Go to explore, select Loki, select Query Builder, open the labels select-component while your browser's dev-tools are opened. Observe that the response to this request is now gzip-compressed:
    image

Copy link
Contributor

@matyax matyax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes looks good. Just wanted to leave a note that I'm trying to get more involved in go code.

@matyax matyax requested a review from a team November 21, 2022 16:18
@grafanabot
Copy link
Contributor

Copy link
Contributor

@gabor gabor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@grafanabot
Copy link
Contributor

@svennergr svennergr merged commit 08e87a2 into main Nov 22, 2022
@svennergr svennergr deleted the svennergr/loki-add-compression branch November 22, 2022 13:02
grafanabot pushed a commit that referenced this pull request Nov 22, 2022
* Loki: Add compression to `callResource`

* add missing tests

* fix formatting

(cherry picked from commit 08e87a2)
svennergr added a commit that referenced this pull request Nov 22, 2022
Loki: Add `gzip` compression to resource calls (#59059)

* Loki: Add compression to `callResource`

* add missing tests

* fix formatting

(cherry picked from commit 08e87a2)

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
GuYounes pushed a commit to paul-wurth/BIXpert that referenced this pull request Feb 8, 2023
Loki: Add `gzip` compression to resource calls (grafana#59059)

* Loki: Add compression to `callResource`

* add missing tests

* fix formatting

(cherry picked from commit 08e87a2)

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants