dannykopping/remove cache stats #14
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- milli -> nano conversion in the querier - add deletes to store requests from the ingester
* Add some comments Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use deletes in tail Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Change error message Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
* fix(gateway): make DNS resolver configurable Signed-off-by: Ivan Rizzante <i.rizzante@gmail.com> * fix: lint
This fixes a bug in the index gateway when querying values for a label.
Since the gRPC handler for LabelValuesForMetricName in the index gateway
allows empty matchers in the LabelValuesForMetricNameRequest, we need to
check if the matchers string is an empty matcher (`{}`) before we parse
the string.
This bug was introduced with the index gateway api refactoring in #5892
Fixes: #5965
Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
when index gateway does not implement the new API. This bug did not yield an error but only returned incorrect results because both functions `c.IndexStore.LabelNamesForMetricName` and `c.IndexStore.LabelValuesForMetricName` return `([]string, error)`. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Signed-off-by: irizzant <i.rizzante@gmail.com>
* Restrict gRPC calls to ingesters when requesting series if there is no overlap within the query_ingesters_within value * Add tests for isWithinIngesterMaxLookbackPeriod * Adding tests for calculateIngesterMaxLookbackPeriod * Only send query for range overlapping with ingesters * Restrict gRPC calls to ingesters when requesting series if there is no overlap within the query_ingesters_within value * Add tests for isWithinIngesterMaxLookbackPeriod * Adding tests for calculateIngesterMaxLookbackPeriod * Only send query for range overlapping with ingesters * Add test for when to query ingester and storage * Fix deadlock in Series when just queriying the ingesters * Refactored slightly for better readability * Added CHANGELOG entry * Fix typo * Rephrase test cases names * Move block to outer loop Co-authored-by: Salva Corts <salva.corts@grafana.com>
* Generate cache invalidation numbers in the delete store * Get cache generation numbers from the store on request * changlog * rename tombstones to something more meaningful * User invisible module * query frontend relies on a compactor to get the cache generation number * fix serialization * source -> name * lint errors * lint errors * log non-200 responses * add jsonnet changes * lint * review feedback * review feedback * client rename
This is part 1 of 2 PRs and updates the build image to version 0.10.4 and adding the buf utility which will later be used for formatting *.proto files
* Disable calling new index-gateway client's API. * Remove infinite recursive call * Correct mock setup for test
* Remove empty match within concat of regex * add back tests
* Revendor grafana/tail. - This revendoring brings the fix to relative symlinks, as described in #3374. * Prune package from `go mod tidy`. * noop
…ranteed query sort order when receiving entries for the same stream that have duplicate timestamps. (#6042) * Add a per tenant ability to fudge duplicate timestamps in the distributor Signed-off-by: Edward Welch <edward.welch@grafana.com> * update docs Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com> * update changelog Signed-off-by: Edward Welch <edward.welch@grafana.com>
* Add two metrics to the IndexGateway. - Add a new `query_readiness_duration_seconds` metric, that reports query readiness duration of a tablemanager/index gateway instance. We should use it later to report performance against the ring mode - Add a new `usersToBeQueryReadyForTotal` metric, that reports number of users involved in the query readiness operation. We should use it later to correlate number of users with the query readiness duration. * Remove `usersToBeQueryReadyForTotal`. - It will report all users always for now, so it isn't too helpful the way it is. * Rename metric help text to not mislead people. * Log queryReadiness duration. * Fix where log message and duration and triggered. * Join users list in a single string. - This is necessary since go-kit doesn't support array type. * Tweak queryReadiness log messages. - As suggested by Ed on #5972 (comment) and #5972 (comment) * Ensure queryReadinessDuration metric. - It is redundant with a recently added log line. * noop
* Revise the getting started section
* Docs: revise getting started guide
* Intermediate commit of getting started material.
* Intermediate revision of the getting started guide.
Add instructions to download, configure, and deploy the test environment.
* Add sample queries and instructions for breaking down the test env
* Revise prose using reviewer comments.
* update to use promtail and flog in the docker compose file
Signed-off-by: Edward Welch <edward.welch@grafana.com>
* Revise guide to put all parts into Docker containers.
* Add README files for examples directory. Revise getting started instructions.
* Address DylanGuedes review comments
* Getting started guide: incorporate suggestions from ssncferreira
Co-authored-by: Edward Welch <edward.welch@grafana.com>
This component has been around for a while but has not been added to the loki-mixin yet. Signed-off-by: Christian Haudum <christian.haudum@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
…tor (#5879) * Add filter parameter to rebound Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix linting issues Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add filter function to delete request Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix linting issues Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Enable api for filter and delete mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add settings for retention Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use labels to check and add test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Simplify filter function Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Also enable filter mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove test settings in config file for docker Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add extra (unused) param for ProcessString in filter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Empty commit to trigger CI again Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix flapping test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove commented out unit tests and add some more Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add extra test case for delete request without line filter Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use chunk bounds Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * check if the log selector has a filter if the whole chunk is selected Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * fix lint issue: use correct go-kit import Signed-off-by: Michel Hollands <michel.hollands@grafana.com>
…by the AST (#6346) * Panic with an actual error if clone (which calls range aggregation expr's validate) errors, instead of swallowing the error and then attempting to execute a nil function, which panics anyways. Signed-off-by: Callum Styan <callumstyan@gmail.com> * Call the validation function on a modified expression before attempting to execute it. If the validation fails we should return the original expression. Signed-off-by: Callum Styan <callumstyan@gmail.com>
Signed-off-by: Christopher Norris <christopher.norris@grafana.com>
So that you can do quantile calculation on metadata queries. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com>
This updates to a version that contains the fix to CVE-2022-28948.
#6310) * Add support for client-side TLS certs to loki-canary * fix lint errors, make the case of using TLS but not client certs more clear, check that -tls is set if client certs are used * move changelog to canary section instead of main * wrap httpClient errors and simplify dialer creation for websocket
* index sampling endpoint * fix bug in withTenantLabelMatcher * show zero values in stats * handles new errcase * fixes comments
Signed-off-by: Trevor Wood <Trevor.G.Wood@gmail.com>
* cmd/loki: minor consistency patch * cmd/loki: fixed a typo
* begins building queryrange compatible types for index sampling * adds MatcherGroups() method to SampleExpr * MatcherGroups test + ShardResolver interface * index stats codec wiring * millisecond precision on index stats codec * lint * cleanMatchers in tsdb.IndexClient for consistency + adds catchall matcher if nil
* Add metrics for journal target There are no metrics for the journal target yet, this adds two basic ones: - Total number of lines processed - Total number of errors processing lines (with a reason) Because of the way the journal works, tailing one file with many possible process writing to it, I chose note to expose counts per unit. * Add changelog entry * Fix compilation of (unsupported) journalmanager target on windows * Extract possible errors to constants & add tests
* Backport memberlist dependency version pin to k102 Signed-off-by: JordanRushing <rushing.jordan@gmail.com> * Revert "Loki Prometheus mixin: templatize cluster label" (#6377) * Revert "Loki Prometheus mixin: templatize cluster label (#6353)" This reverts commit 459036f. * Replace `loki-mixin` dependency to working commit. Signed-off-by: Kaviraj <kavirajkanagaraj@gmail.com> (cherry picked from commit 666bac2) Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com> (cherry picked from commit 372961d) Co-authored-by: JordanRushing <rushing.jordan@gmail.com>
* Add more logging for delete requests Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix typo Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rephrase logging message and add chunk ID Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename userID field to user Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * rename more userID Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix existing log messages Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit 20db4df) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com> (cherry picked from commit c707faf)
…) (#6496) * Compactor: add per tenant compaction delete enabled flag (#6410) * Add per tenant compaction delete enabled flag Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove changes in wrong place Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add compactor deletion enabled field Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use limit in compactor Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Use http middleware and add test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix lint issue Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Revert to default setting if no override Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add default value command line option Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Enable access to deletion API for integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename flag to allow_deletes Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update per review comments Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit b4e6c59) * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit df48f93) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
) * Remove whole stream deletion mode (#6435) * Remove whole-stream-deletion mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove whole-stream-deletion from docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Update the changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Sort changelog entries Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove link to wrong configuration Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix integration test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Set default deletion mode to disabled Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Remove extra white line in documentation Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix default value in docs Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Add DeletionEnabled method on mode Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Rename test Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit f80e487) * Fix changelog Signed-off-by: Michel Hollands <michel.hollands@grafana.com> * Fix changelog link Signed-off-by: Michel Hollands <michel.hollands@grafana.com> (cherry picked from commit da3fbd9) Co-authored-by: Michel Hollands <42814411+MichelHollands@users.noreply.github.com>
This reverts commit 36e0979.
* disable tests which are failing from hardcoded certs which are now expired. Signed-off-by: Edward Welch <edward.welch@grafana.com> * lint Signed-off-by: Edward Welch <edward.welch@grafana.com>
Signed-off-by: Danny Kopping <danny.kopping@grafana.com>
dannykopping
force-pushed
the
dannykopping/remove-cache-stats
branch
from
2a5bb86
to
7c90e53
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
grafana/tailto fix promtail symlink behavior (Revendorgrafana/tailto fix promtail symlink behavior grafana/loki#6034)Loki - Reads Resourcesdashboard in production/loki-mixin (Add IndexGateway Query Readiness Duration panel toLoki - Reads Resourcesdashboard in production/loki-mixin grafana/loki#6014)static_config(promtail: Add support for exclusion patterns instatic_configgrafana/loki#5943)/label/<name>/valuesendpoint to use right set of middlewares (Fix(query-frontend):/label/<name>/valuesendpoint to use right set of middlewares grafana/loki#6072)sources/fundamentals/architecture/deployment-modes.md(Docs: fix pic location in `sources/fundamentals/architecture/deployme… grafana/loki#6089)metrics.gosupport for metadata queries(labels and series) (metrics.gosupport for metadata queries(labels and series) grafana/loki#5971)sources/fundamentals/architecture/deployment-modes.md(Docs: fix pic location in `sources/fundamentals/architecture/deployme… grafana/loki#6089)" (Revert "Docs: fix pic location in `sources/fundamentals/architecture/deployme…" grafana/loki#6093)/servicesAPI endpoint (Document/servicesAPI endpoint grafana/loki#6111)defaultsprig template function to logql label/line formatter (LogQL: add thedefaultsprig template function to logql label/line formatter grafana/loki#6163)querier_forget_delay(query-scheduler: Now proper support forquerier_forget_delaygrafana/loki#6174)resultlength after response validation. (fix(querier): Recordresultlength after response validation. grafana/loki#6176)faketenant (Logcli: fix boltdb shipper local query in logcli and supportfaketenant grafana/loki#6282)multi_kv_configblock to overrides ConfigMap when multi-kv mode is enabled grafana/loki#6323)MetricsNamespaceconfigurable for MemberlistKV module in Loki (SetMetricsNamespaceconfigurable for MemberlistKV module in Loki grafana/loki#6330)series|labelsquery_type inlogql_query_duration(Supportseries|labelsquery_type inlogql_query_durationgrafana/loki#6341)