v0.10.0

Changelog

New Features

• d51c1b8: feat(chstorage): offload sampling to Clickhouse (@tdakkota)
• ef690b6: feat(logql): parse or in line filters (@tdakkota)
• 2250aa6: feat(logql): parse logfmt flags (@tdakkota)
• df1dc3a: feat(logql): parse pattern line filters (@tdakkota)
• f29231d: feat(logql-compliance-tester): add a flag to print diffs to stdout (@tdakkota)
• 46887fc: feat(logql-compliance-tester): add an option to exclude unsupported (@tdakkota)
• bce26df: feat(logql-compliance-tester): add tool to test LogQL compliance (@tdakkota)
• d6da84e: feat(logql-compliance-tester): count unsuccessful results (@tdakkota)
• 07bdbf0: feat(logql-compliance-tester): generate logs outside the query range (@tdakkota)
• 2ce3c99: feat(logql-compliance-tester): print diff of incompatiable results (@tdakkota)
• 57b524b: feat(logqlabels): add label utilities package (@tdakkota)
• b781f38: feat(logqlmetric): implement absent_over_time (@tdakkota)
• 7498c4b: feat(logqlmetric): implement rate_counter (@tdakkota)
• bb3c124: feat(logqlpattern): support line filter patterns (@tdakkota)
• d25e5d1: feat(lokicompliance): add more fields to generated logs (@tdakkota)
• a6a326b: feat(lokicompliance): add package to test LogQL engine compliance (@tdakkota)
• 6b0d5ca: feat(lokicompliance): add responses to resulting json (@tdakkota)
• 0d46f36: feat(lokicompliance): add simple log generator (@tdakkota)
• fd40d8d: feat(lokicompliance): add unexpected non-empty result to report (@tdakkota)
• db662d2: feat(lokicompliance): check if result is empty (@tdakkota)
• c5e4fb5: feat(lokicompliance): detect template args automatically (@tdakkota)
• 748eb1b: feat(lokicompliance): disallow unknown config fields (@tdakkota)
• 9b5755a: feat(lokicompliance): emit more fields (@tdakkota)
• d7fb867: feat(lokicompliance): generate IP labels (@tdakkota)
• bc1f436: feat(lokicompliance): generate duration and bytes labels (@tdakkota)
• caeb80f: feat(otelbench): add SubmitReport stub (@ernado)
• 097737a: feat(otelbench): add otel logs benchmark (@tdakkota)
• 64762cb: feat(otelbench): init submitReport (@ernado)
• 5f7d972: feat(otelbench): report written entries and bytes per second (@tdakkota)
• e6dce74: feat(oteldb): enable chstorage optimizer (@tdakkota)
• 3539891: feat(prometheusremotewrite): map exemplars too (@tdakkota)
• a5bb9bf: feat(prometheusremotewrite): support native histograms (@tdakkota)
• adc6b2b: feat(prompb): implement exemplar and histogram parsing (@tdakkota)
• 30cedaf: feat(prompb): implement timestamp getters (@tdakkota)

Bug fixes

• e3817d1: fix(chstorage): do not offload heap aggregation sampling (@tdakkota)
• b7a7e5a: fix(chstorage): do not offload new line filters (@tdakkota)
• 7d7c2ef: fix(chstorage): do not optimize sampling for unwrapped range aggregation (@tdakkota)
• c949711: fix(chstorage): insert observed timetamp in case if timestamp is zero (@tdakkota)
• 3059bfc: fix(chstorage): map grouping labels too (@tdakkota)
• 4073dd8: fix(chstorage): properly instrument sample query (@tdakkota)
• 63f1c76: fix(chstorage): remove async metric batching (@ernado)
• a721e64: fix(chstorage): use right field name for log message (@tdakkota)
• dd7d201: fix(logql): allow single = when parsing IP label matcher (@tdakkota)
• ddc2c07: fix(logql): binary expression can be applied only to metric expressions (@tdakkota)
• fbb5ade: fix(logql): properly parse rename in label_format (@tdakkota)
• 4fc933f: fix(logql): properly parse right-assoc operators (@tdakkota)
• ebb92a4: fix(logql-compliance): add non-literal queries (@tdakkota)
• 0e9c2a1: fix(logql-compliance): adjust Loki query limit (@tdakkota)
• 9f321f5: fix(logql-compliance): collect logs from promtail (@tdakkota)
• 62c9eb1: fix(logql-compliance): remove invalid queries (@tdakkota)
• a7c61c7: fix(logql-compliance-tester): do not try to read nil result (@tdakkota)
• 37980bd: fix(logql-compliance-tester): print actual total (@tdakkota)
• 302b214: fix(logql-compliance-tester): print test summary anyway (@tdakkota)
• b2e4186: fix(logql-compliance-tester): properly expand test cases (@tdakkota)
• e430dbc: fix(logql-compliance-tester): respect output-passing flag (@tdakkota)
• 5b5d31a: fix(logql-compliance-tester): respect parallelism setting (@tdakkota)
• 41b7fad: fix(logql-compliance-tester): use exponential back off when pinging (@tdakkota)
• 7765ec3: fix(logql-compliance-tester): use predicatable log generator (@tdakkota)
• 33eb5c6: fix(logql-compliance-tester): use proper time range to do ping query (@tdakkota)
• 284a6d0: fix(logql-compliance-tester): wait until ingester is available (@tdakkota)
• 445e781: fix(logql-compliance-tester): wait until instance gets some data (@tdakkota)
• 8dda095: fix(logqlabels): sort entries after adding a new one (@tdakkota)
• 4b76a40: fix(logqlengine): do not set message as label (@tdakkota)
• 39bcfb9: fix(logqlengine): include one step after end (@tdakkota)
• b08f77d: fix(logqlengine): make otel adapter opt-in (@tdakkota)
• d75a852: fix(logqlengine): remove unwrap label from sampled entry (@tdakkota)
• 09724e8: fix(logqlengine): remove unwrap label from sampled entry (@tdakkota)
• fc5759d: fix(logqlengine): respect direction parameter (@tdakkota)
• 1c0e7ad: fix(logqlengine): restore otel adapter (@tdakkota)
• 650fc73: fix(logqlengine): sort labels before computing hash (@tdakkota)
• d7f11b3: fix(logqlengine): truncate range by step (@tdakkota)
• c8ee43d: fix(logqlmetric): do not group vector aggregation by default (@tdakkota)
• 87ce232: fix(logqlmetric): make range aggregation Loki-compatiable (@tdakkota)
• 585e4e1: fix(logqlmetric): properly handle bool modifier (@tdakkota)
• d51b6fb: fix(logqlpattern): validate UTF-8 in pattern (@tdakkota)
• efea944: fix(lokiapi): return all errors as JSON string (@tdakkota)
• daabe93: fix(lokicompliance): add more expand variables (@tdakkota)
• e1f28e5: fix(lokicompliance): convert fpoint strings to floats (@tdakkota)
• 49eae85: fix(lokicompliance): exclude absent_over_time from simple range aggregations (@tdakkota)
• d6ece80: fix(lokicompliance): ignore stats (@tdakkota)
• 75ae255: fix(lokicompliance): properly handle not implemented errors (@tdakkota)
• 2d626a9: fix(lokicompliance): sort collected variant args (@tdakkota)
• 51f7842: fix(lokicompliance): sort response data before compare (@tdakkota)
• d9bbeb8: fix(lokicompliance): use only valid quantiles (@tdakkota)
• 2c14d78: fix(lokihandler): properly handle unsupported error (@tdakkota)
• 57c2fde: fix(oas): remove log entries limit from spec (@tdakkota)
• aa18f7e: fix(otelbench): report formatting (@tdakkota)
• 052a57e: fix(otelbench): run clickhouse reporter (@tdakkota)
• af125d8: fix(otelbench): wait for receiver readiness (@tdakkota)
• 8ba294f: fix(prometheusremotewrite): update code due to collector changes (@tdakkota)
• a2c0505: fix(prometheusremotewritereceiver): do not use deprecated error (@tdakkota)
• 1ff96e6: fix(prompb): re-use timeseries (@tdakkota)

Performance updates

• 826502b: perf(prompb): properly re-use slice pools (@tdakkota)

Benchmarking tooling changes

• d35f88c: ci(bench): add another cleanup step (@ernado)
• 8542ccb: ci(bench): add cleanup step (@ernado)
• 1a076cb: ci(bench): remove docker from check api job (@ernado)
• 3fb2973: ci(bench): try self-hosted runner (@ernado)
• 4f3186c: ci(bench): try warming up (@ernado)
• a2f026d: ci(ch-log-bench): add scripts to run logs benchmark (@tdakkota)
• 511b242: ci(ch-log-bench): write 10k entries per second (@tdakkota)

CI/CD updates

• bee5e36: ci(Dockerfile): add .git folder to .dockerignore (@tdakkota)
• f26ebf3: ci(ch-logql-compliance): add more binary operation tests (@tdakkota)
• e970d71: ci(ch-logql-compliance): add absent_over_time tests (@tdakkota)
• 8fb6d7e: ci(ch-logql-compliance): add label_replace test (@tdakkota)
• cc91485: ci(ch-logql-compliance): add quantile_over_time test (@tdakkota)
• 5ac42e1: ci(ch-logql-compliance): add without grouping test (@tdakkota)
• 6e7ceb6: ci(ch-logql-compliance): add heap aggregation and offset tests (@tdakkota)
• 7083784: ci(ch-logql-compliance): add more label filter and extractor tests (@tdakkota)
• 505c0a5: ci(ch-logql-compliance): add more precedence tests (@tdakkota)
• 4085ef0: ci(ch-logql-compliance): add more topk/bottomk tests (@tdakkota)
• e227759: ci(ch-logql-compliance): add more unwrap sampler tests (@tdakkota)
• 92ba4b5: ci(ch-logql-compliance): add more vector aggregation grouping tests (@tdakkota)
• 020d66d: ci(ch-logql-compliance): add pattern line filter queries (@tdakkota)
• 5872eef: ci(ch-logql-compliance): add range aggregation grouping test (@tdakkota)
• da4ea0d: ci(ch-logql-compliance): add vector aggregation without grouping (@tdakkota)
• 39996d9: ci(ch-logql-compliance): test and in label filters (@tdakkota)
• 87c7455: ci(ch-logql-compliance): test against Loki 3.0.0 (@tdakkota)
• c6a617e: ci(compliance): add LogQL compliance job (@tdakkota)
• a1a038a: ci(compliance): avoid artifact collision (@tdakkota)
• 4eb791f: ci(compliance): fail pipeline if compliance test fails (@tdakkota)
• 470d9ae: ci(compliance): include all testcases in compliance report (@tdakkota)
• 22201a1: ci(compliance): reduce query parallelism (@tdakkota)
• c14a6d4: ci(compliance): update LogQL compliance percent (@tdakkota)
• def1708: ci(compliance): update LogQL target percent (@tdakkota)
• e061e0d: ci(compliance): update target LogQL percent (@tdakkota)
• 1917403: ci(dep-review): allow loki push module (@tdakkota)
• c3cf8aa: ci(dockerignore): add dev folder (@tdakkota)
• cf1f03e: ci(e2e): run all tests (@tdakkota)
• 6678e10: ci(e2e): run only integration tests (@tdakkota)
• 4cb33a5: ci(golangci): disable ptrToRefParam check (@tdakkota)
• 25e5cb7: ci(golangci-lint): ignore some gosec issues (@tdakkota)
• 0ea9a83: ci(golangci-lint): update exclude rules due to new revive checks (@tdakkota)
• 80f03be: ci(goreleaser): update benchmarking category (@ernado)
• 0ddad5b: ci: add compose files to test LogQL compliance locally (@tdakkota)

Updates to tests

• b94eaae: test(logql): update tests due to parser changes (@tdakkota)
• f801b66: test(logqlmetric): log failed query (@tdakkota)
• c9a1820: test(logqlmetric): update tests due to bool operator changes (@tdakkota)
• af24e6f: test(lokie2e): add forward/backward tests (@tdakkota)
• 84e41c4: test(prometheusremotewrite): add unmarshal benchmark (@tdakkota)
• f5e43f7: test(prompb): add conformance test (@tdakkota)
• dcecc60: test(prompb): ensure that slicepool is properly re-used (@tdakkota)

Other work

• d0f83d4: refactor(chstorage): provide new planner API as storage (@tdakkota)
• f4f52af: refactor(chstorage): use new logqlabels package (@tdakkota)
• dc9ef38: refactor(logqlengine): introduce planner (@tdakkota)
• 1c1f142: refactor(logqlengine): make newLabelSet public (@tdakkota)
• 6da1105: refactor(logqlengine): move all errors to logqlerrors package (@tdakkota)
• 8bfead6: refactor(logqlengine): use new logqlabels package (@tdakkota)
• 5440bcd: refactor(lokihandler): make timestamp and duration parser public (@tdakkota)
• 096beee: refactor(lokihandler): use new LogQL engine API (@tdakkota)
• fea93e0: refactor(prompb): make Pools private (@tdakkota)
• 9acf542: refactor(prompb): use easyproto to parse protobuf (@tdakkota)

Docker images

ghcr.io/go-faster/oteldb:v0.10.0