Add Summary metric type #67

palash25 · 2022-06-26T17:40:27Z

Not ready for review yet, just opening a draft for visibility

Closes: #40

palash25 · 2022-07-29T09:30:23Z

sorry for the radio silence on this one, I was moving to another city and completely forgot about it, will try to get back to this next week.

palash25 · 2022-11-20T14:27:34Z

sorry i completely forgot about this PR @mxinden can you please take a look

Signed-off-by: Palash Nigam <npalash25@gmail.com>

palash25 · 2022-11-21T14:26:32Z

i will add some comments in the summary.rs file and some doctests shortly

gmossessian · 2022-12-06T01:33:29Z

@palash25 do you need a hand here? I'm also wanting this metric, I started implementing it myself before I saw your PR. Looks like you still need to implement changes to proto.rs, can I help with this?

@mxinden in your opinion is anything besides this needed here before releasing?

palash25 · 2022-12-06T06:32:47Z

oh you are right, i missed the proto.rs file thanks for pointing it out, i will try to make the remaining changes in the next 2 days, if I can't i will ping you for help.

palash25 · 2022-12-06T06:45:39Z

moving this back to draft till i make the missing changes

mxinden · 2022-12-14T15:25:17Z

@palash25 #105 introduced a fairly large change. I am sorry for the conflict. Let me know in case you need any help updating.

palash25 · 2023-01-28T19:14:41Z

sorry for the late response @mxinden the notification got buried. I tried to make the changes according to the new encoding API but I am stuck here.

Right now my code errors with this

error[E0282]: type annotations needed
   --> src/encoding.rs:145:15
    |
145 |             e.encode_summary(sum, count, quantiles)
    |               ^^^^^^^^^^^^^^ cannot infer type of the type parameter `S` declared on the associated function `encode_summary`
    |
help: consider specifying the generic argument
    |
145 |             e.encode_summary::<S>(sum, count, quantiles)
    |                             +++++

if I do as the compiler says and add this annotation e.encode_summary::<S>(sum, count, quantiles) all the tests start passing but then my code looks a bit inconsistent in comparison to other encode_* methods in the file.

Do you know what I might be doing wrong here?

palash25 · 2023-02-09T20:17:47Z

Hi @mxinden could you take a look at the comment? ☝🏽

Signed-off-by: Palash Nigam <npalash25@gmail.com>

mxinden

Sorry for the delay here. Implementation of encoding in text and protobuf looks good to me. Couple minor comments.

Before I dive deeper into the implementation of Summary, can you expand on why you chose CKMS and how it differs to the golang implementation?

src/metrics/summary.rs

mxinden · 2023-02-24T18:01:19Z

Before I dive deeper into the implementation of Summary, can you expand on why you chose CKMS and how it differs to the golang implementation?

@beorn7 given that you wrote most of the client-golang Summary implementation and given your involvement in OpenObservability/OpenMetrics#256 I would appreciate your input here. Do you have opinions on which quantile algorithm to use?

palash25 · 2023-02-24T18:42:09Z

so about using CKMS, i went through both go and java libraries, the underlying quantile library that go uses has some inacurracies mentioned here https://github.com/beorn7/perks/blob/master/quantile/stream.go#L174 so I looked at the java client implementation and it seemed to be using a CKMS library https://github.com/prometheus/client_java/blob/c28b901225e35e7c1df0eacae8b58fdfbb390162/simpleclient/src/main/java/io/prometheus/client/TimeWindowQuantiles.java#L14 at the same time I was able to find a rust crate implementing CKMS so I just went ahead with it.

Thats all I can remember since it was a long time ago

how it differs to the golang implementation?

Not sure I remember the differences but this is what I wrote in my notes at the time which seems to be a similarity between the go library and the rust crate: "Rust crate uses the LowBiased invariant defined in the golang one quantiles/store.rs at master · blt/quantiles · GitHub"

beorn7 · 2023-02-26T22:15:34Z

tl;dr: Use whatever algorithm you find appropriate. It would require a lot of research to assess all the algorithms out there and pick the "best" one for client_rust.

The algorithm client_golang uses was picked even before my time. I don't have strong opinions about it. It's perfectly possible that there are "better" algorithms out there. It has been a hot topic of research for a while, so there are many algorithms available. It's of course also hard to decide about the "best" algorithm because that surely depends on your use case.

There was an attempt to add a now algorithm or replace the existing one, see prometheus/client_golang#859 . It was closed for lack of follow-up when we tried to discuss the aspects in which it would be a better algorithm.

I do believe that the algorithm in client_golang is implemented correctly (i.e. as it is described in the paper). client_golang uses my fork, which includes fixes. The Merge method @palash25 has mentioned above is still broken, and it cannot be fixed, because the used algorithm doesn't support merging. But that method is also not used in client_golang.

Signed-off-by: Palash Nigam <npalash25@gmail.com>

mxinden

Thanks @palash25 and @beorn7 for the input 🙏

Overall this looks good to me. I have a couple of smaller comments, nothing major. Let me know in case you need help with the feature flagging.

mxinden · 2023-03-23T09:42:48Z

src/metrics/summary.rs

+    pub fn observe(&self, v: f64) {
+        self.rotate_buckets();
+
+        let mut inner = self.inner.write();


Both rotate_buckets and later on observe itself grab the write lock. The lock will be unlocked in between. Thus another thread could potentially execute in between. Does that violate any consistency guarantees. In other words does observe depend on rotate_buckets to be called right before?

See similar race condition in #102.

Are we blocked by this issue?

@cz-cube definitely needs some investigation.

mxinden · 2023-03-23T09:45:27Z

Cargo.toml

@@ -24,6 +24,7 @@ parking_lot = "0.12"
 prometheus-client-derive-encode = { version = "0.4.1", path = "derive-encode" }
 prost = { version = "0.11.0", optional = true }
 prost-types = { version = "0.11.0", optional = true }
+quantiles = "0.7.1"


I would guess that most users of this library don't need the Summary type. What do you think of feature flagging the type as well as the quantiles dependency. The latter will help keeping ones dependency tree lean, thus compile times short.

mxinden · 2023-03-23T09:47:51Z

src/metrics/summary.rs

+use super::{MetricType, TypedMetric};
+use parking_lot::RwLock;
+use std::sync::Arc;
+use std::time::{Duration, Instant};


std::time::Instant is problematic with wasm. Consider using https://crates.io/crates/instant instead.

mxinden · 2023-03-23T09:49:39Z

src/metrics/summary.rs

+    }
+
+    /// Retrieve the values of the summary metric.
+    pub fn get(&self) -> (f64, u64, Vec<(f64, f64)>) {


Suggested change

pub fn get(&self) -> (f64, u64, Vec<(f64, f64)>) {

pub(crate) fn get(&self) -> (f64, u64, Vec<(f64, f64)>) {

Is this needed outside of the encode module?

mxinden · 2023-03-23T09:51:40Z

src/metrics/summary.rs

+        let inner = self.inner.read();
+        let sum = inner.sum;
+        let count = inner.count;
+        let mut quantile_values: Vec<(f64, f64)> = Vec::new();


Instead of allocating, you could consider returning an Iterator instead. Though likely premature optimization without any proof that this is an issue.

mxinden · 2023-03-23T09:54:06Z

src/metrics/summary.rs

+    quantile_streams: Vec<CKMS<f64>>,
+    // head_stream is like a cursor which carries the index
+    // of the stream in the quantile_streams that we want to query.
+    head_stream_idx: u64,


Suggested change

head_stream_idx: u64,

head_stream_idx: usize,

Why not use a usize here as it is an index into a datastructure? You compare it to max_age_buckets, but that could be a usize as well.

cz-cube · 2024-05-09T15:53:32Z

We want to observe multiple sample at once. Please consider adding fn observe(value: f64, times: usize).

palash25 force-pushed the summary-metric branch from dc2cba3 to 1455fcc Compare October 14, 2022 17:50

palash25 force-pushed the summary-metric branch 3 times, most recently from c4a57ff to 6458eb3 Compare November 20, 2022 14:26

palash25 marked this pull request as ready for review November 20, 2022 14:27

palash25 force-pushed the summary-metric branch from 6458eb3 to a0d7e4d Compare November 20, 2022 15:16

Add Summary metric

29661c0

Signed-off-by: Palash Nigam <npalash25@gmail.com>

palash25 force-pushed the summary-metric branch from a0d7e4d to 29661c0 Compare November 21, 2022 14:24

palash25 marked this pull request as draft December 6, 2022 06:45

Fix conflicts

2ab4ac1

palash25 force-pushed the summary-metric branch from 6af2018 to e038f4e Compare January 28, 2023 19:12

palash25 marked this pull request as ready for review February 2, 2023 13:02

Add serde style encoding for Summary

9462cd6

Signed-off-by: Palash Nigam <npalash25@gmail.com>

palash25 force-pushed the summary-metric branch from e038f4e to 9462cd6 Compare February 20, 2023 13:26

palash25 mentioned this pull request Feb 23, 2023

Implement Summary metric #40

Open

mxinden reviewed Feb 24, 2023

View reviewed changes

src/metrics/summary.rs Outdated Show resolved Hide resolved

src/metrics/summary.rs Outdated Show resolved Hide resolved

src/metrics/summary.rs Outdated Show resolved Hide resolved

Address review comments

29f3d4b

Signed-off-by: Palash Nigam <npalash25@gmail.com>

palash25 force-pushed the summary-metric branch from 6862bde to 29f3d4b Compare March 6, 2023 07:19

mxinden reviewed Mar 23, 2023

View reviewed changes

youngsofun mentioned this pull request May 17, 2023

Feature: Refactoring Databend Metrics Component with Prometheus datafuselabs/databend#9422

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Summary metric type #67

Add Summary metric type #67

palash25 commented Jun 26, 2022

palash25 commented Jul 29, 2022

palash25 commented Nov 20, 2022

palash25 commented Nov 21, 2022

gmossessian commented Dec 6, 2022

palash25 commented Dec 6, 2022

palash25 commented Dec 6, 2022

mxinden commented Dec 14, 2022

palash25 commented Jan 28, 2023

palash25 commented Feb 9, 2023

mxinden left a comment

mxinden commented Feb 24, 2023

palash25 commented Feb 24, 2023 •

edited

beorn7 commented Feb 26, 2023

mxinden left a comment

mxinden Mar 23, 2023

cz-cube May 3, 2024

mxinden May 4, 2024

mxinden Mar 23, 2023

mxinden Mar 23, 2023

mxinden Mar 23, 2023

mxinden Mar 23, 2023

mxinden Mar 23, 2023

cz-cube commented May 9, 2024

	pub fn get(&self) -> (f64, u64, Vec<(f64, f64)>) {
	pub(crate) fn get(&self) -> (f64, u64, Vec<(f64, f64)>) {

Add Summary metric type #67

Are you sure you want to change the base?

Add Summary metric type #67

Conversation

palash25 commented Jun 26, 2022

palash25 commented Jul 29, 2022

palash25 commented Nov 20, 2022

palash25 commented Nov 21, 2022

gmossessian commented Dec 6, 2022

palash25 commented Dec 6, 2022

palash25 commented Dec 6, 2022

mxinden commented Dec 14, 2022

palash25 commented Jan 28, 2023

palash25 commented Feb 9, 2023

mxinden left a comment

Choose a reason for hiding this comment

mxinden commented Feb 24, 2023

palash25 commented Feb 24, 2023 • edited

beorn7 commented Feb 26, 2023

mxinden left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cz-cube commented May 9, 2024

palash25 commented Feb 24, 2023 •

edited