Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support seting dynamic time duration #12

Open
ColinChen2 opened this issue Mar 28, 2023 · 3 comments
Open

Support seting dynamic time duration #12

ColinChen2 opened this issue Mar 28, 2023 · 3 comments

Comments

@ColinChen2
Copy link

ColinChen2 commented Mar 28, 2023

We want set dynamic time duration when generate recoding rules, support using variable during e.g.

  • with (a = 2h) (up{job="samples"}[a])
  • with (a = "2h") (up{job="samples"}[a])
  • with (hours = (day_of_month() - 1) * 24 + hour() + "h") (up{job="samples"}[hours])

related issue: dynamic time duration

valyala added a commit that referenced this issue Jul 19, 2023
…ions

Support using WITH template vars in the following places where duration can be passed:

- Lookbehind windows in square brackets: `with (w=5m) m[w]` is transformed to `m[5m]`
- Steps in square brackets: `with (step=5m) m[1h:step]` is transformed to `m[1h:5m]`
- Offsets: `with (off=5m) m offset off` is transformed to `m offset 5m`

Updates VictoriaMetrics/VictoriaMetrics#4025
Updates #12

Thanks to @lujiajing1126 for the initial implementation at #13

Note that this feature doesn't allow specifying dynamic durations in the following way:

  with (w = ((day_of_month()-1) * 24 + hour()) + "h") m[w]

It allows using only static durations

Support for dynamic durations requires significant refactoring of the code responsible for calculating rollups in `/api/v1/query_range` handler.
It will be needed to use different lookbehind windows per each calculated data point per each `step`. Currently the code assumes
that the lookbehind window is static across every calculated data point.
valyala added a commit to VictoriaMetrics/VictoriaMetrics that referenced this issue Jul 19, 2023
…0.61.1

This adds support for passing durations via WITH template vars:

- `WITH (w = 5m) m[w]` is transformed to `m[5m]`
- `WITH (f(w, step, off) = m[w:step] offset off) f(5m, 10s, 1h)` is transformed to `m[5m:10s] offset 1h`

Updates #4025
Updates VictoriaMetrics/metricsql#12

See also the initial implementation by @lujiajing1126 at VictoriaMetrics/metricsql#13
valyala added a commit to VictoriaMetrics/VictoriaMetrics that referenced this issue Jul 19, 2023
…0.61.1

This adds support for passing durations via WITH template vars:

- `WITH (w = 5m) m[w]` is transformed to `m[5m]`
- `WITH (f(w, step, off) = m[w:step] offset off) f(5m, 10s, 1h)` is transformed to `m[5m:10s] offset 1h`

Updates #4025
Updates VictoriaMetrics/metricsql#12

See also the initial implementation by @lujiajing1126 at VictoriaMetrics/metricsql#13
@valyala
Copy link
Contributor

valyala commented Jul 19, 2023

FYI, the commit f3dd382 adds support for specifying duration values via WITH templates. However, it doesn't support the ability to specify dynamic duration, which depend on the evaluation timestamp. See this comment for details.

@lujiajing1126
Copy link

lujiajing1126 commented Jul 20, 2023

FYI, the commit f3dd382 adds support for specifying duration values via WITH templates. However, it doesn't support the ability to specify dynamic duration, which depend on the evaluation timestamp. See this comment for details.

Thanks for the reply!

To the clear, our case is to calculate SLO/SLI based on a calendar-window, for example, SLO for July. And due to the complexity of the query expression, we would like to use RecordingRules to pre-calculate the result (by aggregating windows). So the dynamic window is necessary in this case.

But what you proposed in the comment seems to be promising and fit our case well. I would like to try to implement this in MetricsQL.

@valyala
Copy link
Contributor

valyala commented Jul 28, 2023

FYI, the functionality described in this comment has been included in VictoriaMetrics v1.92.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants