Added PromQL compatibility automatic e2e test suite. #4781
Conversation
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Additionally updated docs. Motivation for this is that we can now set automated tests where in future we might want to provide docker images with exact set of test cases in versioned image. See automated use of it in Thanos: thanos-io/thanos#4781 Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
|
cc @juliusv |
bwplotka
requested review from
brancz,
bill3tt,
kakkoyun,
metalmatze,
squat,
wiardvanrij and
yeya24
test/e2e/compatibility_test.go
Outdated
| }) | ||
| } | ||
|
|
||
| func promLablsPromQLConfig(reference *e2edb.Prometheus, target e2e.Runnable, dropLabels []string) string { |
There was a problem hiding this comment.
Very minor nit: why not promLabelsPromQLConfig?
|
We should also probably document these optional tests 馃槃 Finally, did you know that you can just ask for a review from |
| - targets: | ||
| - 'demo.promlabs.com:10000' | ||
| - 'demo.promlabs.com:10001' | ||
| - 'demo.promlabs.com:10002' |
There was a problem hiding this comment.
I can see these addresses are from https://github.com/prometheus/compliance/blob/main/promql/promql-compliance-tester.yml#L88-L90.
But are they stable? Will they change in the future?
There was a problem hiding this comment.
They're stable for now, at least I don't have any plans of turning the demo service down. But it would still be better if we could either host something official from the Prometheus project at some point or somehow make people run their own.
There was a problem hiding this comment.
For now, let's keep it I guess - we don't plan to do this on CI yet (:
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
馃憢馃徑 Adding e2e test so it's easier to run PromQL compliance tests.
It's skipped by default because:
If we/https://github.com/prometheus/compliance project will automate above (provide static dataset as we do in
thanos/examples/interactive/interactive_test.go
Line 45 in b894fd6
Signed-off-by: Bartlomiej Plotka bwplotka@gmail.com