docs: add post-mortem for avoid global sort and move it #7130
Conversation
GiedriusS
force-pushed
the
global_sort_postmortem
branch
2 times, most recently
from
7fe2034
to
49e5d0f
Compare
Long overdue TODO task for me - adding a post-mortem for the avoid global sort proposal and move it to the "proposals-done" directory. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
GiedriusS
force-pushed
the
global_sort_postmortem
branch
from
49e5d0f
to
5b6a3f0
Compare
|
|
||
| Theoretically, we could address this in the distant future by modifying the ordering rules within the TSDB, but I'm uncertain if that will ever materialize. | ||
|
|
||
| Nevertheless, despite the initial challenges faced in implementation and subsequent reverting, we have made significant improvements to the querying internals. For example, we now use a ProxyHeap in a multitude of components, rather than repeating the same logic across each individual component. |
There was a problem hiding this comment.
I mean we are still avoiding global sort by stripping out replica labels on stores, we just have to perform resorting in stores afterwards, but the sorting is now not done in global context but on the store APIs right? So its not full revert I think.
There was a problem hiding this comment.
Mhm, why do we https://github.com/thanos-io/thanos/blob/main/pkg/store/proxy_heap.go#L757-L761 force resorting here? Shouldn't we check whether the server supports without replica labels?
There was a problem hiding this comment.
That is a good question, stores should always return sorted responses.
Edit: Ah, I added that, its because this is used for block clients in bucket store. We remove labels in the block clients and then put them onto a proxy heap; if we wouldnt resort there we would have the same issues we have in querier in the bucket store because it acts like a proxy heap for blocks. So we need to make them return sorted responses after removal of the external labels. We cannot rely on sorted responses there. Maybe it would have been more elegant to do the sorting in the block series client because technically it is the one violating the sorting constraint for Store APIs I think.
Long overdue TODO task for me - adding a post-mortem for the avoid global sort proposal and move it to the "proposals-done" directory.