You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Sep 8, 2018. It is now read-only.
Much like Prometheus, oklog does not intend to be a long-term storage for logs. Thanos is a aggregation and long-term storage solution for Prometheus based on arbitrary object storage. Experiences with Thanos have shown that it is well possible to use object storage as a highly cost-efficient real-time database (at the order of "real-time" for human interaction).
This should be even more valid for the logging use case, where the majority of reads are linear scans. Thus, object storage seems like a suitable and very portable solution to solve long-term storage for oklog as well. Depending on evaluated performance, it could even be feasible to shrink oklog's own retention to a few days to hours.
In general it would probably work based on a client to oklog's store nodes, that reads, deduplicates, and uploads data to the object storage. At query time, data from object storage and store nodes will be merged transparently.
This obviously will need a full design discussion.
The text was updated successfully, but these errors were encountered:
Much like Prometheus, oklog does not intend to be a long-term storage for logs. Thanos is a aggregation and long-term storage solution for Prometheus based on arbitrary object storage. Experiences with Thanos have shown that it is well possible to use object storage as a highly cost-efficient real-time database (at the order of "real-time" for human interaction).
This should be even more valid for the logging use case, where the majority of reads are linear scans. Thus, object storage seems like a suitable and very portable solution to solve long-term storage for oklog as well. Depending on evaluated performance, it could even be feasible to shrink oklog's own retention to a few days to hours.
In general it would probably work based on a client to oklog's store nodes, that reads, deduplicates, and uploads data to the object storage. At query time, data from object storage and store nodes will be merged transparently.
This obviously will need a full design discussion.
The text was updated successfully, but these errors were encountered: