You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#6984 adds support for lazy downloaded index headers. It can be useful if only some blocks are accessed more often than other blocks. For example, data from recent one month are usually accessed more often than data older than one year. With lazy downloaded index header feature, we can speed up store gateway start up time and index header disk space usage because we download less index headers.
However, there is no way to clean up index headers on local disk if they are not accessed anymore. If an index header file is downloaded and accessed only once, it remain on the disk forever. We only do mmap unload but there is no local data retention.
Describe the solution you'd like
Purge local index headers periodically if files are not used after some time.
Implement size based strategy as well for ^
We can start with option 1 and add size based strategy later.
Describe alternatives you've considered
NA
The text was updated successfully, but these errors were encountered:
Is your proposal related to a problem?
#6984 adds support for lazy downloaded index headers. It can be useful if only some blocks are accessed more often than other blocks. For example, data from recent one month are usually accessed more often than data older than one year. With lazy downloaded index header feature, we can speed up store gateway start up time and index header disk space usage because we download less index headers.
However, there is no way to clean up index headers on local disk if they are not accessed anymore. If an index header file is downloaded and accessed only once, it remain on the disk forever. We only do mmap unload but there is no local data retention.
Describe the solution you'd like
We can start with option 1 and add size based strategy later.
Describe alternatives you've considered
NA
The text was updated successfully, but these errors were encountered: