You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Defaultly, LRU cache uses a policy kFullChargeCacheMetadata which will take memory usage of meta into account.
While using it as file cache where we hope 1 file takes only 1 charge, we should use the policy kDontChargeCacheMetadata.
Here is where the cache instance of BlobFileCahe is created:
auto file_cache_size = db_options_.max_open_files;
if (file_cache_size < 0) {
file_cache_size = kMaxFileCacheSize;
}
file_cache_ = NewLRUCache(file_cache_size);
Under such setting, each file will be count as 89 charge. In my test case, i set max_open_files to 950 so only 10 files can be cached in LRU. Thus most of Get operations need to open a blob file again and again and i observed significant performance degradation.
file_number: 54 cacheusage: 89 total insert count: 1
file_number: 53 cacheusage: 178 total insert count: 2
file_number: 197 cacheusage: 267 total insert count: 3
file_number: 109 cacheusage: 356 total insert count: 4
file_number: 149 cacheusage: 445 total insert count: 5
file_number: 6 cacheusage: 534 total insert count: 6
file_number: 85 cacheusage: 623 total insert count: 7
file_number: 5 cacheusage: 712 total insert count: 8
file_number: 73 cacheusage: 801 total insert count: 9
file_number: 30 cacheusage: 890 total insert count: 10
file_number: 88 cacheusage: 890 total insert count: 11
file_number: 118 cacheusage: 890 total insert count: 12
Defaultly, LRU cache uses a policy
kFullChargeCacheMetadata
which will take memory usage of meta into account.While using it as file cache where we hope 1 file takes only 1 charge, we should use the policy
kDontChargeCacheMetadata
.Here is where the cache instance of BlobFileCahe is created:
titan/src/blob_file_set.cc
Lines 22 to 26 in ff85fd1
Under such setting, each file will be count as 89 charge. In my test case, i set
max_open_files
to 950 so only 10 files can be cached in LRU. Thus most of Get operations need to open a blob file again and again and i observed significant performance degradation.We should use it like this:
The text was updated successfully, but these errors were encountered: