Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mvfs: Replace moka with lru due to memory leak #48

Merged
merged 1 commit into from
Sep 1, 2022
Merged

Conversation

losfair
Copy link
Owner

@losfair losfair commented Sep 1, 2022

Seems that the moka crate has a memory leak issue when moka::sync::Cache is dropped. Detected in production, and confirmed with Valgrind:

==905778== 82,960 (560 direct, 82,400 indirect) bytes in 10 blocks are definitely lost in loss record 194 of 199
==905778==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==905778==    by 0x503005B: moka::sync_base::base_cache::BaseCache<K,V,S>::do_insert_with_hash::{{closure}} (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x502FD32: moka::cht::map::bucket::InsertOrModifyState<K,V,F>::into_insert_bucket (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x502CF77: moka::cht::map::bucket::BucketArray<K,V>::insert_or_modify (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4BC49EE: moka::cht::map::bucket_array_ref::BucketArrayRef<K,V,S>::insert_with_or_modify_entry_and (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x503A7F6: moka::sync::cache::Cache<K,V,S>::insert (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4BC8DDB: mvfs::vfs::Connection::insert_to_page_cache (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4B08D3A: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4B0A24A: _ZN97_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$4poll17h50e20491b908d0c9E.llvm.3961465344180881740 (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmv
sqlite_preload.so)
==905778==    by 0x4B1042B: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4A722A2: std::thread::local::LocalKey<T>::with (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4A97131: tokio::park::thread::CachedParkThread::block_on (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==
==905778== 165,920 (1,120 direct, 164,800 indirect) bytes in 20 blocks are definitely lost in loss record 197 of 199
==905778==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==905778==    by 0x503005B: moka::sync_base::base_cache::BaseCache<K,V,S>::do_insert_with_hash::{{closure}} (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x502FD32: moka::cht::map::bucket::InsertOrModifyState<K,V,F>::into_insert_bucket (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x502CF77: moka::cht::map::bucket::BucketArray<K,V>::insert_or_modify (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4BC49EE: moka::cht::map::bucket_array_ref::BucketArrayRef<K,V,S>::insert_with_or_modify_entry_and (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x503A7F6: moka::sync::cache::Cache<K,V,S>::insert (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4BC8DDB: mvfs::vfs::Connection::insert_to_page_cache (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4B08D3A: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4B0A24A: _ZN97_$LT$core..future..from_generator..GenFuture$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$4poll17h50e20491b908d0c9E.llvm.3961465344180881740 (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmv
sqlite_preload.so)
==905778==    by 0x4B11852: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4A717CC: std::thread::local::LocalKey<T>::with (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==    by 0x4A96D7E: tokio::park::thread::CachedParkThread::block_on (in /home/ubuntu/Projects/mvsqlite/mvsqlite-preload/libmvsqlite_preload.so)
==905778==

This PR replaces moka with lru to resolve the issue.

@losfair
Copy link
Owner Author

losfair commented Sep 1, 2022

Related: moka-rs/moka#176

@losfair losfair merged commit d275610 into main Sep 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant