New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
asMap() compute and the Weigher #513
asMap() compute and the Weigher #513
Comments
I have have to jump into a meeting, so here is an immediate answer before I dig in for moore details. I believe you are assuming LRU eviction semantics, but Caffeine may evict entries early. At a very tiny cache of 3, we may more aggressively evict from the window region causing the last insert/update to be dropped. In most cases the size is large enough that we can use this window region to retain very recent items, but at a tiny size the rejections are more noticeable. This also is non-deterministic as we introduce a small amount of jitter into the eviction decisions to protect against a HashDoS attack. My immediate guess is that while the cache is honoring its bound, it is not choosing the entry you expected for eviction. You can debug this by adding a removal listener and outputting the current size, .removalListener((k, v, cause) -> {
System.err.printf("%s: %s=%s%n", cause, k, v);
})
// before and after write
System.out.println("Weighted Size: " + cache.policy().eviction().get().weightedSize().getAsLong()); For re-weighing, a compute or conditional replace should be used to avoid a race, as you suggested. |
Ahh thanks! That makes sense. I tried with more entries this time, and things are more well behaved.
|
What does this refer to?
|
Sorry, I'm still only able to partially focus.
Is there any case where you observe the value/weight to be incorrect, or only that the cache evicted an entry that you did not expect?
|
Regarding the comment: When doing a compute(), the original entry (1L:2) is replaced with another entry (1L:3). In addition it would appear one other entry is removed as well (I don't check which, but the final size is 99 and the updated entry is preserved). The total eviction weight stat count is 2 however. There could be a case made that the total should be 4. It doesn't matter much to my use case, and I suspect some internal reason for this with compute(), but maybe just FYI for you if you didn't know. Other than that, it would appear things are good to go for me. Just needed a bigger test size to see the behavior. For reference here is my final @test confirming that computeIfPresent() is used to replace an entry that eviction takes place as expected.
|
A few findings,
Any thoughts? |
I don't really have much input as this is pretty deep in the Cache inner workings for me. The 4 vs 2 makes sense, i'm fine with it as defined. The only part that was a little strange is the effect of a MaxWeight=3, add 3x elements of weight 1, then update an element to weight 2, actually removes the element, and leave the cache with the 2 original elements. I realize now why this happens, but it was a little gotcha that happens with low weight values and entries. Probably not worth fixing unless there was a larger issue a play here. |
If we reorder in probation, then it would select K2 instead of K1, which would mask the problem a little bit better for you. It would pass your original test, except that the eviction weight would be 1. That's a benign change, so while it doesn't impact much real-world I agree having things behave less surprisingly when debugging through a test case is worthwhile. |
In v3 your original tests passes, if ignoring the eviction weight misunderstanding, because we now search the admission window for additional candidates. This change was done to improve the handling for excessively large entries (464bc19) to avoid flushing a region because the newly added entry is in the MRU position. In your case, the candidate (3) and the victim (1) are compared by TinyLFU, where the victim wins due to already being promoted so it is discarded only if the candidate has a higher frequency. Regardless, I ported over your test case and add the reordering as they seem like nice change that might avoid this type of confusion. |
Excellent. Thanks for the great support as always Ben! Cheers
…On Wed, Mar 10, 2021, 5:12 PM Ben Manes ***@***.***> wrote:
In v3 your original tests passes, if ignoring the eviction weight
misunderstanding, because we now search the admission window for additional
candidates. This change was done to improve the handling for excessively
large entries (464bc19
<464bc19>)
to avoid flushing a region because the newly added entry is in the MRU
position. In your case, the candidate (3) and the victim (1) are compared
by TinyLFU, where the victim wins due to already being promoted so it is
discarded only if the candidate has a higher frequency.
Regardless, I ported over your test case and add the reordering as they
seem like nice change that might avoid this type of confusion.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#513 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA2TKJOSSSRWMM6AXVK5APLTDADHTANCNFSM4Y6SKNCA>
.
|
For weighted caches, if an entry in the probation queue is too large to be promoted to the protected queue, it was skipped. This could leave it as the victim, making it eligable for eviction. In small caches, often used by test code, the premature eviction can be surprising if one still assumes LRU. To reduce this confusion, we can reorder within the probation queue and treat it as an LRU instead of FIFO. This test case already passes in v3 due to changes in how it selects candidates in order to handle excessively large entries. However, codifying the intent seems worthwhile and, if it helps reduce unit test confusion, then a nice benefit.
For weighted caches, if an entry in the probation queue is too large to be promoted to the protected queue, it was skipped. This could leave it as the victim, making it eligable for eviction. In small caches, often used by test code, the premature eviction can be surprising if one still assumes LRU. To reduce this confusion, we can reorder within the probation queue and treat it as an LRU instead of FIFO. This test case already passes in v3 due to changes in how it selects candidates in order to handle excessively large entries. However, codifying the intent seems worthwhile and, if it helps reduce unit test confusion, then a nice benefit.
Hi Ben,
Using Caffeine 2.8.5 currently.
I'm trying to utilize the 'compute' varients (specifically computeIfPresent()) of the concurrent map from asMap() such that it will update the weight for the Weigher based eviction. I'm writing tests to see the functionality, but it's hard to know if Eviction is working correctly when things are updated in this way. Here is one strange behavior that I found already:
Essentially I want to re-weigh a value and it appears the safest way to do that is to to update the value with a new value. Ideally in a compute() context, but I suppose I could settle for PUT in this case and just accept the race condition. Put appears none better however.
Any pointers on getting this update to values and weight correct?
Also I noticed that calling cleanUp() allows eviction to happen when utilizing compute() changes, but I have no idea how heavy a hammer this is? Does cleanUp() internally re-compute all weights?
The text was updated successfully, but these errors were encountered: