TimeLimitIteration calls close() asynchronously from another thread #4806
Comments
|
A possibly related stack trace: |
kenwenzel
changed the title
|
Potential fix for LmdbRecordIterator: The code uses the already existing txnLock and checks if the calling thread is different from the owner thread of the iterator. If this is the case then an exclusive write-lock is acquired. |
|
Are we sure that any other read or write locks will be released, or is there a chance that some other code would wait to release its lock until this iterator is closed? |
|
The My colleague is currently running a load test and it seems that the above code fixes the problem and at least prevents the JVM from completely crashing. Should we also think about changing the logic of |
|
The problem is bigger than first anticipated. Closing the data set triggers flush on the SAIL store from the wrong thread. |
|
If I understand correctly, thread A created the iteration, it times out and is interrupted by the monitoring thread B. Thread B calls interrupt and also calls the close() method, which causes issues because it should be thread A that calls close(). Is that correct? We could have thread B only interrupt thread A, and not close the iteration. For safety we could always have thread B close the iteration anyways after a specified amount of time. So, thread A starts the iteration, it times out so thread B interrupts thread A. Thread B sleeps for 30 seconds (or until interrupted) and checks if the iteration is closed, if not then it closes the iteration. If things work out they way they are supposed to, then thread A is interrupted and calls close(), then interrupts thread B that wakes up and sees that the iteration is already closed and then does nothing. |
Yes, that is correct. The interrupt method of TimeLimitIteration directly calls close on the underlying iteration.
Yes, that should work. The method
Yes, this could work. It is only a problem with LMDB as only one thread may use a write transaction for example. If |
|
The same logic could be added to |
Current Behavior
Instead of just setting the
isInterruptedflag, theclose()method is directly called frominterrupt().The latter method is executed within an asynchronous thread which leads to race conditions if the iterator expects non-multithreaded usage (which is the usual case).
This seems to affect the
LmdbRecordIteratorthat causes aA heap has been corruptedexception at least under Windows.Expected Behavior
The
close()method should not be called asynchronously from another thread.Maybe a global
ThreadLocalshould be introduced which provides access toisInterruptedor directly acheckInterrupted()method that throws an exception. Then its up to the individual iterators to check the interrupted state and stop the execution.Another solution would be to explicitly support the asynchronous invocation of
close()inLmdbRecordIterator.Steps To Reproduce
No response
Version
4.2.2
Are you interested in contributing a solution yourself?
Perhaps?
Anything else?
No response
The text was updated successfully, but these errors were encountered: