You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The window operators should indicate if the returned Flux for each window is safe to subscribe to multiple times.
Documentation Issue
I had a data processing pipeline that I introduced batching to, using the window operator. In this pipeline, it will retry a batch if it faces a transient error. When batching was added, and it retried, I saw a failure of IllegalStateException: UnicastProcessor allows only a single Subscriber . Apparently the internal implementation details of window use a unicast processor!
Improvement Suggestion
Similar to how operators indicate how they handle discards, if an operator is transforming a Flux in a way that would change semantics of the Flux it is operating on, it could call that out.
In practical terms, the window operators should indicate that a window can only be subscribed to once. This may also be true for other operators that do a transformation of Flux<?> to Flux<Flux<?>> such as groupBy
The text was updated successfully, but these errors were encountered:
indeed, all these operators produce Unicast windows. in terms of retry, the expectation is that it occurs at the outer level rather than the inner level, because windowing is just splitting a live source, so there's no predefined way of solving what "retry a subset of the source" means.
on the other hand, I think this kind of use cases could work better with buffer rather than window, although the compromise here is that buffer emit the whole subsequence at the end, not as matching elements get published.
👍 My fix was to use buffer rather than window. I believe I could have used window and cache on the subsequences to the same effect.
I wasn't really thinking things through; I was expecting each window to act like a Flux.just and support multiple subscriptions, thus the documentation suggestion.
This commit clarifies that window operators produce inner Flux (windows)
that are unicast. This is also true of `GroupedFlux` produced by the
`groupBy` operator.
Fixes#2568.
The
window
operators should indicate if the returned Flux for each window is safe to subscribe to multiple times.Documentation Issue
I had a data processing pipeline that I introduced batching to, using the
window
operator. In this pipeline, it will retry a batch if it faces a transient error. When batching was added, and it retried, I saw a failure ofIllegalStateException: UnicastProcessor allows only a single Subscriber
. Apparently the internal implementation details ofwindow
use a unicast processor!Improvement Suggestion
Similar to how operators indicate how they handle discards, if an operator is transforming a Flux in a way that would change semantics of the Flux it is operating on, it could call that out.
In practical terms, the
window
operators should indicate that a window can only be subscribed to once. This may also be true for other operators that do a transformation ofFlux<?>
toFlux<Flux<?>>
such asgroupBy
The text was updated successfully, but these errors were encountered: