Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support lazy enumerators for @stream #4920

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

rmosolgo
Copy link
Owner

@rmosolgo rmosolgo commented Apr 17, 2024

Fixes #4907

TODO:

  • Maintain compatibility with existing batching
  • Add a test for this behavior in particular
  • Find an implementation other than recursion

@rmosolgo
Copy link
Owner Author

Looking at how this played out, I have a few big questions still to work out:

  • Currently, each list item calls the next one via recursion. This will make really big Ruby stacks, hogging memory. A better approach would be "trampolining", something I spiked several years ago, where different steps of execution actually return to the main caller, which dispatches steps. That change might be necessary to support this feature effectively.
  • The current batching behavior expects each list item to request its own data, then resolve requests for all list items in a single batch. That's impossible with this approach where each item is handled separately. How can I support both without expensive .is_a? checks on every object? Maybe detect @stream in particular?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Lazy Enumerators with Stream
1 participant