Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a ES Point in time to keep the same index status across V4 Search API cursor pagination. #3999

Open
albertisfu opened this issue Apr 29, 2024 · 0 comments

Comments

@albertisfu
Copy link
Contributor

In #3975 we implemented cursor pagination for the V4 Search API using the search_after parameter.

There can be some corner cases where the cursor pagination can lead to inconsistent results, for instance, if the last or first document on the current page used as the cursor is updated before moving to the next page and the field used as the sorting key (like 'dateFiled') or if sorting by relevance, the update affects the document score.

This can lead to inconsistencies in results when moving to the next or previous page, where the updated document or documents can be displayed again. To solve this issue, ES documentation recommends to use a Point in time.

A point in time (PIT) is a lightweight snapshot of the index, so all requests performed using the same PIT will query the same index states, maintaining consistency during the lifetime of the PIT.

Thus, we can consider implementing this in the future as an improvement if required. It is important to note that although the PIT is lightweight, it still represents an additional resource load. Therefore, some assessment and evaluation in terms of resource usage are recommended before deciding whether to implement it for all users or only for those requiring high consistency in results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

1 participant