Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for pagination with Query API in Redis State Store #3329

Open
gspadotto opened this issue Feb 2, 2024 · 1 comment
Open

Add support for pagination with Query API in Redis State Store #3329

gspadotto opened this issue Feb 2, 2024 · 1 comment
Labels
help wanted Extra attention is needed kind/enhancement New feature or request

Comments

@gspadotto
Copy link
Contributor

gspadotto commented Feb 2, 2024

It seems that the paginated Query API is not being unit tested for Redis as - for example - is done here for Postgresql:

{
input: "../../../../tests/state/query/q2.json",
query: "SELECT key, value, xmin as etag FROM state WHERE value->>'state'=$1 LIMIT 2",
},
{
input: "../../../../tests/state/query/q2-token.json",
query: "SELECT key, value, xmin as etag FROM state WHERE value->>'state'=$1 LIMIT 2 OFFSET 2",
},

Expected Behavior

Add a unit test that checks that pagination works with Query API on Redis State Store

Actual Behavior

Query API on Redis State store seems not to be working correctly (at least on 1.12.4) when using "limit" and "token" parameters.

Steps to Reproduce the Problem

Try Query API with pagination parameters (token and limit) on a Redis State Store anche check that the correct responses are received.

Release Note

RELEASE NOTE: FIX Pagination management on Query API for Redis.

@gspadotto gspadotto added the kind/bug Something isn't working label Feb 2, 2024
@berndverst
Copy link
Member

berndverst commented Feb 7, 2024

Query API is poorly designed and will never become stable, so I don't advise relying on this. It will remain Alpha forever. Some other maintainers didn't agree labeling it as deprecated - but in reality, it's quasi deprecated.

Some features like pagination only works for a subset of the few state stores that implement Query API support. This is expected. From our understanding, Redis has no mechanism that would allow us to implement pagination - and we also do not consider this to be a priority.

EDIT: We use HGET and JSON.GET functions from REDIS which do not support paging. To support paging in Query API properly we would also have to change how Dapr stores data in redis using indices.

One option is to create an index from your search parameters using https://redis.io/commands/ft.create/ and then using https://redis.io/commands/ft.aggregate/ with Cursor to begin the search on that index, and then paging via https://redis.io/commands/ft.cursor-read/

PRs welcome.

@berndverst berndverst added kind/enhancement New feature or request and removed kind/bug Something isn't working labels Feb 7, 2024
@berndverst berndverst changed the title Missing unit test for pagination with Query API on Redis State Store Add support for pagination with Query API in Redis State Store Feb 7, 2024
@berndverst berndverst added the help wanted Extra attention is needed label Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed kind/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants