Introduce Infinite Query Support

[riqts]

This PR aims to introduce the InfiniteQuery to RTKQ. My current strategy is to basically follow Lenz's suggestion in the RTKQ Infinite Query thread.

This PR is a starting point for discussion. Feedback is welcome on the overall approach and implementation details. Its current state is just my initial method of adding the new endpoint definition and I am currently working out the best way to turn the infiniteQuery initiate into something that can fetch multiple args from selection.

This new definition allows for the following:

• selection Function: Handles the core logic of fetching data in an infinite fashion, determining when to make subsequent requests.
• nestedQuery Option: Provides the ability to integrate dependent queries that retrieve additional details for each chunk of data.
• Added two Options that can be declared in the endpoint or the hook (similar to react-queries answer for having the user provide a function to select):

export type GetNextPageParamFunction<TPageParam, TQueryFnData = unknown> = (
  lastPage: TQueryFnData,
  allPages: Array<TQueryFnData>,
  lastPageParam: TPageParam,
  allPageParams: Array<TPageParam>,
) => TPageParam | undefined | null

export type InfiniteQueryConfigOptions<TQueryFnData = unknown, TPageParam = unknown> = {
  getPreviousPageParam?: GetPreviousPageParamFunction<TPageParam, TQueryFnData>
  getNextPageParam: GetNextPageParamFunction<TPageParam, TQueryFnData>
}

• Integration into EndpointBuilder: The infiniteQuery method is added to the builder, aligning with the standard RTKQ pattern.

Example Usage:

import { createApi, fetchBaseQuery } from '@reduxjs/toolkit/query/react';

const api = createApi({
  // ... other api properties
  endpoints: (builder) => ({
    getInfiniteItems: builder.infiniteQuery({
      getNextPageParam: async ({ startCursor, limit }, { getState, dispatch }) => {
        // ... logic to determine the args to be passed to the query fn
      },
      Query: { 
        query: (initialPage) => `/items/page/${initialPage}`,
      },
    }),
  }),
});

Open Questions

• Type Structure: Is the InfiniteQueryDefinition structure the most suitable approach, or should infinite query logic be incorporated differently within existing definitions?
• Error Handling: What are best practices for error handling and retry logic within the selection function?
• The current Draft implementation uses its own thunk that I made just for the sake of having TS stop complaining and get the core query logic in. However, I eventually plan to utilise the existing query functionality more. What logic should primarily be handled by middleware in this?

Is there any query logic/options that we would NOT want to be accessible by the infiniteQuery? Or should I default to no access to additional options i.e. subscription

Code Changes

• Added InfiniteQueryDefinition, modifications to EndpointDefinitions, etc.
• Integration of infiniteQuery into the EndpointBuilder.
• Initiate + InfiniteQueryThunk/ExecuteEndpoint logic to fetch all the pages

Draft Status: This PR is a starting point for discussion. Feedback is welcome on the overall approach and implementation details.

[codesandbox]

Review or Edit in CodeSandbox

Open the branch in Web Editor • VS Code • Insiders

Open Preview

[netlify]

✅ Deploy Preview for redux-starter-kit-docs ready!

Name	Link
🔨 Latest commit	e0c952f
🔍 Latest deploy log	https://app.netlify.com/sites/redux-starter-kit-docs/deploys/66279bbdb2c2eb0008c262d8
😎 Deploy Preview	https://deploy-preview-4249--redux-starter-kit-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[phryneas]

Interesting!

Is there a particular reason you went for nestedQuery here with another builder.query call instead of having a query/queryFn in infiniteQuery itself?

The more this API would look like query/mutation, the better it would probably be 🤔

[riqts]

Is there a particular reason you went for nestedQuery here with another builder.query call instead of having a query/queryFn in infiniteQuery itself?

The more this API would look like query/mutation, the better it would probably be 🤔

Ah, actually just a mistake in my examples! I was trying to represent that the "query" part of the new definition runs using the standard query initiate/thunk. My intention is to have it act very very similarly to the query definition with the only difference being:

1. Runs a new added option selection() BEFORE the queryInitiate to determine args and potentially fires multiple queryInitiate
2. Selector merges all args result

And then the list will become much bigger once we start addressing the options :D

[riqts]

I updated the top comment as an overview as well, but I'll leave a timeline comment too for better visibility/discussion purposes.

@phryneas Would it be more likely that the query/queryFn would have access to all the normal extraOptions that a query/queryFn has? i.e.merge,polling. Is there any obvious reason to restrict the API?

Changes and Current process:

1. Added to the initiate two functions:

              refetch: () =>
                dispatch(
                  infiniteQueryAction(arg, { subscribe: false, forceRefetch: true })
                ),
              fetchNextPage: () =>
                dispatch(
                  infiniteQueryAction(arg, { subscribe: false, forceRefetch: true, direction: "forward"})
                ),
              fetchPreviousPage: () =>
                dispatch(
                  infiniteQueryAction(arg, {subscribe: false, forceRefetch: true, direction: "backwards"})
                ),

2. added new infiniteQueryThunk and changed data to return { pages: [], pagesParam: [] } for infiniteQueries

          const thunk = infiniteQueryThunk({
            type: 'query',
            ...
            queryCacheKey,
            data,
            param,
            previous,
            direction
          })         

3. fetchPage loop happens within executeEndpoint

if (arg.direction && arg.data.pages.length) {

            const previous = arg.direction === 'backwards'
            const pageParamFn = previous ? getPreviousPageParam : getNextPageParam
            const oldData = arg.data
            const param = pageParamFn(arg.infiniteQueryOptions, oldData)

            result = await fetchPage(oldData, param, previous)
          } else {
            // Fetch first page
            result = await fetchPage(
              { pages: [], pageParams: [] },
              oldPageParams[0] ?? arg.originalArgs,
            )

            //original
            // const remainingPages = pages ?? oldPages.length
            const remainingPages = oldPages.length

            // Fetch remaining pages
            for (let i = 1; i < remainingPages; i++) {
              const param = getNextPageParam(arg.infiniteQueryOptions, result.data as InfiniteData<unknown>)
              result = await fetchPage(result.data as InfiniteData<unknown>, param)
            }
          }

Notes/Considerations

• This is a fairly primitive implementation of react-queries method of having the next/prev page selector provided by the user as an option
• I don't use a slice or middleware yet, its entirely handled in the queryThunk/execute endpoint
• it currently is a completely new infiniteQueryThunk action. Depending on whether full access to the extraOptions for a query should be provided in the API or not, this will change to either just sharing the default queryThunk with some changed options, or it will have to have new functionality added as well for substate/lifecycle etc