blog/react-query-data-transformations

[utterances-bot]

React Query Data Transformations | TkDodo's blog

Learn the possibilities to perform the quite common and important task of transforming your data with react-query

https://tkdodo.eu/blog/react-query-data-transformations

[hrafnkellpalsson]

The select is nifty! Haven't tried out v3 yet, looking forward to trying out.

As a slight variation on 1 I've sometimes put both the untouched server data and the derived data into the cache, something like

const fetchTodos = async (): Promise<Todos> => {
    const response = await axios.get('todos')
    const data: Todos = response.data
      return {
        server: data,
        derived: data.map((todo) => todo.name.toUpperCase())
      }
}

[TkDodo]

That looks good if you need both the raw data and the derived data in the cache 👍

[bebiangel]

Wow!! That's really good.👍 Thanks Dominik!🙏

[onkarj422]

Hello, thanks for sharing these useful insights!

What different does it make if the select function is an inline one vs a stable function reference?

[TkDodo]

What different does it make if the select function is an inline one vs a stable function reference?

if it's an inline function, it will be executed on every render, even if neither data nor any other input to the function has changed, because the identity of the function itself has changed. This is only relevant if your component renders for another reason, and also only if your transformation is very expensive.

I suggest to just start with an inline function, and if you measure some slowness, you can improve it. It's likely perfectly fine to inline it :)

[loia5tqd001]

for derived selectors (selectors that based on another selectors), I'm making a custom hook that return React.useMemo wrapping around the expensive calculation. Is it recommended? Or should I use something like reselect to create stable selectors @TkDodo?

notice use useTodoWithId and useTodoWithId2 in my example

export const useTodosQuery = (select) =>
  useQuery({
    queryKey: ['todos'],
    queryFn: fetchTodos,
    select,
  })
export const useTodosData = (select) => useTodosQuery(select).data; // don't care about the statuses
export const useTodosCount = () => useTodosData((data) => data.length);
export const useTodoWithId = (id) => {
  const data = useTodosData(); // subscribe to the whole todos
  return React.useMemo(() => data.find((todo) => todo.id === id), [data, id]);
}
export const useTodoWithId2 = (id) => useTodosData((data) => data.find((todo) => todo.id === id));

function ToDoCount() {
  const count = useTodosCount(); // subscribe to the count, if items change but count doesn't, this component won't re-render (WOW)
  return <div>{count}</div>;
}
function ToDo() {
  const todo = useTodoWithId(id); // subscribe to the the whole todos because the hook itself subscribes to the whole todos
  return <div>{todo.name}</div>;
}
function ToDo2() {
  const todo = useTodoWithId2(id); // subscribe to ...???
  return <div>{todo.name}</div>;
}

EDIT: I've just read your other comment. Looks like you are slightly against the useMemo approach. Thanks.

[jacobworrel]

Thank you for this great article series!

Correct me if I'm wrong, but since the select option runs once per component instance, what would you recommend if you only want a data transformation to run once for multiple component instances? For example, let's say we have a useQuery call with a select option that runs an expensive computation wrapped in a custom hook and that custom hook is called by many components throughout an application. Is the only option then to move the data transformation to the queryFn and have it run once per fetch?

[TkDodo]

If the select function is stable, it will only execute when data changes. So even if you have it in multiple components, it will not run when re-rendering, and even if you have a background refetch, it will not run unless the data is different (after structural sharing). So yeah I'd do that first, because moving it to the queryFn means it will run with every request, which might be more often.

[loia5tqd001]

unless the data is different

I expect that the data's value might be the same but the reference would be different after each refetch, no? @TkDodo

EDIT: I've just read your other comment. Thanks.

[jacobworrel]

Thanks for the clarification! I made a codesandbox to better illustrate the example I'm talking about where the select function is called once per component that's consuming the custom hook with the useQuery call, despite having a stable function reference (I think). What am I missing?

[TkDodo]

You’re right. select is a per-observer option, so every usage will have it's own select memory, which means it has to at least run once per observer.

It seems that indeed the only other option is to move it into the queryFn. I don’t think there is a way to transform there only once, but something like memoize-one could help.

[loia5tqd001]

Hmm I've tried memoize-one and it doesn't work also @TkDodo

https://codesandbox.io/p/sandbox/sweet-hellman-zs9ql8?file=%2Fsrc%2Findex.js

It only works (expensive calculation is calculated only once) when I use constate to leverage context to distribute the query (can see in the codestandbox as well)

EDIT: replaced with proxy-memoize instead and it works like charm

[wobsoriano]

I'm having problems with useInfiniteQuery and select option.

export function useConversation(to: string) {
  const user = useUser();

  return useInfiniteQuery<MessageListResponse>(
    [QueryKeys.MessageList, { userId: user?.guid, to }],
    ({ pageParam = 1 }) => {
      return $axios.$post(`/api/${QueryKeys.MessageList}`, {
        guid: user.value?.guid,
        to: to,
        offset: pageParam
      });
    },
    {
      getNextPageParam,
      enabled: !!to,
      select: (data) => { // type error
        const pages = data.pages.map((page) => {
          return  transformList(page.payload.list);
        });

        return {
          ...data,
          pages,
        };
      },
    }
  );
}

Without the select property, the type I'm getting is

undefined | InfiniteData<MessageListResponse>

But with the select option, I'm getting type errors.

[TkDodo]

@wobsoriano I can take a closer look at it if you can create a minimal example in codesandbox

[wobsoriano]

@wobsoriano I can take a closer look at it if you can create a minimal example in codesandbox

Here you go https://codesandbox.io/s/use-query-infinite-ts-583sp

[TkDodo]

Ah yes, you cannot use select to change the whole structure with an InfiniteQuery. You need to still return an object with pages and pageParams (known as InfiniteData). You can structure the pages however you like though. See also this discussion: TanStack/query#1410

[wobsoriano]

Ah yes, you cannot use select to change the whole structure with an InfiniteQuery. You need to still return an object with pages and pageParams (known as InfiniteData). You can structure the pages however you like though. See also this discussion: tannerlinsley/react-query#1410

Ah, that makes sense. I updated my example and used your 2nd option instead. Works fine.

[phutngo]

Thank you so much for this guide! I find myself keep coming back here as a reference.
A couple of months back, we chose to use the "select" method to do our data transformations and it's been working out great. It seems like it's the most flexible pattern, and the code is easy to read and understand as well.

[ver1fy]

Great job on these series of posts, thanks! I've been using the second option in one of my projects and wondered if this is still relevant in v4? The reason being is that using the spread operator on ...queryInfo appears to break tracking and leads to unnecessary renders. There is some discussion about it in this thread.

To give some context, I'm trying to use local state as a fallback if the server state isn't available (recommended here). Perhaps a selector is the way forward, although it doesn't feel 'right' to closure over data in select that isn't stored in the query cache. Any advice/insight is much appreciated.

[TkDodo]

yeah I would avoid ...queryInfo in v4 because of tracked queries. I would use select over useMemo almost always.

[jamezening]

Awesome article as usual 🔥
have been using react-query for a while and now see the need for selectors.

so basically do we create a new query hook for each selected slice of data?
just worried that I will be creating too many selectors - is there another pattern to manage many different selected pieces of data without creating too many files / separate query hooks?

[ver1fy]

You can simply pass a selector to your custom hook as described in section 3:

export const useTodosQuery = (select) =>
  useQuery({ queryKey: ['todos'], queryFn: fetchTodos, select })

[tielushko]

Thank you for such informative article. I had a question regarding it. I am trying to implement the select in my TypeScript project. I feel like the return type of the select can be very generic. What would be the suggested way of approaching this?

[TkDodo]

export const useTodosQuery = <TData = Todo[])(select?: (data: Todo[]) => TData) =>
  useQuery({ queryKey: ['todos'], queryFn: fetchTodos, select })

[tielushko]

Thank you for the reply and this example. I noticed the usage of TData but wasn't sure of it because it wasn't an exported type or anything. This makes sense now, thanks again!

[bradreelee]

Hi, thank you for introducing great features of React Query with this great article.

I have one question regarding the last example code.
In the code, the selector functions are again inline functions. Does this mean that the selector functions are then ran on every redner?

thanks in advance!

[TkDodo]

yes, inline selectors are run on every render. But that's usually not a problem unless the selector itself is expensive.

(data) => data.find((todo) => todo.id === id)

is likely not expensive, it's a linear search on an array that hopefully doesn't have thousands of records :)

[joel-daros]

When using the select function to filter data, is there a better way to get the original data than returning it back in a object along with the filtered data in the select function?

[ver1fy]

I’d go with two separate queries: one for the original data and one for the filtered data. If you are using the same query key, the library won’t instigate another network request. Here’s an example of this in practice from the blog post:

export const useTodosQuery = (select) =>
  useQuery({ queryKey: ['todos'], queryFn: fetchTodos, select })

export const useTodosCount = () => useTodosQuery((data) => data.length)
export const useTodo = (id) =>
  useTodosQuery((data) => data.find((todo) => todo.id === id))

[AayushKarki714]

Thank you for the amazing write up,
When does the data stored by use query actually get's changed

"data inside the queryInfo will be referentially stable unless something really changed"

does that changes every time the function passed to the useQuery get's invoked

[TkDodo]

no, there's a process of structurally sharing data that comes back from the network with previous data already stored in the cache. You can read more about it here:

https://tkdodo.eu/blog/react-query-render-optimizations#structural-sharing

[mosnicholas]

@TkDodo, this looks great. I'm curious about typescript implementation. When I try something like:

const fetchSingleTodo = (data, id) => data.find((todo) => todo.id === id);

export const useTodosQuery = (select) =>
  useQuery({ queryKey: ['todos'], queryFn: fetchTodos, select })

export const useTodosCount = () => useTodosQuery((data) => data.length)
export const useTodo = (id) =>
  useTodosQuery((data) => fetchSingleTodo(data, id))

I keep getting typescript errors on the useTodo-like function:

Type 'Todo' is missing the following properties from type 'Todo[]': length, pop, push, concat, and 31 more

Have I got something incorrectly typed? Or is this expected?

Also -- when I console.log in fetchSingleTodo, it logs multiple times for the same ID and data. Is it not cached?

[joel-daros]

I think your select function should return a filtered array of Todos, not just a number value like data.length you are returning in useTodosCount. The same applies to fetchSingleTodo, instead of data.find() you should use data.filter() that returns an array rather than a primitive.

[mosnicholas]

This feels wrong (1) because the examples return a single record ie. using data.find, and (2) because I want the structure of the data response to be a single record, and not have to do ?.[0] on the output each time.

[joel-daros]

Yes, you’re right, sorry for my mistake. This is a suggestion of your code using TS implementation:

import { useQuery } from '@tanstack/react-query';

type Todo = {
  id: number;
  name: string;
};

const fetchTodos = (): Todo[] => [
  {
    id: 1,
    name: 'wash dishes',
  },
  {
    id: 2,
    name: 'clean the house',
  },
];

export const useTodosQuery = <
  T extends number | Todo | Todo[] | undefined = Todo[]
>(
  select: (data: Todo[]) => T
) => useQuery({ queryKey: ['todos'], queryFn: fetchTodos, select });

export const useTodosCount = () => useTodosQuery((data: Todo[]) => data.length);

export const useTodo = (id: number) =>
  useTodosQuery((data) => data.find((todo) => todo.id === id));

[TkDodo]

export const useTodo = (id) =>
  useTodosQuery((data) => fetchSingleTodo(data, id))

this doesn't work. You can't have a select function be async and do another fetch. If you have a separate endpoint to fetch a single todo, you need a separate useTodo(id: number) hook with its own queryKey and queryFn.

[hadson172]

Hi TkDodo,
Maybe you wil be help me with this small confusion.
We are using react-query as state manager (all refetch disabled).

We heavly depend on select option for useQuery. One thing bothers me is that calculation inside select are always made when the hook is called.

So for example if we mapping 1000 objects inside (select) prop then reusing this hooks in 10 different places causes mapping 10 000 objects for the scope of whole application.

Does it mean that we have to use different state manager for this ?

Is there any possibility to make sure that select is executed once for the scope of whole app ? Thanks

I was thinking maybe if would be good idea to put mapped data into cache instead of select but is it only solution?
Thanks

[TkDodo]

You can combine select with something like reselect:

import { createSelector } from 'reselect'

const mySelector = createSelector(
  data => data,
  data => doYourTransformation(data)
)

const useMyQuery = useQuery({
  queryKey,
  queryFn,
  select: mySelector
})

createSelector will create a separate selector cache that only depends on data, so no matter how often and in how many components you will call this selector, it will only be executed if data changed.

[georgehua]

Hi TkDodo,
I have a need to initiate an API call when user click a button, so I set useQuery() enabled property to false and use refetch() to trigger the query. And I also want to create some computed properties based on the raw data from API, so I tried to use the "select" option you introduced in this article. However I keep getting undefined value for useVideoTitle(). I'm suspecting it's caused by the enabled property as false since that's the only difference to your example. But I can't find a way to trigger API call with button click as well as using "select" option properly, can you help me on this?
Thank you very much!!

use-video.ts
export const useVideoQuery = (videoUrl = null, select = null) =>
    useQuery({
        queryKey: ["video", videoUrl],
        queryFn: async () => {
            const response = await fetch(videoUrl);
            const data = response.json();
            return data;
        },
        enabled: false,
        select
    });

export const useVideoTitle = () => useVideoQuery(null, data => data.title)

page.tsx
const { data: videoData, isLoading, error, refetch } = useVideoQuery(url);
const { data: videoTitle } = useVideoTitle();
useEffect(() => {
   console.log(videoTitle);  // always print undefined
}, [videoTitle]);

<Button onClick={() => {refetch()}} />

[georgehua]

Thank you for the response, and sorry for not making the question clear. I made a github repo to demo the problem: https://github.com/georgehua/TanStack-Query-select-option-problem-demo
I think I did return in the "select" function, but it seems not picking up the react-query results, would you mind take a look at the repo? Many Thanks!!

[TkDodo]

for me, the query is in error state with the error:

{"name":"TypeError","message":"Failed to fetch"}

[georgehua]

Are you able to access this Codesandbox? The query works in their environment.

[georgehua]

I think I figured it out... The queryKey I passed in contains the user input url. And when I create the select function, I pass the url as null because I don't want to re-trigger the query. So I assume react-query can't access the cached query result because of the key changing. After I removed the dynamic part of the key, the select returning values now.

[ak274]

I have two APIs One returns all Users in all orgs and other returns only ids of users in a org.

Now as I have already all the users from one query and filtered out user Ids of specific org from other, I want to filter out users that belong to one org from all users with all their details. I'm following the approach but it doesn't feel good(Now i have to check in my component whether both returned resolved or if any of them in error state). Is there a better way to combine these quires and when both are successful return one single query with desired data and other statues?

const { data: userIdsByOrg = [] } = useUsersByOrgId(orgId);
const { data: users = [] } = useUsersByOrganizationId(id);

const assignedUsers = users.filter((user) => userIdsByOrg.includes(user.id));

 return <div>LicensedUser {JSON.stringify(assignedUsers)}</div>;

[lrodriguez-rsi]

Hi community and @TkDodo, I've been using selector to transform my data, it is great, however, I was wondering how should I test it?

In my test I have this

  beforeEach(() =>
    (useSearchByIndexQuery as jest.Mock).mockImplementation(() => ({
      data: mockSearchData,
      refetch: jest.fn(),
      isFetching: false,
    }))

So my mocked data has the shape as the server shape, but with select I'm transforming it.
I noticed that when using select to transform my data, the select function is not run (of course) because I'm injecting my data directly in my test,
So my test fails because the component is working with the transformed data, but the test is injecting the server-shape data.

I do not want to use the transformed data as my Mock because that won't test the transform function.

what I did was extract the Transform function and export it,

export function transformSearchResults(data: SearchByIndexQuery) {}


useQuery(...
    select: transformSearchResults,

and finally in my test imported the transform function

(useSearchByIndexQuery as jest.Mock).mockImplementation(() => ({
      data: transformSearchResults(mockSearchData),
      refetch: jest.fn(),
      isFetching: false,
    }))

in that way, the test is also testing the transform function, but is this the correct approach? is there other better?

[joel-daros]

You shouldn’t mock your useSearchByIndexQuery hook in a test. The best way to test your function is let it execute the same ways as it does in your real app. You can do this using MSW (https://mswjs.io) to mock the backend responses. I think this Tkdodo post has all the information you need: https://tkdodo.eu/blog/testing-react-query

[lrodriguez-rsi]

Awesome and thanks for the quick response, I've mocked the network layer with MSW before, but for some reason in a lot of tests we've been mocking our functions xD maybe one did it and the other ones followed that pattern. It is true now with your answer I'm more inclined to think that the best approach is using MSW, in that way the transform function and the query will invoke everything that normally would invoke.
Thanks @joel-daros!

[nunoplima]

Hey TKDodo! Awesome blog post!

I'm wondering if the select fn depends on other dynamic data to have its result enriched, having that dynamic data as a dependency of the select wrapped in a useCallback, will make it rerun once the dynamic data changes without triggering a refetch? is it a good practice?

Thank you

[TkDodo]

will make it rerun once the dynamic data changes without triggering a refetch?

yes.

is it a good practice?

it's necessary to re-run select if a dependency for it changes, so yes.

[canidam]

TkDodo thanks for making this blog, it's been a great resource for learning!

I've encountered an issue I can't figure out, I hope the community here can show me where I'm wrong.
I've a custom hook where I fetch and transform the data using select. The data doesn't change often, so I wanted to use useCallback/useMemo to memoize the result. I added a log print to see when my useCallback is executed, and was surprised to see it is executed on every render. I thought it should run only when the data actually changes, no?

Thanks!

export const useDataQuery = () => {
    return useQuery({
        queryKey: ["/data"],
        queryFn: getData,
        select: useCallback((data: DataProps[]): Map<string, DataProps[]> => {
            const dataMap = new Map<string, dataProps[]>()
            data?.forEach((element) => {
                const dataDate = getDataDate(new Date(element.time))

                if (!dataMap.has(dataDate)) {
                    dataMap.set(dataDate, [])
                }

                const dataOnDate = dataMap.get(dataDate)
                dataMap.push(element)
            })

            console.log("new DataMap")
            return dataMap
        }, []),
    })
}

// component.js
export const ListData = () => {
    const { data: dataMap } = useDataQuery()

    ... component logic ...

[canidam]

The way I checked if it renders is seeing the console.log print in the console when navigating between pages. Is my assumption wrong?

[TkDodo]

when you navigate between pages, one page unmounts and the other remounts. I think you have a misunderstanding between rendering and mounting in React.

[canidam]

I see. Thanks for the reply. Is there a right way to check if useCallback/useMemo works properly?

[TkDodo]

it works properly if it specifies all dependencies in the dependency array.

[canidam]

I had a misunderstanding of remounts. This helped me realize that, thanks for the comments

[jamezening]

Continuing to come back to various articles in your awesome blog - thanks again for these resources.

When implementing 3. using the select option using the version that passes select to the query in the select-partial-subscriptions example, what would be the best way to manage the query key so that data is cached properly for when there is no selector passed, and for when any multiple numbers of select functions are passed?

  return useQuery<IMemberFollow[] | boolean>({
    queryKey: [QueryKeys.following, memberId, select],
    queryFn: () => new Promise(() => {}),
    enabled: !!memberId,
    select,
  });

we are using v5 - does passing the select function to the query key properly cache each situation?

[TkDodo]

select has nothing to do with caching. You'll always cache what the queryFn returns. select then lets components subscribe to parts of the state. If you have experience with redux, it's a similar concept: There's a store and you then leverage selectors with useSelector to subscribe your component to the part that you're selecting.

So select doesn't go into the queryKey (it's a function, that doesn't work anyway), because only things used in the queryFn need to go into the key.

[jamezening]

Awesome - that is how I expected it to work.

Another related question:
our team is a bit squirmish about having to add the select related useQuery return type.
Is there another way / best practice for the select return type?

[TkDodo]

generics: https://stackoverflow.com/questions/75712953/react-query-select-inference

[Harsh-Sharma25]

I want to use this select option in an enabled query & I'm facing type issue with select option & can't get my head around it
I've this query

  const useGradeTax = (id: string, options?: UseQueryOptions<TaxonomyData[]>) =>
    useQuery<TaxonomyData[]>(['grade', id], () => getGradeTaxonomy(id), options);

& I want to possibly use it like this

  const { data, isLoading: gradeLoading } = useGradeTax(`${gradeSelected}`, {
    ...DEFAULT_OPTIONS,
    enabled: !!gradeSelected,
    select: (data) => data?.find((item) => item.id === +(subjectId ?? 0)),
  });

and this is the type of function
const getGradeTaxonomy: (gradeId: string) => Promise<TaxonomyData[]>
and it basically says that it is expecting an array return
Type 'TaxonomyData' is missing the following properties from type 'TaxonomyData[]': length, pop, push, concat

[TkDodo]

same answer as above:

• https://stackoverflow.com/questions/75712953/react-query-select-inference

additionally:

• don't try to use UseQueryOptions unless you specify all 4 type parameters
• don't pass the type parameter directly to useQuery<TaxonomyData[]>, see the docs on type inference: https://tanstack.com/query/v5/docs/react/typescript#type-inference

[pranavdhamanage-work]

absolutely 🔥 blog post.. just loving it 👍🏽