RQL inside index definition

[marcuslindblom]

Is it possible to execute a RQL query inside an index definition like this?

map("Homes", (doc) => {
    let posts = from "Posts" as p limit 2 // <--
    return {
        name: doc.displayName,
        posts: posts
    }
})

[ppekrol]

Hi @marcuslindblom

No, there is no possibility to do that. Only pure JS can be used there, what are you trying to do?

[marcuslindblom]

@ppekrol I'm trying to create a predefined filter on a specific collection, labels, ordered and limited by a parameters set by a user in my app.
The UI used by the users looks like this

and when the user saves the filter I save something like this on the document:

{
    "displayName": "Home",
    "labels": [],
    "origin": null,
    "slug": "/",
    "createdAt": "2024-04-08T09:22:59.9620000Z",
    "createdBy": "ffcb6a79-f1f3-4e21-b332-2713f31eeb81",
    "publishedDate": "2024-04-08T09:43:58.4782990Z",
    "deleted": false,
    "heading": "A simple blog built with Astro, Tailwind CSS and Strife",
    "posts": {
        "take": 6,
        "query": "from Posts as c where c.deleted != true AND c.labels in ('4a66f653-be65-4be6-b8be-23f3e911bfe0') order by c.publishedDate desc limit 6",
        "collection": "Posts"
    },
    "@metadata": {
        "@collection": "Landings",
        "Raven-Clr-Type": "null"
    }
}

This works but I need to create multiple queries to get the Landing document and the posts.
Something like this:

// Get the home page content
const session = store.openSession();
const home = await session
  .query({ indexName: 'Content/ByUrl' })
  .whereEquals('url', Astro.url.pathname)
  .firstOrNull();

// Get selected posts
const sortedPosts = await session.advanced.rawQuery(home.posts.query).all();

Also, this approach is not that flexible so I would rather see a different solution. Maybe create an index that is updated when the landing is changed or a new post is added, deleted or edited. In the case of creating an index, is is possible to index posts based on if a label exist and limit the indexing to a number, based on a specfic sort order? Like this query from Posts as c where c.deleted != true AND c.labels in ('4a66f653-be65-4be6-b8be-23f3e911bfe0') order by c.publishedDate desc limit 6.

Maybe you have another solution in mind after reading this?

[ayende]

In indexing, that can never happen. Remember that indexing happens before the query.
So imagine that I indexed something, and a new post comes in, what happens then? Do I saw the query results before the query, after it, etc?

At query time, technically, it is something that we can do. I'm talking about something like allowing you to execute a separate query at query time. Something like:

from index 'Content/ByUrl' as home
where home.url = $url
select {
   posts : query(home.posts.query),
   url: home.url,
   heading: home.heading
}

Then you get that in one shot.

The problem with that is that let's say that you want to return 100 results, that would mean that you issue 100s of additional queries.
And the size of the result is a problem, etc.

Is the additional roundtrip such a big issue in your scenario?

[marcuslindblom]

In this specific case one roundtrip is maybe not a big deal. But let's say we have a list of these queries like this:

    "sections": [
        {
            "@strife": {
                "template": "post-list",
                "type": "Post list",
                "name": "Awesome blog posts"
            },
            "posts": {
                "collection": "Posts",
                "take": 6,
                "query": "from Posts as c where c.deleted != true order by c.publishedDate desc limit 6"
            }
        },
        ... // more entries
    ],

This could potentially be a lot of queries.

The best solution I can come up to is keeping a list of id:s. But if I want it to be based on a filter I need to keep that list of id:s up to date with some other logic which can be hard maybe.

How would you approach this @ppekrol @ayende

[ayende]

In this case, that is still just a single round trip.
You load the document, then use lazy queries to run all of those.

The catch from our perspective is that this feature is opening the door to unbounded internal queries, which can make analyzing query costs harder, figuring out what is taking resources nearly impossible and likely going to cause additional hidden perf issues.

[marcuslindblom]

Yes, of course, using lazy would fix the extra roundtrips.