Structuring the Active Record Query Guide

[Ridhwana]

We're currently in the process of refining our extensive Active Record Query Guide. As it stands, this guide spans almost 3,000 lines of code and comprises 22 sections, covering a comprehensive range of querying techniques.

We recognize the importance of ensuring that this guide is not only comprehensive but also well-structured for ease of understanding and navigation. To achieve this goal, we're looking for insights and feedback on the guide's structural layout.

Specifically, we're interested in hearing your thoughts on:

• Suggestions for restructuring existing sections to improve clarity and accessibility
• Identifying any missing sections that could enhance the guide's completeness
• General ideas on how to enhance the overall structure of the guide

[AmandaPerino]

Current chapters:

1. What is the Active Record Query Interface?
2. Retrieving Objects from the Database

• Retrieving a Single Object
• Retrieving Multiple Objects in Batches

3. Conditions

• Pure String Conditions
• Array Conditions
• Hash Conditions
• NOT Conditions
• OR Conditions
• AND Conditions

4. Ordering
5. Selecting Specific Fields
6. Limit and Offset
7. Grouping

• Total of Grouped Items
• HAVING Conditions

8. Overriding Conditions

• unscope
• only
• reselect
• reorder
• reverse_order
• rewhere
• regroup

9. Null Relation
10. Readonly Objects
11. Locking Records for Update

• Optimistic Locking
• Pessimistic Locking

12. Joining Tables

• joins
• left_outer_joins
• where.associated and where.missing

13. Eager Loading Associations

• N + 1 Queries Problem
• includes
• preload
• eager_load
• strict_loading
• strict_loading!

14. Scopes

• Passing in Arguments
• Using Conditionals
• Applying a Default Scope
• Merging of Scopes
• Removing All Scoping

15. Dynamic Finders
16. Enums
17. Understanding Method Chaining

• Retrieving Filtered Data from Multiple Tables
• Retrieving Specific Data from Multiple Tables

18. Find or Build a New Object

• find_or_create_by
• find_or_create_by!
• find_or_initialize_by

19. Finding by SQL

• select_all
• pluck
• pick
• ids

20. Existence of Objects
21. Calculations

• count
• average
• minimum
• maximum
• sum

22. Running EXPLAIN

• Explain Options
• Interpreting EXPLAIN

[Rovel]

Hi great initiative, about the:

• Identifying any missing sections that could enhance the guide's completeness
  I would love more info on querying polymorphic relationships since there is one mention of it the docs and in association_basics there is no query example in it.

[biske]

I would ask who's target audience. Are we trying to improve documentation for the beginners or advanced users?
@AmandaPerino

[AmandaPerino]

I would ask who's target audience. Are we trying to improve documentation for the beginners or advanced users? @AmandaPerino

That's a great question @biske. These guides are meant to be more for beginners/mid-level.

Writing aimed at beginners (with more context) will not put off a more experienced developer, so essentially it will serve both audiences. It is further likely that the more advanced user would head straight to the API docs first anyway.

We are hoping that beginners will come to these guides searching for an answer (and find it), but also discover a load of other possibilities on how Active Record (in this example) can be utilized.

[brunoprietog]

I don't have a strong opinion on this, but maybe it would be good to change the order of the chapters. Kind of like going up in level of complexity a little at a time and by level of importance or by the most frequent things first. For example, scopes or enums are way down and are things that are used quite a bit. Likewise, selecting specific fields, unscope, I don't feel like it's something that a beginner would use to present it so early on. I don't want it to be misunderstood in the sense that this shouldn't be. I'm just saying that maybe the order in which it's taught can be quite influential in a junior developer being able to understand things more quickly.

[viniciusalonso]

I took a quick look at stackoverflow researching by the terms activerecord and ruby on rails in most frequent tab. The first question is about case-insensitive search in rails model an example of something with could more explicit in docs.

https://stackoverflow.com/questions/tagged/activerecord%2bruby-on-rails?tab=Frequent

I believe that stackoverflow would be useful for this kind of research.

[borama]

Hi, I love this effort! In the guide, I sometimes miss a brief mention of async queries and calculations, possibly also its strengths and limits.

[yoav-epstein]

maybe a subquery section close to the join section.

[mkrisher]

I would second the comment about changing the order of the chapters. Maybe a consistent example that progresses through the chapters would be helpful?

A quick example regarding the chapters, maybe moving pluck up towards the top. I find that to be used pretty frequently in relatively simple use cases. I can see beginners thinking "how do I just get the :id of a record?" for example.

Some chapters could probably use some more examples and/or clarity. JOINS and INCLUDES for example. Joins currently only has examples for a single association. Maybe add an example with nested associations?

This is a big effort. I'm happy to help contribute.

[noccor]

Hey @Ridhwana ,

Wow this is a long piece of (much needed) guide documentation. I have some feedback based on how I try and structure the technical copy at AppSignal. This feedback is mostly related to the structure of the copy rather than the structure of the documentation.

As the guide is quite long, and this is not a PR I thought I'd put some focus here and try and keep my as feedback brief as possible. Please let me know if anything I say is unclear!

1. Explain the aim of documentation in the intro

In the section 1 What is the Active Record Query Interface? I'd say there is no harm in re-asserting the aim of the document:

"This guide will give you a deeper understanding of the Active Record Query Interface, introducing the core concepts and methods needed to efficiently manage Active Record queries."

Of course, there is a "this guide will help you" section at the top of the page, but adding a sentence or two outlining the guide's intent will help further clarify the guide, make the focus clear, and catch the attention of people who may scroll past the bullet list at the top of the page.

2. Model introduction:

Regarding the introduction to example models in the section 1 What is the Active Record Query Interface?, I would recommend putting this content in its own sub-section (ie "Example models") and having the following structure:

1. Brief explanation of the example app: "In this guide, we'll use a book store app as an example..."
2. The diagram of the db_schema (consider adding created_at, updated_at columns to the models to make it more explicit)
3. The example model code blocks

Also note: With domain-specific terms like ISBN, I'd recommend explaining quickly what this is/stands for, so in the event, a reader is uncertain they don't have to leave the guide for more context.

3. Descriptive tone of voice and accessible language

Using the section Retrieving Objects from the Database as an example, when describing an action, I always favour a descriptive tone of voice (using the ing form of verbs). I'd also advise using more accessible language. For example, terms like "fire a query" or "Instantiate a class" may not be clear to all English speakers, we can use words like "create" and "execute" instead:

• Converting the supplied options to an equivalent SQL query.
• Executing the SQL query and retrieving the corresponding results from the database.
• Creating the equivalent Ruby object of the appropriate model for every resulting row.
• Running after_find and then after_initialize callbacks, if any.

4. Describe concepts before demonstrating them

It can be beneficial to quickly explain what something is before demonstrating it. For example, in the 3.4 NOT Conditions section, a short explanation could be added before the example:

A not condition fetches records that do not match your Active Record query, for example when trying to find customers that do not have an order_count of 1, 3 or 5...

Another example is with N+1 queries problem, where it may be beneficial to explain what an N+1 query is before walking a reader through an example of one.

This helps keep the docs explicit and scannable and provides greater context to lesser senior developers before introducing them to code samples.

5. Lists that link

When a list is listing concepts the documentation describes, make that list item link to the relevant docs section to avoid the reader having to scroll down toward it.

Final tiny nit-pick: In Retrieving Multiple Objects in Batches it says "as" when I think it is meant to say "such as".

[AmandaPerino]

Hey @Ridhwana ,

Wow this is a long piece of (much needed) guide documentation. I have some feedback based on how I try and structure the technical copy at AppSignal. This feedback is mostly related to the structure of the copy rather than the structure of the documentation.

Hi @noccor thanks for your thoughts on the structure of the copy. We were specifically posting to request information about the structure of topics, but it never hurts to hear feedback about the copy, too, or processes that other technical writers follow.

@Ridhwana and @bhumi1102 have been working on updating the guides since early February (Guides that have been merged), so I think you'll be pleased to see that they follow a similar approach with their own technical writing. In particular you can see all of your advice reflected in the Active Model Basics Guide which Ridhwana recently worked on.

This level of feedback would be really appreciated once the writers have edited each guide and pushed their PRs to GitHub for community review, so we would love it if you could keep an eye out for those. We have approximately 1-2 a week up for review.

Thanks!

[noccor]

No problem @AmandaPerino, thanks for your response. I'll keep an eye out for up-coming PRs where this feedback is more valuable/well placed.

I'd say my only current structure feedback at this point is based on the order of the example models where I'd flip the ordering so the schema comes before the models.

It's definitely impressive work from @Ridhwana and @bhumi1102 - look forward to following future PRs!

[mstroming]

@Ridhwana @AmandaPerino Excellent idea!

Back in 2020, there was the May of WTFs, where a lot of great feedback was given. Some of it pertains to Active Record and documentation.

https://discuss.rubyonrails.org/c/a-may-of-wtfs

Do you happen to know if anyone has gone through those threads to see if any of the action items from those pertain to this project as well?

[bradly]

13.1.1 Solution to N + 1 Queries Problem

Active Record lets you specify in advance all the associations that are going to be loaded.

The methods are:

[includes](https://api.rubyonrails.org/v7.1.3.2/classes/ActiveRecord/QueryMethods.html#method-i-includes)
[preload](https://api.rubyonrails.org/v7.1.3.2/classes/ActiveRecord/QueryMethods.html#method-i-preload)
[eager_load](https://api.rubyonrails.org/v7.1.3.2/classes/ActiveRecord/QueryMethods.html#method-i-eager_load)

Some added guidance on when to use each of the three condensed down in single place would be helpful. I also think join should be included in this discussion as well. Selfishly these are the kinda of things I personal use the guides for in my day-to-day, but I'm guessing others would find this useful as well.

Thank you for your work on this.

[andynu]

I notice that neither the "Pure string conditions" section, nor the linked security guide mention Arel.sql which I believe is still the required method for more complex pure sql conditions. If this is indeed still the recommended approach, I think it deserves a mention in one of these two documents. Thanks!!!

[Oli0li]

Hello @Ridhwana and @AmandaPerino,

Thank you so much for dedicating your time to work on this, I look forward to reading the new version!
I'm a junior developer who just finished reading this section of the guides and here are a few things I thought concerning the structure:

• Chapter 2 is one big chunk with many methods, but on the index on the right side there are just 2 subdivisions visible. I think it would be good to add more links on the index so we can easily access the explanation of each method. As for the order in which methods are presented, I think it would make sense for #find_by (2.1.5) to come right after #find (2.1.1). I also felt like Chapter 15 about Dynamic arrays could have been included in Chapter 2 since it's also a way to find a record in the database. I wonder if Chapter 18 (find_or_create) could also benefit from being presented closer to the other methods to find an object.

• Scopes are introduced in chapter 14 but I think it would make sense to introduce them earlier, before the methods such as #rewhere in chapter 8 which, if my understanding is correct, would mainly be useful if chaining on a scope. As someone who has just 1 year experience with Rails, I have used scopes many times and when I learned about it last year I wish I had learned it sooner, but I have never used or even seen any of the methods mentioned in chapter 8. To be honest, I think Chapter 8 could go towards the end, after the explanations on #pluck, #ids, #exists? or #count which are used much more often.

I also have some thoughts about the clarity of some examples and descriptions from a junior's perspective. Let me know if that could be helpful, I'm happy to share if needed. :)

[AmandaPerino]

Hi @Oli0li thanks for weighing in here, junior insight is so valuable!
Please feel free to share those further thoughts about the examples and descriptions. We appreciate it!

[Oli0li]

Thank you @AmandaPerino! This is what I noted:

Differences between methods that seem similar

#take vs #limit

The difference between the two wasn't clear to me, as it seems that the SQL is the same in both cases. From what I read here and tested afterwards, #take does not necessarily need an argument and it returns an Array, but #limit requires an argument and returns an Active Record relation. So, for example, I can write User.take but not User.limit, and I can write User.limit(3).order(:created_at) but not User.take(3).order(:created_at). I think this should be explained.

This reminds me that when I first started writing queries, I didn't know there was a difference between an Array and an Active Record relation because it looks similar in the console, so I got errors many times for example when trying to use #order on an Array instead of a Relation. This is a bit alluded to on the section on #pluck, but I think it would be very helpful to explain in its own section what the difference between an Array of instances and an Active Record Relation is, and to clearly flag which methods return an Array and which ones return a Relation.

#joins vs #includes

I think that for beginners with little knowledge of SQL, it would be helpful to compare the different results one could get with #joins that uses INNER JOIN and #includes that uses LEFT OUTER JOIN. For example, if I do Author.includes(:books).pluck(:id), I may get an array with more ids than if I do Author.joins(:books).pluck(:id) if some Authors are registered in the database but haven't currently any Book associated to them.

I was a bit confused when I read this explanation about #includes in 13.2.2:

Even though Active Record lets you specify conditions on the eager loaded associations just like joins, the recommended way is to use joins instead. However if you must do this, you may use where as you would normally.
Author.includes(:books).where(books: { out_of_print: true })

I wondered why #joins was considered preferable over #includes and I was wondering what they consider a case where we "must do this". I was all the more confused that a few sections later in section 21, we see an example with #includes and #where despite the warning in 13.2.2 to use #joins instead, even though in this case it seems to me that #joins would have the same result:

Customer.includes("orders").where(first_name: 'Ryan', orders: { status: 'shipped' }).count

This made me wonder if this was one of the cases where we "must do this" but it was unclear to me why. This advice to use #joins was also mentioned as confusing in Episode 10 of The Tightly Coupled Book Club Podcast.

#where.associated vs #joins

I believe they are the same thing, and as such, we need to use #distinct if we want to filter out duplicated records. There is a note about using #distinct at the end of 12.1.2.1 in the section about #joins, but I think it could be beneficial to add it too for #where.associated, as I fell into that trap the first time I used #where.associated to count some records.

#where vs #find with an array as argument

When passed an array of ids, I think the difference between #where and #find is that #find will keep the order but #where won't, but that wasn't clear to me reading the guides. For example, User.where(id: [1, 5, 3]) could return the objects in the order of 1, 3, 5, but User.find(id: [1, 5, 3]) would return them in the order of 1, 5, 3.

Examples I found hard to understand

Overriding conditions (unscope, rewhere, etc)

I found the examples in section 8 such as Book.where(out_of_print: true).rewhere(out_of_print: false) confusing for a beginner because it just made me wonder: "but why not just remove the first 'where' if I don't need it?". I understand this was probably for the sake of simplicity but I think it would be helpful to show an example that could be encountered "in the wild" in order to understand what the use case would be. The example for #reorder was helpful because we can see it is used to override the ordering that was set in the model, that is implicit in Author.books.

pick (19.3)

pick makes it possible to replace code like:
Customer.where(id: 1).pluck(:id).first
with:
Customer.where(id: 1).pick(:id)

This example confused me because I don't understand in what case we would need this. We already know the id since we are passing it in the where condition, so why do we need to pick it? Even if we were using #pick on a different attribute, for example Customer.where(id: 1).pick(:email), I wondered what the benefit of using this would be over Customer.find_by(id: 1)&.email. After looking into this, I saw that it can return several attributes so I guess that could be a reason to use it, but it would be good to come up with clearer examples to make the use case easier to understand.

In that regard, I found the example provided for #none in section 9 very helpful because it gives a clear example of a use case.

Also, as a general comment, I noticed that at the beginning of the guide, there are examples showing what would be returned in the console, but towards the end it often just shows what SQL is produced without showing what exactly would be returned. That's the case for example for #select: even though it does say we are only selecting a subset of fields, I think it'd be easier to visualise and understand if we could see what it looks like in the console.

Unclear reference in section 21

Throughout section 21, the descriptions of all method end with this sentence:

For options, please see the parent section, Calculations.

But following the link, it wasn't clear to me what these "options" were. What is meant here by "options"? The only thing I understood from this section is that we can pass a column's name as an argument for all these methods.

... and that's all that comes to mind at the moment. Sorry for the wall of text! 😅 Let me know if anything is unclear.