-
Notifications
You must be signed in to change notification settings - Fork 10.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Creation of documentation on how to implement js-search with gatsby #11030
Changes from all commits
1b3d918
3cedcd7
bc28b1f
1f7dee0
6d219fd
46b6b8b
7b046b1
f59e1c5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -1,8 +1,357 @@ | ||||||
--- | ||||||
title: Adding search with js-search | ||||||
title: "Adding search with js-search" | ||||||
--- | ||||||
|
||||||
This is a stub. Help our community expand it. | ||||||
## Prerequisites | ||||||
|
||||||
Please use the [Gatsby Style Guide](/docs/gatsby-style-guide/) to ensure your | ||||||
pull request gets accepted. | ||||||
Before we go through the steps needed for adding client side search to your Gatsby website, it would be advised | ||||||
if the reader is not familiar on how Gatsby works and is set up to follow through the [tutorial](https://www.gatsbyjs.org/tutorial/) and brush up on the [documentation](https://www.gatsbyjs.org/docs/). | ||||||
|
||||||
Otherwise just skip this part and move onto the next part. | ||||||
|
||||||
## What is JS Search | ||||||
|
||||||
[JS Search](https://github.com/bvaughn/js-search) is a library created by Brian Vaughn, a member of the core team at Facebook. It provides an efficient way to search for data on the client using JavaScript and JSON objects. It also has extensive customisation options, check out their docs for more details. | ||||||
|
||||||
The full code and documentation for this library is [available on GitHub](https://github.com/bvaughn/js-search). This guide is based on the official js-search example but has been adapted to work with your Gatsby site. | ||||||
|
||||||
## Setup | ||||||
|
||||||
Let's start by creating a new Gatsby site to work with. Open up a terminal and create the folder that you'll use for this project: | ||||||
|
||||||
```bash | ||||||
mkdir client-search-with-gatsby | ||||||
jonniebigodes marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
``` | ||||||
|
||||||
Inside that folder, create a new Gatsby website using a starter template, using the command below: | ||||||
|
||||||
```bash | ||||||
npx gatsby new jsSearchExample https://github.com/gatsbyjs/gatsby-starter-default | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nice use of npx :) what do you think about changing the directory name to all lowercase? No big deal either way but I think that will more closely match other guides in the site. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not propose a compromise, instead of all lowercase how about There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @jonniebigodes that sounds great! I think we should generally use dash-case (often called kebab-case), so yeah, good call! |
||||||
``` | ||||||
|
||||||
After the process is complete, some additional packages are needed. | ||||||
|
||||||
Navigate into the `jsSearchExample` folder and issue the following command: | ||||||
|
||||||
```bash | ||||||
npm install --save js-search axios | ||||||
``` | ||||||
|
||||||
Or if Yarn is being used: | ||||||
|
||||||
```bash | ||||||
yarn add js-search axios | ||||||
``` | ||||||
|
||||||
Note: | ||||||
|
||||||
For this particular example [axios](https://github.com/axios/axios) will be used, to handle all of the promise based HTTP requests. | ||||||
|
||||||
After all of this is done the actual implementation can be started. | ||||||
|
||||||
Both approaches documented here are fairly generalistic so that most of the options offered by the library can be experimented with. | ||||||
|
||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This could be a good spot to add a couple of lines about the two approaches, describing when either approach might be more appropriate. You've got this info at the end of the second approach section already 👍, moving it up here means people could choose an approach before reading through the whole guide. |
||||||
## First approach | ||||||
|
||||||
The approach documented below is a fairly simple one, by having the component fetch the data and create the search engine. | ||||||
|
||||||
Start by creating a file in the `components` folder, for this particular case the name will be `SearchContainer.js` and inside of it the following baseline code will be added to get started: | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add
Suggested change
|
||||||
|
||||||
```javascript | ||||||
import React, { Component } from "react" | ||||||
import Axios from "axios" | ||||||
import * as JsSearch from "js-search" | ||||||
import DataTable from "./DataTable" | ||||||
import "./search.css" | ||||||
class Search extends Component { | ||||||
state = { | ||||||
bookList: [], // the data that will be fetched | ||||||
search: [], // the js-search engine | ||||||
searchResults: [], // results of the search | ||||||
isError: false, // property to notify the component that something bad happened and let the user know | ||||||
indexByTitle: true, // one of the indexes used to search the data recieved | ||||||
indexByAuthor: true, // another of the indexes used to search the data recieved | ||||||
termFrequency: true, // something something | ||||||
removeStopWords: false, | ||||||
searchQuery: "", // the actual query text to be made | ||||||
selectedStrategy: "Prefix match", // the strategy used by js-search to speed up the process | ||||||
selectedSanitizer: "Lower Case", | ||||||
} | ||||||
/** | ||||||
* React lifecycle method to fetch the data | ||||||
*/ | ||||||
async componentDidMount() { | ||||||
// fetches the data to be used changes the component state based on the result of the request and data | ||||||
Axios.get("http://bvaughn.github.io/js-search/books.json") | ||||||
.then(result => { | ||||||
const bookData = result.data | ||||||
this.setState({ bookList: bookData.books }) | ||||||
/** | ||||||
* calls the rebuildIndex es6 fat arrow function generate the engine with the appropriate options | ||||||
*/ | ||||||
this.rebuildIndex() | ||||||
}) | ||||||
.catch(err => { | ||||||
this.setState({ isError: true }) | ||||||
console.log("====================================") | ||||||
console.log(`Something bad happened while fetching the data\n${err}`) | ||||||
console.log("====================================") | ||||||
}) | ||||||
} | ||||||
/** | ||||||
* es6 fat arrow function to instantiate the search engine | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think you could make these comments clearer by removing the es6 fat arrow function section and adding a note to the Prerequisites section that you'll need some familiarity with es6 syntax like arrow functions. |
||||||
* according to the options defined in the component state | ||||||
* creates the index and injects the data | ||||||
*/ | ||||||
rebuildIndex = () => { | ||||||
const { | ||||||
bookList, | ||||||
selectedStrategy, | ||||||
selectedSanitizer, | ||||||
removeStopWords, | ||||||
termFrequency, | ||||||
indexByTitle, | ||||||
indexByAuthor, | ||||||
} = this.state | ||||||
const dataToSearch = new JsSearch.Search("isbn") | ||||||
|
||||||
if (removeStopWords) { | ||||||
dataToSearch.tokenizer = new JsSearch.StopWordsTokenizer( | ||||||
dataToSearch.tokenizer | ||||||
) | ||||||
} | ||||||
/** | ||||||
* defines a indexing strategy for the data | ||||||
* more more about it in here https://github.com/bvaughn/js-search#configuring-the-index-strategy | ||||||
*/ | ||||||
if (selectedStrategy === "All") { | ||||||
dataToSearch.indexStrategy = new JsSearch.AllSubstringsIndexStrategy() | ||||||
} | ||||||
if (selectedStrategy === "Exact match") { | ||||||
dataToSearch.indexStrategy = new JsSearch.ExactWordIndexStrategy() | ||||||
} | ||||||
if (selectedStrategy === "Prefix match") { | ||||||
dataToSearch.indexStrategy = new JsSearch.PrefixIndexStrategy() | ||||||
} | ||||||
|
||||||
/** | ||||||
* defines the sanitizer for the search | ||||||
* to prevent some of the words from being excluded | ||||||
* | ||||||
*/ | ||||||
selectedSanitizer === "Case Sensitive" | ||||||
? (dataToSearch.sanitizer = new JsSearch.CaseSensitiveSanitizer()) | ||||||
: (dataToSearch.sanitizer = new JsSearch.LowerCaseSanitizer()) | ||||||
|
||||||
/** | ||||||
* defines the search index | ||||||
* read more in here https://github.com/bvaughn/js-search#configuring-the-search-index | ||||||
*/ | ||||||
termFrequency === true | ||||||
? (dataToSearch.searchIndex = new JsSearch.TfIdfSearchIndex("isbn")) | ||||||
: (dataToSearch.searchIndex = new JsSearch.UnorderedSearchIndex()) | ||||||
|
||||||
/** | ||||||
* checks the values in the state and indexes the data revieved based on it | ||||||
*/ | ||||||
if (indexByTitle) { | ||||||
dataToSearch.addIndex("title") | ||||||
} | ||||||
if (indexByAuthor) { | ||||||
dataToSearch.addIndex("author") | ||||||
} | ||||||
|
||||||
dataToSearch.addDocuments(bookList) // adds the data | ||||||
this.setState({ search: dataToSearch }) | ||||||
} | ||||||
|
||||||
/** | ||||||
* es6 fat arrow function to handle the input change and perfom a search with js-search | ||||||
* in which the results will be added to the state | ||||||
*/ | ||||||
searchData = e => { | ||||||
const { search } = this.state | ||||||
const queryResult = search.search(e.target.value) | ||||||
this.setState({ searchQuery: e.target.value, searchResults: queryResult }) | ||||||
} | ||||||
render() { | ||||||
const { bookList, searchResults, searchQuery, indexByAuthor } = this.state | ||||||
return ( | ||||||
<div> | ||||||
<div> | ||||||
<form onSubmit={this.handleSubmit}> | ||||||
<div className="form-group"> | ||||||
<label htmlFor="Search" className="form-label"> | ||||||
Enter your search here | ||||||
</label> | ||||||
<input | ||||||
id="Search" | ||||||
value={searchQuery} | ||||||
onChange={this.searchData} | ||||||
placeholder="Enter your search here" | ||||||
className="searchQuery" | ||||||
/> | ||||||
</div> | ||||||
..... | ||||||
</form> | ||||||
<div> | ||||||
Number of items: | ||||||
{searchQuery === "" ? books.length : searchResults.length} | ||||||
<DataTable data={searchQuery === "" ? books : searchResults} /> | ||||||
</div> | ||||||
</div> | ||||||
</div> | ||||||
) | ||||||
} | ||||||
} | ||||||
|
||||||
export default Search | ||||||
``` | ||||||
|
||||||
Breaking down the code into smaller parts: | ||||||
|
||||||
1. When the component is mounted, the `componentDidMount()` lifecycle method is triggered and the data will be fetched. | ||||||
2. If no errors occur, the data received is added to the state and the `rebuildIndex()` function is invoked. | ||||||
3. The search engine is then created and configured with the options provided in the component's state. | ||||||
4. The data that was received is then added and indexed using js-search. | ||||||
5. When the contents of the input change, `js-search` starts the search process based on the updated input and returns the contents, which is then presented to the user via the `DataTable` component. | ||||||
|
||||||
Note: | ||||||
|
||||||
For brevity purposes, most of the component implementation is omited, with the exception of the `searchData()` function, as this one is responsible for making the search, also the implementation of the `DataTable` component and the page holding this component is not shown. The full code of this example is available [in the js-search repo](https://github.com/gatsbyjs/gatsby/tree/master/examples/using-js-search). | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like how you've highlighted the relevant parts of the code in the tutorial here. But I can see this part being confusing for someone that's followed along so far, I got a bit stuck working out which files I needed to copy from the example. An approach that could make this clearer is to reduce the amount of files used in the example site. So you could do something like this:
Which might not be strictly best practice, but we can add comments explaining how you'd do it in a "real" site, and it'll make the follow up instructions shorter. Doing this means that someone following this tutorial only needs to copy Then the instructions here can clearly describe which files should be copied from the example site to your local site. After making the previous changes, this could be something like To get this running in your site, copy the SearchContainer.js file to your site, and then update the file Edit: I added some more thoughts about tightening up this section in the overall PR comment. |
||||||
|
||||||
## Second approach | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if there's a more precise headers than "first approach" and "second approach". Because sometimes people share URLs with people and the URL will look more readable the headers are specific. |
||||||
|
||||||
This approach takes advantage of Gatsby's APIs to prefetch the data, process it if needed and generate the search engine options beforehand, all of this achieved during the build process. In order to accomplish this, some changes are required to the project. | ||||||
|
||||||
Starting by modifying the `gatsby-node.js` file by adding the following code: | ||||||
|
||||||
```javascript | ||||||
const path = require("path") | ||||||
const axios = require("axios") | ||||||
|
||||||
exports.createPages = ({ actions }) => { | ||||||
const { createPage } = actions | ||||||
return new Promise((resolve, reject) => { | ||||||
axios | ||||||
.get("https://bvaughn.github.io/js-search/books.json") | ||||||
.then(result => { | ||||||
const { data } = result | ||||||
/** | ||||||
* creates a page dynamic page with the data recieved | ||||||
* injects the data recived into the context object alongside with some options | ||||||
* to configure js-search | ||||||
*/ | ||||||
createPage({ | ||||||
path: "/search", | ||||||
component: path.resolve(`./src/templates/ClientSearchTemplate.js`), | ||||||
context: { | ||||||
bookData: { | ||||||
allBooks: data.books, | ||||||
options: { | ||||||
indexStrategy: "Prefix match", | ||||||
searchSanitizer: "Lower Case", | ||||||
TitleIndex: true, | ||||||
AuthorIndex: true, | ||||||
SearchByTerm: true, | ||||||
}, | ||||||
}, | ||||||
}, | ||||||
}) | ||||||
resolve() | ||||||
}) | ||||||
.catch(err => { | ||||||
console.log("====================================") | ||||||
console.log(`error creating Page:${err}`) | ||||||
console.log("====================================") | ||||||
reject(new Error(`error on page creation:\n${err}`)) | ||||||
}) | ||||||
}) | ||||||
} | ||||||
``` | ||||||
|
||||||
Contrary to our earlier approach, instead of letting the component do all of the work, it's Gatsby's job to do the work and pass all the data to a page defined by the path object, via [pageContext](https://www.gatsbyjs.org/docs/behind-the-scenes-terminology/#pagecontext). | ||||||
|
||||||
Let's do this by adding a couple of new components. The first will be your page template component. | ||||||
|
||||||
It will be created in the following location `/src/templates`, under the name `ClientSearchTemplate.js`, with the following code: | ||||||
|
||||||
```javascript | ||||||
import React from "react" | ||||||
import Layout from "../components/layout" | ||||||
import ClientSearch from "../components/ClientSearch" | ||||||
|
||||||
const SearchTemplate = props => { | ||||||
const { pageContext } = props | ||||||
const { bookData } = pageContext | ||||||
const { allBooks, options } = bookData | ||||||
return ( | ||||||
<Layout> | ||||||
<h1 style={{ marginTop: "3em", textAlign: "center" }}> | ||||||
Search data using JS Search using Gatsby Api | ||||||
</h1> | ||||||
<h3 style={{ marginTop: "2em", padding: "2em 0em", textAlign: "center" }}> | ||||||
Books Indexed by: | ||||||
</h3> | ||||||
|
||||||
<div> | ||||||
<ClientSearch books={allBooks} engine={options} /> | ||||||
</div> | ||||||
</Layout> | ||||||
) | ||||||
} | ||||||
|
||||||
export default SearchTemplate | ||||||
``` | ||||||
|
||||||
Now add the second component. Add a file called `ClientSearch.js` to the `components` folder, with the following code as a baseline: | ||||||
|
||||||
```javascript | ||||||
import React, { Component } from 'react' | ||||||
import { Segment, Form, Header } from 'semantic-ui-react' | ||||||
import * as JsSearch from 'js-search' | ||||||
import DataTable from './DataTable' | ||||||
import './search.css' | ||||||
class ClientSearch extends Component { | ||||||
state = { | ||||||
isLoading: true, | ||||||
searchResults: [], | ||||||
search: null, | ||||||
isError: false, | ||||||
indexByTitle: false, | ||||||
indexByAuthor: false, | ||||||
termFrequency: true, | ||||||
removeStopWords: false, | ||||||
searchQuery: '', | ||||||
selectedStrategy: '', | ||||||
selectedSanitizer: '', | ||||||
} | ||||||
|
||||||
static getDerivedStateFromProps(nextProps, prevState) { | ||||||
if (prevState.search === null) { | ||||||
const { engine } = nextProps | ||||||
return { | ||||||
indexByTitle: engine.TitleIndex, | ||||||
indexByAuthor: engine.AuthorIndex, | ||||||
termFrequency: engine.SearchByTerm, | ||||||
selectedSanitizer: engine.searchSanitizer, | ||||||
selectedStrategy: engine.indexStrategy, | ||||||
} | ||||||
} | ||||||
return null | ||||||
} | ||||||
async componentDidMount() { | ||||||
this.rebuildIndex() | ||||||
} | ||||||
..... | ||||||
} | ||||||
export default ClientSearch | ||||||
``` | ||||||
|
||||||
The code used in here is almost identical to the one documented in the first approach and for the same reason stated in the first approach, here also the full implementation will not be shown. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Similar to my earlier comment, let's make it very clear how to get from this step of the tutorial to a working local site:
|
||||||
|
||||||
But there is a slight diference although, it will be used the React `getDerivedStateFromProps()` lifecycle method to adjust the component state accordingly and then `componentDidMount()` to instatiate the client side search, based on the options defined by the state. With this, all of the data will be already available as soon as the specified endpoint is reached, and searching can be made almost instantly. | ||||||
|
||||||
Hopefully this guide has shed some insights on how you can implement client search using js-search. | ||||||
|
||||||
Now go and make something great! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Part of the Gatsby Style Guide is to use "you" as the pronoun in the docs. Here's an explanation of how to make this change and why! https://www.gatsbyjs.org/docs/gatsby-style-guide/#use-you-as-the-pronoun