Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search boosts #1930

Merged
merged 29 commits into from May 30, 2022
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions example/src/classes/Customer.ts
Expand Up @@ -2,6 +2,8 @@
* An abstract base class for the customer entity in our application.
*
* Notice how TypeDoc shows the inheritance hierarchy for our class.
*
* @category Model
*/
export abstract class Customer {
/** A public readonly property. */
Expand Down
5 changes: 5 additions & 0 deletions example/src/reactComponents.tsx
Expand Up @@ -36,6 +36,8 @@ export interface CardAProps {
* This is our recommended way to define React components as it makes your code
* more readable. The minor drawback is you must click the `CardAProps` link to
* see the component's props.
*
* @category Component
*/
export function CardA({ children, variant = "primary" }: PropsWithChildren<CardAProps>): ReactElement {
return <div className={`card card-${variant}`}>{children}</div>;
Expand Down Expand Up @@ -66,6 +68,8 @@ export function CardA({ children, variant = "primary" }: PropsWithChildren<CardA
*
* This can make the TypeDoc documentation a bit cleaner for very simple components,
* but it makes your code less readable.
*
* @category Component
*/
export function CardB({
children,
Expand Down Expand Up @@ -245,6 +249,7 @@ export interface EasyFormDialogProps {
* )
* }
* ```
* @category Component
*/
export function EasyFormDialog(props: PropsWithChildren<EasyFormDialogProps>): ReactElement {
return <div />;
Expand Down
10 changes: 9 additions & 1 deletion example/typedoc.json
Expand Up @@ -2,5 +2,13 @@
"$schema": "https://typedoc.org/schema.json",
"entryPoints": ["./src"],
"sort": ["source-order"],
"media": "media"
"media": "media",
"categorizeByGroup": false,
"searchCategoryBoosts": {
"Component": 2,
"Model": 1.2
},
"searchGroupBoosts": {
"class": 1.5
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I'm lowercasing everything, but the actual kind is "Class". Any preference on casing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed all the toLowerCase stuff, so that the kind names align with requiredToBeDocumented. d6e5b18

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking of which, are you sure that searchKindBoosts wouldn't be a better name for this? You refer to "kind" in the description for requiredToBeDocumented...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With 0.22, yes, searchKindBoosts does look like the better name - I'm looking ahead to 0.23 though, where TypeDoc accepts an @group tag. By default, groups are determined by kind, but this gives additional flexibility. I think I'm happy with this, just need to find the time to pull it down and play with it a bit :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okie-doke

}
}
12 changes: 12 additions & 0 deletions src/lib/converter/context.ts
Expand Up @@ -14,6 +14,7 @@ import type { Converter } from "./converter";
import { isNamedNode } from "./utils/nodes";
import { ConverterEvents } from "./converter-events";
import { resolveAliasedSymbol } from "./utils/symbols";
import type { SearchConfig } from "../utils/options/declaration";

/**
* The context describes the current state the converter is in.
Expand Down Expand Up @@ -118,6 +119,17 @@ export class Context {
return this.converter.application.options.getCompilerOptions();
}

getSearchOptions(): SearchConfig {
return {
searchCategoryBoosts: this.converter.application.options.getValue(
"searchCategoryBoosts"
) as SearchConfig["searchCategoryBoosts"],
searchGroupBoosts: this.converter.application.options.getValue(
"searchGroupBoosts"
) as SearchConfig["searchGroupBoosts"],
};
}

/**
* Return the type declaration of the given node.
*
Expand Down
55 changes: 42 additions & 13 deletions src/lib/converter/plugins/CategoryPlugin.ts
Expand Up @@ -4,12 +4,12 @@ import {
DeclarationReflection,
CommentTag,
} from "../../models";
import { ReflectionCategory } from "../../models/ReflectionCategory";
import { ReflectionCategory } from "../../models";
import { Component, ConverterComponent } from "../components";
import { Converter } from "../converter";
import type { Context } from "../context";
import { BindOption } from "../../utils";
import type { Comment } from "../../models/comments/index";
import type { Comment } from "../../models";

/**
* A handler that sorts and categorizes the found reflections in the resolving phase.
Expand Down Expand Up @@ -66,9 +66,12 @@ export class CategoryPlugin extends ConverterComponent {
* @param context The context object describing the current state the converter is in.
* @param reflection The reflection that is currently resolved.
*/
private onResolve(_context: Context, reflection: Reflection) {
private onResolve(context: Context, reflection: Reflection) {
if (reflection instanceof ContainerReflection) {
this.categorize(reflection);
this.categorize(
reflection,
context.getSearchOptions()?.searchCategoryBoosts ?? {}
);
}
}

Expand All @@ -79,26 +82,36 @@ export class CategoryPlugin extends ConverterComponent {
*/
private onEndResolve(context: Context) {
const project = context.project;
this.categorize(project);
this.categorize(
project,
context.getSearchOptions()?.searchCategoryBoosts ?? {}
);
}

private categorize(obj: ContainerReflection) {
private categorize(
obj: ContainerReflection,
categorySearchBoosts: { [key: string]: number }
) {
if (this.categorizeByGroup) {
this.groupCategorize(obj);
this.groupCategorize(obj, categorySearchBoosts);
} else {
this.lumpCategorize(obj);
CategoryPlugin.lumpCategorize(obj, categorySearchBoosts);
}
}

private groupCategorize(obj: ContainerReflection) {
private groupCategorize(
obj: ContainerReflection,
categorySearchBoosts: { [key: string]: number }
) {
if (!obj.groups || obj.groups.length === 0) {
return;
}
obj.groups.forEach((group) => {
if (group.categories) return;

group.categories = CategoryPlugin.getReflectionCategories(
group.children
group.children,
categorySearchBoosts
);
if (group.categories && group.categories.length > 1) {
group.categories.sort(CategoryPlugin.sortCatCallback);
Expand All @@ -112,11 +125,17 @@ export class CategoryPlugin extends ConverterComponent {
});
}

private lumpCategorize(obj: ContainerReflection) {
static lumpCategorize(
obj: ContainerReflection,
categorySearchBoosts: { [key: string]: number }
) {
if (!obj.children || obj.children.length === 0 || obj.categories) {
return;
}
obj.categories = CategoryPlugin.getReflectionCategories(obj.children);
obj.categories = CategoryPlugin.getReflectionCategories(
obj.children,
categorySearchBoosts
);
if (obj.categories && obj.categories.length > 1) {
obj.categories.sort(CategoryPlugin.sortCatCallback);
} else if (
Expand All @@ -132,10 +151,13 @@ export class CategoryPlugin extends ConverterComponent {
* Create a categorized representation of the given list of reflections.
*
* @param reflections The reflections that should be categorized.
* @param categorySearchBoosts A user-supplied map of category titles, for computing a
* relevance boost to be used when searching
* @returns An array containing all children of the given reflection categorized
*/
static getReflectionCategories(
reflections: DeclarationReflection[]
reflections: DeclarationReflection[],
categorySearchBoosts: { [key: string]: number }
): ReflectionCategory[] {
const categories: ReflectionCategory[] = [];
let defaultCat: ReflectionCategory | undefined;
Expand All @@ -154,11 +176,18 @@ export class CategoryPlugin extends ConverterComponent {
categories.push(defaultCat);
}
}

defaultCat.children.push(child);
return;
}
for (const childCat of childCategories) {
let category = categories.find((cat) => cat.title === childCat);

const catBoost = categorySearchBoosts[category?.title ?? -1];
if (catBoost != undefined) {
child.relevanceBoost =
(child.relevanceBoost ?? 1) * catBoost;
}
if (category) {
category.children.push(child);
continue;
Expand Down
6 changes: 6 additions & 0 deletions src/lib/models/reflections/container.ts
Expand Up @@ -20,6 +20,12 @@ export class ContainerReflection extends Reflection {
*/
categories?: ReflectionCategory[];

/**
* A precomputed boost derived from the searchCategoryBoosts typedoc.json setting, to be used when
* boosting search relevance scores at runtime.
*/
relevanceBoost?: number;

/**
* Return a list of all children of a certain kind.
*
Expand Down
30 changes: 22 additions & 8 deletions src/lib/output/plugins/JavascriptIndexPlugin.ts
Expand Up @@ -5,8 +5,8 @@ import {
DeclarationReflection,
ProjectReflection,
ReflectionKind,
} from "../../models/reflections/index";
import { GroupPlugin } from "../../converter/plugins/GroupPlugin";
} from "../../models";
import { GroupPlugin } from "../../converter/plugins";
import { Component, RendererComponent } from "../components";
import { RendererEvent } from "../events";
import { writeFileSync } from "../../utils";
Expand Down Expand Up @@ -42,6 +42,11 @@ export class JavascriptIndexPlugin extends RendererComponent {
const rows: any[] = [];
const kinds: { [K in ReflectionKind]?: string } = {};

const kindBoosts =
(this.application.options.getValue("searchGroupBoosts") as {
[key: string]: number;
}) ?? {};

for (const reflection of event.project.getReflectionsByKind(
ReflectionKind.All
)) {
Expand All @@ -63,24 +68,32 @@ export class JavascriptIndexPlugin extends RendererComponent {
parent = undefined;
}

if (!kinds[reflection.kind]) {
kinds[reflection.kind] = GroupPlugin.getKindSingular(
reflection.kind
);

const boost =
kindBoosts[(kinds[reflection.kind] ?? "").toLowerCase()];
if (boost != undefined) {
reflection.relevanceBoost =
(reflection.relevanceBoost ?? 1) * boost;
}
}

const row: any = {
id: rows.length,
kind: reflection.kind,
name: reflection.name,
url: reflection.url,
classes: reflection.cssClasses,
relevanceBoost: reflection.relevanceBoost,
};

if (parent) {
row.parent = parent.getFullName();
}

if (!kinds[reflection.kind]) {
kinds[reflection.kind] = GroupPlugin.getKindSingular(
reflection.kind
);
}

rows.push(row);
}

Expand All @@ -100,6 +113,7 @@ export class JavascriptIndexPlugin extends RendererComponent {
"assets",
"search.js"
);

const jsonData = JSON.stringify({
kinds,
rows,
Expand Down
34 changes: 29 additions & 5 deletions src/lib/output/themes/default/assets/typedoc/components/Search.ts
Expand Up @@ -6,8 +6,9 @@ interface IDocument {
kind: number;
name: string;
url: string;
classes: string;
classes?: string;
parent?: string;
relevanceBoost?: number;
}

interface IData {
Expand Down Expand Up @@ -154,7 +155,30 @@ function updateResults(
// Perform a wildcard search
// Set empty `res` to prevent getting random results with wildcard search
// when the `searchText` is empty.
const res = searchText ? state.index.search(`*${searchText}*`) : [];
let res = searchText ? state.index.search(`*${searchText}*`) : [];

for (let i = 0; i < res.length; i++) {
const item = res[i];
const row = state.data.rows[Number(item.ref)];
let boost = 1;

// boost by exact match on name
if (row.name.toLowerCase().startsWith(searchText.toLowerCase())) {
boost *=
1 + 1 / (Math.abs(row.name.length - searchText.length) * 10);
}

// boost by relevanceBoost
if ((row.relevanceBoost ?? 1) > 1) {
debugger;
}

boost *= row.relevanceBoost ?? 1;

item.score *= boost;
}

res.sort((a, b) => b.score - a.score);

for (let i = 0, c = Math.min(10, res.length); i < c; i++) {
const row = state.data.rows[Number(res[i].ref)];
Expand All @@ -169,7 +193,7 @@ function updateResults(
}

const item = document.createElement("li");
item.classList.value = row.classes;
item.classList.value = row.classes ?? "";

const anchor = document.createElement("a");
anchor.href = state.base + row.url;
Expand Down Expand Up @@ -199,11 +223,11 @@ function setCurrentResult(results: HTMLElement, dir: number) {
// current with the arrow keys.
if (dir === 1) {
do {
rel = rel.nextElementSibling;
rel = rel.nextElementSibling ?? undefined;
} while (rel instanceof HTMLElement && rel.offsetParent == null);
} else {
do {
rel = rel.previousElementSibling;
rel = rel.previousElementSibling ?? undefined;
} while (rel instanceof HTMLElement && rel.offsetParent == null);
}

Expand Down
10 changes: 9 additions & 1 deletion src/lib/utils/options/declaration.ts
Expand Up @@ -3,7 +3,7 @@ import type { LogLevel } from "../loggers";
import type { SortStrategy } from "../sort";
import { isAbsolute, join, resolve } from "path";
import type { EntryPointStrategy } from "../entry-point";
import type { ReflectionKind } from "../../models/reflections/kind";
import { ReflectionKind } from "../../models/reflections/kind";

export const EmitStrategy = {
true: true, // Alias for both, for backwards compatibility until 0.23
Expand Down Expand Up @@ -50,6 +50,12 @@ export type TypeDocOptionValues = {
: TypeDocOptionMap[K][keyof TypeDocOptionMap[K]];
};

const Kinds = Object.values(ReflectionKind);
export interface SearchConfig {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd like to split this up into a few options:

  • maxSearchResults
  • searchCategoryBoosts
  • searchGroupBoosts

I think it makes sense to always give exact matches a large boost, probably needs experimentation to figure out what that should be.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed exactMatch from the config stuff, and now it's on by default.

545fc88

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8864deb

(removed the numResults thing; we don't need it right now, and I don't want to needlessly prolong this review)

searchGroupBoosts?: { [key: typeof Kinds[number]]: number };
searchCategoryBoosts?: { [key: string]: number };
}

/**
* Describes all TypeDoc options. Used internally to provide better types when fetching options.
* External consumers should likely use {@link TypeDocOptions} instead.
Expand Down Expand Up @@ -107,6 +113,8 @@ export interface TypeDocOptionMap {
version: boolean;
showConfig: boolean;
plugin: string[];
searchCategoryBoosts: unknown;
searchGroupBoosts: unknown;
logger: unknown; // string | Function
logLevel: typeof LogLevel;
markedOptions: unknown;
Expand Down