HTML Search: omit anchor reference from document titles in the search index.

[jayaddison]

Feature or Bugfix

• Bugfix

Purpose

• Adjusts the contents of the search-index provided to the client so that the browser correctly de-duplicates search results that link to the top-level title in documents.

Detail

• Adapts the transformation of docutils nodes.title nodes into the title index by omitting their id that is used as the hyperlink anchor in the special-case of the document title.
• Note: there is no explicit markup for the document title in reStructuredText.

Relates

• Resolves HTML Search: Contains duplicates based on title and content search #11961.
• Alternative approach to client-side fix in HTML Search: Fix duplicate results #11942.

[wlach]

This seems preferable to my solution in #11942, just tested and it has the same results while making the search index smaller. 👍 from me.

[picnixz]

Fine for me (I'm not an expert in that search-related aspect so I'll leave it to you guys). Should I understand that this PR would replace #11942 ? @jayaddison ping me when you want it to be merged

[jayaddison]

Fine for me (I'm not an expert in that search-related aspect so I'll leave it to you guys). Should I understand that this PR would replace #11942 ?

That's correct. I think there are three things that I don't like about this pull request, in order of priority:

1. It doesn't offer JavaScript test coverage to demonstrate the fix -- only Python test coverage that confirms the expected index format.
2. I haven't been able to think of a way that this would happen yet, but in theory I expect some downstream tools that use the current index format could be broken by this - in particular the possibility of null values where previously everything was a string.
3. Related to (2), the index format isn't as small as it could be. In raw text, the JavaScript empty string can be represented by either '' or "" (two characters), whereas a null value is represented by the token null (four characters). It may not matter a lot, but over a long enough duration of time the cumulative cost could be significant (or not! maybe it's a waste of time considering it).

[jayaddison]

It doesn't offer JavaScript test coverage to demonstrate the fix -- only Python test coverage that confirms the expected index format.

I think I'll begin work on a small JavaScript refactor PR to make test coverage easier to add.

[picnixz]

particular the possibility of null values where previously everything was a string.

Is it possible to keep a string or is the null type needed?

[jayaddison]

particular the possibility of null values where previously everything was a string.

Is it possible to keep a string or is the null type needed?

If I remember correctly, this conditional needs to be adjusted if we're using empty strings, but otherwise it should be fine.

What I would really like is a way to generate the indexes used in the JavaScript tests from the same Python code that builds searchindex.js when projects are built. It feels fragile that the test indexes are written by hand.

[jayaddison]

What I would really like is a way to generate the indexes used in the JavaScript tests from the same Python code that builds searchindex.js when projects are built. It feels fragile that the test indexes are written by hand.

Moved into #12099.

[jayaddison]

I'd like to check whether we can get #12102 in place before progressing this pull request further. If that can be added, then I think adding test coverage here will be much easier and more reliable (I'll be able to create a sample Sphinx project that returns duplicate search results, and add test coverage against that).

[jayaddison]

(and maybe do the null to '' refactoring at the same time and more safely, given the test coverage)

[picnixz]

Removing the "awaiting review" label until this PR is ready

[jayaddison]

Removing the "awaiting review" label until this PR is ready

Oops, thanks. I forgot about that.

[jayaddison]

While I think it would be sensible to merge #12102 first (if-and-when that pull request is considered acceptable!), I'm going to remove the blocker label and draft status from this pull request.