Tests: generate the indexes in the HTML search JavaScript tests using the application code. #12099
Labels
html search
internals:refactoring
javascript
Pull requests that update Javascript code
type:proposal
a feature suggestion
type:tests
Is your feature request related to a problem? Please describe.
We have JavaScript test coverage of our HTML search functionality, and that's great.
However: the tests use inline-declared index datastructures to represent what each test index is expected to contain.
That means that the format of the search indexes produced by the application code could go out of sync with the JavaScript tests unnoticed, and we may not catch regressions -- or, equally as annoyingly, it might be very difficult to accurately write additional tests from scratch in future.
Describe the solution you'd like
Although it would add Python, andsphinx
, and its requirements to the dependencies for the JavaScript test workflow, I think we should do that and build test index fixtures dynamically.They should be checked-in to source control, and the test would fail if the generated-index didn't match the one currently checked-in.An approach that does not require a Python runtime dependency for the JavaScript tests was found instead; the indices can be built using Python, then committed to source control and read as fixture data when the JavaScript tests run.
To catch changes to the index format, a unit test in the Python test suite attempts to regenerate them using the latest application code, and fails when differences are detected in the content.
Describe alternatives you've considered
Allowing the status-quo to continue seems fine, but the fact that we lack continuous committers / experts in the search area (it has taken me a while to begin to understand it, and I still don't feel completely confident about it) makes me want to guard it a bit against lack of continuous maintenance/knowledge.
Additional context
This would require good documentation too.
The text was updated successfully, but these errors were encountered: