v7.16.1

7.16.1 - 2024-04-09

Bugs

• You can delete a user who has existing judgements, and then that messes up the main Books page because their judgements hang around. Now you are warned about this, and you can choose to make their judgements anonymous. (or just Lock their user account). #999 by @epugh.

Full Changelog: v7.16.0...v7.16.1

7.16.0

7.16.0 - 2024-03-28

Features

• Now have a dropdown for Books similar to Cases that leverages our existing tracking of what books you have viewed. #986 by @epugh.

• We now run scorers over queries that have no ratings and queries that are ZSR. This lets us have smarter logic about how you want to score those situations. #993 by @epugh.

Improvements

• One less query to determine what a user has access to via being an owner or a member of a team while ensuring no duplicate cases/books show up. #982 by @epugh with input from @reid-rigo.

• Remove extranous faraday logs when running tests. #983 by @epugh with input form @reid-rigo fixes #964.

• Nicer onboarding using TMDB dataset for Algolia users. #987 by @chuckmeyer.

Bugs

• Book Importing was broken. Plus now we have nicer formatted error message when you validate a book to import. #989 by @epugh.

Full Changelog: v7.15.1...v7.16.0

v7.15.1

We found a very annoying bug in 7.15.0, so a quick 7.15.1 release was needed!

7.15.1 - 2024-03-13

Bugs

• Looking up Cases for a user is returning duplicates due to missing "distinct" keyword on SQL Query. #980 by @epugh.

• The "bundle exec thor case:create" is NOW updated to handle SearchEndpoints. #981 by @epugh.

7.15.0 - 2024-03-11

Features

• Now track an Explanation for a judgement. If a human clicks "I can't rate this" then they are prompted for an optional explanation why. In the near future, LLM powered judgements will communicate a explanation as well! #978 by @epugh.

• Algolia integration! We have a wonderful contribution from @sumitsarkar that adds Algolia as one of the supported Search engines. #933 by @sumitsarker.

Improvements

• Move to using Thor (an existing CLI tool) for loading all our sample data files, and less custom Rake code. #966 by @epugh.

• Better UX when working with custom search API's and the mapper code. #965 and #963 by @epugh, fixing #959, #960, #961 by @wrigleyDan and #950 by @erikhatcher.

• Proxied Search API requests should follow Redirects. #962 by @epugh fixes #958 by @wrigleyDan.

• Improved performance of checking case access permissions. #955 by @reid-rigo. Thanks Reid!

• Updated the Jaccard, RBO, and Judge comparison notebooks that ship with Quepid. Introduced more robust testing for them. #956 by @epugh.

• Improve the Books API. #951 by @epugh.

• Snapshots used to have "Best Snapshot" option, but it hasn't worked in years, so stripped it out. #948 by @epugh.

Bugs

• Scoring logic was all somewhat broken. Add a query, it doesn't rescore. Delete a query, it doesn't rescore. Have a ZSR query, it impacts scoring in a "wonky" way. Then, when you introduce snapshot comparisons, it all gets weirder. We've gone through at a pretty line by line debugger level and fixed the various interaction bugs. #975 by @epugh.

• Only ship Web Open Font fonts, which simplifies our CSS and set up and eliminates a warning in Firefox console. #techdebt. #974 by @epugh.

• Fixed some bit rot in handling of Tries in the web UI, and do some polish of UX. #967 by @epugh.

• Check length of query_text. #952 by @epugh and @mkr.

• No email provider? No problem! Fixed issue with inviting folks and no email provider setup. #949 by @epugh.

## 7.14.0 - 2024-02-06

7.14.0 - 2024-02-06

Features

• SCALING SCALING SCALING. Now possible, though painful still, to run 5000 queries by moving lots of activities to background processing. Thanks to my colleagues on the Cville Slack communities #ruby channel for lots of advice and help. #916 by @epugh.

• Quepid truncates the document fields in the core display, either via using highlighting to pluck out a section, or just snippeting to 200 characters. However some days you just want to see the entire field! Now you can control this in your field specifciation by prepending the field name with unabridged: tag. #946 by @epugh.

• Need to communicate with your users about a new Scorer or upcoming downtime? You can now post Annoucements to your users. #911 by @mkr and #941 by @epugh fixes #848 for @flaxsearch.

Improvements

• Snapshots require you to be able to "lookup" the document by id from the search engine. However some search engines like Vectara and the Search API don't support this. Reworked how snapshots work to require you to save the document fields at snapshot creation time in order to power the snapshot comparison UI. This may end up being spread to other search engines as well. #945 and #931 by @epugh.

• The logs in Quepid are being overwhelmed by the size of the documents. #942 by @reid-rigo cleans this up. Thank you Reid!

• Stripped out Threshold based alerting on queries, the community never used them. #927 by @epugh.

• Now tracking when a Book was viewed by a user, which powers the dashboard. #925 by @epugh.

• Removed the QUEPID_PROPHET_ANALYTICS environment variable, as Prophet was never a performance issue. #924 by @epugh.

• Dropped old database columns in the tries table that had moved to search_endpoints table. #930 by @epugh.

Bugs

• Restore the thor commands for creating an admin user. #922 by @epugh fixes #888 by @vincetrumental.

• Bug fix for static search endpoint when Quepid is running in a nested host context. #917 by @epugh.

• We were not export notes and options for a query. #926 by @epugh fixes #923 by @peterdm.

Plus we updated to the latest of Ruby and Ruby gems, Mysql, various NPM packages.

v7.13.0

7.13.0 - 2024-01-03

Just in time for the New Year!

A number of changes all wrapped up in one large PR #906 by @epugh. This one is for you @david-fisher ;-).

• Now when you judge a query doc pair for a Judgement Book we update in the background any associated Cases that have that query. This eliminates the need to manually refresh the ratings for a Case from the Judgement Book! Thanks @david-fisher for the suggestion in #829.

• Provide an option to allow queries that exist on a Book to be created in the Case if they are missing. The default behavior is to NOT create queries in a Case from the Book. Thanks to @jvia for this suggestion. #871.

• Band-aid the awkwardness that Cases have Queries/Ratings while Books have QueryDocPairs + Judgements, and that there are attributes on a Query like information need, notes, or options that don't exist on a QueryDocPair. So now we preserve them, which facilitates round tripping better. Upload a Book? Upload a Case? It doesn't matter ;-) Long term we need to merge these two data structures...

• When Books were first added to Quepid, they didn't follow the same permissions/sharing structure that Scores and Cases did. Now you can share a Book with multiple Teams! Fixes #907

• And as an aside, anyone can edit a custom scorer, not just the Owner of it. #762 suggested by @david-fisher.

v7.12.2

7.12.2 - 2023-12-01

We discovered that your custom headers can be very long! Thanks @tiagoshin for your patch. #903 by @tiagoshin.

7.12.1 - 2023-12-01

7.12.0 introduced a change in splainer-search that breaks searchapi :-(.

• Rollback splainer changes, but also some polish on the UX for going from the Case judgement screen to the Share with a Team. Also make parsing the csv file for the static api more robust. #894

7.12.0 - 2023-11-30

7.12.0 - 2023-11-30

A flurry of changes based on feedback from the community. This is basically a "improve" Quepid release.

Features

• Now show the last run on using friendly times, like "one hour ago". #892 by @epugh.

Improvements

• Restored in the homepage and the case listing the "last run on" and "last run by" information. Adding this had been causing performance issues, but hoping we worked this out. #891 by @epugh.

• Some general fixes to handling static defined search case and search endpoints. #886 by @epugh.

• Also, the usual upgrades to Rails, Node etc. We are now on Rails 7.1.3, though we still need to update some configuration settings from Rails 7.0 to 7.1.

Bugs

• Respect the proxy setting for a search end point in doing a snapshot compare lookup, and in linking to individual docs. #893 by @epugh. Thanks @david-fisher for finding this.

• We now support custom headers for search apis. Thanks @dacox for the fix in o19s/splainer-search#138.

• Made more robust the adding of queries for a static CSV case. #889 by @epugh. Thanks to @tiagoshin for identifying the issue.

v7.9.2 - Finally we can import and export Cases and Books cleanly

7.9.2 - 2023-09-20

You can now export and import Case's and Book's cleanly from one Quepid to another Quepid, with proper error handling. You can also use this to export a Case or Book, make a bunch of changes externally using other tools, and then bring it back in.

Get a dump file named case_25.json via:

python case_export.py --case_id=25 --root_url=https://app.quepid.com --access_token=YOUR_ACCESS_TOKEN

And bring it back in via:

python case_import.py --case_file=case_25.json --root_url=http://localhost:3000 --access_token=YOUR_ACCESS_TOKEN

More docs and some sample scripts coming soon.

#834 by @epugh.

v7.7.0 - a Dashboard style home page

7.7.0 - 2023-08-31

We finally have a brand new homepage! Quepid started as a tool for relevancy engineers only, but today we have human judges using Quepid to rate documents, Search Managers who are keeping an eye on the offline metrics, and of course in more complex setups, networking types who configure the connection between Quepid and the Search Engine. In the past, regardless of who you were, we dropped you right into a Case. If you had never created one before, well, you got shoved into the Case creation wizard, whether you wanted it or not. It made for a unpleasant first experience for anyone other than the hard core relevancy engineer.

The new dashboard is an attempt to change that UX. It's just a first cut, and honestly, it probably doesn't quite make anyone perfectly happy. I'm hoping that we can get a lot of good feedback and learn from it. I could imagine in the future that based on what you do, we surface information you care about. For example, a network engineer would want to know that the Quepid can talk to each Search Engine. A Search Manager would be more interested in insights and summary progress information. Human judges want to know "what is my next task to do". So please share feedback!

Features

• New home page (and sidebar!) in #681 is added via #684 by @epugh.

Improvements

• The feedback message after creating an empty book is now smart about if there are related cases or not, and gives better next steps instructions. #796 by @OkkeKlein fixed by #797 by @epugh. Thanks Okke!

• The Cases listing page shows your cases from oldest to newest. @atarora opened #708 for sortable columns. Implementing this didn't go well, so as an alternative, changed the sorting to be from newest to oldest, taking into account the case update time as well as the last time you viewed a specific case. #795 by @epugh. We may want to redo the entire Cases listing page at some point.... Hopefully this is better @atarora ;-).

• Reworked the messageing (and error handling) when you attempt to open a case that hasn't been shared with you (or doesn't exist!). We had been showing the "You need to reload Quepid in the HTTPS" type message, when actually it was a "Make sure this case has been shared with you!" situation. Also reduced the amount of error messages in the browser and the server in this situation by being smarter. Thanks @cmorley for surfacing this. #792 by @epugh fixed by #793.

• Thumb images can use a prefix_url, just like regular images. #790 by @OkkeKlein is fixed in #805 by @epugh.

• Using Nginx and have huge documents being snapshotted etc. New configuration suggested by @OkkeKlein allows Nginx to be happy. #804 by @epugh.

Bugs

• Drop a link in Slack to team 3, and you see details about case 3;-). #733 by @epugh fixed by #794 by @epugh to only do unfurling for cases.

• Ace code editor insists on loading two javascript files from specific paths in Quepid. Despite best effort, couldn't change this. So #793 just mounts those javascript where Ace wants it to shut up some very loud error messages in the browser console. If you can't beat'em, join'em.

• When judging ratings using the Book of Judgements, you could sometimes get a situation where you create a judgement, and then tried to create a new one, hitting a constraint. #809 by @epugh makes this more robust.

• Guess what? 7 GB out of our 9 GB production database is scores for cases being inserted OVER AND OVER again! The AngularJS app uses event emitting to signal when scoring is completed, this causes extra events to be triggered, causing extra PUT of the case scores. This shouldn't matter as we have logic on the server side. However, due to a bug, that logic doesn't actually work. The unit tests for the front end and the back end each independently validate the logic, it's just when you put them together it doesn't work. This didn't matter for years, but the new Dashboard surfaces scores ;-). Fixed in #807 by @epugh.

7.6.1: Round Trip Human Ratings!

Lots has happened since the last release notes, so doing one big combined one below!

7.6.1 - 2023-07-06

• Chased down bug with showing you the previous judgement on the Human Judgement page. #779 by @epugh.

7.6.0 - 2023-07-05

Big overhaul on the human rating side of things. We now roundtrip ratings that you have made from the core Case screen into your Book of Judgements when you populate it. (We are also now tracking who the last person was to rate a doc in the Case screen). You can now also merge ratings from multiple Book's into a new Book, which lets you reuse your judgements in new combinations.

Additionally, to encourage your human judges, a progress screen has been added every 20 judgements showing progress and a leaderboard.

• #778 by @epugh introduces some "fun" into the rating process ;-).

• #766 introduced the synthesizing of a Book, and fixes #765, track who rates in the main quepid ui, #763, link from list of books in judgements to the book itself, and #761, Combining books loses judgments but still produces scores.

• #760 changes Quepid to use floats for ratings and judgements, which opens the door to implicit judgements.

7.5.0 - 2023-06-15

Need to interact with Quepid API's from outside of Quepid? We now support Personal Access Tokens! From your Profile page you can generate your own token and then use that to authenticate against Quepid. For example, you can programatically load a judgement directly into Quepid:

curl -X POST http://localhost:3000/api/books/2/judgements/ -H 'Authorization: Bearer 4a82040bf1b2d255c63833cb59fa9275' -H 'Content-Type: application/json' -d '{
  "judgement": {
    "query_doc_pair_id": 201
    "rating": 1
  }
}'

#759 and #756 by @epugh.

In support of this, we added new API's for judgements and ratings in #757 by @epugh.

Bugs

• Exporting Ratings in CSV format was returning the same rating for every single rater, regardless of what they did! Fixed in by @epugh with special help from @grahamwren. I randomly sat next to him on plane flight home and we paired on it ;-).

• Looking up the wrong info need for a Query Doc Pair! Fixed in #755 by @epugh.

• Missed a relative path for the tries history visualization. Fixed in #754.

7.4.1 - 2023-06-09

Lots of small things from working with Quepid using templates with OpenSearch. Also, if you haven't used https://github.com/o19s/agent_q, it's been updated to work with Quepid 7.x line. I use it to automate pulling my relevance metrics daily from Quepid cases ;-).

Features

• Rework how we handle OS/ES templates in Quepid so that you don't need to append /template to the url, instead, use the existence of the {"id":"my-template"} in the query parameters to decide what URL to use. This simplifies life when you are using a mix of templates and not since you don't need to change the url constantly. This was mostly in the Splainer-Search v2.22, v2.22.1, and v2.22.2 releases. https://github.com/o19s/splainer-search/

• Now, under Explain Query modal, you can see the rendered template query for a specific query! #751 by @epugh.

Improvements

• Kind of in the weeds, but early database migrations in Quepid back in Rails 4 days didn't have a version spec. Rails 7 complains about this. #739 fixed by #746 by @epugh.

• Introduced a new environment variable QUEPID_CONSIDER_ALL_REQUESTS_LOCAL that lets you bubble up errors message in a production Rails environment, which is helpful in troubleshooting deployments when you can't see the logs. #750 by @epugh.

• Package Jupyterlite to not require external network calls. #721 by @epugh fixed by #728 by @mkr.

Bugs

• A three point custom scorer should support keyboard shortcuts. #738 by @epugh fixed by #752 by @depahelix2021.

• Missing Documents Modal doesn't work when an ES/OS Query Template is defined. #747 fixed by #753 by @epugh.

• Populating a book of judgements would blow up. Plus the data model allowed a single user to rate a query/doc pair multiple times, which was icky. #734 fixed by #745 by @epugh.

• Use relative paths everywhere so when Quepid is behind proxies it works properly. #754 by @epugh.

7.3.3 - 2023-05-30

• When populating a book of judgements, if the title field wasn't title, then it wouldn't show up with the correct name. #737 by @epugh fixes this.

7.3.1 - 2023-05-26

• Fixed some additional API paths for use with a nested Quepid context. #736 by @epugh. Follow up to #719.