Skip to content

Troubleshooting Solr and Quepid

Jump to bottom Edit New page
Eric Pugh edited this page Dec 5, 2023 · 38 revisions

Connecting Solr to Quepid

Here's a blog written for Quepid beginners, explaining how to set up a connection to a Solr index (ignore the bits about the free trial and single Case, this was written before Quepid was free to use): https://www.flax.co.uk/index.html@p=3316.html

Quepid talks to Solr using the JSONP method, which basically means sending a big JSON request wrapped in some Ajax. It's not as reliable as using CORS, but avoids having to enable CORS in Solr. Your Solr should be ready out of the box!

Note: As of Solr 8.4.1, and the latest browser security enhancements, JSONP may also require additional configuration. Please review the Connectivity section for more information.

Demo Solr Indexes with the TMDB dataset

We now have a number of different versions of Solr deployed with the same TMDB dataset, as well as some others, which is useful if you have a new feature that you want to make sure is backwards compatible. solr:SolrRocks

Highlighting of Results in Quepid Results Page

Quepid by default tries to be smart with how it handles the fields returned. If you are searching on a field, then it automatically creates a snippet of the field using highlighting. If you aren't searching on the field, then it returns the entire field. You can pass in hl parameters as part of your query to tweak the behaviors.

Connectivity

As of Solr 8.4.1, Solr ships with more restrictive security options by default. This, along with a early 2020 change by all the browser vendors has tightened up the rules for browser CORS interaction. The new default of nosniff for X-Content-Type-Options appears to be breaking this functionality, which interferes with outside websites accessing a Solr instance directly. The default configuration that ships with 8.4.1 now only allows such requests to originate from the Solr host itself.

HTTPS Only Mode

If your browser is set up with HTTPS Only mode, and you are connecting to a non HTTPS Solr, then of course it won't work ;-)

Solr w/ Basic Auth

If you are using Basic Authentication, ie a URL in the format of [http|https]://[username:password]@[host][:port]/solr/[collectionName]/select then you will NOT be prompted by your browser to authenticate.

Otherwise, if you skip setting the username and password as part of the URL, then the browser will prompt you to login.

Be aware of the fact that browsers behave differently. Using Chrome may block Quepid queries with embedded credentials without any details. See https://chromestatus.com/feature/5669008342777856 for reference. On the other hand, Firefox may work without issues.

Workarounds

The following workarounds are recommended to establish connectivity between Quepid and your Solr instances.

Compatibility with nosniff

In many solrconfig.xml example files we overrode the output format of json to make it easy to read by a human, but this triggers the nosniff issue. If you delete that override, or explicitly configure the json response writer to use a Content-Type of application/javascript then you avoid this security issue:

  <queryResponseWriter name="json" class="solr.JSONResponseWriter">
    <str name="content-type">application/javascript; charset=UTF-8</str>
  </queryResponseWriter>

Assuming you are using SolrCloud, you can actually make this change via the Config API:

curl -X POST -H 'Content-type:application/json' -d '{
  "update-queryresponsewriter": {
    "name": "json",
    "class": "solr.JSONResponseWriter",
    "content-type": "application/javascript;charset=UTF-8"
  }
}' http://localhost:8983/solr/gettingstarted/config

Confirm the change by issuing a curl command similar to curl "http://localhost:8983/solr/gettingstarted/select?q=*:*&rows=0&wt=json" -v -i to confirm you are getting Content-Type: application/javascript; charset=UTF-8 results.

This will satisfy the nosniff header as the Content-Type will match the expected content for the request.

Run a private proxy that sets custom CORS/CSP headers

There are numerous proxy solutions that can customize the headers to be compatible with Quepid. We are working to supply such an option with Quepid itself.

Allow CORS requests

This is a tricky option because the CORS specification provides for allowing all hosts via a wildcard or only allowing a specific host. Specifying multiple hosts is not supported within the header itself, however it can be supported by custom code that generates the header on the fly by checking a allow list of domains.

It is generally not recommended to permit a wildcard as that will enable any website with malicious Javascript to access your Solr instances. If you're able to whitelist a domain via configuration, try adding the app.quepid.com domain. If you need to use the wildcard you can follow the process documented here.

etc/jetty.xml

<!-- security-related headers -->
<Call name="addRule">
 <Arg>
   <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule">
     <Set name="pattern">*</Set>
     <Set name="name">Content-Security-Policy</Set>
     <Set name="value">default-src 'none'; base-uri 'none'; connect-src 'self'; form-action 'self'; font-src 'self'; frame-ancestors 'none'; img-src 'self'; media-src 'self'; style-src 'self' 'unsafe-    inline'; script-src 'self'; worker-src 'self';</Set>
   </New>
 </Arg>
</Call>
<Call name="addRule">
 <Arg>
   <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule">
     <Set name="pattern">*</Set>
     <Set name="name">X-Content-Type-Options</Set>
     <Set name="value">nosniff</Set>
   </New>
 </Arg>
</Call>
<Call name="addRule">
 <Arg>
   <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule">
     <Set name="pattern">*</Set>
     <Set name="name">X-Frame-Options</Set>
     <Set name="value">SAMEORIGIN</Set>
   </New>
 </Arg>
</Call>
<Call name="addRule">
 <Arg>
   <New class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule">
     <Set name="pattern">*</Set>
     <Set name="name">X-XSS-Protection</Set>
     <Set name="value">1; mode=block</Set>
   </New>
 </Arg>
</Call>

Other Notes

Web browsers and best security practices are always evolving. With Quepid, we strive to keep up with these security policies while maintaining compatibility with the application.

It is never recommended to have a Solr instance exposed to the public web. It is also never recommended to "loosen" security on private instances unless it is a last resort. Practicing safe practices with your sandbox instances will lead to better practices with your production instances.

As always, Happy (and safe) searching!

Add a custom footer
Add a custom sidebar
Clone this wiki locally