Releases: pat/thinking-sphinx
v5.5.0
Upgrading
No breaking or major changes.
New Features
- ThinkingSphinx::Processor, a public interface to perform index-related operations on model instances or model name/id combinations. In collaboration with @akostadinov (#1215).
Changes to behaviour
- Confirmed support by testing against Ruby 3.1 and 3.2 by @jdelStrother (#1237).
Fixes
v5.4.0
Upgrading
No breaking or major changes.
New Features
- Rails 7 support, including contributions from @anthonyshull in #1205.
Changes to behaviour
- Confirmed support by testing against Manticore 4.0 and Sphinx 3.4.
Fixes
- Include instance_exec in ThinkingSphinx::Search::CORE_METHODS by @jdelStrother in #1210.
- Use File.exist? instead of the deprecated File.exists? (#1211).
v5.3.0
Upgrading
No breaking or major changes.
Changes to behaviour
- StaleIdsExceptions now include a URL in their error message with recommendations on how to resolve the problem.
- Fire real-time callbacks on
after_commit
(including deletions) instead ofafter_save
/after_destroy
to ensure data is fully persisted to the database before updating Sphinx. More details in #1204.
Fixes
- Ensure Thinking Sphinx's ActiveRecord components are loaded by either Rails' after_initialise hook or ActiveSupport's on_load notification, because the order of these two events are not consistent.
- Remove
app/indices
from eager_load_paths in Rails 4.2 and 5, to match the behaviour in 6.
Both of these fixes are evolutions/improvements to changes introduced in v5.2.0/5.2.1.
v5.2.1
v5.2.0
Upgrading
No breaking or major changes.
New features
- Confirmed support for Ruby 3.0.
- Orphaned records in real-time indices can now be cleaned up without running
rails ts:rebuild
. Disabled by default, can be enabled by settingreal_time_tidy
to true per environment inconfig/thinking_sphinx.yml
(and will needts:rebuild
to restructure indices upon initial deploy). More details in #1192.
Bug fixes
- Avoid loading ActiveRecord during Rails initialisation so app configuration can still have an impact (@jdelStrother in #1194).
- Remove
app/indices
(in both the Rails app and engines) from Rails' eager load paths, which was otherwise leading to indices being loaded more than once. (See #1191 and #1195).
v5.1.0
Upgrading
No breaking or major changes.
New features
- Support for Sphinx v3.3 and Manticore v3.5.
- Support for Rails 6.1 (via joiner v0.6.0).
Changes to behaviour
enable_star
is no longer available as a configuration option, as it's been enabled by default in Sphinx since v2.2.2, and is no longer allowed in Sphinx v3.3.1.- All timestamp attributes are now considered plain integer values from Sphinx's perspective. Sphinx was already expecting integers, but since Sphinx v3.3.1 it doesn't recognise timestamps as a data type. There is no functional difference with this change - Thinking Sphinx was always converting times to their UNIX epoch integer values.
- Allow configuration of the maximum statement length (@kalsan in #1179).
- Respect
:path
values to navigate associations for Thinking Sphinx callbacks on SQL-backed indices. Discussed in #1182.
Bug fixes
- Don't attempt to update delta flags on frozen model instances.
v5.0.0
Major Features and Breaking Changes
Thinking Sphinx v5.0 has one significant change - explicit callbacks - plus drops support for old versions of Rails/Ruby/Sphinx, and adds a few other smaller improvements.
Explicit Callbacks
Previous versions of Thinking Sphinx automatically added callbacks to all ActiveRecord models, for the purpose of persisting changes back to Sphinx (whether that be inserts, updates, or deletions). And while the actual overhead for non-indexed models wasn't super slow, it's still far from ideal.
So now, you need to add callbacks yourself, to just the models you're indexing.
With SQL-backed models (defined using :with => :active_record
), you'll very likely want to add one of the two following lines inside your model:
class Article < ApplicationRecord
# If you're not using delta indices:
ThinkingSphinx::Callbacks.append(self, :behaviours => [:sql])
# If you *are* using delta indices:
ThinkingSphinx::Callbacks.append(self, :behaviours => [:sql, :deltas])
end
If you're using real-time indices, you very likely already have callbacks defined in your models, but you can replace them with the new calls:
class Article < ApplicationRecord
# Instead of this...
after_save ThinkingSphinx::RealTime.callback_for(:article)
# use this...
ThinkingSphinx::Callbacks.append(self, :behaviours => [:real_time])
end
For associated models which still fire real-time callbacks, you can use the :path
option with the same call:
class Comment < ApplicationRecord
belongs_to :article
ThinkingSphinx::Callbacks.append self,
:behaviours => [:real_time],
:path => [:article]
end
And if you're using a custom block with your old real-time callback, you can pass that same block to the new approach as well:
class Article < ApplicationRecord
ThinkingSphinx::Callbacks.append(
self, :behaviours => [:real_time]
) do |instance|
# returning an array of instances to index. You could add
# custom logic here if you don't want indexing to happen
# in some cases.
end
end
At this point in time, the older callback style for real-time indices will continue to work, but it's still recommended to update your code to the new style instead.
On the off chance you are using SQL-backed indices and you have attribute_updates
enabled in config/thinking_sphinx.yml
, you'll want to specify that in your :behaviours
option:
ThinkingSphinx::Callbacks.append(self, :behaviours => [:sql, :updates])
Sphinx 2.2.11 or newer is required
Sphinx 2.1 is no longer supported - and ideally, it's best to upgrade any 2.2.x release to 2.2.11.
Sphinx 3.x releases are supported, but there are known issues with indexing SQL-backed indices on a PostgreSQL database (real-time indices are fine though).
As part of this change, Sphinx's docinfo setting is no longer configured, so the skip_docinfo
setting in config/thinking_sphinx.yml
can be removed.
When it comes to Manticore as a drop-in replacement for Sphinx, we're testing against the latest 2.x and 3.x releases, which are currently 2.8.2 and 3.4.2 respectively.
Ruby 2.4 or newer is required
Versions of Ruby less than 2.3 are no longer supported, sorry. We're currently testing against 2.4 through to 2.7.
Rails 4.2 or newer is required
It's been a long time coming, but Rails 3.2 (and 4.0 and 4.1) are no longer supported. The current supported versions are 4.2 through to 6.0 (and 6.1 will likely work as well, once it's released).
Other changes to behaviour
- Remove internal uses of
send
, replaced withpublic_send
as that's available in all supported Ruby versions. - Custom index_set_class implementations can now expect the
:instances
option to be set alongside:classes
, which is useful in cases to limit the indices returned if you're splitting index data for given classes/models into shards. (Introduced in PR #1171 after discussions with @lunaru in #1166.) - Deletion statements are simplified by avoiding the need to calculate document keys/offsets (@njakobsen via #1134).
- Real-time data is deleted before replacing it, to avoid duplicate data when offsets change (@njakobsen via #1134).
- Use
reference_name
as per customindex_set_class
definitions. Previously, the class method was called onThinkingSphinx::IndexSet
even if a custom subclass was configured. (As per discussions with @kalsan in #1172.) - Fields and attributes can be overriden - whichever's defined last with a given name is the definition that's used. This is an edge case, but useful if you want to override any of the default fields/indices. (Requested by @kalsan in #1172.)
Bug fixes
None.
v4.4.1
v4.4.0
Upgrading
No breaking or major changes.
New features
- Confirmed Rails 6.0 support.
- Added ability to have custom real-time index processors (which handles all indices) and populators (which handles a particular index). These are available to get/set via
ThinkingSphinx::RealTime.processor
andThinkingSphinx::RealTime.populator
.
The processor should accept call
with two arguments: an array of index objects, and a block to invoke after each index is processed. Here is a simple example for parallel processing of indices:
# Add the 'parallel' gem to your Gemfile.
ThinkingSphinx::RealTime.processor = Proc.new do |indices, &block|
Parallel.map(indices) do |index|
puts "Populating index #{index.name}"
ThinkingSphinx::RealTime.populator.populate index
puts "Populated index #{index.name}"
block.call
end
end
And the populator should respond to populate
, accepting a single argument which is the index object. Here is a simple example for parallel processing.
# Add the 'parallel' gem to your Gemfile.
class ParallelPopulator
def self.populate(index)
new(index).call
end
def initialize(index)
@index = index
end
def call
Parallel.each(index.scope.find_in_batches) do |instances|
transcriber.copy *instances
true # Don't emit any large object because results are accumulated
end
ActiveRecord::Base.connection.reconnect!
end
private
attr_reader :index
def transcriber
@transcriber ||= ThinkingSphinx::RealTime::Transcriber.new index
end
end
ThinkingSphinx::RealTime.populator = ParallelPopulator
Instead of building your own procs/classes from scratch, you may instead wish to subclass the default classes to tweak behaviour - or at the very least, both classes are useful as reference points for your own replacements:
These changes were influenced by discussions in #1134 with @njakobsen about parallel processing of real-time indices.
Changes to behaviour
- Improve failure message when tables don't exist for models associated with Sphinx indices (Kiril Mitov in #1139).
Bug fixes
- Injected has-many/habtm collection search calls as default extensions to associations in Rails 5+, as it's a more reliable approach in Rails 6.0.0.