Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remaining FT.* commands - set FIRST_KEY_INDEX to 1 #2444

Closed

Conversation

carlhopf
Copy link
Contributor

Description

remaining FT.* commands also select the cluster node based on FIRST_KEY_INDEX


Checklist

  • Does npm test pass with this change (including linting)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?

Copy link
Collaborator

@leibale leibale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, I need to make sure about some commands (see review comments)

packages/search/lib/commands/CONFIG_GET.ts Outdated Show resolved Hide resolved
packages/search/lib/commands/DICTADD.ts Outdated Show resolved Hide resolved
packages/search/lib/commands/DICTDEL.ts Outdated Show resolved Hide resolved
packages/search/lib/commands/DICTDUMP.ts Outdated Show resolved Hide resolved
@carlhopf carlhopf force-pushed the cluster-search-first-key-indexes branch from fbdb5d1 to 76f7e1b Compare March 17, 2023 13:30
@carlhopf carlhopf force-pushed the cluster-search-first-key-indexes branch from 76f7e1b to da7233a Compare March 17, 2023 13:39
@@ -1,3 +1,5 @@
export const FIRST_KEY_INDEX = 2;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually 2 is incorrect as well, this should be executed on all shards (same for all alias commands)

@@ -1,5 +1,7 @@
import { RediSearchSchema, pushSchema } from '.';

export const FIRST_KEY_INDEX = 1;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RediSearch indexes should be created on all nodes (this should be removed)

@carlhopf carlhopf closed this Mar 17, 2023
@leibale
Copy link
Collaborator

leibale commented Mar 17, 2023

@carlhopf I guess you close it by mistake?

If you don't have the time to make the changes I can do them.. :)

@leibale leibale reopened this Mar 17, 2023
@codecov-commenter
Copy link

codecov-commenter commented Mar 17, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.01 🎉

Comparison is base (ca1d04f) 95.60% compared to head (2ffedc8) 95.61%.

❗ Current head 2ffedc8 differs from pull request most recent head da7233a. Consider uploading reports for the commit da7233a to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2444      +/-   ##
==========================================
+ Coverage   95.60%   95.61%   +0.01%     
==========================================
  Files         455      455              
  Lines        4548     4563      +15     
  Branches      520      520              
==========================================
+ Hits         4348     4363      +15     
  Misses        129      129              
  Partials       71       71              
Impacted Files Coverage Δ
packages/search/lib/commands/ALIASADD.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/ALIASDEL.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/ALIASUPDATE.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/DROPINDEX.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/EXPLAINCLI.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/PROFILE_AGGREGATE.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/PROFILE_SEARCH.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/SPELLCHECK.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/SUGADD.ts 100.00% <100.00%> (ø)
packages/search/lib/commands/SUGDEL.ts 100.00% <100.00%> (ø)
... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@carlhopf
Copy link
Contributor Author

carlhopf commented Mar 17, 2023

the disadvantage i see with creating all indexes (high number) on all cluster nodes, would be that it will not scale horizontally when increasing cluster size.

the base index memory (even if contains no data) would always be occupied on all nodes?


my original reason to shard FT.* across the cluster with FIRST_KEY_INDEX, was to distribute indexes and hashes to individual nodes (using /redisearch.so only and sharding manually across the cluster with {hash}:slots)

but if i understand correctly, this is done automatically under the hood by using RediSearch+coordinator https://redis.io/docs/stack/search/design/overview/#scalable-distributed-search

hence it might not be useful to set FIRST_KEY_INDEX, and instead simply let coordinator take care of it?

@leibale leibale closed this Apr 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants