Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add async supoort for SEARCH commands #2096

Merged
merged 6 commits into from Apr 28, 2022
Merged

Conversation

dvora-h
Copy link
Collaborator

@dvora-h dvora-h commented Apr 7, 2022

Pull Request check-list

Please make sure to review and check all of these items:

  • Does $ tox pass with this change (including linting)?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?
  • Was the change added to CHANGES file?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

Please provide a description of the change here.

@dvora-h dvora-h added the feature New feature label Apr 7, 2022
@dvora-h dvora-h marked this pull request as draft April 7, 2022 15:48
@codecov-commenter
Copy link

codecov-commenter commented Apr 7, 2022

Codecov Report

Merging #2096 (1f29a42) into master (cb91eed) will increase coverage by 0.14%.
The diff coverage is 97.23%.

@@            Coverage Diff             @@
##           master    #2096      +/-   ##
==========================================
+ Coverage   92.45%   92.59%   +0.14%     
==========================================
  Files         104      105       +1     
  Lines       24364    25085     +721     
==========================================
+ Hits        22526    23228     +702     
- Misses       1838     1857      +19     
Impacted Files Coverage Δ
redis/asyncio/client.py 92.10% <ø> (+0.17%) ⬆️
redis/commands/search/commands.py 88.59% <87.09%> (-0.44%) ⬇️
tests/test_asyncio/test_search.py 98.68% <98.68%> (ø)
redis/commands/__init__.py 100.00% <100.00%> (ø)
redis/commands/redismodules.py 100.00% <100.00%> (ø)
redis/commands/search/__init__.py 89.28% <100.00%> (+4.28%) ⬆️
tests/test_asyncio/conftest.py 94.44% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb91eed...1f29a42. Read the comment docs.

@dvora-h dvora-h marked this pull request as ready for review April 7, 2022 16:39
@dvora-h dvora-h requested a review from chayim April 7, 2022 16:39
@@ -857,3 +857,245 @@ def syndump(self):
""" # noqa
raw = self.execute_command(SYNDUMP_CMD, self.index_name)
return {raw[i]: raw[i + 1] for i in range(0, len(raw), 2)}


class AsyncSearchCommands(SearchCommands):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could see this becoming too difficult to maintain. WDYT about moving this into async_commands.py?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That might get messy once you start thinking about the other Redis modules like AI and their own async counterparts.

It might help to choose either of these options instead:

  1. Under the current commands folder, create an asyncio module
  2. Under the asyncio module, create a new commands directory

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true - commands are becoming unwieldy. Is there a more accepted pattern for maintaining async support? What about the following pattern (note my being new to async), yes this might break things:

  1. Create functions for each command (i.e set) prefixed with double underscores to denote private
  2. Create their implementations in both async and sync versions, with execute_command being different, to support async/await.

Gut check: Does this seem painful as you'd now need to create one implementation and two wrappers per function? Alternatively we could generate #2 quite frankly, easily enough.

@Andrew-Chen-Wang thoughts? I'm just spitballing here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the only patterns that I've seen adopted are to use an a- or async- prefix (eg Python itself, django) or to have separated mixins (eg Elasticsearch-py).

And mm the current implementation of an abstraction of execute_command, to me, is the most viable method of maintainence. I think adding too many ways of separation between sync and async will make redis-py more bloated and unmaintainable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I think that's out of scope for this (obviously!) but let's not lose track. I'm literally going to open a GH issue, and link to this comment.

@chayim chayim self-requested a review April 24, 2022 08:27
Copy link
Contributor

@chayim chayim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dvora-h see the duplication of testsdata. Beyond that, it's clean given our current implementation.

@dvora-h dvora-h merged commit 1475e5c into redis:master Apr 28, 2022
@dvora-h dvora-h deleted the async-search branch June 18, 2023 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants