Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V5 #2485

Draft
wants to merge 327 commits into
base: master
Choose a base branch
from
Draft

V5 #2485

wants to merge 327 commits into from

Conversation

leibale
Copy link
Collaborator

@leibale leibale commented Apr 26, 2023

@codecov-commenter
Copy link

codecov-commenter commented Apr 27, 2023

Codecov Report

Patch coverage: 39.13% and project coverage change: -41.59 ⚠️

Comparison is base (3273c85) 95.67% compared to head (784bd0d) 54.08%.

❗ Current head 784bd0d differs from pull request most recent head e04609c. Consider uploading reports for the commit e04609c to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #2485       +/-   ##
===========================================
- Coverage   95.67%   54.08%   -41.59%     
===========================================
  Files         455      264      -191     
  Lines        4552     2777     -1775     
  Branches      522      447       -75     
===========================================
- Hits         4355     1502     -2853     
- Misses        128     1161     +1033     
- Partials       69      114       +45     
Impacted Files Coverage Δ
packages/client/lib/commands/GETBIT.ts 50.00% <ø> (-50.00%) ⬇️
packages/client/lib/commands/GETDEL.ts 100.00% <ø> (ø)
packages/client/lib/commands/GETEX.ts 38.46% <ø> (-61.54%) ⬇️
packages/client/lib/commands/GETRANGE.ts 100.00% <ø> (ø)
packages/client/lib/commands/GETSET.ts 100.00% <ø> (ø)
packages/client/lib/commands/HDEL.ts 100.00% <ø> (ø)
packages/client/lib/commands/HELLO.ts 12.50% <ø> (-87.50%) ⬇️
packages/client/lib/commands/HEXISTS.ts 100.00% <ø> (ø)
packages/client/lib/commands/HGET.ts 100.00% <ø> (ø)
packages/client/lib/commands/HGETALL.ts 100.00% <ø> (ø)
... and 159 more

... and 241 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@varadero varadero mentioned this pull request Dec 29, 2023
leibale and others added 13 commits January 3, 2024 11:46
* redis client socket changes needed for sentinel

* Sentinel Implementation [EXPERIMENTAL]

* add pooling

* improve typing with SENTINEL_ client members

* cleanup - remove unused comments / commented code

* small sendCommand change + revert change to tsconfig

* add more sentinel commands needed for testing.

* lots of fixups and a reasonable first pass test suite

* add a timer option to update topology in background

+ don't need both sentinel client and pubsubclient
+ nits

* format all the things

* more progress

* small cleanup

* try to group promises together to minimize the internal await points

* redo events, to keep a single topology event to listen on

* nits + readme

* add RedisSentinelFactory to provide lower level access to sentinel

* nit

* update

* add RedisSentinelClient/Type for leased clients
	returned by aquire()
	used by function passed to use()

* add self for private access + improve emitting

* nit

* nits

* improve testing

- improve steady state waiting between tests
- get masternode from client, not from sentinels themselves (not consistent and then client isn't changing as we expect
- provide extensive logging/tracing on test errors
	- provide a very low impact tracing mechanism withinthe code that only really impacts code when tracing is in use.

* ismall nit for typing

* bunch of changes

- harden testing
	- don't use sentinel[0] for debug error dump as could be downed by a test
	- increase time for sentinel down test to 30s (caused a long taking failover)
- add client-error even / don't pass throuh client errors as errors option for pubsub proxy
- when passing through cient errors as error events, dont pass the event, but the Error object, as only Error objects are supposed to be on 'error'
	-

* improve pub sub proxy.

save the refference to all channel/pattern listeners up front on creation, dont hve to fetch the object each time, as it doesn't change.

removes race condition between setting up the listener and the pub sub node going down and being recreated.

* wrap the passed through RedisClient error to make clear where its coming from.

* refactor sentinel object / factory tests apart

* harden tests a little bit more

* add pipeline test

* add scripts/function tests + fixups / cleanups to get them to work

* change to use redis-stack-server for redis nodes to enable module testing

* fix test, forgot to return in use function with module

* rename test

* improve tests to test with redis/sentinel nodes with and withput passwords

this tests that we are handling the nodeClientOptions and sentinelClientOptions correctly

* cleanup for RedisSentinel type generic typing in tests

* remove debugLog, just rely on traace mechanism

* added multi tests for script/function/modules

* don't emit errors on lease object, only on main object

* improve testing

* extract out common code to reduce duplication

* nit

* nits

* nit

* remove SENTINEL_... commands from main client, load them via module interface

* missed adding RedisSentinelModule to correct places in RedisSentinelFactory

* nits

* fix test logging on error

1) it takes a lot of time now, so needs larger timeout
2) docker logs can be large, so need to increase maxBuffer size so doesn't error (and break test clean up)

* invalidate watches when client reconnects

+ provide API for other wrapper clients to also create invalid watch states programatically.

Reasoning: if a user does a WATCH and then the client reconnects, the watch is no longer active, but if a user does a MULTI/EXEC after that, they wont know, and since the WATCH is no longer active, the request has no protection.

The API is needed for when a wrapper client (say sentinel, cluster) might close the underlying client and reopen a new one transparently to the user.  Just like in the reconnection case, this should result in an error, but its up to the wrapping client to provide the appropriate error

* remove WATCH and UNWATCH command files, fix WATCH and UNWATCH return type, some more cleanups

* missing file in last commit :P

* support for custom message in `WatchError`

* setDirtyWatch

* update watch docs

* fixes needed

* wip

* get functions/modules to work again

self -> _self change

* reuse leased client on pipelined commands.

though I realize this implementation, really only works after the first write command.

unsure this is worth it.

* test tweaks

* nit

* change how "sentinel" object client works, allow it to be reserved

no more semaphore type counting

* review

* fixes to get more tests to pass

* handle dirtyWatch and watchEpoch in reset and resetIfDirty

* "fix", but not correct, needs more work

* fix pubsub proxy

* remove timeout from steadyState function in test, caused problems

* improve restarting nodes

* fix pubsub proxy and test

---------

Co-authored-by: Leibale Eidelman <me@leibale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants