Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memberlist: expose configuration of memberlist handoff queue depth #4402

Closed
cabrinha opened this issue Aug 5, 2021 · 2 comments
Closed

Memberlist: expose configuration of memberlist handoff queue depth #4402

cabrinha opened this issue Aug 5, 2021 · 2 comments
Labels

Comments

@cabrinha
Copy link

cabrinha commented Aug 5, 2021

Is your feature request related to a problem? Please describe.
Memberlist errors out with handler queue full, dropping message (8) from=192.168.25.7:7946 very often.

Describe the solution you'd like
I'd like to not see this error.

Describe alternatives you've considered
Replication factor of memberlist may need changing?

Additional context
Oh, you know... The usual errors here and there.

@cabrinha
Copy link
Author

cabrinha commented Sep 21, 2021

Also getting errors like these on distributor startup:

{"caller":"lifecycler.go:448","err":"failed to CAS-update key collectors/distributor: too many retries","level":"error","msg":"failed to write to the KV store, sleeping","ring":"distributor","ts":"2021-09-21T20:15:40.887303649Z"}

Which is followed by:

{"caller":"memberlist_logger.go:74","level":"error","msg":"Failed to decode dead message: codec.decoder: Only encoded map or array can be decoded into a struct. (valueType: c) from=192.168.48.69:7946","ts":"2021-09-21T20:25:17.845920625Z"}
{"caller":"memberlist_logger.go:74","level":"error","msg":"msg type (6) not supported from=192.168.48.69:7946","ts":"2021-09-21T20:25:17.846028565Z"}
{"caller":"memberlist_logger.go:74","level":"error","msg":"msg type (16) not supported from=192.168.48.69:7946","ts":"2021-09-21T20:25:17.84604389Z"}

Related / same issue? #4326

@stale
Copy link

stale bot commented Dec 21, 2021

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. It will be closed in 15 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Dec 21, 2021
@stale stale bot closed this as completed Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant