You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Create a nest microservice with ClientKafka configured to communicate via MessagePattern
Create a topic with 2 partitions in Kafka
Subscribe to the topic via subscribeToResponseOf() method
Run two instances of the microservice
Nest will create a “reply” topic for receiving responses (increase the number of partitions to 2 if the reply topic has only one)
Consumer of the first microservice is assigned to the partition 0, and consumer of the second one to the partition 1
Run one more microservice, Nest will leave its consumer without assigned partition (because we have just 2)
Shutdown the first microservice
Nest will start rebalancing using KafkaReplyPartitionAssigner
Result: the consumer of the second microservice is assigned to two partitions, and the consumer of third one to zero partitions. Because of that the rebalancing is launched continuously after rebalanceTimeout period.
The reason of this behavior is in the logic which tries to retain previous assignments. The assignment of the 3rd consumer comes with previous value null and Nest successfully re-assigns it again to this consumer.
As far as I understand it’s necessary to improve the condition in this line if (assignment[assignee][topic].length === 0) { to check additionally for null assignments.
Is there an existing issue for this?
Current behavior
Consumer is not assigned to a topic partition after rollout in k8s.
Precondition: it’s necessary to have 2+ replicas of microservice with Kafka client.
Minimum reproduction code
nest/packages/microservices/helpers/kafka-reply-partition-assigner.ts
Line 123 in 2d13d08
Steps to reproduce
subscribeToResponseOf()
methodKafkaReplyPartitionAssigner
Result: the consumer of the second microservice is assigned to two partitions, and the consumer of third one to zero partitions. Because of that the rebalancing is launched continuously after
rebalanceTimeout
period.The reason of this behavior is in the logic which tries to retain previous assignments. The assignment of the 3rd consumer comes with previous value
null
and Nest successfully re-assigns it again to this consumer.As far as I understand it’s necessary to improve the condition in this line
if (assignment[assignee][topic].length === 0) {
to check additionally for null assignments.nest/packages/microservices/helpers/kafka-reply-partition-assigner.ts
Line 123 in 2d13d08
Expected behavior
All consumers are assigned to at least one partition after the rebalancing.
Package
@nestjs/common
@nestjs/core
@nestjs/microservices
@nestjs/platform-express
@nestjs/platform-fastify
@nestjs/platform-socket.io
@nestjs/platform-ws
@nestjs/testing
@nestjs/websockets
Other package
No response
NestJS version
9 (but I guess it’s actual for 10th as well)
Packages versions
Node.js version
No response
In which operating systems have you tested?
Other
No response
The text was updated successfully, but these errors were encountered: