Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION] Consumer group doesn't get any partitions by the RoundRobinPartitionAssignor #906

Open
MaximMukhametov opened this issue Jul 13, 2023 · 0 comments
Labels

Comments

@MaximMukhametov
Copy link

Hi 馃憢馃徎
I have a pytest test with parametrization (7 tests), that produce and consume simple msg with simple logic.
It uses kafka cluster with 3 nodes, each test creates topic with new name test_toipic<鈩朹test>with 3 partition and 2 replica-factor, and delete this topic after each test.
Each test I run 3 consumers with new group test_group<鈩朹test>
Every time I run this test, 1/7 or 2/7 fails, because there are no partitions for the consumer group:
[WARNING] No partition metadata for topic test_topic_4 (roundrobin.py:61)

I use KafkaAdminClient to inspect some details, so I get:
topic:

{'error_code': 0,
  'topic': 'test_topic_4',
  'is_internal': False,
  'partitions': [{'error_code': 0,
    'partition': 0,
    'leader': 4,
    'replicas': [4, 2],
    'isr': [4, 2],
    'offline_replicas': []},
   {'error_code': 0,
    'partition': 2,
    'leader': 3,
    'replicas': [3, 4],
    'isr': [3, 4],
    'offline_replicas': []},
   {'error_code': 0,
    'partition': 1,
    'leader': 2,
    'replicas': [2, 3],
    'isr': [2, 3],
    'offline_replicas': []}]},

group:

 GroupInformation(error_code=0, group='test_group4', state='Stable', protocol_type='consumer', protocol='roundrobin', members=[MemberInformation(member_id='aiokafka-0.8.0-f8ed0518-2198-4f40-84e4-35263e3796cd', client_id='aiokafka-0.8.0', client_host='/172.31.0.1', member_metadata=ConsumerProtocolMemberMetadata(version=0, subscription=['test_topic_4'], user_data=b''), member_assignment=ConsumerProtocolMemberAssignment(version=0, assignment=[], user_data=b'')), MemberInformation(member_id='aiokafka-0.8.0-6c0c504d-8a40-4ab8-9d24-8304da934bb0', client_id='aiokafka-0.8.0', client_host='/172.31.0.1', member_metadata=ConsumerProtocolMemberMetadata(version=0, subscription=['test_topic_4'], user_data=b''), member_assignment=ConsumerProtocolMemberAssignment(version=0, assignment=[], user_data=b'')), MemberInformation(member_id='aiokafka-0.8.0-c4f69127-7feb-4a19-b9f3-0270e57745b2', client_id='aiokafka-0.8.0', client_host='/172.31.0.1', member_metadata=ConsumerProtocolMemberMetadata(version=0, subscription=['test_topic_4'], user_data=b''), member_assignment=ConsumerProtocolMemberAssignment(version=0, assignment=[], user_data=b''))], authorized_operations=None),

As you can see, the topic has 3 partitions, by all members of the group have assignment=[]

I also use kafka-ui and can share some details from there:
Screenshot 2023-07-13 at 12 25 05
Screenshot 2023-07-13 at 12 18 04
Screenshot 2023-07-13 at 12 18 48
Screenshot 2023-07-13 at 12 19 31

Do you have any idea why is this happening馃檹馃徎?
Let me know If you need any additional information

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant