Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: When segmentCount > 200K [channel_store][ResourceExhausted desc = trying to send message larger than max (2235554 vs. 2097152) #33125

Open
1 task done
XuanYang-cn opened this issue May 17, 2024 · 2 comments
Open
1 task done

[Bug]: When segmentCount > 200K [channel_store][ResourceExhausted desc = trying to send message larger than max (2235554 vs. 2097152) #33125

XuanYang-cn opened this issue May 17, 2024 · 2 comments
Assignees
@ThreadDao
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone
2.4.4

Comments

@XuanYang-cn
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version:
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

[2024/05/16 23:30:18.106 +00:00] [INFO] [datacoord/channel_manager_v2.go:268] ["deregister node"] [nodeID=7] [updates="["{type=Delete,nodeID=7,channels=\"[compact-opt-op-5-3844-rootcoord-dml_0_449802983197901109v0]\"}","{type=Watch,nodeID=-9223372036854775808,channels=\"[compact-opt-op-5-3844-rootcoord-dml_0_449802983197901109v0]\"}"]"]

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response
@XuanYang-cn XuanYang-cn added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 17, 2024
@XuanYang-cn XuanYang-cn self-assigned this May 17, 2024
@XuanYang-cn
Copy link
Contributor Author

[2024/05/16 23:30:18.581 +00:00] [WARN] [datacoord/channel_manager_v2.go:272] ["fail to update channel operation updates into meta"] [error="rpc error: code = ResourceExhausted desc = trying to send message larger than max (2235554 vs. 2097152)"]

@ThreadDao ThreadDao self-assigned this May 17, 2024
@yanliang567 yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 17, 2024
@yanliang567 yanliang567 added this to the 2.4.2 milestone May 17, 2024
@ThreadDao ThreadDao mentioned this issue May 17, 2024
Closed
1 task
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue May 22, 2024
@XuanYang-cn
enhance: Make channel meta able to writer 200k plus segments
352b5b9 
See also: milvus-io#33125

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
@XuanYang-cn XuanYang-cn mentioned this issue May 22, 2024
Merged
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue May 23, 2024
@XuanYang-cn
enhance: Make channel meta able to writer 200k plus segments
fb586dd 
See also: milvus-io#33125

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue May 23, 2024
@XuanYang-cn
enhance: Make channel meta able to writer 200k plus segments
f88173b 
See also: milvus-io#33125
pr: milvus-io#33279

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
@XuanYang-cn XuanYang-cn mentioned this issue May 23, 2024
Merged
@yanliang567 yanliang567 modified the milestones: 2.4.2, 2.4.3 May 24, 2024
sre-ci-robot pushed a commit that referenced this issue May 28, 2024
@XuanYang-cn
enhance: Make channel meta able to writer 200k plus segments (#33300)
0c5aaeb 
See also: #33125
pr: #33279

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue May 28, 2024
@XuanYang-cn
enhance: Make channel meta able to writer 200k plus segments (#33279)
5e39aa9 
See also: #33125

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
@XuanYang-cn
Copy link
Contributor Author

/assign @ThreadDao Please help verify
/unassign

@yanliang567 yanliang567 modified the milestones: 2.4.3, 2.4.4 May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ThreadDao ThreadDao

Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
2.4.4
Development

No branches or pull requests
3 participants
@ThreadDao @XuanYang-cn @yanliang567