Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement]: Add a param item to ignore bad message id from checkpoint #33122

Open
1 task done
congqixia opened this issue May 17, 2024 · 0 comments
Open
1 task done

[Enhancement]: Add a param item to ignore bad message id from checkpoint #33122

congqixia opened this issue May 17, 2024 · 0 comments
Assignees
@congqixia
Labels
kind/enhancement Issues or changes related to enhancement

Comments

@congqixia
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

What would you like to be added?

Add a param item to ignore bad message id from checkpoint

Why is this needed?

When users try to switch mq type(say kafka to pulsar) the checkpoint left in metastore may cause new cluster failed to seek and some function may go abnormal
We need a switch for user to control behavior when mq client meet bad message id here.

Anything else?

No response
@congqixia congqixia added the kind/enhancement Issues or changes related to enhancement label May 17, 2024
@congqixia congqixia self-assigned this May 17, 2024
congqixia added a commit to congqixia/milvus that referenced this issue May 17, 2024
@congqixia
enhance: Add param item to ignore bad message id in checkpoint
4c5fcdc 
See also milvus-io#33122

This pr add param item `mq.ignoreBadPosition` to control behavior when
mq failed to parse message id from checkpoint

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
@congqixia congqixia mentioned this issue May 17, 2024
Merged
congqixia added a commit to congqixia/milvus that referenced this issue May 17, 2024
@congqixia
enhance: Add param item to ignore bad message id in checkpoint
05ce8c5 
See also milvus-io#33122

This pr add param item `mq.ignoreBadPosition` to control behavior when
mq failed to parse message id from checkpoint

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
@congqixia congqixia mentioned this issue May 17, 2024
Merged
sre-ci-robot pushed a commit that referenced this issue May 20, 2024
@congqixia
enhance: Add param item to ignore bad message id in checkpoint (#33123)
c2ac692 
See also #33122

This pr add param item `mq.ignoreBadPosition` to control behavior when
mq failed to parse message id from checkpoint

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue May 20, 2024
@congqixia
enhance: [2.3] Add param item to ignore bad message id in checkpoint (#…
f848e82 
…33128)

Cherry-pick from master
pr: #33123
See also #33122

This pr add param item `mq.ignoreBadPosition` to control behavior when
mq failed to parse message id from checkpoint

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
congqixia added a commit to congqixia/milvus that referenced this issue May 20, 2024
@congqixia
enhance: Try LatestMessageID when checkpoint unmarshal fails
e4cb10b 
See also milvus-io#33122

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
@congqixia congqixia mentioned this issue May 20, 2024
Merged
congqixia added a commit to congqixia/milvus that referenced this issue May 20, 2024
@congqixia
enhance: Try LatestMessageID when checkpoint unmarshal fails
2e1774e 
See also milvus-io#33122

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
@congqixia congqixia mentioned this issue May 20, 2024
Merged
sre-ci-robot pushed a commit that referenced this issue May 21, 2024
@congqixia
enhance: [2.3]Try LatestMessageID when checkpoint unmarshal fails (#3…
e325ec0 
…3159)

Cherry-pick from master
pr: #33158
See also #33122

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
sre-ci-robot pushed a commit that referenced this issue May 21, 2024
@congqixia
enhance: Try LatestMessageID when checkpoint unmarshal fails (#33158)
12e8c6c 
See also #33122

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
congqixia added a commit to congqixia/milvus that referenced this issue May 22, 2024
@congqixia
enhance: Add param item to ignore bad message id in checkpoint (milvu…
51764b3 
…s-io#33123)

See also milvus-io#33122

This pr add param item `mq.ignoreBadPosition` to control behavior when
mq failed to parse message id from checkpoint

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
congqixia added a commit to congqixia/milvus that referenced this issue May 22, 2024
@congqixia
enhance: Try LatestMessageID when checkpoint unmarshal fails (milvus-…
ffd14a9 
…io#33158)

See also milvus-io#33122

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
@congqixia congqixia mentioned this issue May 22, 2024
Merged
sre-ci-robot pushed a commit that referenced this issue May 23, 2024
@congqixia
enhance: [2.4] Add param item to ignore bad message id in checkpoint (#…
d6bc95d 
…33123) (#33249)

Cherry-pick from master
pr: #33123 #33158
See also  #33122

This pr add param item `mq.ignoreBadPosition` to control behavior when
mq failed to parse message id from checkpoint

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@congqixia congqixia

Labels
kind/enhancement Issues or changes related to enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@congqixia