Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Kafka Consumer Package #342

Merged
merged 71 commits into from Apr 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
1f5a823
feat: add skeleton for kafka raw consumer
oss92 Oct 28, 2020
6b4eec6
feat: options
oss92 Jan 27, 2021
9557e62
feat: consumer component
oss92 Feb 4, 2021
186e421
feat: more options and flexibility for Kafka consumer
oss92 Feb 4, 2021
ff10865
feat: add retries on message/batch level
oss92 Feb 4, 2021
be97a5f
fix: compile
oss92 Feb 4, 2021
5f71522
feat: implement failure strategy
oss92 Feb 4, 2021
16ddd45
fix: stylecheck
oss92 Feb 4, 2021
132e284
feat: add correlation
oss92 Feb 6, 2021
be363d0
docs: add example
oss92 Feb 6, 2021
b0943b5
docs: finalized example
oss92 Feb 7, 2021
080ef76
fix: ExitStrategy
oss92 Feb 7, 2021
4b3c17d
docs: fix example location
oss92 Feb 7, 2021
5bc743a
fix: remove unnecessary handling of signals
oss92 Feb 7, 2021
b183ae6
fix: prevent ConsumeClaim from reclaiming messages before cancelling …
oss92 Feb 7, 2021
bb40a01
docs: more details to context cancellation
oss92 Feb 7, 2021
f026199
test: kafka component initialization
oss92 Feb 8, 2021
aacfe77
feat: add more metrics
oss92 Feb 8, 2021
a510c88
fix: add default sarama config
oss92 Feb 8, 2021
d9b1f2b
test: with sarama mock
oss92 Feb 8, 2021
e558d4d
test: remove sarama mocks
oss92 Feb 9, 2021
88517a6
test: integration test
oss92 Feb 9, 2021
16c6581
test: fix integration test names
oss92 Feb 9, 2021
d9a0681
refactor: better method placement
oss92 Feb 9, 2021
00b50c0
fix: gauge vector name
oss92 Feb 9, 2021
f1beb47
test: kafka consumer handler
oss92 Feb 9, 2021
bd8ca59
test: fix string decoder
oss92 Feb 10, 2021
6e9a64b
test: differentiate loggers for patron instances
oss92 Feb 10, 2021
b416c9b
test: start from offset oldest to avoid race
oss92 Feb 10, 2021
c43bcfe
fix: error when the component could not process a message
oss92 Feb 10, 2021
2aeb582
test: ticker and batching
oss92 Feb 10, 2021
ef520bb
chore: tidy
oss92 Feb 10, 2021
41d4e70
Merge branch 'master' into kafka-raw-consumer
Feb 13, 2021
2ff99ce
Merge branch 'master' into kafka-raw-consumer
Feb 16, 2021
deafe1f
feat: add Context() for each Kafka message
oss92 Feb 16, 2021
4ed89b3
feat: add kafka.Batch
oss92 Feb 16, 2021
87d01c0
chore: address PR review
oss92 Feb 16, 2021
50f6021
fix: optimize truncate slice
oss92 Feb 17, 2021
2f562d7
feat: add Commit and MarkBatchOffset to Batch interface
oss92 Feb 17, 2021
5e67ba2
refactor: no need to write Batch again
oss92 Feb 17, 2021
cd6fa7e
Merge branch 'master' of github.com:beatlabs/patron into kafka-raw-co…
oss92 Feb 22, 2021
08161e3
feat: kafka component OptionFunc instead of builder
oss92 Feb 22, 2021
49d0373
fix: lint
oss92 Feb 22, 2021
7d550b9
test: kafka OptionFunc
oss92 Feb 22, 2021
c2d525b
chore: address PR review to simplify Run for kafka component
oss92 Mar 2, 2021
a21521b
docs: kafka component
oss92 Mar 2, 2021
9f28dfb
fix: kafka example to commit batch in case message could not be decoded
oss92 Mar 2, 2021
37e1572
fix: remove unused channel
oss92 Mar 2, 2021
12bad5f
fix: example to MarkOffset
oss92 Mar 2, 2021
174f4d0
feat: add debug line when no correlation header is present in Kafka c…
oss92 Mar 2, 2021
27bd0ef
fix: comment
oss92 Mar 2, 2021
0ef2f17
chore: update sarama to 1.28.0
oss92 Mar 2, 2021
2c177ba
fix: remove error log
oss92 Mar 3, 2021
5c8f877
fix: remove ability to mark offsets and commit from message interface
oss92 Mar 18, 2021
37be59e
refactor: move consumer group implementation to its own package
oss92 Mar 19, 2021
fb3dbc9
fix: comment
oss92 Mar 19, 2021
ece7cac
Merge branch 'master' of github.com:beatlabs/patron into kafka-raw-co…
oss92 Mar 19, 2021
c84b622
fix: lint
oss92 Mar 19, 2021
48d5ed6
fix: fail if a nil sarama config is supplied to OptionFunc
oss92 Mar 19, 2021
f1c2b8b
fix: retries on component level
oss92 Mar 25, 2021
ecedf90
refactor: cleanup failure strat
oss92 Mar 25, 2021
311f7dd
fix: test, skip never fails
oss92 Mar 25, 2021
9f4c1ae
test: higher coverage kafka package
oss92 Mar 26, 2021
cd6a68f
docs: add more documentation for FailureStrategy and retries
oss92 Mar 26, 2021
f1bc754
fix: unlock even on flush error
oss92 Mar 28, 2021
53edcf3
cleanup: remove empty line
oss92 Mar 28, 2021
88523c2
feat: reset retry counter on processing success
oss92 Mar 28, 2021
c757f14
feat: default retries and wait
oss92 Mar 28, 2021
37f3d8b
chore: add default to executeFailureStrategy
oss92 Mar 29, 2021
46882bc
fix: bug with parent context cancelling and handle pr review
oss92 Mar 29, 2021
6541bb3
Merge branch 'master' into kafka-raw-consumer
tpaschalis Mar 30, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions component/async/kafka/group/group_test.go
Expand Up @@ -159,6 +159,7 @@ func (m *mockConsumerSession) MemberID() string { return "" }
func (m *mockConsumerSession) GenerationID() int32 { return 0 }
func (m *mockConsumerSession) MarkOffset(string, int32, int64, string) {
}
func (m *mockConsumerSession) Commit() {}
func (m *mockConsumerSession) ResetOffset(string, int32, int64, string) {
}
func (m *mockConsumerSession) MarkMessage(*sarama.ConsumerMessage, string) {}
Expand Down