Releases: confluentinc/parallel-consumer
Releases · confluentinc/parallel-consumer
0.5.2.8
Fixes
- fix: Fix equality and hash code for ShardKey with array key (#638), resolves (#579)
- fix: Fix target loading computation for inflight records (#662)
- fix: Fix synchronisation logic for transactional producer commit affecting non-transactional usage (#665), resolves (#637)
- fix: Fix for race condition in partition state clean/dirty tracking (#666), resolves (#664)
Improvements
- feature: Make PC message buffer size configurable - two new configuration options for controlling buffer size added (#682)
GH generated
- Fix key equality by @rkolesnev in #638
- Remove duplicated batchSize factor from target load; add a test case by @cloudhunter89 in #662
- Fix synchronisation logic between commits by @rkolesnev in #665
- Fix for race condition in partition state clean/dirty tracking by @rkolesnev in #666
- use https for repositories by @rkolesnev in #683
- Make PC message buffer size configurable by @rkolesnev in #682
Dependency updates omitted for readability
New Contributors
- @cloudhunter89 made their first contribution in #662
Full Changelog: 0.5.2.7...0.5.2.8
Contributors
cloudhunter89 and rkolesnev
Assets 2
0.5.2.7
Fixes
- fix: Return cached pausedPartitionSet (#620), resolves (#618)
- fix: Parallel consumer stops processing data sometimes (#623), fixes (#606)
- fix: Add synchronization to ensure proper intializaiton and closing of PCMetrics singleton (#627), fixes (#617)
- fix: Readme - metrics example correction (#614)
- fix: Remove micrometer-atlas dependency (#628), fixes (#625)
Improvements
- Refactored metrics implementation to not use singleton - improves meter separation, allows correct metrics subsystem operation when multiple parallel consumer instances are running in same java process (#630), fixes (#617) improves on (#627)
GH generated
- fix : for issue 606 : exclude stale containers from counting and add remove add remove staled work containers function in ProcessingShard by @sangreal in #623
- fix: Return cached pausedPartitionSet by @acktsap in #620
- PL-464 remove micrometer-atlas dependency by @rkolesnev in #628
- minor: Change Supplier -> SupplierUtils for consistency by @acktsap in #619
- add synchronization to ensure proper intializaiton and closing of PCM… by @antonmos in #627
- fix readme doc by @flashmouse in #624
- Fix/refactor metrics singleton by @rkolesnev in #630
- build(deps): bump io.projectreactor:reactor-core from 3.5.7 to 3.5.9 by @dependabot in #626
- build(deps): bump org.xerial.snappy:snappy-java from 1.1.10.1 to 1.1.10.3 by @dependabot in #610
- build(deps-dev): bump junit.platform.version from 1.10.0-RC1 to 1.10.0 by @dependabot in #612
- build(deps): bump maven-site-plugin from 4.0.0-M8 to 4.0.0-M9 by @dependabot in #603
New Contributors
Full Changelog: 0.5.2.6...0.5.2.7
0.5.2.6
Improvements
- feature: Micrometer metrics (#594)
Fixes
- fix: Don’t drain mode shutdown kills inflight threads (#559)
- fix: Drain mode shutdown doesn’t pause consumption correctly (#552)
- fix: RunLength offset decoding returns 0 base offset after no-progress commit - related to (#546)
- fix: Transactional PConsumer stuck while rebalancing - related to (#541)
GH generated:
What's Changed
- fix: runlength decoding after no-progress commit by @rkolesnev in #563
- Fix deadlock between pc-control and pc-broker-poll threads where partitions are revoked by @nachomdo in #548
- build(deps-dev): bump mockito.version from 4.9.0 to 5.1.1 by @dependabot in #556
- build(deps): bump vertx.version from 4.3.6 to 4.4.1 by @dependabot in #566
- build(deps): bump versions-maven-plugin from 2.14.1 to 2.15.0 by @dependabot in #555
- build(deps-dev): bump threeten-extra from 1.7.1 to 1.7.2 by @dependabot in #528
- Do not create trace log message when trace logging is not enabled by @lennehendrickx in #544
- build(deps): bump maven-release-plugin from 3.0.0-M7 to 3.0.1 by @dependabot in #580
- PL-211: Update dependencies from dependabot, Add mvnw, use mvnw in jenkins by @eddyv in #583
- PL-211: update dependencies from dependabot by @eddyv in #589
- add instance id into thread name by @flashmouse in #578
- workerThreadPool is initialized lazily by @pweso in #531
- build(deps): bump reactor-core from 3.5.1 to 3.5.7 by @dependabot in #585
- Pl 175 DRAIN mode fix by @niamhthornbury in #591
- build(deps): bump junit.version from 5.10.0-M1 to 5.10.0-RC1 by @dependabot in #598
- build(deps): bump guava from 32.0.1-jre to 32.1.1-jre by @dependabot in #595
- build(deps): bump vertx.version from 4.4.3 to 4.4.4 by @dependabot in #590
- build(deps-dev): bump junit.platform.version from 1.10.0-M1 to 1.10.0-RC1 by @dependabot in #599
- improves: Error in onPartitionsAssigned in parallel consumer by @nachomdo in #537
- Feature/health metrics micrometer by @rkolesnev in #613
- fix: example in readme, license headers by @rkolesnev in #614
- PL-176 handle close dont drain mode gracefully by @rkolesnev in #615
New Contributors
- @lennehendrickx made their first contribution in #544
- @eddyv made their first contribution in #583
- @flashmouse made their first contribution in #578
- @pweso made their first contribution in #531
- @niamhthornbury made their first contribution in #591
Full Changelog: 0.5.2.5...0.5.2.6
0.5.2.5
Improvements
- perf: Adds a caching layer to work management to alleviate O(n) counting (#530)
Fixes
- fixes: #195 NoSuchFieldException when using consumer inherited from KafkaConsumer (#469)
- fix: After new performance fix PR#530 merges - corner case could cause out of order processing (#534)
- fix: Cleanup WorkManager's count of in-progress work, when work is stale after partition revocation (#547)
GH generated:
What's Changed
- fixes: #195 NoSuchFieldException when using consumer inherited from KafkaConsumer by @astubbs in #469
- tests: Demonstrates how to use MockConsumer with PC for issue #176 by @astubbs in #470
- logs: Add partition to warning upon log Truncation by @Ehud-Lev-Forter in #489
- minor: fixes #486: Missing generics in JStreamParallelStreamProcessor by @astubbs in #491
- build(deps): wiremock, mvn-release, reactor, podam, postgresql, junit-pioneer by @astubbs in #495
- minor: tests: Removes empty/not implemented tests by @astubbs in #493
- tests: Extra test coverage for when there is less keys than threads by @astubbs in #450
- docs: Remove legacy config maxUncommittedMessagesToHandle from readme by @astubbs in #503
- build(deps): vertx, maven-dependency, testcontainers, logback, mockito by @astubbs in #504
- docs: Changelog update by @astubbs in #507
- minor: JStream review / generics fixes by @astubbs in #505
- build: Update JenkinsFile to match upstream changes by @astubbs in #521
- minor: Remove dangling API methods which were kept by accident by @astubbs in #522
- minor: Fix enum naming, tests tweak by @astubbs in #517
- build(deps): bump vertx.version, maven-install, slf4j-api, reactor-core, versions-maven-plugin by @astubbs in #523
- fixes #327: Improve error message when Kafka Streams magic number found by @nachomdo in #482
- Add missing @UtilityClass to utils class by @acktsap in #518
- docs: Yourkit Attribution by @astubbs in #529
- fix: After new performance fix PR#530 merges - corner case could cause out of order processing by @astubbs in #534
- Fixes #547 - Cleanup WorkManager#numberRecordsOutForProcessing when w… by @RaviKalasapur in #549
New Contributors
- @Ehud-Lev-Forter made their first contribution in #489
- @nachomdo made their first contribution in #482
- @acktsap made their first contribution in #518
- @RaviKalasapur made their first contribution in #549
Full Changelog: 0.5.2.4...0.5.2.5
0.5.2.4
Improvements
- feature: Simple PCRetriableException to remove error spam from logs (#444)
Fixes
- fixes #409: Adds support for compacted topics and commit offset resetting (#425)
- Truncate the offset state when bootstrap polled offset higher or lower than committed
- Prune missing records from the tracked incomplete offset state, when they're missing from polled batches
- fix: Improvements to encoding ranges (int vs long) #439
- Replace integer offset references with long - use Long everywhere we deal with offsets, and where we truncate down, do it exactly, detect and handle truncation issues.
GH Generated:
What's Changed
- build: Fix maven scm tag to HEAD by @astubbs in #434
- fixes #409: Adds support for compacted topics and commit offset resetting by @astubbs in #425
- build(deps): bump vertx.version from 4.3.3 to 4.3.4 by @dependabot in #427
- build(deps): bump maven-jar-plugin from 3.2.2 to 3.3.0 by @dependabot in #426
- build(deps): bump testcontainers.version from 1.17.3 to 1.17.5 by @dependabot in #438
- docs: Various javadoc and @StandardException by @astubbs in #343
- build:(deps) Bump JUnit, Reactor by @astubbs in #447
- build:(deps) Bump Slf4j, Logback by @astubbs in #448
- build(deps): bump jabel from 0.4.2 to 1.0.0 (Java 19 support) by @astubbs in #449
- refactor #409: Clarify truncation code by @astubbs in #445
- fixes: NoSuchElementException in LoopingResumingIterator by @HiremathPanchaxari in #435
- feature: Simple retriable exception to remove error spam from logs by @astubbs in #444
- build(deps-dev): Fix Slf4j setup caused by shaded Wiremock dep (slf4j/logback collisions) by @astubbs in #451
- Update service.yml - disable semaphore by @vinnybod in #458
- fix: Improvements to encoding ranges (int vs long) by @astubbs in #439
- refactor: SortedSet's all the way down (Set->SortedSet) by @astubbs in #463
- Fixes/test improvements by @astubbs in #465
- docs: Add Reactor example and clarify bulk transaction system by @astubbs in #452
New Contributors
- @HiremathPanchaxari made their first contribution in #435
- @vinnybod made their first contribution in #458
Full Changelog: 0.5.2.3...0.5.2.4
0.5.2.3
Improvements
-
Transactional commit mode system improvements and docs (#355)
- Clarifies transaction system with much better documentation.
- Fixes a potential race condition which could cause offset leaks between transactions boundaries.
- Introduces lock acquisition timeouts.
- Fixes a potential issue with removing records from the retry queue incorrectly, by having an inconsistency between compareTo and equals in the retry TreeMap.
-
Adds a very simple Dependency Injection system modeled on Dagger (#398)
-
Various refactorings e.g. new ProducerWrap
-
Dependencies
- build(deps): prod: zstd, reactor, dev: podam, progressbar, postgresql maven-plugins: versions, help (#420)
- build(deps-dev): bump postgresql from 42.4.1 to 42.5.0
- bump podam, progressbar, zstd, reactor
- build(deps): bump versions-maven-plugin from 2.11.0 to 2.12.0
- build(deps): bump maven-help-plugin from 3.2.0 to 3.3.0
- build(deps-dev): bump Confluent Platform Kafka Broker to 7.2.2 (#421)
- build(deps): Upgrade to AK 3.3.0 (#309)
Fixes
- fixes #419: NoSuchElementException during race condition in PartitionState (#422)
- Fixes #412: ClassCastException with retryDelayProvider (#417)
- fixes ShardManager retryQueue ordering and set issues due to poor Comparator implementation (#423)
GH generated:
What's Changed
- Improvements/DI module by @astubbs in #398
- Improvements/transaction docs large refactorings #399 - Fix for merge error by @astubbs in #406
- Transaction system improvements and docs by @astubbs in #355
- build(deps): Upgrade to AK 3.3.0 by @astubbs in #309
- build(deps-dev): bump Confluent Platform Kafka Broker to 7.2.2 by @astubbs in #421
- build(deps): prod: zstd, reactor, dev: podam, progressbar, postgresql maven-plugins: versions, help by @astubbs in #420
- Fixes #412: ClassCastException with retryDelayProvider by @astubbs in #417
- ide: Add in ignored bisect helper and run targets by @astubbs in #347
- fixes #419 NoSuchElementException during race condition in PartitionState by @astubbs in #422
- docs: Clarify offset storage behaviour by @astubbs in #424
- fixes #327: Friendlier error for encoding problems & Exception name normalisation by @astubbs in #354
- fixes ShardManager retryQueue ordering and set issues due to poor Comparator implementation by @astubbs in #423
Full Changelog: 0.5.2.2...0.5.2.3
0.5.2.2
What's Changed
Dependencies
- build(deps-dev): bump postgresql, install, info, unit, deploy, resource by @astubbs in #370
- build(deps): bump reactor, mockito, junit-platorm, site, threeten (#386)
- build(deps): bump vertx.version from 4.3.2 to 4.3.3 by @dependabot in #387
- build(deps): bump maven-javadoc-plugin from 3.4.0 to 3.4.1 by @dependabot in #388
New Contributors
Full Changelog: 0.5.2.1...0.5.2.2
0.5.2.1
0.5.2.0
Fixes and Improvements
- fixes #184: Fix multi topic subscription with KEY order by adding topic to shard key (#315)
- fixes #329: Committing around transaction markers causes encoder to crash (#328)
- build: Upgrade Truth-Generator to 0.1.1 for user Subject discovery (#332)
Build
- build: Allow snapshots locally, fail in CI (#331)
- build: OSS Index scan change to warn only and exclude Guava CVE-2020-8908 as it's WONT_FIX (#330)
Dependencies
- build(deps): bump reactor-core from 3.4.19 to 3.4.21 (#344)
- build(deps): dependabot bump Mockito, Surefire, Reactor, AssertJ, Release (#342) (#342)
- build(deps): dependabot bump TestContainers, Vert.x, Enforcer, Versions, JUnit, Postgress (#336)
Linked issues
0.5.1.0
Features
- #193: Pause / Resume PC (circuit breaker) without unsubscribing from topics
Fixes and Improvements
- #225: Build and runtime support for Java 16+ (#289)
- #306: Change Truth-Generator dependency from compile to test
- #298: Improve PollAndProduce performance by first producing all records, and then waiting for the produce results.Previously, this was done for each ProduceRecord individually.
Raw
What's Changed
- build(deps-dev): bump postgresql from 42.3.4 to 42.3.5 by @dependabot in #286
- fixes: #225 Support Java 16+ by @astubbs in #289
- docs: #292 Clarify project status by @astubbs in #294
- docs: Changelog update by @astubbs in #287
- build(deps): bump vertx.version from 4.2.7 to 4.3.0 by @dependabot in #296
- build(deps): bump reactor-core from 3.4.17 to 3.4.18 by @dependabot in #295
- build(deps): bump maven-help-plugin from 3.0.1 to 3.2.0 by @dependabot in #293
- build(deps): bump zstd-jni from 1.5.2-2 to 1.5.2-3 by @dependabot in #305
- docs: Correctly render includes when editing README template by @astubbs in #240
- feature #193: Pause/Resume ParallelConsumer by @nioertel in #198
- #301: Add .editorconfig support by @astubbs in #302
- pollAndProduce: Wait for all futures at once by @birdayz in #298
- fix #306: Change Truth-Generator dependency from compile to test by @astubbs in #307
New Contributors
Full Changelog: 0.5.0.0...0.5.1.0
