Clean up StreamingPlatform / Engine #9727
Assignees
Labels
kind/toil
Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc.
version:8.1.0-alpha5
Marks an issue as being completely or in parts released in 8.1.0-alpha5
version:8.1.0
Marks an issue as being completely or in parts released in 8.1.0
Comments
Zelldon
added
kind/toil
This was referenced Aug 4, 2022
Closed
Merged
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Aug 5, 2022
9994: Remove writers from TypedRecordProcessor r=Zelldon a=Zelldon ## Description * Replaces the usage of the StreamWriter and ResponseWriter from the parameters with the Writers and their "sub-writers" * Discussable whether we always want to have Writers in the processor classes or just the specific writers, but I guess we can here improve also later * Adjust several test cases that they also are filled with the Writers object * Finally, remove the writers from the processRecord interface <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #9727 closes #8002 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Aug 5, 2022
10000: Remove proxy writers r=Zelldon a=Zelldon ## Description After #9967 is merged we can look at this PR. <!-- Please explain the changes you made here. --> Removes the proxy writers, which have been introduced for not writing on reprocessing, which we no longer do. ## Related issues <!-- Which issues are closed by this PR or are related --> related to #9727 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Aug 8, 2022
10000: Remove proxy writers r=Zelldon a=Zelldon ## Description After #9967 is merged we can look at this PR. <!-- Please explain the changes you made here. --> Removes the proxy writers, which have been introduced for not writing on reprocessing, which we no longer do. ## Related issues <!-- Which issues are closed by this PR or are related --> related to #9727 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Aug 8, 2022
10000: Remove proxy writers r=Zelldon a=Zelldon ## Description After #9967 is merged we can look at this PR. <!-- Please explain the changes you made here. --> Removes the proxy writers, which have been introduced for not writing on reprocessing, which we no longer do. ## Related issues <!-- Which issues are closed by this PR or are related --> related to #9727 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Aug 8, 2022
10000: Remove proxy writers r=Zelldon a=Zelldon ## Description After #9967 is merged we can look at this PR. <!-- Please explain the changes you made here. --> Removes the proxy writers, which have been introduced for not writing on reprocessing, which we no longer do. ## Related issues <!-- Which issues are closed by this PR or are related --> related to #9727 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Aug 8, 2022
10021: Engine abstraction code cleanup r=pihme a=pihme ## Description * Removes unnecessary references * Moves `LegacyTypedResponseWriter` to stream processor, as it is only used in this package * Renames `LegacyTypedResponseWriter` ## Related issues related to #9727 Co-authored-by: pihme <pihme@users.noreply.github.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Aug 8, 2022
9753: Document events and event scopes r=korthout a=korthout ## Description <!-- Please explain the changes you made here. --> This provides an initial description of events and event scopes. It uses both the BPMN specification as well as the terminology used in Zeebe to describe events, triggers, and event scopes. ## Related issues <!-- Which issues are closed by this PR or are related --> NA 10021: Engine abstraction code cleanup r=pihme a=pihme ## Description * Removes unnecessary references * Moves `LegacyTypedResponseWriter` to stream processor, as it is only used in this package * Renames `LegacyTypedResponseWriter` ## Related issues related to #9727 10028: Revert "Merge pull request #9985 from camunda/zell-clean-up-abstracti… r=Zelldon a=Zelldon ## Description This reverts commit 155de10, reversing changes made to 47c3d3a. <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10027 Co-authored-by: Nico Korthout <nico.korthout@camunda.com> Co-authored-by: pihme <pihme@users.noreply.github.com> Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
10 tasks
This was referenced Aug 17, 2022
deepthidevaki
added
release/8.1.0-alpha5
version:8.1.0-alpha5
10 tasks
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 20, 2022
10398: Remove multi-partition support from StreamPlatform test extension r=Zelldon a=Zelldon ## Description Remove the multi partition support in the StreamPlatform, and related extension. This is not necessary for testing the StreamPlatform code <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #9727 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
10 tasks
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10412: Add a standalone app that can be used to restore a broker from a backup r=deepthidevaki a=deepthidevaki ## Description * Added a new module for restore. * Adds a RestoreManager that coordinates restoring of all partitions. It first determines which partitions to restore from based on the given `BrokerCfg`. This configuration must be the same one used by the Broker when it is started after the restore. Because of this, restore module depens on broker module. But since it needs access to the configuration and `RaftPartitionGroupFactory`, there is no other way. There is scope of splitting broker to take these shared configs out of it. * Added a `RestoreApp` in dist. My plan was to add it in restore module. But it needs access to BrokerConfig, which is built in dist. So the app is added to dist module. Alternative is to copy components `BrokerConfiguration` and `WorkerDirectory` to restore module. * The backup id to restore from is passed as a command line argument `--backupId=x`. This is parsed by spring boot. I decided to keep it simple for now, since we are planning to release it as experimental feature anyway. If we decide to support this app long term, then I suggest to switch to something like picocli that we use in zdb. A basic test to verify a broker can be restored is added. More scenarios should be added later. PS:- App is not added to the distribution yet. ## Related issues related #10263 10440: Extract KeyGenerator from ZeebeState r=oleschoenburg a=Zelldon ## Description As preparation for the StreamProcessor - Engine module split we have to move out the Keygenerator out from the zeebeState, since this part of the stream processing (key is part of RecordMetadata). We have to reset the keys during replay and for that we need access to the key generator. The Key generator is now created in the StreamProcessor and passed into the ZeebeState (as dependency), this allowed to make minimal changes. At the start I removed completely the KEygenerator from the ZeebeState, but then we would need to adjust a LOT of setup code, which doesn't felt necessary tbh since it just about where this generator is created. The change also allowed us to remove the ZeebeState from the StreamProcessorContext 💪 `@npepinpe` `@saig0` who ever has time to review this. I think `@npepinpe` is currently a bit busy with other stuff related to backup and reviews, maybe you have time to review it `@saig0` <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10130 related to #9727 10483: deps(maven): bump checkstyle from 10.3.3 to 10.3.4 r=github-actions[bot] a=dependabot[bot] Bumps [checkstyle](https://github.com/checkstyle/checkstyle) from 10.3.3 to 10.3.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/checkstyle/checkstyle/releases">checkstyle's releases</a>.</em></p> <blockquote> <h2>checkstyle-10.3.4</h2> <p><a href="https://checkstyle.org/releasenotes.html#Release_10.3.4">https://checkstyle.org/releasenotes.html#Release_10.3.4</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/checkstyle/checkstyle/commit/6de3b9f1feb5c80450af2ad646f82ba34dc250bb"><code>6de3b9f</code></a> [maven-release-plugin] prepare release checkstyle-10.3.4</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/a4497de578d5a72b386e7ca88676da190584eece"><code>a4497de</code></a> doc: release notes 10.3.4</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/32e8d374e67e5ef5c19a688064bb2a77b5947651"><code>32e8d37</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/12145">#12145</a>: corrected tokens so all are required</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/96e3e05a526bca3f7a157ca929666530eb0b089f"><code>96e3e05</code></a> dependency: bump pitest-accelerator-junit5 from 1.0.1 to 1.0.2</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/37842d2f291b138b2e9d44b33350add3f9562f34"><code>37842d2</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/3955">#3955</a>: corrected tokens so all are required</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/38ee347a49376e80858c969f742e6461dfe7d731"><code>38ee347</code></a> minor: remove unnecessary checkstyle versions to diff.groovy</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/318770dc1343e9be90b8304c275d9adaf4ad8d45"><code>318770d</code></a> dependency: bump slf4j-simple from 2.0.1 to 2.0.2</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/119453600ff9f6accc247f748df8c0dd6c65a2af"><code>1194536</code></a> dependency: bump junit.version from 5.9.0 to 5.9.1</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/5a56c16882b68e0aaedf3a2a29737f412bbd66be"><code>5a56c16</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/12132">#12132</a>: Fix ArrayIndexOutOfBoundsException in pitest-survival-check-xml...</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/701bd65dec286ab75fed35b99a580fc77c8ea03f"><code>701bd65</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/12210">#12210</a>: Add method to ignore unstable checker framework violations</li> <li>Additional commits viewable in <a href="https://github.com/checkstyle/checkstyle/compare/checkstyle-10.3.3...checkstyle-10.3.4">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: Deepthi Devaki Akkoorath <deepthidevaki@gmail.com> Co-authored-by: Christopher Zell <zelldon91@googlemail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10412: Add a standalone app that can be used to restore a broker from a backup r=deepthidevaki a=deepthidevaki ## Description * Added a new module for restore. * Adds a RestoreManager that coordinates restoring of all partitions. It first determines which partitions to restore from based on the given `BrokerCfg`. This configuration must be the same one used by the Broker when it is started after the restore. Because of this, restore module depens on broker module. But since it needs access to the configuration and `RaftPartitionGroupFactory`, there is no other way. There is scope of splitting broker to take these shared configs out of it. * Added a `RestoreApp` in dist. My plan was to add it in restore module. But it needs access to BrokerConfig, which is built in dist. So the app is added to dist module. Alternative is to copy components `BrokerConfiguration` and `WorkerDirectory` to restore module. * The backup id to restore from is passed as a command line argument `--backupId=x`. This is parsed by spring boot. I decided to keep it simple for now, since we are planning to release it as experimental feature anyway. If we decide to support this app long term, then I suggest to switch to something like picocli that we use in zdb. A basic test to verify a broker can be restored is added. More scenarios should be added later. PS:- App is not added to the distribution yet. ## Related issues related #10263 10440: Extract KeyGenerator from ZeebeState r=oleschoenburg a=Zelldon ## Description As preparation for the StreamProcessor - Engine module split we have to move out the Keygenerator out from the zeebeState, since this part of the stream processing (key is part of RecordMetadata). We have to reset the keys during replay and for that we need access to the key generator. The Key generator is now created in the StreamProcessor and passed into the ZeebeState (as dependency), this allowed to make minimal changes. At the start I removed completely the KEygenerator from the ZeebeState, but then we would need to adjust a LOT of setup code, which doesn't felt necessary tbh since it just about where this generator is created. The change also allowed us to remove the ZeebeState from the StreamProcessorContext 💪 `@npepinpe` `@saig0` who ever has time to review this. I think `@npepinpe` is currently a bit busy with other stuff related to backup and reviews, maybe you have time to review it `@saig0` <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10130 related to #9727 10483: deps(maven): bump checkstyle from 10.3.3 to 10.3.4 r=github-actions[bot] a=dependabot[bot] Bumps [checkstyle](https://github.com/checkstyle/checkstyle) from 10.3.3 to 10.3.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/checkstyle/checkstyle/releases">checkstyle's releases</a>.</em></p> <blockquote> <h2>checkstyle-10.3.4</h2> <p><a href="https://checkstyle.org/releasenotes.html#Release_10.3.4">https://checkstyle.org/releasenotes.html#Release_10.3.4</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/checkstyle/checkstyle/commit/6de3b9f1feb5c80450af2ad646f82ba34dc250bb"><code>6de3b9f</code></a> [maven-release-plugin] prepare release checkstyle-10.3.4</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/a4497de578d5a72b386e7ca88676da190584eece"><code>a4497de</code></a> doc: release notes 10.3.4</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/32e8d374e67e5ef5c19a688064bb2a77b5947651"><code>32e8d37</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/12145">#12145</a>: corrected tokens so all are required</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/96e3e05a526bca3f7a157ca929666530eb0b089f"><code>96e3e05</code></a> dependency: bump pitest-accelerator-junit5 from 1.0.1 to 1.0.2</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/37842d2f291b138b2e9d44b33350add3f9562f34"><code>37842d2</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/3955">#3955</a>: corrected tokens so all are required</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/38ee347a49376e80858c969f742e6461dfe7d731"><code>38ee347</code></a> minor: remove unnecessary checkstyle versions to diff.groovy</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/318770dc1343e9be90b8304c275d9adaf4ad8d45"><code>318770d</code></a> dependency: bump slf4j-simple from 2.0.1 to 2.0.2</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/119453600ff9f6accc247f748df8c0dd6c65a2af"><code>1194536</code></a> dependency: bump junit.version from 5.9.0 to 5.9.1</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/5a56c16882b68e0aaedf3a2a29737f412bbd66be"><code>5a56c16</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/12132">#12132</a>: Fix ArrayIndexOutOfBoundsException in pitest-survival-check-xml...</li> <li><a href="https://github.com/checkstyle/checkstyle/commit/701bd65dec286ab75fed35b99a580fc77c8ea03f"><code>701bd65</code></a> Issue <a href="https://github-redirect.dependabot.com/checkstyle/checkstyle/issues/12210">#12210</a>: Add method to ignore unstable checker framework violations</li> <li>Additional commits viewable in <a href="https://github.com/checkstyle/checkstyle/compare/checkstyle-10.3.3...checkstyle-10.3.4">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> 10485: ci(go): use self hosted runner r=megglos a=megglos ## Description <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> relates #10370 Co-authored-by: Deepthi Devaki Akkoorath <deepthidevaki@gmail.com> Co-authored-by: Christopher Zell <zelldon91@googlemail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Meggle (Sebastian Bathke) <sebastian.bathke@camunda.com>
Zelldon
added
the
version:8.1.0
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Part of #9600
There a multiple things we can remove or clean up during #9600, some of these might be only possible at the end, but some are already possible at the beginning.
See
Findingin #9602 (comment)Todo:
io.camunda.zeebe.engine.apito another package so that it's clearer it is a shared API and not only used by the engineThe text was updated successfully, but these errors were encountered: