Support deploying multiple DMN files at once

[remcowesterhoud]

Description

When deploying multiple DMN resources in a single command users would receive the error

io.camunda.zeebe.client.api.command.ClientStatusException: Command 'CREATE' rejected with code 'INVALID_ARGUMENT': Expected to deploy new resources, but encountered the following errors:
'dummy2.dmn': Key DbLong{2251799813685250} in ColumnFamily DMN_DECISION_REQUIREMENTS already exists

This error is caused by a bug in Zeebe where we are sending multiple events for the same DRG and decisions. This only occurs when the command contains multiple DMN resources.

Root cause
In the DmnResourceTransformer we will transform each resource, so each dmn file. In here we have the method appendMetadataToDeploymentEvent . This method will add a DecisionRequirementsMetadataRecord to the DeploymentRecord. Once the metadata has been added we write the records:

private void writeRecords(final DeploymentRecord deployment, final DeploymentResource resource) {

    for (final DecisionRequirementsMetadataRecord drg : deployment.decisionRequirementsMetadata()) {
      if (!drg.isDuplicate()) {
        stateWriter.appendFollowUpEvent(
            drg.getDecisionRequirementsKey(),
            DecisionRequirementsIntent.CREATED,
            new DecisionRequirementsRecord()
                .setDecisionRequirementsKey(drg.getDecisionRequirementsKey())
                .setDecisionRequirementsId(drg.getDecisionRequirementsId())
                .setDecisionRequirementsName(drg.getDecisionRequirementsName())
                .setDecisionRequirementsVersion(drg.getDecisionRequirementsVersion())
                .setNamespace(drg.getNamespace())
                .setResourceName(drg.getResourceName())
                .setChecksum(drg.getChecksumBuffer())
                .setResource(resource.getResourceBuffer()));
      }
    }

As you can see this method takes the DeploymentRecord as a parameter and iterates over all the DecisionRequirementsMetadataRecords that have been added. So what happens when we deploy multiple dmns:

1. Send the deploy resource command. We create a new DeploymentRecord with an empty list of DecisionRequirementsMetadataRecords.
2. We start transforming dmn 1. A new DecisionRequirementsMetadataRecord gets appended to the DeploymentRecord
3. We write a record for all metadatas in the DeploymentRecord . At this point this is only the metadata of dmn 1
4. We start transforming dmn 2. A new DecisionRequirementsMetadataRecord gets appended to the DeploymentRecord
5. We write a record for all metadatas in the DeploymentRecord. At this point this record contains the metadata of both dmn 1 and dmn 2, resulting in a CREATED event for dmn 2 containing the metadata keys of dmn 1

Related issues

closes #9337
closes #9115
Related to camunda/zeebe-process-test#357

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

• The changes are backwards compatibility with previous versions
• If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/1.3) to the PR, in case that fails you need to create backports manually.

Testing:

• There are unit/integration tests that verify all acceptance criterias of the issue
• New tests are written to ensure backwards compatibility with further versions
• The behavior is tested manually
• The change has been verified by a QA run
• The impact of the changes is verified by a benchmark

Documentation:

• The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
• New content is added to the release announcement
• If the PR changes how BPMN processes are validated (e.g. support new BPMN element) then the Camunda modeling team should be informed to adjust the BPMN linting.

Please refer to our review guidelines.

[saig0]

@remcowesterhoud thanks for fixing 🍪 LGTM 👍

[saig0]

bors merge

[zeebe-bors-camunda]

Build succeeded:

• continuous-integration/jenkins/branch
• Smoke tests on macos-latest
• Smoke tests on windows-latest
• Smoke tests on ubuntu-latest

[backport-action]

Successfully created backport PR #9458 for stable/8.0.