java.lang.ClassCastException: class ModelElementInstanceImpl cannot be cast to class BpmnModelElementInstance

[npepinpe]

Describe the bug

The error occurred in Camunda Cloud for Zeebe 0.23.3: https://console.cloud.google.com/errors/CM_ZxM226tXn3wE?service=zeebe&version=0.23.3&time=P1D&project=camunda-cloud-240911

From what I can tell the severity is quite low - it occurred only 2 in the last week (so low likelihood it seems, though unsure) and no user reported any issues.

To Reproduce

As it is an incident, I'm unsure how to reproduce.

Expected behavior

As BpmnModelElementInstanceImpl inherits from ModelElementInstanceImpl, it seems possible that some ModelElementInstanceImpl could not be cast to BpmnModelElementInstanceImpl, so either we have to prevent those cases, or we should expect and handle them.

Log/Stacktrace

Full Stacktrace

ava.lang.ClassCastException: class org.camunda.bpm.model.xml.impl.instance.ModelElementInstanceImpl cannot be cast to class io.zeebe.model.bpmn.instance.BpmnModelElementInstance (org.camunda.bpm.model.xml.impl.instance.ModelElementInstanceImpl and io.zeebe.model.bpmn.instance.BpmnModelElementInstance are in unnamed module of loader 'app')
	at java.util.ArrayList.forEach(Unknown Source) ~[?:?]
	at io.zeebe.model.bpmn.traversal.ModelWalker.walk(ModelWalker.java:62) ~[zeebe-bpmn-model-0.23.3.jar:0.23.3]
	at io.zeebe.engine.processor.workflow.deployment.transform.BpmnValidator.validate(BpmnValidator.java:40) ~[zeebe-workflow-engine-0.23.3.jar:0.23.3]
	at io.zeebe.engine.processor.workflow.deployment.transform.DeploymentTransformer.transformResource(DeploymentTransformer.java:104) ~[zeebe-workflow-engine-0.23.3.jar:0.23.3]
	at io.zeebe.engine.processor.workflow.deployment.transform.DeploymentTransformer.transform(DeploymentTransformer.java:81) ~[zeebe-workflow-engine-0.23.3.jar:0.23.3]
	at io.zeebe.engine.processor.workflow.deployment.TransformingDeploymentCreateProcessor.processRecord(TransformingDeploymentCreateProcessor.java:68) ~[zeebe-workflow-engine-0.23.3.jar:0.23.3]
	at io.zeebe.engine.processor.TypedRecordProcessor.processRecord(TypedRecordProcessor.java:48) ~[zeebe-workflow-engine-0.23.3.jar:0.23.3]
	at io.zeebe.engine.processor.ReProcessingStateMachine.lambda$chooseOperationForEvent$5(ReProcessingStateMachine.java:287) ~[zeebe-workflow-engine-0.23.3.jar:0.23.3]
	at io.zeebe.db.impl.rocksdb.transaction.ZeebeTransaction.run(ZeebeTransaction.java:79) ~[zeebe-db-0.23.3.jar:0.23.3]
	at io.zeebe.engine.processor.ReProcessingStateMachine.lambda$processUntilDone$2(ReProcessingStateMachine.java:262) ~[zeebe-workflow-engine-0.23.3.jar:0.23.3]
	at io.zeebe.util.retry.ActorRetryMechanism.run(ActorRetryMechanism.java:36) ~[zeebe-util-0.23.3.jar:0.23.3]
	at io.zeebe.util.retry.EndlessRetryStrategy.run(EndlessRetryStrategy.java:50) ~[zeebe-util-0.23.3.jar:0.23.3]
	at io.zeebe.util.sched.ActorJob.invoke(ActorJob.java:73) [zeebe-util-0.23.3.jar:0.23.3]
	at io.zeebe.util.sched.ActorJob.execute(ActorJob.java:39) [zeebe-util-0.23.3.jar:0.23.3]
	at io.zeebe.util.sched.ActorTask.execute(ActorTask.java:115) [zeebe-util-0.23.3.jar:0.23.3]
	at io.zeebe.util.sched.ActorThread.executeCurrentTask(ActorThread.java:107) [zeebe-util-0.23.3.jar:0.23.3]
	at io.zeebe.util.sched.ActorThread.doWork(ActorThread.java:91) [zeebe-util-0.23.3.jar:0.23.3]
	at io.zeebe.util.sched.ActorThread.run(ActorThread.java:195) [zeebe-util-0.23.3.jar:0.23.3]

Environment:

• OS: Docker distribution
• Zeebe Version: 0.23.3
• Configuration: cloud configuration

Related Support Case: SUPPORT-13739

[saig0]

This sounds familiar. It might be the same issue as #3278.

We see this error if we deploy a workflow that contains an unexpected (XML) element (i.e. unknown by the Zeebe model API).

[Zelldon]

Describe the bug
Found that in a production cluster log. Unfortunately the cluster was already delete and I cant say what kind of model caused that. I'm not sure whether we really need to log that as error or not. Ideally I would say we return a good error message to the client, but Idk whether this happened in this case.

Error log

Error group https://console.cloud.google.com/errors/CPLX0sCVkLWzugE?service=zeebe&time=P7D&project=camunda-cloud-240911&authuser=1

To Reproduce

I assume something like having multiple models in one? It seems model element registers for "" https://github.com/camunda/camunda-bpm-platform/blob/master/model-api/xml-model/src/main/java/org/camunda/bpm/model/xml/impl/instance/ModelElementInstanceImpl.java#L58 not 100% sure what this means, see https://github.com/zeebe-io/zeebe/blob/develop/bpmn-model/src/main/java/io/zeebe/model/bpmn/Bpmn.java#L536.

Expected behavior

Errors like this can be detected and catched. A good error message is returned to the client, which shows what the problem is.

Log/Stacktrace
If possible add the full stacktrace or Zeebe log which contains the issue.

Full Stacktrace

java.lang.ClassCastException: class org.camunda.bpm.model.xml.impl.instance.ModelElementInstanceImpl cannot be cast to class io.zeebe.model.bpmn.instance.BpmnModelElementInstance (org.camunda.bpm.model.xml.impl.instance.ModelElementInstanceImpl and io.zeebe.model.bpmn.instance.BpmnModelElementInstance are in unnamed module of loader 'app')
at java.util.ArrayList.forEach (Unknown Source)
at io.zeebe.model.bpmn.traversal.ModelWalker.walk (ModelWalker.java:62)
at io.zeebe.engine.processing.deployment.transform.BpmnValidator.validate (BpmnValidator.java:40)
at io.zeebe.engine.processing.deployment.transform.DeploymentTransformer.transformResource (DeploymentTransformer.java:104)
at io.zeebe.engine.processing.deployment.transform.DeploymentTransformer.transform (DeploymentTransformer.java:81)
at io.zeebe.engine.processing.deployment.TransformingDeploymentCreateProcessor.processRecord (TransformingDeploymentCreateProcessor.java:71)
at io.zeebe.engine.processing.streamprocessor.TypedRecordProcessor.processRecord (TypedRecordProcessor.java:51)
at io.zeebe.engine.processing.streamprocessor.ProcessingStateMachine.lambda$processInTransaction$2 (ProcessingStateMachine.java:267)
at io.zeebe.db.impl.rocksdb.transaction.ZeebeTransaction.run (ZeebeTransaction.java:79)
at io.zeebe.engine.processing.streamprocessor.ProcessingStateMachine.processInTransaction (ProcessingStateMachine.java:258)
at io.zeebe.engine.processing.streamprocessor.ProcessingStateMachine.processEvent (ProcessingStateMachine.java:227)
at io.zeebe.engine.processing.streamprocessor.ProcessingStateMachine.tryToReadNextEvent (ProcessingStateMachine.java:203)
at io.zeebe.engine.processing.streamprocessor.ProcessingStateMachine.readNextEvent (ProcessingStateMachine.java:194)
at io.zeebe.util.sched.ActorJob.invoke (ActorJob.java:76)
at io.zeebe.util.sched.ActorJob.execute (ActorJob.java:39)
at io.zeebe.util.sched.ActorTask.execute (ActorTask.java:122)
at io.zeebe.util.sched.ActorThread.executeCurrentTask (ActorThread.java:107)
at io.zeebe.util.sched.ActorThread.doWork (ActorThread.java:91)
at io.zeebe.util.sched.ActorThread.run (ActorThread.java:204)

Environment:

• OS: k8s production
• Zeebe Version: 0.25.1
• Configuration: default

Migrated from #5896

[Zelldon]

I was able to reproduce this with message throw events. The error we get on the client side is quite similar we see in the log.

Error: Command 'CREATE' rejected with code 'INVALID_ARGUMENT': Expected to deploy new resources, but encountered the following errors: 'yolo.bpmn': class org.camunda.bpm.model.xml.impl.instance.ModelElementInstanceImpl cannot be cast to class io.zeebe.model.bpmn.instance.BpmnModelElementInstance (org.camunda.bpm.model.xml.impl.instance.ModelElementInstanceImpl and io.zeebe.model.bpmn.instance.BpmnModelElementInstance are in unnamed module of loader 'app')

This is not really helpful from the user point of view. Even for us in the logs it brings us no value.

[npepinpe]

As this occurred yet again, and users (also internal Camundos) run into this, I would propose we fix this for the next release (8.0.1).

[saig0]

Related support case: https://jira.camunda.com/browse/SUPPORT-13739