Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
10418: Add monitor backup API endpoint r=deepthidevaki a=npepinpe

## Description

This PR extends the backup management endpoint to add a monitoring API under `GET /{id}` (e.g. if the actuator is at `http://localhost:9600/actuator/backups`, then `GET http://localhost:9600/actuator/backups/{id}`).

At the moment we use the internal types provided by the broker client as the response types, but in the future this will be replaced by the OpenAPI generated types. As such, I didn't think it was necessary to create user facing types and write some mapping code, since we will scrape it and replace it. However, I understand that we don't know when that will done, so I'm open to doing it anyway for safety.

The acceptance tests were updated (and renamed to better reflect what they do), and as they overlap with the old `BackupIT` tests, those were deleted.

## Related issues

closes #9902 



10442: feat: support terminate end events r=saig0 a=saig0

## Description

Add support for BPMN terminate end events. See #8789 (comment) on how the BPMN element should work.

The implementation doesn't follow the BPMN spec in one point: the flow scope that contains the terminate end event is not terminated but completed. Reasoning:
- The state of the flow scope is a detail that doesn't influence the core behavior. In both cases, the process instance should continue, for example, by taking the outgoing sequence flow. The difference is not visible to process participants but only when monitoring the process instance, for example, in Operate.
- It fits better with the existing implementation. It would be a bigger effort to continue the process instance (e.g. taking the outgoing sequence flow) when the flow scope is terminated. As a result, we would end up in more complex code.
- It aligns with the behavior of Camunda Platform 7. 

Side note: I implemented the major parts during a [Live Hacking session](https://www.twitch.tv/videos/1584245006). 🎥 

## Related issues

closes #8789



10464: Add restore app to the distribution r=deepthidevaki a=deepthidevaki

## Description

* Create a binary `bin/restore` when building the project. 
* Configure docker image to run restore, when a specific env variable is set. This is done similarly to chose between broker and gateway.
* In addition, extended the restore app to delete data directory if restore failed.

Depends on #10412 

## Related issues

closes #10263 



Co-authored-by: Nicolas Pepin-Perreault <nicolas.pepin-perreault@camunda.com>
Co-authored-by: Philipp Ossler <philipp.ossler@gmail.com>
Co-authored-by: Deepthi Devaki Akkoorath <deepthidevaki@gmail.com>
  • Loading branch information
4 people committed Sep 26, 2022
4 parents db48127 + 2cf33e7 + 1668ad1 + cfeb276 commit 721b6b4
Show file tree
Hide file tree
Showing 26 changed files with 1,085 additions and 260 deletions.
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,8 @@ LABEL io.openshift.min-cpu="1"

ENV ZB_HOME=/usr/local/zeebe \
ZEEBE_BROKER_GATEWAY_NETWORK_HOST=0.0.0.0 \
ZEEBE_STANDALONE_GATEWAY=false
ZEEBE_STANDALONE_GATEWAY=false \
ZEEBE_RESTORE=false
ENV PATH "${ZB_HOME}/bin:${PATH}"

WORKDIR ${ZB_HOME}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import io.camunda.zeebe.model.bpmn.BpmnModelInstance;
import io.camunda.zeebe.model.bpmn.instance.EndEvent;
import io.camunda.zeebe.model.bpmn.instance.ErrorEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.TerminateEventDefinition;

/**
* @author Sebastian Menski
Expand Down Expand Up @@ -71,4 +72,17 @@ public ErrorEventDefinitionBuilder errorEventDefinition() {
element.getEventDefinitions().add(errorEventDefinition);
return new ErrorEventDefinitionBuilder(modelInstance, errorEventDefinition);
}

/**
* Creates a terminate event definition and add it to the end event. It morphs the end event into
* a terminate end event.
*
* @return the builder object
*/
public B terminate() {
final TerminateEventDefinition terminateEventDefinition =
createInstance(TerminateEventDefinition.class);
element.getEventDefinitions().add(terminateEventDefinition);
return myself;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import io.camunda.zeebe.model.bpmn.instance.ErrorEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.EventDefinition;
import io.camunda.zeebe.model.bpmn.instance.MessageEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.TerminateEventDefinition;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
Expand All @@ -28,7 +29,8 @@
public class EndEventValidator implements ModelElementValidator<EndEvent> {

private static final List<Class<? extends EventDefinition>> SUPPORTED_EVENT_DEFINITIONS =
Arrays.asList(ErrorEventDefinition.class, MessageEventDefinition.class);
Arrays.asList(
ErrorEventDefinition.class, MessageEventDefinition.class, TerminateEventDefinition.class);

@Override
public Class<EndEvent> getElementType() {
Expand Down Expand Up @@ -59,7 +61,7 @@ private void validateEventDefinition(
def -> {
if (SUPPORTED_EVENT_DEFINITIONS.stream().noneMatch(type -> type.isInstance(def))) {
validationResultCollector.addError(
0, "End events must be one of: none, error or message");
0, "End events must be one of: none, error, message, or terminate");
}
});
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import io.camunda.zeebe.model.bpmn.instance.ErrorEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.EventDefinition;
import io.camunda.zeebe.model.bpmn.instance.MessageEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.TerminateEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.TimerEventDefinition;
import java.util.Arrays;
import java.util.List;
Expand All @@ -28,7 +29,10 @@ public class EventDefinitionValidator implements ModelElementValidator<EventDefi

private static final List<Class<? extends EventDefinition>> SUPPORTED_EVENT_DEFINITIONS =
Arrays.asList(
MessageEventDefinition.class, TimerEventDefinition.class, ErrorEventDefinition.class);
MessageEventDefinition.class,
TimerEventDefinition.class,
ErrorEventDefinition.class,
TerminateEventDefinition.class);

@Override
public Class<EventDefinition> getElementType() {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
/*
* Copyright © 2017 camunda services GmbH (info@camunda.com)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.camunda.zeebe.model.bpmn.validation;

import static io.camunda.zeebe.model.bpmn.validation.ExpectedValidationResult.expect;

import io.camunda.zeebe.model.bpmn.Bpmn;
import io.camunda.zeebe.model.bpmn.BpmnModelInstance;
import io.camunda.zeebe.model.bpmn.builder.EndEventBuilder;
import io.camunda.zeebe.model.bpmn.builder.StartEventBuilder;
import io.camunda.zeebe.model.bpmn.instance.CancelEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.CompensateEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.EndEvent;
import io.camunda.zeebe.model.bpmn.instance.ErrorEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.EscalationEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.EventDefinition;
import io.camunda.zeebe.model.bpmn.instance.MessageEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.SignalEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.TerminateEventDefinition;
import java.util.function.UnaryOperator;
import java.util.stream.Stream;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;

class ZeebeEndEventValidationTest {

private static final String END_EVENT_ID = "end";

@ParameterizedTest(name = "[{index}] event type = {0}")
@MethodSource("supportedEndEventTypes")
@DisplayName("Should support end event of the given type")
void supportedEndEventTypes(final EndEventTypeBuilder endEventTypeBuilder) {
// given
final BpmnModelInstance process = createProcessWithEndEvent(endEventTypeBuilder);

// when/then
ProcessValidationUtil.assertThatProcessIsValid(process);
}

@ParameterizedTest(name = "[{index}] event type = {0}")
@MethodSource("unsupportedEndEventTypes")
@DisplayName("Should not support end event of the given type")
void unsupportedEndEventTypes(final EndEventTypeBuilder endEventTypeBuilder) {
// given
final BpmnModelInstance process = createProcessWithEndEvent(endEventTypeBuilder);

// when/then
ProcessValidationUtil.assertThatProcessHasViolations(
process,
expect(END_EVENT_ID, "End events must be one of: none, error, message, or terminate"),
expect(endEventTypeBuilder.eventType, "Event definition of this type is not supported"));
}

@Test
@DisplayName("An end event should not have an outgoing sequence flow")
void outgoingSequenceFlow() {
// given
final BpmnModelInstance process =
Bpmn.createExecutableProcess("process")
.startEvent()
.endEvent(END_EVENT_ID)
// an activity after the end event
.manualTask()
.done();

// when/then
ProcessValidationUtil.assertThatProcessHasViolations(
process,
expect(
EndEvent.class, "End events must not have outgoing sequence flows to other elements."));
}

private static BpmnModelInstance createProcessWithEndEvent(
final EndEventTypeBuilder endEventTypeBuilder) {
final StartEventBuilder processBuilder = Bpmn.createExecutableProcess("process").startEvent();
endEventTypeBuilder.build(processBuilder.endEvent(END_EVENT_ID));
return processBuilder.done();
}

private static Stream<EndEventTypeBuilder> supportedEndEventTypes() {
return Stream.of(
new EndEventTypeBuilder(null, endEvent -> endEvent),
new EndEventTypeBuilder(
ErrorEventDefinition.class, endEvent -> endEvent.error("error-code")),
new EndEventTypeBuilder(
MessageEventDefinition.class,
endEvent -> endEvent.message("message-name").zeebeJobType("job-type")),
new EndEventTypeBuilder(TerminateEventDefinition.class, EndEventBuilder::terminate));
}

private static Stream<EndEventTypeBuilder> unsupportedEndEventTypes() {
return Stream.of(
new EndEventTypeBuilder(
SignalEventDefinition.class, endEvent -> endEvent.signal("signal-name")),
new EndEventTypeBuilder(
EscalationEventDefinition.class, endEvent -> endEvent.escalation("escalation-code")),
new EndEventTypeBuilder(
CompensateEventDefinition.class,
endEvent -> endEvent.compensateEventDefinition().compensateEventDefinitionDone()),
new EndEventTypeBuilder(
CancelEventDefinition.class,
endEvent -> {
// currently, we don't have a builder for cancel events
final CancelEventDefinition cancelEventDefinition =
endEvent.getElement().getModelInstance().newInstance(CancelEventDefinition.class);
endEvent.getElement().getEventDefinitions().add(cancelEventDefinition);
return endEvent;
}));
}

private static final class EndEventTypeBuilder {
private final String eventTypeName;
private final Class<? extends EventDefinition> eventType;
private final UnaryOperator<EndEventBuilder> elementModifier;

private EndEventTypeBuilder(
final Class<? extends EventDefinition> eventType,
final UnaryOperator<EndEventBuilder> elementModifier) {
this.eventType = eventType;
eventTypeName = eventType == null ? "none" : eventType.getSimpleName();
this.elementModifier = elementModifier;
}

public EndEventBuilder build(final EndEventBuilder endEventBuilder) {
return elementModifier.apply(endEventBuilder);
}

@Override
public String toString() {
return eventTypeName;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
import io.camunda.zeebe.model.bpmn.builder.AbstractCatchEventBuilder;
import io.camunda.zeebe.model.bpmn.builder.ProcessBuilder;
import io.camunda.zeebe.model.bpmn.instance.CompensateEventDefinition;
import io.camunda.zeebe.model.bpmn.instance.EndEvent;
import io.camunda.zeebe.model.bpmn.instance.IntermediateCatchEvent;
import io.camunda.zeebe.model.bpmn.instance.SignalEventDefinition;
import java.util.Arrays;
Expand All @@ -38,28 +37,6 @@ public static Object[][] parameters() {
Bpmn.createExecutableProcess("process").done(),
singletonList(expect("process", "Must have at least one start event"))
},
{
Bpmn.createExecutableProcess("process")
.startEvent()
.endEvent("end")
.signalEventDefinition("foo")
.id("eventDefinition")
.done(),
Arrays.asList(
expect("end", "End events must be one of: none, error or message"),
expect("eventDefinition", "Event definition of this type is not supported"))
},
{
Bpmn.createExecutableProcess("process")
.startEvent()
.endEvent()
.serviceTask("task", tb -> tb.zeebeJobType("task"))
.done(),
singletonList(
expect(
EndEvent.class,
"End events must not have outgoing sequence flows to other elements."))
},
{
Bpmn.createExecutableProcess("process")
.startEvent()
Expand Down
10 changes: 10 additions & 0 deletions dist/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,11 @@
<artifactId>agrona</artifactId>
</dependency>

<dependency>
<groupId>io.camunda</groupId>
<artifactId>zeebe-protocol</artifactId>
</dependency>

<!-- /end -->

<dependency>
Expand Down Expand Up @@ -248,6 +253,10 @@
<id>gateway</id>
<mainClass>io.camunda.zeebe.gateway.StandaloneGateway</mainClass>
</program>
<program>
<id>restore</id>
<mainClass>io.camunda.zeebe.restore.RestoreApp</mainClass>
</program>
</programs>
<repositoryLayout>flat</repositoryLayout>
<repositoryName>lib</repositoryName>
Expand Down Expand Up @@ -313,6 +322,7 @@
<ignoredNonTestScopedDependencies>
<ignoredNonTestScopedDependency>org.agrona:agrona</ignoredNonTestScopedDependency>
<ignoredNonTestScopedDependency>io.netty:netty-handler</ignoredNonTestScopedDependency>
<ignoredNonTestScopedDependency>io.camunda:zeebe-protocol</ignoredNonTestScopedDependency>
</ignoredNonTestScopedDependencies>
<!-- dependencies only packaged but not explicitly used -->
<usedDependencies>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,12 @@

import io.camunda.zeebe.gateway.admin.backup.BackupApi;
import io.camunda.zeebe.gateway.admin.backup.BackupRequestHandler;
import io.camunda.zeebe.gateway.admin.backup.BackupStatus;
import io.camunda.zeebe.gateway.impl.broker.BrokerClient;
import io.camunda.zeebe.util.VisibleForTesting;
import java.util.concurrent.CompletionException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.actuate.endpoint.annotation.ReadOperation;
import org.springframework.boot.actuate.endpoint.annotation.Selector;
import org.springframework.boot.actuate.endpoint.annotation.WriteOperation;
import org.springframework.boot.actuate.endpoint.web.WebEndpointResponse;
Expand Down Expand Up @@ -42,18 +44,34 @@ public WebEndpointResponse<?> take(@Selector @NonNull final long id) {
return new WebEndpointResponse<>(new TakeBackupResponse(backupId));
} catch (final CompletionException e) {
return new WebEndpointResponse<>(
new TakeBackupError(id, e.getCause().getMessage()),
new ErrorResponse(id, e.getCause().getMessage()),
WebEndpointResponse.STATUS_INTERNAL_SERVER_ERROR);
} catch (final Exception e) {
return new WebEndpointResponse<>(
new TakeBackupError(id, e.getMessage()),
new ErrorResponse(id, e.getMessage()), WebEndpointResponse.STATUS_INTERNAL_SERVER_ERROR);
}
}

// TODO: do not use the internal data type directly, but later use the OpenAPI generated models on
// both the client and server side
@ReadOperation
public WebEndpointResponse<?> status(@Selector @NonNull final long id) {
try {
final BackupStatus status = api.getStatus(id).toCompletableFuture().join();
return new WebEndpointResponse<>(status);
} catch (final CompletionException e) {
return new WebEndpointResponse<>(
new ErrorResponse(id, e.getCause().getMessage()),
WebEndpointResponse.STATUS_INTERNAL_SERVER_ERROR);
} catch (final Exception e) {
return new WebEndpointResponse<>(
new ErrorResponse(id, e.getMessage()), WebEndpointResponse.STATUS_INTERNAL_SERVER_ERROR);
}
}

@VisibleForTesting
record TakeBackupResponse(long id) {}
record ErrorResponse(long id, String failure) {}

@VisibleForTesting
record TakeBackupError(long id, String failure) {}
record TakeBackupResponse(long id) {}
}

0 comments on commit 721b6b4

Please sign in to comment.