Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PartitionId in the logging context #7885

Merged
18 commits merged into from
Sep 24, 2021
Merged

PartitionId in the logging context #7885

18 commits merged into from
Sep 24, 2021

Conversation

Zelldon
Copy link
Member

@Zelldon Zelldon commented Sep 21, 2021

Description

  • Add the partition id explicitly to the raft context
  • Set the partition id to the MDC on the first raft thread execution
  • Add a context map to the actor which is used for the MDC on exection

@npepinpe please take a look at this and let me know whether this approach makes sense to you. If this is the case I will change sub classes of the Actor class to add there partition id to the context.

Related issues

closes #7859

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

  • The changes are backwards compatibility with previous versions
  • If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/0.25) to the PR, in case that fails you need to create backports manually.

Testing:

  • There are unit/integration tests that verify all acceptance criterias of the issue
  • New tests are written to ensure backwards compatibility with further versions
  • The behavior is tested manually
  • The change has been verified by a QA run
  • The impact of the changes is verified by a benchmark

Documentation:

  • The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
  • New content is added to the release announcement

We have in the RaftContext only the name, which contains the raft group and partition id. Sometimes we just want to have the partition id as an int this introduces this property.
This context map is used on execution to set the logging context. Per default it just contains the actor name. Later sub classes should fill it with more context. It should make it easier on debugging and improve observability.
In order to not recreate the map object all the time, we create a map only once on the first call and return that afterwards.
Copy link
Member

@npepinpe npepinpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Code looks fine, just some questions.

💭 I'm not sure this is the best approach in the long term, but it should work well enough until we have some form of actor hierarchy (whether with Akka or in our scheduler). See for example this Akka tutorial. Propagating your context is one of the advantages of having a hierarchy. Anyway, just a thought.

import java.util.concurrent.TimeUnit;
import java.util.function.BiConsumer;
import java.util.function.Consumer;

public abstract class Actor implements CloseableSilently, AsyncClosable, ConcurrencyControl {

public static final String ACTOR_PROP_NAME = "actor-name";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔧 Can be private

@Zelldon
Copy link
Member Author

Zelldon commented Sep 22, 2021

@npepinpe please have another look

@Zelldon
Copy link
Member Author

Zelldon commented Sep 24, 2021

@npepinpe ping

Copy link
Member

@npepinpe npepinpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I'm not sure why it re-ordered some things (e.g. in ZeebePartition, AsyncSnapshotDirector), can you verify before merging that it's now correctly ordered according to the style guide?

💭 I'm wondering if it might not be interesting to pass in context information when spawning an actor/submitting it to the scheduler? That way you could propagate context information.

@Zelldon
Copy link
Member Author

Zelldon commented Sep 24, 2021

Bors r+

@Zelldon
Copy link
Member Author

Zelldon commented Sep 24, 2021

I'm pretty sure since otherwise the other 20 files would have been reordered.

Regarding the other thing I think we can do it if we need it

ghost pushed a commit that referenced this pull request Sep 24, 2021
7885: PartitionId in the logging context r=Zelldon a=Zelldon

## Description

 * Add the partition id explicitly to the raft context 
 * Set the partition id to the MDC on the first raft thread execution
 * Add a context map to the actor which is used for the MDC on exection

@npepinpe please take a look at this and let me know whether this approach makes sense to you. If this is the case I will change sub classes of the Actor class to add there partition id to the context.

<!-- Please explain the changes you made here. -->

## Related issues

<!-- Which issues are closed by this PR or are related -->

closes #7859



Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
@ghost
Copy link

ghost commented Sep 24, 2021

Build failed:

@Zelldon
Copy link
Member Author

Zelldon commented Sep 24, 2021

Bors r+

@ghost
Copy link

ghost commented Sep 24, 2021

Build succeeded:

@ghost ghost merged commit 6905d5c into develop Sep 24, 2021
@ghost ghost deleted the zell-partition-id-ctx branch September 24, 2021 16:41
@github-actions
Copy link
Contributor

Successfully created backport PR #7910 for release-1.2.0.

ghost pushed a commit that referenced this pull request Sep 24, 2021
7910: [Backport release-1.2.0] PartitionId in the logging context r=Zelldon a=github-actions[bot]

# Description
Backport of #7885 to `release-1.2.0`.

relates to #7859

Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
ghost pushed a commit that referenced this pull request Sep 24, 2021
7910: [Backport release-1.2.0] PartitionId in the logging context r=Zelldon a=github-actions[bot]

# Description
Backport of #7885 to `release-1.2.0`.

relates to #7859

Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Partition ID or name to the Logging context
3 participants