Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Add Idempotency Support #1503

Open
yordis opened this issue Aug 23, 2023 · 1 comment
Open

[Feature Request] Add Idempotency Support #1503

yordis opened this issue Aug 23, 2023 · 1 comment
Labels
area/api v1 Affects the v1 API area/datastore Affects the storage system kind/proposal Something fundamentally needs to change state/needs discussion This can't be worked on yet

Comments

@yordis
Copy link
Contributor

yordis commented Aug 23, 2023

Description:

Based on a conversation between @yordis and @vroldanbet in Discord, there seems to be a need for first-class idempotency support in the WriteRelationship API.

This is crucial for achieving reliable operations, especially in scenarios involving event processing retries, and ensuring the linearization of processing.

Context:

The WriteRelationship API already supports batch writes of relationships, which allow transactional creation, touching, or deletion of relationships. However, there are challenges around ensuring idempotent writes, especially when there could be serialization conflicts if transactions overlap.

In scenarios described by Yordis Prieto:

T1: Add Relationship - takes too long.
T2: Remove Relationship - happens faster than T1.
T1: Timeout - timeouts.
T1: Retry Add Relationship - redo.
In such a case, the sequence of operations can lead to unexpected results.

Suggested Solution:

A proposed solution by @vroldanbet is to combine CREATE with an idempotency key. By doing this, if the WriteRelationships operation fails because the idempotency-key-relationship already exists, it would signify that the operation was previously carried out.

However, this solution has limitations. For instance, it returns an error result, and we don't get the zedtoken returned by the operation.

Proposal

Integrate first-class support for idempotency keys in the WriteRelationship API.
Ensure that with the idempotency key, repeated calls to WriteRelationship with the same key will result in the same effect without unintended side effects.

@jzelinskie jzelinskie added area/api v1 Affects the v1 API area/datastore Affects the storage system state/needs discussion This can't be worked on yet kind/proposal Something fundamentally needs to change labels Sep 23, 2023
@n3ziniuka5
Copy link

A proposed solution by @vroldanbet is to combine CREATE with an idempotency key. By doing this, if the WriteRelationships operation fails because the idempotency-key-relationship already exists, it would signify that the operation was previously carried out.
However, this solution has limitations. For instance, it returns an error result, and we don't get the zedtoken returned by the operation.

Have you considered a solution where repeated calls with the same key would return the same exact response as the original request? That's the most common implementation of idempotency keys, and would have the issue with zedtoken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api v1 Affects the v1 API area/datastore Affects the storage system kind/proposal Something fundamentally needs to change state/needs discussion This can't be worked on yet
Projects
None yet
Development

No branches or pull requests

3 participants