Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dal,rebaser): store pending dvu values on snapshot #3790

Merged
merged 1 commit into from
May 20, 2024

Conversation

zacharyhamm
Copy link
Contributor

Instead of sending the set of values to use as the starting point for calculating the dependent value graph in the DependentValuesUpdate job, place those values on the graph off a new category node. Then, when the job executes them, remove them from the graph.

self.find_existing_dependent_value(value_id).await?;

if existing_value_id.is_some() {
return Ok(());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we want to "touch" the existing node somehow to force conflict resolution? In the case where there's a DVU running that hasn't had the removal of these go through the rebaser, and we modify something (that won't be visible to the running DVU), we don't want to lose the fact that things still need to run. The conflict should always be a "modified a deleted" (or "deleted a modified") one, and the resolution would be to keep the modified version. In the case of a "conflicting modify", we can deduplicate down to one (doesn't really matter which). Same if we detect that two change sets tried to add the same one when neither started from a base that had it.

Copy link
Contributor

@fnichol fnichol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good to me and agreed we can follow up on @jhelwig's comment

EdgeWeight::new(change_set, EdgeWeightKind::new_use())?,
action_node_index,
)?;
for category_node_kind in CategoryNodeKind::iter() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That feels like this will scale better over time

@zacharyhamm
Copy link
Contributor Author

bors r+

@zacharyhamm
Copy link
Contributor Author

bors r-

@si-bors-ng
Copy link
Contributor

si-bors-ng bot commented May 15, 2024

Canceled.

@zacharyhamm zacharyhamm force-pushed the zack/add-dependent-values-to-root branch from 90c8298 to 66b1d95 Compare May 15, 2024 16:51
@zacharyhamm
Copy link
Contributor Author

bors r+

@si-bors-ng
Copy link
Contributor

si-bors-ng bot commented May 15, 2024

🕐 Waiting for PR status (Github check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set.

@zacharyhamm
Copy link
Contributor Author

bors r-

@zacharyhamm
Copy link
Contributor Author

forgot i wanted to ensure this was backwards compat

@si-bors-ng
Copy link
Contributor

si-bors-ng bot commented May 15, 2024

Merge conflict.

@zacharyhamm zacharyhamm force-pushed the zack/add-dependent-values-to-root branch 6 times, most recently from be48b12 to f33c19c Compare May 20, 2024 17:00
@zacharyhamm
Copy link
Contributor Author

bors r+

@si-bors-ng
Copy link
Contributor

si-bors-ng bot commented May 20, 2024

🕐 Waiting for PR status (Github check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set.

si-bors-ng bot added a commit that referenced this pull request May 20, 2024
3790: feat(dal,rebaser): store pending dvu values on snapshot r=zacharyhamm a=zacharyhamm

Instead of sending the set of values to use as the starting point for calculating the dependent value graph in the DependentValuesUpdate job, place those values on the graph off a new category node. Then, when the job executes them, remove them from the graph.

Co-authored-by: Zachary Hamm <zack@systeminit.com>
@zacharyhamm
Copy link
Contributor Author

bors r-

@zacharyhamm zacharyhamm reopened this May 20, 2024
@si-bors-ng
Copy link
Contributor

si-bors-ng bot commented May 20, 2024

Canceled.

Instead of sending the set of values to use as the starting point for
calculating the dependent value graph in the DependentValuesUpdate job,
place those values on the graph off a new category node. Then, when the
job executes them, remove them from the graph.
@zacharyhamm zacharyhamm force-pushed the zack/add-dependent-values-to-root branch from f33c19c to 9c5e486 Compare May 20, 2024 17:15
@zacharyhamm
Copy link
Contributor Author

bors r+

@si-bors-ng
Copy link
Contributor

si-bors-ng bot commented May 20, 2024

Build succeeded:

@si-bors-ng si-bors-ng bot merged commit b004e37 into main May 20, 2024
8 checks passed
@si-bors-ng si-bors-ng bot deleted the zack/add-dependent-values-to-root branch May 20, 2024 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants