Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update doc/persistence.md #641

Merged
merged 2 commits into from
Dec 14, 2022
Merged

Update doc/persistence.md #641

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a737a27
typofix: heck --> check
donhui Dec 13, 2022
fb7fd7f
Update doc/persistence.md
donhui Dec 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 2 additions & 3 deletions doc/persistence.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ Running pipelines persist in 3 pieces:
- If null, probably the build dates back to before this field was added - we check to see if this is running in a highly persistent DurabilityMode (Max_survivability generally)
* done - if true, this execution completed, if false or un-set, the pipeline is a candidate to resume unless its only head is a FlowEndNode
- The handling of false is for legacy reasons, since it was only recently made persistent.
*
* various other boolean flags & settings for the execution (durability setting, user that started the build, is it sandboxed, etc)
3. The Program -- this is the current execution state of the Pipeline
* This holds the Groovy state - the `CpsThreadGroup` - with runtime calls transformed by CPS so they can persist
Expand All @@ -29,12 +28,12 @@ Running pipelines persist in 3 pieces:

Some basic rules:

1. If the FlowNodeStorage is corrupt, incomplete, or un-persisted, all manner of heck will break loose
1. If the FlowNodeStorage is corrupt, incomplete, or un-persisted, all manner of check will break loose
- In terms of Pipeline execution, the impact is like the Resonance Cascade from the Half-Life games
donhui marked this conversation as resolved.
Show resolved Hide resolved
- The pipeline can never be resumed (the key piece is missing)
- Usually we fake up some placeholder FlowNodes to cover this situation and save them
2. Whenever persisting data, the Pipeline *must* have the FlowNodes persisted on disk (via `storage.flush()` generally)
in order to be able to load the heads and restore the program.
3. Once we've set persistedClean as false and saved the FlowExecution, then it doesn't matter what we do -- the Pipeline will assume
it already has incomplete persistence data (as with 1) when trying to resume. This is how we handle the low-durability modes, to
avoid resuming a stale state of the Pipeline simply because we have old data persisted and are not updating it.
avoid resuming a stale state of the Pipeline simply because we have old data persisted and are not updating it.