Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Check if processed data already exists in --production mode #1024

Merged
merged 6 commits into from
May 16, 2022
Merged

Check if processed data already exists in --production mode #1024

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b3e1d91
Add check if live_data exists, assuming live_data first type
mflierm Apr 20, 2022
59dedd2
Check now for bootstrax_state instead. Not sure about the 'else' in l…
mflierm Apr 20, 2022
2a1a3c9
Removed 'else' statement
mflierm Apr 21, 2022
3afeb7f
Update bin/bootstrax -> typos
mflierm Apr 21, 2022
373d5e1
Change 'processing' to 'busy' in possible bootstrax_states
mflierm Apr 21, 2022
a4794ba
Merge branch 'master' into production_check
JoranAngevaare May 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 12 additions & 0 deletions bin/bootstrax
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -345,11 +345,23 @@ def main():
elif args.process:
t_start = now()
number = args.process

# Check whether the run is already processed
bootstrax_state = run_coll.find_one(
{'number': number},
projection={'bootstrax': True}).get('bootstrax', {}
).get('state', 'no-state')
if args.production and bootstrax_state in ['done', 'busy', 'considering']:
message = f"It looks like run {number} is already processed."
log_warning(message, priority='fatal')
raise ValueError(message)

rd = consider_run({'number': number})
if rd is None:
message = f"Trying to process single run but no run numbered {number} exists"
log_warning(message, priority='fatal')
raise ValueError(message)

set_state('busy')
process_run(rd)
log.info(
Expand Down