Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add further debug logging #691

Draft
wants to merge 48 commits into
base: dev
Choose a base branch
from
Draft

Add further debug logging #691

wants to merge 48 commits into from

Conversation

ddobie
Copy link
Contributor

@ddobie ddobie commented Jun 29, 2023

No description provided.

@github-actions github-actions bot added this to In progress in Pipeline Backlog Jun 29, 2023
github-actions[bot]
github-actions bot reviewed Jun 29, 2023
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congrats on your first PR: please assign the PR to a milestone, and to a reviewer if not auto-assigned. Add relevant labels if possible

@ddobie ddobie added this to the v1.0.0 milestone Jun 29, 2023
ddobie and others added 26 commits July 4, 2023 10:37
@ddobie
Bump to 1.0.1 for reference
93c4e42
@ddobie
Added forced photometry timer debug statements
1a0eebb
@ddobie
Fixed previous commit
a05bb86
@ddobie
Log number of sources being forced fit
fa0bda6
@ddobie
Updated forced fitting scheduler to processes
1f5c7c6
@ddobie
Added timing debug loggging to new_sources.py
84c4423
@ddobie
Fixed f-strings
9b41f7b
@ddobie
Added association_models_generator debug logging
63e80c3
@ddobie
Drop associations batch size to 100
8ab0f67
@ddobie
Added get_memory_usage function
d7f95d1
@ddobie
Added get_memory_usage to finalise.py
ba4f91b
@ddobie
Added get_memory_usage to loading.py
0e6bf3a
@ddobie
Added get_memory_usage to main.py
1cc2f0e
@ddobie
Cleaned things up
b181cd7
@ddobie
Hardcode n_cpu
43d9d2f
@ddobie
Shorten forced fit measurment name
9c30c43
@ddobie
Add some further logging
029418e
@ddobie
Only log if long
2cf9970
Fix merge conflicts
8faefdf
@ddobie
Fixed indent
e95351f
@ddobie
Fix syntax error
ea30f62
@ddobie
Revert association size to 1e5
cb22d33
@ddobie
Temporarily hardcode n_cpu=10
e77d538
@ddobie
Added some more logging
72d1aeb
@ddobie
Attempt to limit memory usage
b3e217b
@ddobie
Attempt to limit memory usage
41bd240
ddobie and others added 21 commits February 15, 2024 09:30
@ddobie
Update parallel_grouppby to use chunksize
83b60cf
@ddobie
Correctly set chunk size
9102656
@ddobie
Remove superfluous logging
19741ba
@ddobie
Further dask groupby fixes
1598169
@ddobie
Added django.db.reset_queries
a3e4135
@ddobie
Added chunking for associations upload
f3fb2b8
@ddobie
Fixes
868d518
@ddobie
calculate measurement pairs with sensible partitions
c06b2e5
@ddobie
parallel_groupby_coord with sensible partitions
55b8a88
@ddobie
parallel_groupby_coord with sensible partitions
6b0455c
@ddobie
parallel_association with sensible partitions
75d2fb0
@ddobie
parallel_get_rms_measurements with sensible partitions
9da2404
@ddobie
Fixed new_sources issue
1af0a9f
@ddobie
commit all changes
7d93b31
@ddobie
Remove unnecessary debug logging statements
40283e1
@ddobie
Remove superfluous debug statements
2355d9e
@ddobie
Lots of debugging statements
4f4eb8b
@ddobie
Correctly estimate partition size
edae5f8
@ddobie
More playing
f0cffcc
@ddobie
Stash?
1082596
@ddobie
More changes to make this work
ec66300
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions left review comments

Assignees
No one assigned
Labels
None yet
Projects
Pipeline Backlog
  
In progress
Milestone
v1.0.0
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@ddobie