Skip to content
This repository has been archived by the owner on Nov 29, 2023. It is now read-only.

Improve performance of usage command #607

Closed
lars-reimann opened this issue Jun 11, 2022 · 0 comments · Fixed by #611
Closed

Improve performance of usage command #607

lars-reimann opened this issue Jun 11, 2022 · 0 comments · Fixed by #611
Assignees
Labels
performance 🏃 Speed things up

Comments

@lars-reimann
Copy link
Member

lars-reimann commented Jun 11, 2022

Currently, there is a lot of file I/O happening. After a single program is processed, its usages are written to disk. Once everything is processed, the usages are aggregated. This ensure that in case the program crashes we do not need to start from scratch and allows for parallelization. However, it's still really inefficient.

Alternative solution:
Have workers just return their results and continuously aggregate them. Ensure the program does not crash 🙃.

Additional Context

Screenshot 2022-06-12 120010

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
performance 🏃 Speed things up
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant