Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(metrics): add process collector #15591

Merged
merged 8 commits into from
May 31, 2024
Merged

feat(metrics): add process collector #15591

merged 8 commits into from
May 31, 2024

Conversation

flaneur2020
Copy link
Member

@flaneur2020 flaneur2020 commented May 20, 2024 •
edited

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

fixes #15538

this PR adds the following process metrics, they're considered as the standard process metrics:

# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 138100.24
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.048576e+06
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 26
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 3.7982208e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.70893894953e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 1.346695168e+09
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19

as prometheus/rust-client still has not builtin support for process collector, this PR adds a vanilla implementation which only works in linux. we can switch to the builtin implementation after prometheus/client_rust#29 got resolved.

Tests

  • Unit Test

Type of change

  • Other (please describe): metrics improvement

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label May 20, 2024
@flaneur2020 flaneur2020 force-pushed the add-process-collector branch 2 times, most recently from bf15244 to e075981 Compare May 21, 2024 06:53
@flaneur2020 flaneur2020 marked this pull request as ready for review May 21, 2024 09:06
@BohuTANG BohuTANG requested a review from zhang2014 May 21, 2024 11:27
@zhang2014
Copy link
Member

Perhaps it can be add process level metrics when reading system.metrics.

@flaneur2020
Copy link
Member Author

Perhaps it can be add process level metrics when reading system.metrics.

we need find a way to let system.metrics able to expose the metrics from Collector, maybe we can add it in another PR to make a refactor about it.

@BohuTANG BohuTANG merged commit e2add77 into datafuselabs:main May 31, 2024
75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhang2014 zhang2014 zhang2014 approved these changes

Assignees
No one assigned
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

feat: add process level metrics like process_resident_memory_bytes
3 participants
@flaneur2020 @zhang2014 @BohuTANG