push memory profiler when rss rapidly increase

[zdyj3170101136]

the agent could push memory profiler immediately when rss rapidly increase.

like:
https://github.com/mosn/holmes

[zdyj3170101136]

@felixge could you check it?

[felixge]

Do you want this for the datadog-agent or when using dd-trace-go in your own applications?

Have you used holmes successfully in the past? If yes, for which use case? OOMs?

[zdyj3170101136]

https://github.com/mosn/holmes

dd-trace-go on my own applications.

no, just a proposal.

now, per minute push of memory profiler is not enough for oom, especially for pod.

the k8s pod memory = rss + cache (not accurate.).

if a pod oom killed.

1, caused by rss memory. we could push a immediately memory profiler if memory rapidly increase to debug that.

2, caused by high cache memory.may be the dd-trace-go could provide some information on how to debug that.
use linux syscall mincore to get file cached memory.
2.1, check all file open by pid first, like log ~.
2.2, check all file in the pid file directory /root/pid/root.