Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Many zombie processes into cadvisor container #3518

Open
rc5hack opened this issue Apr 19, 2024 · 0 comments
Open

Many zombie processes into cadvisor container #3518

rc5hack opened this issue Apr 19, 2024 · 0 comments

Comments

@rc5hack
Copy link

rc5hack commented Apr 19, 2024

After upgrading cadvisor from v0.47.2 to v0.49.1, I found huge number of zombie processes running on the server. After cadvisor stop, they disappear.

Is it normal or not?

root@media:/opt/cadvisor$ ps auxww | grep -F -c ' Z'
17611

root@media:/opt/cadvisor$ ps auxww | grep -F ' Z' | tail
root     4192204  0.0  0.0      0     0 ?        Zs   Apr18   0:00 [timeout] <defunct>
root     4192490  0.0  0.0      0     0 ?        Zs   Apr16   0:00 [timeout] <defunct>
root     4192629  0.0  0.0      0     0 ?        Zs   Apr18   0:00 [timeout] <defunct>
root     4192942  0.0  0.0      0     0 ?        Zs   Apr16   0:00 [timeout] <defunct>
root     4193064  0.0  0.0      0     0 ?        Zs   Apr18   0:00 [timeout] <defunct>
root     4193399  0.0  0.0      0     0 ?        Zs   Apr16   0:00 [timeout] <defunct>
root     4193440  0.0  0.0      0     0 ?        Zs   Apr18   0:00 [timeout] <defunct>
root     4193810  0.0  0.0      0     0 ?        Zs   Apr16   0:00 [timeout] <defunct>
root     4193887  0.0  0.0      0     0 ?        Zs   Apr18   0:00 [timeout] <defunct>
root     4194260  0.0  0.0      0     0 ?        Zs   Apr16   0:00 [timeout] <defunct>

root@media:/opt/cadvisor$ pstree -p -s 4194260
systemd(1)───containerd-shim(2009543)───cadvisor(2009562)───timeout(4194260)

root@media:/opt/cadvisor$ docker compose down
[+] Running 2/2
 ✔ Container cadvisor                    Removed                                                                                                       1.7s
 ✔ Network docker-host-cadvisor_default  Removed                                                                                                       0.3s

root@media:/opt/cadvisor$ ps auxww | grep -F -c ' Z'
1

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.4 LTS
Release:        22.04
Codename:       jammy

$ uname -r
5.15.0-100-generic

docker-compose.yml:

version: '2.4'
name: docker-host-cadvisor
services:
  cadvisor:
    container_name: cadvisor
    hostname: cadvisor
    image: gcr.io/cadvisor/cadvisor:v0.49.1
    volumes:
      - ./cadvisor.htpasswd:/opt/cadvisor.htpasswd:ro
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
    devices:
      - /dev/kmsg
    privileged: true
    command:
      - --http_auth_file=/opt/cadvisor.htpasswd
      - --http_auth_realm=cadv
      - --prometheus_endpoint=/cadvisor_metrics_random_string_here # see https://github.com/google/cadvisor/issues/3401
      - --housekeeping_interval=5s
      - --disable_metrics=advtcp,app,cpu_topology,cpuset,disk,diskIO,hugetlb,memory_numa,percpu,process,referenced_memory,resctrl,sched,tcp,udp
      - --docker_only=true
    ports:
      - 0.0.0.0:9999:8080
    restart: unless-stopped
    cpus: 0.50
    mem_limit: 768m
    healthcheck:
      test: timeout 2 wget -q -t1 -O- http://localhost:8080/cadvisor_metrics_random_string_here | grep cadvisor_version_info
      interval: 20s
      retries: 3
      start_period: 10s
      timeout: 5s
    logging:
      driver: "json-file"
      options:
        max-size: "128m"
        max-file: "4"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant