Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tokio::process::Command leaves zombies when child future is dropped #2685

Closed
flumm opened this issue Jul 24, 2020 · 5 comments
Closed

tokio::process::Command leaves zombies when child future is dropped #2685

flumm opened this issue Jul 24, 2020 · 5 comments
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-process Module: tokio/process T-docs Topic: documentation

Comments

@flumm
Copy link

flumm commented Jul 24, 2020

Version

└── tokio v0.2.22
    └── tokio-macros v0.2.5

Platform
Linux zita 5.4.44-2-pve #1 SMP PVE 5.4.44-2 (Wed, 01 Jul 2020 16:37:57 +0200) x86_64 GNU/Linux

Description
when i start a child process with 'kill_on_drop(true)'
and before the future is finished i drop it, there is a zombie
left until a new Child future is polled.

short reproducer: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=6746973d696b5122bc212e4bf6436092

i commented out the stdin code for the playground
when run locally it is easier to see that there is a defunct process when looking at e.g., 'ps ax | grep sleep' output
as long as no new child future is polled

i would expect that someone (e.g. the drop handler?) polls the future once again
so that the reaper can call waitpid once more...

@flumm flumm added A-tokio Area: The main tokio crate C-bug Category: This is a bug. labels Jul 24, 2020
@davidbarsky
Copy link
Member

Does this playground show the behavior that you're expecting? https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=2fb1919dae4c46a305d328cb333cd79c

You might need to .await the task you spawn.

@flumm
Copy link
Author

flumm commented Jul 24, 2020

No actually the problem is that after the child in the first task is dropped, the child process is killed but waitpid is never called for the first spawned child. Only when there is another command spawned and its future is polled, waitpid is called again

You can verify this by removing the second command spawning and waiting eg. on an input. You can now see that the child process is a zombie

@Darksonn Darksonn added the M-process Module: tokio/process label Jul 24, 2020
@ipetkov
Copy link
Member

ipetkov commented Jul 24, 2020

Hi @flumm thanks for the report! It is is a known limitation that we do not call waitpid on drop since we found in the past that there is no guarantee the kernel will not block even immediately after sigkill is sent to the child: alexcrichton/tokio-process#51

Tokio will perform a best-effort attempt to clean up zombie processes; if no more processes are spawned via tokio, then it is possible some of the processes remain as zombies.

One possible work around is to avoid dropping child structs and await them to completion.

@ipetkov ipetkov added the T-docs Topic: documentation label Jul 24, 2020
@flumm
Copy link
Author

flumm commented Jul 27, 2020

hmm... ok so this is 'known' behaviour
i would argue though that it makes the 'kill_on_drop' feature a little hard to use, since it will often
leave zombies in long running applications that are not guaranteed to spawn another process
(since there is no way to await a dropped future)

it probably would be good to somehow extract a handle to to kill it, like mentioned in #2512
but the solution there with tokio::select is probably good enough for us now

@ipetkov
Copy link
Member

ipetkov commented Oct 13, 2020

Docs around this have been updated in #2952 and we've revamped the orphan reaping implementation in #2907 so hopefully this should be less of a problem now. Going to close this out, but feel free to reopen if there are other questions!

@ipetkov ipetkov closed this as completed Oct 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-process Module: tokio/process T-docs Topic: documentation
Projects
None yet
Development

No branches or pull requests

4 participants