Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support subprocesses with the dump command #491

Open
benfred opened this issue May 15, 2022 · 2 comments
Open

Support subprocesses with the dump command #491

benfred opened this issue May 15, 2022 · 2 comments

Comments

@benfred
Copy link
Owner

benfred commented May 15, 2022

We can profile subprocesses with the record or top commands, but not yet with the dump command. We should add support for that as well (see https://twitter.com/StasBekman/status/1525636315936485376 )

@stas00
Copy link

stas00 commented May 15, 2022

that would be very useful, thank you, @benfred!

and additionally controlling how many generations of ancestry to descend might be useful as well.

e.g. currently I have a need to py-spy the first generation of children and not the grandchildren (which are just helper processes). Here is a pstree output to show what I mean:

$ pstree -lT `whoami`
slurm_script─┬─srun───srun
             └─tee

python─┬─2*[python───60*[python]]
       └─6*[python]

sshd───bash───bash───pstree

So I want just the 2+6 python processes.

The 1 main python process is the pytorch launcher, the 60 grandchildren are helper dataloaders. the hanging usually happens in the 8 children.

Currently extracting this 2nd only generation with:

pgrep -P $(pgrep -o python) | xargs -I {} py-spy dump --pid {}

but, of course, I'm totally fine with the working pgrep solution.

@benfred
Copy link
Owner Author

benfred commented May 16, 2022

I've got the first part of this done in #492 - this adds support for the subprocesses flag in dump (but not limiting the depth of the subprocesses). This will be in the next release, but in the meantime you can grab the prebuilt wheels from the CI artifacts https://github.com/benfred/py-spy/actions/runs/2333647967#artifacts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants