fix:Before killing a process under high concurrency, it is necessary …

[nuonuonn]

…to check if the process exists.

In high concurrency or asynchronous situations, the ProxyCommand process may have been killed by other means.

[bskinn]

Are there other errors that could be raised during these stream closure actions? If anything other than a ProcessLookupError occurs, then the except won't catch correctly.

If I read the docs right, close() is idempotent and won't raise on an already-closed stream... so, there may not actually be any additional errors to worry about?

If so, augmenting your comment here to note that close() won't raise even if the streams are already closed would be good.

[nuonuonn]

close() is redundant, I have deleted it.
But it is necessary to check whether a process is still present before killing it.

Because when I use distributed task queues Celery, this process is usually killed by the system.
It is possible to report an error ProcessLookupError.

In short, it is necessary to check whether a process is still alive before killing it.

[bskinn]

I'm sorry, I think I expressed myself poorly -- I wasn't requesting that you change your implementation, only that you add some comments &c. to document it more thoroughly.

Although, if that if self.closed: check will do just as well as the full set of stream .close() calls and the try/except, that's certainly much cleaner.

[nuonuonn]

There is no need to close IO as it will kill directly

[jun66j5]

We could use Popen.terminate() simply, in order to terminate the process if alive.

See

• posix - https://github.com/python/cpython/blob/v3.6.15/Lib/subprocess.py#L1596-L1605
• nt - https://github.com/python/cpython/blob/v3.6.15/Lib/subprocess.py#L1153-L1166

        self.process.terminate()

[nuonuonn]

Thank you for your guidance.

[jun66j5]

@nuonuonn Thanks for the explain.

The issue has been fixed in Python 3.10.0, python/cpython@01a202a

However, the issue occurs for the version or early. We could change like the following:

diff --git a/paramiko/proxy.py b/paramiko/proxy.py
index f7609c984..111e91627 100644
--- a/paramiko/proxy.py
+++ b/paramiko/proxy.py
@@ -119,7 +119,13 @@ class ProxyCommand(ClosingContextManager):
             raise ProxyCommandFailure(" ".join(self.cmd), e.strerror)

     def close(self):
-        os.kill(self.process.pid, signal.SIGTERM)
+        try:
+            self.process.terminate()
+        except ProcessLookupError:
+            pass  # Suppress the error if the process is not existent
+        # Close pipes and avoid zombie process
+        with self.process:
+            pass

     @property
     def closed(self):

[jun66j5]

The ProxyCommand class has other issues:

• The .close method doesn't close PIPE file descriptors for stdin, stdout and stderr of the process.
• The process becomes a zombie process caused by .wait is not invoked in .close method.

I think that we should invoke Popen.__exit__ using with self.process in the .close method.

See https://docs.python.org/3/library/subprocess.html?highlight=Added%20context%20manager%20support

[jun66j5]

In high concurrency or asynchronous situations, the ProxyCommand process may have been killed by other means.

I have questions. In this situation, what have you encountered issues?

It is able to send signals to the process until calling .wait() even if the process is killed by any reasons.

>>> import os, time
>>> from subprocess import Popen, PIPE
>>> proc = Popen(['sleep', '1'], stdin=PIPE, stdout=PIPE, stderr=PIPE)  # the process terminates in 1 second
>>> proc.returncode is None
True
>>> proc.pid
3657305
>>> time.sleep(3)
>>> os.kill(proc.pid, 0)
>>> proc.terminate()
>>> proc.returncode is None
False
>>> proc.wait()
0
>>> proc.returncode is None
False
>>> proc.returncode
0
>>> os.kill(proc.pid, 0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ProcessLookupError: [Errno 3] No such process
>>> proc.terminate()  # do nothing because proc.returncode is already set
>>>

Also, ProxyCommand class doesn't call .wait(), .poll() nor .communicate() so that self.process.returncode is never set.

See https://docs.python.org/3/library/subprocess.html#subprocess.Popen.returncode

[nuonuonn]

@jun66j5
In this situation
I used this code in my project （multi-process）

# ignores the subprocess exit signal and lets the operating system handle it.
signal.signal(signal.SIGCHLD, signal.SIG_IGN)

The subprocess has been killed by the operating system.

But in paramiko (proxy.py)

def close(self):
        os.kill(self.process.pid, signal.SIGTERM)

So there was an error
ProcessLookupError