Handle ZombieProcess and NoSuchProcess from psutil for OSX systems

[flipdazed]

…://github.com//issues/1588

What I did

I added a check to ensure that a process in is actually an active process. There are many circumstances in OSX where one can have hanging/zombie processes. In this instance, the process name does not exist as the process has been terminated. This causes a really confusing psutil error for the user

  ...

  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/brownie/network/main.py", line 48, in connect
    rpc.attach(host)
  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/brownie/network/rpc/__init__.py", line 114, in attach
    pid = self._find_rpc_process_pid(resolved_addr)
  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/brownie/network/rpc/__init__.py", line 193, in _find_rpc_process_pid
    return self._get_pid_from_docker_backend()
  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/brownie/network/rpc/__init__.py", line 241, in _get_pid_from_docker_backend
    proc = self._find_proc_by_name("com.docker.backend")
  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/brownie/network/rpc/__init__.py", line 255, in _find_proc_by_name
    if process_name.lower() in proc.name().lower():
  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/__init__.py", line 628, in name
    cmdline = self.cmdline()
  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/__init__.py", line 681, in cmdline
    return self._proc.cmdline()
  File "/opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/_psosx.py", line 348, in wrapper
    raise NoSuchProcess(self.pid, self._name)
psutil.NoSuchProcess: process no longer exists (pid=696)

Related issue: #1588

How I did it

As with many bugs, this fix is quite simple when the origin of the bug has been pinned down. The following code

brownie/brownie/network/rpc/__init__.py

Lines 253 to 256 in 4ae5f52

	def _find_proc_by_name(self, process_name: str) -> psutil.Process:
	for proc in psutil.process_iter():
	if process_name.lower() in proc.name().lower():
	return proc

will fail for certain psutil cases (yielding ZombieProcess and NoSuchProcess shown below for ease of viewing and linked)

        except ProcessLookupError:
            if is_zombie(self.pid):
                raise ZombieProcess(self.pid, self._name, self._ppid)
            else:
                raise NoSuchProcess(self.pid, self._name)
        except PermissionError:
            raise AccessDenied(self.pid, self._name)
        except cext.ZombieProcessError:
            raise ZombieProcess(self.pid, self._name, self._ppid)

Added a try/except loop to catch the specific error and skip it.

    def _find_proc_by_name(self, process_name: str) -> psutil.Process:
        for proc in psutil.process_iter():
            try:
                proc_name = proc.name().lower()
            except (psutil.NoSuchProcess, psutil.ZombieProcess):
                continue

            if process_name.lower() in proc_name:
                return proc

This is the most intuitive way of catching this error as there is no way to filter the list of iterated processes to improve efficiency.

How to verify it

This is really tough - you need to somehow create a hanging process. It should be obvious from the code within psutil

https://github.com/giampaolo/psutil/blob/4446e3bd5fd29e0598f2ca6672d5ffe8ca40d442/psutil/_psosx.py#L336-L353

It will be difficult to create zombie processes or hanging processes for a full regression test. I don't think there should be test cases for such an obscure issue as a result. Satisfying the existing tests should be sufficient.

simple verification using existing bad system setup

as shown by ..

In [19]: import psutil
In [22]: [proc for proc in psutil.process_iter() if proc.name()]
---------------------------------------------------------------------------
ProcessLookupError                        Traceback (most recent call last)
File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/_psosx.py:343, in wrap_exceptions.<locals>.wrapper(self, *args, **kwargs)
    342 try:
--> 343     return fun(self, *args, **kwargs)
    344 except ProcessLookupError:

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/_psosx.py:401, in Process.cmdline(self)
    399 @wrap_exceptions
    400 def cmdline(self):
--> 401     return cext.proc_cmdline(self.pid)

ProcessLookupError: [Errno 3] assume no such process (originated from sysctl(KERN_PROCARGS2) -> EINVAL)

During handling of the above exception, another exception occurred:

NoSuchProcess                             Traceback (most recent call last)
Input In [22], in <cell line: 1>()
----> 1 [proc for proc in psutil.process_iter() if proc.name()]

Input In [22], in <listcomp>(.0)
----> 1 [proc for proc in psutil.process_iter() if proc.name()]

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/__init__.py:628, in Process.name(self)
    622 if POSIX and len(name) >= 15:
    623     # On UNIX the name gets truncated to the first 15 characters.
    624     # If it matches the first part of the cmdline we return that
    625     # one instead because it's usually more explicative.
    626     # Examples are "gnome-keyring-d" vs. "gnome-keyring-daemon".
    627     try:
--> 628         cmdline = self.cmdline()
    629     except AccessDenied:
    630         pass

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/__init__.py:681, in Process.cmdline(self)
    679 def cmdline(self):
    680     """The command line this process has been called with."""
--> 681     return self._proc.cmdline()

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/_psosx.py:348, in wrap_exceptions.<locals>.wrapper(self, *args, **kwargs)
    346         raise ZombieProcess(self.pid, self._name, self._ppid)
    347     else:
--> 348         raise NoSuchProcess(self.pid, self._name)
    349 except PermissionError:
    350     raise AccessDenied(self.pid, self._name)

NoSuchProcess: process no longer exists (pid=696)

The fix avoids this problem ...

In [15]:         for proc in psutil.process_iter():
    ...:             try:
    ...:                 proc_name = proc.name().lower()
    ...:             except (psutil.NoSuchProcess, psutil.ZombieProcess):
    ...:                 continue
    ...: 
    ...:             if process_name.lower() in proc_name:
    ...:                 print(proc)
    ...: 
psutil.Process(pid=92742, name='com.docker.backend', status='running', started='21:38:50')
psutil.Process(pid=92745, name='com.docker.backend', status='running', started='21:38:50')

For now it can be manually corrected by removing the bad processes like

In [24]: !kill 696

In [25]: In [19]: import psutil
    ...: In [22]: [proc for proc in psutil.process_iter() if proc.name()]
---------------------------------------------------------------------------
ProcessLookupError                        Traceback (most recent call last)
File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/_psosx.py:343, in wrap_exceptions.<locals>.wrapper(self, *args, **kwargs)
    342 try:
--> 343     return fun(self, *args, **kwargs)
    344 except ProcessLookupError:

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/_psosx.py:401, in Process.cmdline(self)
    399 @wrap_exceptions
    400 def cmdline(self):
--> 401     return cext.proc_cmdline(self.pid)

ProcessLookupError: [Errno 3] assume no such process (originated from sysctl(KERN_PROCARGS2) -> EINVAL)

During handling of the above exception, another exception occurred:

NoSuchProcess                             Traceback (most recent call last)
Input In [25], in <cell line: 2>()
      1 import psutil
----> 2 [proc for proc in psutil.process_iter() if proc.name()]

Input In [25], in <listcomp>(.0)
      1 import psutil
----> 2 [proc for proc in psutil.process_iter() if proc.name()]

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/__init__.py:628, in Process.name(self)
    622 if POSIX and len(name) >= 15:
    623     # On UNIX the name gets truncated to the first 15 characters.
    624     # If it matches the first part of the cmdline we return that
    625     # one instead because it's usually more explicative.
    626     # Examples are "gnome-keyring-d" vs. "gnome-keyring-daemon".
    627     try:
--> 628         cmdline = self.cmdline()
    629     except AccessDenied:
    630         pass

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/__init__.py:681, in Process.cmdline(self)
    679 def cmdline(self):
    680     """The command line this process has been called with."""
--> 681     return self._proc.cmdline()

File /opt/homebrew/anaconda3/envs/keyring-verifier-api/lib/python3.8/site-packages/psutil/_psosx.py:348, in wrap_exceptions.<locals>.wrapper(self, *args, **kwargs)
    346         raise ZombieProcess(self.pid, self._name, self._ppid)
    347     else:
--> 348         raise NoSuchProcess(self.pid, self._name)
    349 except PermissionError:
    350     raise AccessDenied(self.pid, self._name)

NoSuchProcess: process no longer exists (pid=20033)

In [26]: !kill 20033

In [27]: In [19]: import psutil
    ...: In [22]: [proc for proc in psutil.process_iter() if proc.name()]
Out[27]: 
[psutil.Process(pid=0, name='kernel_task', status='running', started='2022-08-18 21:48:21'),
 psutil.Process(pid=1, name='launchd', status='running', started='2022-08-18 21:48:21'),
 psutil.Process(pid=303, name='logd', status='running', started='2022-08-18 21:49:03'),
 psutil.Process(pid=304, name='UserEventAgent', status='running', started='2022-08-18 21:49:03'),
 psutil.Process(pid=306, name='uninstalld', status='running', started='2022-08-18 21:49:03'),
 psutil.Process(pid=307, name='fseventsd', status='running', started='2022-08-18 21:49:03'),
 psutil.Process(pid=308, name='mediaremoted', status='running', started='2022-08-18 21:49:03'),
 psutil.Process(pid=310, name='Karabiner-Driver', status='running', started='2022-08-18 21:49:03'),

...

Checklist

• I have confirmed that my PR passes all linting checks
• I have included test cases
• I have updated the documentation
• I have added an entry to the changelog

[flipdazed]

This function shouldn't have test cases or documentation as it's so niche it would add code bloat