Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

select(...) on Paramiko Channel for writability not viable/consistent #695

Open
ipmcc opened this issue Feb 23, 2016 · 1 comment · May be fixed by #2300
Open

select(...) on Paramiko Channel for writability not viable/consistent #695

ipmcc opened this issue Feb 23, 2016 · 1 comment · May be fixed by #2300

Comments

@ipmcc
Copy link

ipmcc commented Feb 23, 2016

Channels appear to "fake up" support for select() on POSIX OSes by creating a pipe using os.pipe() and handing back the read end of the pipe as the fileno(). A pipe created by the pipe() syscall is, according to the docs, (and ~40 years of convention,) unidirectional (unlike a socket). The Paramiko docs say:

A Channel is meant to behave like a socket, and has an API that should be indistinguishable from the Python socket API.

Empirically speaking, handing back the read end of a unidirectional pipe doesn't universally "behave like a [bidirectional] socket". Specifically, on Linux, select([], [chan], []) will not indicate that the Channel is writable, even if calling chan.send_ready() returns True.

Without coding it up (I'm happy to help, but don't have the bandwidth right this second), I posit that the ideal solution here would probably be to replace the use of os.pipe() in this case, and instead use socket.socketpair(). Once you have a socket pair, you can use setsockopt to set the buffer sizes to (and probably the high and low water marks, etc) allowing you to effectively know/manipulate the state of the buffer using one of the sockets, and therefore manipulate the read- and write-ability of the other socket in the pair, at will. From there, pass out the "other" end of the pair as the fileno() of the Channel (which can be select()ed for both read- and write-ability), and then frob the private end at will so as to signal read- or write-ability on the public end when the Channel state changes.

As a potentially interesting forensic aside, I discovered this problem because of inconsistent behavior of some code that I was running on both OS X and Linux. On OS X, putting this "read end" FD into the list for polling writability signals that the FD is writable. For example, on OS X:

ipmcc@osx:~ $ python
Python 2.7.11 (default, Dec 26 2015, 17:47:15) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, select
>>> r,w = os.pipe()
>>> _, canwrite, _ = select.select([], [r], [], 0)
>>> print(canwrite)
[3]
>>> _, canwrite, _ = select.select([], [w], [], 0)
>>> print canwrite
[4]
>>> 
ipmcc@osx:~ $ 

Whereas, on Linux, it does not:

ipmcc@linux:~$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, select
>>> r,w = os.pipe()
>>> _, canwrite, _ = select.select([], [r], [], 0)
>>> print(canwrite)
[]
>>> _, canwrite, _ = select.select([], [w], [], 0)
>>> print canwrite
[4]
>>> 
ipmcc@linux:~$ 

In addition to this empirical evidence, Googling around indicates that the behavior in this situation is implementation-dependent. For instance, I found an old HP/UX manual that says:

select() returns the same results for a pipe whether a file descriptor associated with the read-only end or the write-only end of the pipe is used, since both file descriptors refer to the same underlying pipe. So a select() of a read-only file descriptor that is associated with a pipe can return ready to write, even though that particular file descriptor cannot be written to.

...and although this aligns with what OS X is doing, it's clearly not how Linux is behaving. Anyway, I'm working around this at the moment, by wrapping select() and polling Channels (with waiting data to write) using chan.send_ready() to short-circuit the select(). It works, but it's not ideal.

For posterity:

OS X version was: 10.11.3
Paramiko version on OS X was: 1.15.2
Linux version was: Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-77-generic x86_64)
Paramiko version on Linux was: 1.10.1
(FWIW, I still see the same pipe()-based code at the top of master here in GitHub, so I'm not particularly worried about my versions being out of sync/date)

@amosonn
Copy link

amosonn commented Sep 11, 2023

Just ran into this. Is there any plan to fix this? Conversely, could anyone point me in the right direction so I can attempt to fix this?

@amosonn amosonn linked a pull request Sep 11, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants