You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Channels appear to "fake up" support for select() on POSIX OSes by creating a pipe using os.pipe() and handing back the read end of the pipe as the fileno(). A pipe created by the pipe() syscall is, according to the docs, (and ~40 years of convention,) unidirectional (unlike a socket). The Paramiko docs say:
A Channel is meant to behave like a socket, and has an API that should be indistinguishable from the Python socket API.
Empirically speaking, handing back the read end of a unidirectional pipe doesn't universally "behave like a [bidirectional] socket". Specifically, on Linux, select([], [chan], []) will not indicate that the Channel is writable, even if calling chan.send_ready() returns True.
Without coding it up (I'm happy to help, but don't have the bandwidth right this second), I posit that the ideal solution here would probably be to replace the use of os.pipe() in this case, and instead use socket.socketpair(). Once you have a socket pair, you can use setsockopt to set the buffer sizes to (and probably the high and low water marks, etc) allowing you to effectively know/manipulate the state of the buffer using one of the sockets, and therefore manipulate the read- and write-ability of the other socket in the pair, at will. From there, pass out the "other" end of the pair as the fileno() of the Channel (which can be select()ed for both read- and write-ability), and then frob the private end at will so as to signal read- or write-ability on the public end when the Channel state changes.
As a potentially interesting forensic aside, I discovered this problem because of inconsistent behavior of some code that I was running on both OS X and Linux. On OS X, putting this "read end" FD into the list for polling writability signals that the FD is writable. For example, on OS X:
ipmcc@osx:~ $ python
Python 2.7.11 (default, Dec 26 2015, 17:47:15)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, select
>>> r,w = os.pipe()
>>> _, canwrite, _ = select.select([], [r], [], 0)
>>> print(canwrite)
[3]
>>> _, canwrite, _ = select.select([], [w], [], 0)
>>> print canwrite
[4]
>>>
ipmcc@osx:~ $
Whereas, on Linux, it does not:
ipmcc@linux:~$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, select
>>> r,w = os.pipe()
>>> _, canwrite, _ = select.select([], [r], [], 0)
>>> print(canwrite)
[]
>>> _, canwrite, _ = select.select([], [w], [], 0)
>>> print canwrite
[4]
>>>
ipmcc@linux:~$
In addition to this empirical evidence, Googling around indicates that the behavior in this situation is implementation-dependent. For instance, I found an old HP/UX manual that says:
select() returns the same results for a pipe whether a file descriptor associated with the read-only end or the write-only end of the pipe is used, since both file descriptors refer to the same underlying pipe. So a select() of a read-only file descriptor that is associated with a pipe can return ready to write, even though that particular file descriptor cannot be written to.
...and although this aligns with what OS X is doing, it's clearly not how Linux is behaving. Anyway, I'm working around this at the moment, by wrapping select() and polling Channels (with waiting data to write) using chan.send_ready() to short-circuit the select(). It works, but it's not ideal.
For posterity:
OS X version was: 10.11.3
Paramiko version on OS X was: 1.15.2
Linux version was: Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-77-generic x86_64)
Paramiko version on Linux was: 1.10.1
(FWIW, I still see the same pipe()-based code at the top of master here in GitHub, so I'm not particularly worried about my versions being out of sync/date)
The text was updated successfully, but these errors were encountered:
Channels appear to "fake up" support for
select()
on POSIX OSes by creating a pipe usingos.pipe()
and handing back the read end of the pipe as thefileno()
. A pipe created by thepipe()
syscall is, according to the docs, (and ~40 years of convention,) unidirectional (unlike asocket
). The Paramiko docs say:Empirically speaking, handing back the read end of a unidirectional pipe doesn't universally "behave like a [bidirectional] socket". Specifically, on Linux,
select([], [chan], [])
will not indicate that the Channel is writable, even if callingchan.send_ready()
returnsTrue
.Without coding it up (I'm happy to help, but don't have the bandwidth right this second), I posit that the ideal solution here would probably be to replace the use of
os.pipe()
in this case, and instead usesocket.socketpair()
. Once you have a socket pair, you can usesetsockopt
to set the buffer sizes to (and probably the high and low water marks, etc) allowing you to effectively know/manipulate the state of the buffer using one of the sockets, and therefore manipulate the read- and write-ability of the other socket in the pair, at will. From there, pass out the "other" end of the pair as thefileno()
of the Channel (which can beselect()
ed for both read- and write-ability), and then frob the private end at will so as to signal read- or write-ability on the public end when the Channel state changes.As a potentially interesting forensic aside, I discovered this problem because of inconsistent behavior of some code that I was running on both OS X and Linux. On OS X, putting this "read end" FD into the list for polling writability signals that the FD is writable. For example, on OS X:
Whereas, on Linux, it does not:
In addition to this empirical evidence, Googling around indicates that the behavior in this situation is implementation-dependent. For instance, I found an old HP/UX manual that says:
...and although this aligns with what OS X is doing, it's clearly not how Linux is behaving. Anyway, I'm working around this at the moment, by wrapping
select()
and polling Channels (with waiting data to write) usingchan.send_ready()
to short-circuit theselect()
. It works, but it's not ideal.For posterity:
OS X version was: 10.11.3
Paramiko version on OS X was: 1.15.2
Linux version was: Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-77-generic x86_64)
Paramiko version on Linux was: 1.10.1
(FWIW, I still see the same
pipe()
-based code at the top of master here in GitHub, so I'm not particularly worried about my versions being out of sync/date)The text was updated successfully, but these errors were encountered: