Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSA key auth failing from paramiko 2.9.x client to dropbear server #1961

Closed
osfrickler opened this issue Dec 29, 2021 · 24 comments
Closed

RSA key auth failing from paramiko 2.9.x client to dropbear server #1961

osfrickler opened this issue Dec 29, 2021 · 24 comments

Comments

@osfrickler
Copy link

This was found in the OpenStack CI which uses Cirros as a small cloud image for testing purposes. Cirros runs dropbear as ssh server. There a no issues connecting with paramiko < 2.9, with the latest versions, authentication with an RSA key is failing, while using an ECDSA key or password login still works. The failure looks like this, sadly I haven't found out yet how to gather more debugging about the connection process:

Traceback (most recent call last):
  File "test.py", line 12, in <module>
    client.connect(host, username=user)
  File "/home/ubuntu/vv/lib/python3.8/site-packages/paramiko/client.py", line 435, in connect
    self._auth(
  File "/home/ubuntu/vv/lib/python3.8/site-packages/paramiko/client.py", line 766, in _auth
    raise saved_exception
  File "/home/ubuntu/vv/lib/python3.8/site-packages/paramiko/client.py", line 742, in _auth
    self._transport.auth_publickey(username, key)
  File "/home/ubuntu/vv/lib/python3.8/site-packages/paramiko/transport.py", line 1634, in auth_publickey
    return self.auth_handler.wait_for_response(my_event)
  File "/home/ubuntu/vv/lib/python3.8/site-packages/paramiko/auth_handler.py", line 263, in wait_for_response
    raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

See also cirros-dev/cirros#74.

@ingwarsw
Copy link

We have the same issue..
RSA stopped working for paramiko > 2.8.1 (both 2.9.0 and 2.9.1 was tested) while other algorithms works ok.
We are connecting to debian open ssh servers.

Will try to get more debugging logs.

@osfrickler
Copy link
Author

After seeing the demo code, I found that using util.log_to_file() gives me the debug output I was looking for. It seems that the server fails to announce any viable algorithm:

DEB [20211229-11:24:51.106] thr=1   paramiko.transport: Our pubkey algorithm list: ['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa']
DEB [20211229-11:24:51.106] thr=1   paramiko.transport: Server-side algorithm list: ['']
DEB [20211229-11:24:51.106] thr=1   paramiko.transport: Agreed upon 'rsa-sha2-512' pubkey algorithm
INF [20211229-11:24:51.180] thr=1   paramiko.transport: Authentication (publickey) failed.

Doing some more testing, disabled_keys = {'pubkeys':['rsa-sha2-512','rsa-sha2-256']} does work around the issue, but I wonder if the default behavior could/should be more backwards compatible at least until a major version step is done, like by falling back to ssh-rsa if the server provides an empty list.

@yoctozepto
Copy link

Does the server actually fail to announce the algorithm list or paramiko fails to parse it?

@marcelorik
Copy link

same issue here, working in version 2.8.1

@jun66j5
Copy link
Contributor

jun66j5 commented Dec 29, 2021

According to https://matt.ucc.asn.au/dropbear/CHANGES, Dropbear 2020.79 supports rsa-sha2 signatures. Are you using what version of dropbear?

Also, changelog of paramiko says:

When the server does not send server-sig-algs, Paramiko will attempt the first algorithm in the above list. Clients connecting to legacy servers should thus use disabled_algorithms to turn off SHA2.

Currently, it is need to use disabled_algorithms to connect to old ssh server.

@ingwarsw
Copy link

ingwarsw commented Dec 29, 2021

As you can see from log its Debian OpenSSH not Dropbear.
The same issue..

2021-12-29 14:50:12 DEBUG    transport/_log starting thread (client mode): 0xb78e8ca0
2021-12-29 14:50:12 DEBUG    transport/_log Local version/idstring: SSH-2.0-paramiko_2.9.1
2021-12-29 14:50:12 DEBUG    transport/_log Remote version/idstring: SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u8
2021-12-29 14:50:12 INFO     transport/_log Connected (version 2.0, client OpenSSH_6.7p1)
2021-12-29 14:50:12 DEBUG    transport/_log === Key exchange possibilities ===
2021-12-29 14:50:12 DEBUG    transport/_log kex algos: curve25519-sha256@libssh.org, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, diffie-hellman-group-exchange-sha256, diffie-hellman-group14-sha1
2021-12-29 14:50:12 DEBUG    transport/_log server key: ssh-rsa, ssh-dss, ecdsa-sha2-nistp256, ssh-ed25519
2021-12-29 14:50:12 DEBUG    transport/_log client encrypt: aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com, chacha20-poly1305@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log server encrypt: aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com, chacha20-poly1305@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log client mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2021-12-29 14:50:12 DEBUG    transport/_log server mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2021-12-29 14:50:12 DEBUG    transport/_log client compress: none, zlib@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log server compress: none, zlib@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log client lang: <none>
2021-12-29 14:50:12 DEBUG    transport/_log server lang: <none>
2021-12-29 14:50:12 DEBUG    transport/_log kex follows: False
2021-12-29 14:50:12 DEBUG    transport/_log === Key exchange agreements ===
2021-12-29 14:50:12 DEBUG    transport/_log Kex: curve25519-sha256@libssh.org
2021-12-29 14:50:12 DEBUG    transport/_log HostKey: ssh-ed25519
2021-12-29 14:50:12 DEBUG    transport/_log Cipher: aes128-ctr
2021-12-29 14:50:12 DEBUG    transport/_log MAC: hmac-sha2-256
2021-12-29 14:50:12 DEBUG    transport/_log Compression: none
2021-12-29 14:50:12 DEBUG    transport/_log === End of kex handshake ===
2021-12-29 14:50:12 DEBUG    transport/_log kex engine KexCurve25519 specified hash_algo <built-in function openssl_sha256>
2021-12-29 14:50:12 DEBUG    transport/_log Switch to new keys ...
2021-12-29 14:50:12 DEBUG    transport/_log Adding ssh-ed25519 host key for **.com: b'**'
2021-12-29 14:50:12 DEBUG    transport/_log Trying SSH agent key b'***'
2021-12-29 14:50:13 DEBUG    transport/_log userauth is OK
2021-12-29 14:50:13 DEBUG    transport/_log Finalizing pubkey algorithm for key of type 'ssh-rsa'
2021-12-29 14:50:13 DEBUG    transport/_log Our pubkey algorithm list: ['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa']
2021-12-29 14:50:13 DEBUG    transport/_log Server-side algorithm list: ['']
2021-12-29 14:50:13 DEBUG    transport/_log Agreed upon 'rsa-sha2-512' pubkey algorithm
2021-12-29 14:50:13 INFO     transport/_log Authentication (publickey) failed.

Server-side algorithm list: [''] looks bad ..

@osfrickler
Copy link
Author

When I connect with ssh -vv the server sends

debug2: peer server KEXINIT proposal                                                       
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha256,diffie-hellman-group14-s
ha1,kexguess2@matt.ucc.asn.au        
debug2: host key algorithms: ecdsa-sha2-nistp256,ssh-rsa

so it looks like paramiko is failing to handle this properly. The dropbear version currently being used in cirros is 2018.76, it will certainly make sense towards an update on that end, too.

@SGRelic
Copy link

SGRelic commented Dec 29, 2021

Same issue here using fabric with a proxy jump.
Connection was granted to the relay host (Ubuntu) but failed on the destination (Debian 9.11) with the same error log and an empty Server-side algorithm list.

@jun66j5
Copy link
Contributor

jun66j5 commented Dec 31, 2021

Workaround is to set transport.server_extensions = {'server-sig-algs': 'ssh-rsa'} after initiating Transport().

t = paramiko.Transport(sock)
# workaround for #1961
if hasattr(t, 'server_extensions'):
    t.server_extensions = {'server-sig-algs': 'ssh-rsa'}
t.start_client()

I consider that the following patch will fix the issue without use of disabled_algorithms by the user.

diff --git a/paramiko/transport.py b/paramiko/transport.py
index b99b3278d..141192eeb 100644
--- a/paramiko/transport.py
+++ b/paramiko/transport.py
@@ -679,6 +679,9 @@ class Transport(threading.Thread, ClosingContextManager):
             `.SSHException` -- if negotiation fails (and no ``event`` was
             passed in)
         """
+        # backwards-compatibility for old ssh servers which doesn't send
+        # server-sig-algs in MSG_EXT_INFO.
+        self.server_extensions = {"server-sig-algs": "ssh-rsa"}
         self.active = True
         if event is not None:
             # async, return immediately and let the app poll for completion

@martinprikryl
Copy link

I believe the problem is that Paramiko does not handle correctly an absence of server-sig-algs extension on the server side. Instead of disabling rsa-sha2-512 and rsa-sha2-256, it actually forces use of the rsa-sha2-512.

See https://stackoverflow.com/q/70565357/850848

@teunis90
Copy link

teunis90 commented Jan 5, 2022

I can confirm that I still hit the same/similar bug on version 2.9.1 using RSA keys. If we switch to password login it does work.

Here's the debug trace:

Fails:

Versions:
Python 3.8.7
paramiko==2.9.1

root@00146b78fd9d:/app# ./test-ssh.py
start
DEBUG:paramiko.transport:starting thread (client mode): 0x167ff280
DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_2.9.1
DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-Cisco-1.25
INFO:paramiko.transport:Connected (version 2.0, client Cisco-1.25)
DEBUG:paramiko.transport:=== Key exchange possibilities ===
DEBUG:paramiko.transport:kex algos: diffie-hellman-group-exchange-sha1, diffie-hellman-group14-sha1
DEBUG:paramiko.transport:server key: ssh-rsa
DEBUG:paramiko.transport:client encrypt: aes128-ctr, aes192-ctr, aes256-ctr, aes128-cbc, 3des-cbc, aes192-cbc, aes256-cbc
DEBUG:paramiko.transport:server encrypt: aes128-ctr, aes192-ctr, aes256-ctr, aes128-cbc, 3des-cbc, aes192-cbc, aes256-cbc
DEBUG:paramiko.transport:client mac: hmac-sha1, hmac-sha1-96
DEBUG:paramiko.transport:server mac: hmac-sha1, hmac-sha1-96
DEBUG:paramiko.transport:client compress: none
DEBUG:paramiko.transport:server compress: none
DEBUG:paramiko.transport:client lang: <none>
DEBUG:paramiko.transport:server lang: <none>
DEBUG:paramiko.transport:kex follows: False
DEBUG:paramiko.transport:=== Key exchange agreements ===
DEBUG:paramiko.transport:Kex: diffie-hellman-group-exchange-sha1
DEBUG:paramiko.transport:HostKey: ssh-rsa
DEBUG:paramiko.transport:Cipher: aes128-ctr
DEBUG:paramiko.transport:MAC: hmac-sha1
DEBUG:paramiko.transport:Compression: none
DEBUG:paramiko.transport:=== End of kex handshake ===
DEBUG:paramiko.transport:Got server p (2048 bits)
DEBUG:paramiko.transport:kex engine KexGex specified hash_algo <built-in function openssl_sha1>
DEBUG:paramiko.transport:Switch to new keys ...
DEBUG:paramiko.transport:Adding ssh-rsa host key for 1.2.3.4: b'0ef9h412d03a152f934d1b06e84c41ba' # changed for security
DEBUG:paramiko.transport:Trying SSH key b'1df1aee321a9af1d34fc6f97bfa24fc9' # changed for security
DEBUG:paramiko.transport:userauth is OK
DEBUG:paramiko.transport:Finalizing pubkey algorithm for key of type 'ssh-rsa'
DEBUG:paramiko.transport:Our pubkey algorithm list: ['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa']
DEBUG:paramiko.transport:Server-side algorithm list: ['']
DEBUG:paramiko.transport:Agreed upon 'rsa-sha2-512' pubkey algorithm
INFO:paramiko.transport:Authentication (publickey) failed.
Traceback (most recent call last):
  File "./testje.py", line 50, in <module>
    ssh.connect('1.2.3.4', username='imp_ro', pkey=ssh_key, timeout=3)
  File "/usr/local/lib/python3.8/site-packages/paramiko/client.py", line 435, in connect
    self._auth(
  File "/usr/local/lib/python3.8/site-packages/paramiko/client.py", line 766, in _auth
    raise saved_exception
  File "/usr/local/lib/python3.8/site-packages/paramiko/client.py", line 666, in _auth
    self._transport.auth_publickey(username, pkey)
  File "/usr/local/lib/python3.8/site-packages/paramiko/transport.py", line 1634, in auth_publickey
    return self.auth_handler.wait_for_response(my_event)
  File "/usr/local/lib/python3.8/site-packages/paramiko/auth_handler.py", line 258, in wait_for_response
    raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

Works:

Versions:
Python 3.8.3
paramiko==2.7.2

root@0f75ba5c03b8:/app# ./test-ssh.py
start
DEBUG:paramiko.transport:starting thread (client mode): 0x29cccdc0
DEBUG:paramiko.transport:Local version/idstring: SSH-2.0-paramiko_2.7.2
DEBUG:paramiko.transport:Remote version/idstring: SSH-2.0-Cisco-1.25
INFO:paramiko.transport:Connected (version 2.0, client Cisco-1.25)
DEBUG:paramiko.transport:kex algos:['diffie-hellman-group-exchange-sha1', 'diffie-hellman-group14-sha1'] server key:['ssh-rsa'] client encrypt:['aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'aes128-cbc', '3des-cbc', 'aes192-cbc', 'aes256-cbc'] server encrypt:['aes128-ctr', 'aes192-ctr', 'aes256-ctr', 'aes128-cbc', '3des-cbc', 'aes192-cbc', 'aes256-cbc'] client mac:['hmac-sha1', 'hmac-sha1-96'] server mac:['hmac-sha1', 'hmac-sha1-96'] client compress:['none'] server compress:['none'] client lang:[''] server lang:[''] kex follows?False
DEBUG:paramiko.transport:Kex agreed: diffie-hellman-group-exchange-sha1
DEBUG:paramiko.transport:HostKey agreed: ssh-rsa
DEBUG:paramiko.transport:Cipher agreed: aes128-ctr
DEBUG:paramiko.transport:MAC agreed: hmac-sha1
DEBUG:paramiko.transport:Compression agreed: none
DEBUG:paramiko.transport:Got server p (2048 bits)
DEBUG:paramiko.transport:kex engine KexGex specified hash_algo <built-in function openssl_sha1>
DEBUG:paramiko.transport:Switch to new keys ...
DEBUG:paramiko.transport:Adding ssh-rsa host key for 1.2.3.4: b'0ef9h412d03a152f934d1b06e84c41ba' # changed for security
DEBUG:paramiko.transport:Trying SSH key b'1df1aee321a9af1d34fc6f97bfa24fc9' # changed for security
DEBUG:paramiko.transport:userauth is OK
INFO:paramiko.transport:Authentication (publickey) successful!
command
DEBUG:paramiko.transport:[chan 0] Max packet in: 32768 bytes
DEBUG:paramiko.transport:[chan 0] Max packet out: 4096 bytes
DEBUG:paramiko.transport:Secsh channel 0 opened.
DEBUG:paramiko.transport:[chan 0] Sesch channel 0 request ok
end

Also tried with settings:

paramiko.Transport._preferred_kex = ('diffie-hellman-group-exchange-sha1', 'diffie-hellman-group14-sha1')
paramiko.Transport._preferred_ciphers = ('aes128-ctr', 'hmac-sha1')

Which does change the Key exchange agreements, but I still get the same error:

...

DEBUG:paramiko.transport:=== Key exchange agreements ===
DEBUG:paramiko.transport:Kex: diffie-hellman-group-exchange-sha1
DEBUG:paramiko.transport:HostKey: ssh-rsa
DEBUG:paramiko.transport:Cipher: aes128-ctr
DEBUG:paramiko.transport:MAC: hmac-sha1
DEBUG:paramiko.transport:Compression: none
DEBUG:paramiko.transport:=== End of kex handshake ===

...

INFO:paramiko.transport:Authentication (publickey) failed.
Traceback (most recent call last):
  File "./testje.py", line 52, in <module>
    ssh.connect('195.238.86.6', username='imp_ro', pkey=ssh_key, timeout=3)
  File "/usr/local/lib/python3.8/site-packages/paramiko/client.py", line 435, in connect
    self._auth(
  File "/usr/local/lib/python3.8/site-packages/paramiko/client.py", line 766, in _auth
    raise saved_exception
  File "/usr/local/lib/python3.8/site-packages/paramiko/client.py", line 666, in _auth
    self._transport.auth_publickey(username, pkey)
  File "/usr/local/lib/python3.8/site-packages/paramiko/transport.py", line 1634, in auth_publickey
    return self.auth_handler.wait_for_response(my_event)
  File "/usr/local/lib/python3.8/site-packages/paramiko/auth_handler.py", line 258, in wait_for_response
    raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

Code in test-ssh.py:

#!/usr/local/bin/python
import sys
import logging

from paramiko import AutoAddPolicy, RSAKey
from paramiko.client import SSHClient
from io import StringIO
import paramiko

root = logging.getLogger()
root.setLevel(logging.DEBUG)
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger("paramiko").setLevel(logging.DEBUG)

ssh_client = SSHClient()

ssh_client.load_system_host_keys()
ssh_client.set_missing_host_key_policy(AutoAddPolicy())
ssh_key_as_string = """-----BEGIN OPENSSH PRIVATE KEY-----
... secret ...
-----END OPENSSH PRIVATE KEY-----"""
ssh_key = RSAKey.from_private_key(StringIO(ssh_key_as_string))

with ssh_client as ssh:
    print('start')
    ssh.connect('1.2.3.4', username='myuser', pkey=ssh_key, timeout=3)
    print('command')
    stdin, stdout, stderr = ssh.exec_command('my ssh command')
    print('end')

Cisco device does not support ssh-dss, and password login is prohibited on production. Can I add anymore information to support resolving this issue?

@martinprikryl
Copy link

@teunis90 My answer as Stack Overflow question linked in my previous comment shows how to workaround the issue.

@teunis90
Copy link

teunis90 commented Jan 5, 2022

Thanks @martinprikryl this fixed my issue!

-        ssh.connect(ip, username=username, pkey=ssh_key, timeout=3)
+        ssh.connect(ip, username=username, pkey=ssh_key, timeout=3, disabled_algorithms=dict(pubkeys=['rsa-sha2-256', 'rsa-sha2-512']))

nsoranzo added a commit to galaxybot/galaxy that referenced this issue Jan 6, 2022
Fix `paramiko.ssh_exception.SSHException: encountered RSA key, expected OPENSSH key`
error, which is due to paramiko/paramiko#1961 .
nsoranzo added a commit to galaxybot/galaxy that referenced this issue Jan 6, 2022
Fix `paramiko.ssh_exception.SSHException: encountered RSA key, expected OPENSSH key`
error, which is due to paramiko/paramiko#1961 .
@marcus-ahle
Copy link

Ran into this issue as well. Have downgraded and pinned 2.8.1 for now.

@bitprophet
Copy link
Member

As @jun66j5 notes, this can be worked around w/o any patches by using the disabled_algorithms feature, which is roughly analogous to using OpenSSH's accepted pubkey algorithms setting. This was in the changelog but clearly not visible enough; I've just updated https://www.paramiko.org/changelog.html#2.9.0 with a backwards incompatibility notice at the top of the entry.

Modern OpenSSH versions' default behaviors, and the RFC this feature is based on, all suggest that at some point this sort of cut-over needs to happen (prefer SHA2 instead of SHA1) and I judged it was worthwhile to do so now, given that users can use the above-mentioned setting to revert to the prior behavior.


That said, some of these comments sound like there are still actual bugs lurking in the implementation, so I'll be trying to replicate some of those scenarios and looking at other recent filed issues in case we need a 2.9.2 release.

@bitprophet
Copy link
Member

bitprophet commented Jan 8, 2022

As you can see from log its Debian OpenSSH not Dropbear. The same issue..

2021-12-29 14:50:12 DEBUG    transport/_log starting thread (client mode): 0xb78e8ca0
2021-12-29 14:50:12 DEBUG    transport/_log Local version/idstring: SSH-2.0-paramiko_2.9.1
2021-12-29 14:50:12 DEBUG    transport/_log Remote version/idstring: SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u8
2021-12-29 14:50:12 INFO     transport/_log Connected (version 2.0, client OpenSSH_6.7p1)
2021-12-29 14:50:12 DEBUG    transport/_log === Key exchange possibilities ===
2021-12-29 14:50:12 DEBUG    transport/_log kex algos: curve25519-sha256@libssh.org, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, diffie-hellman-group-exchange-sha256, diffie-hellman-group14-sha1
2021-12-29 14:50:12 DEBUG    transport/_log server key: ssh-rsa, ssh-dss, ecdsa-sha2-nistp256, ssh-ed25519
2021-12-29 14:50:12 DEBUG    transport/_log client encrypt: aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com, chacha20-poly1305@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log server encrypt: aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com, chacha20-poly1305@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log client mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2021-12-29 14:50:12 DEBUG    transport/_log server mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2021-12-29 14:50:12 DEBUG    transport/_log client compress: none, zlib@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log server compress: none, zlib@openssh.com
2021-12-29 14:50:12 DEBUG    transport/_log client lang: <none>
2021-12-29 14:50:12 DEBUG    transport/_log server lang: <none>
2021-12-29 14:50:12 DEBUG    transport/_log kex follows: False
2021-12-29 14:50:12 DEBUG    transport/_log === Key exchange agreements ===
2021-12-29 14:50:12 DEBUG    transport/_log Kex: curve25519-sha256@libssh.org
2021-12-29 14:50:12 DEBUG    transport/_log HostKey: ssh-ed25519
2021-12-29 14:50:12 DEBUG    transport/_log Cipher: aes128-ctr
2021-12-29 14:50:12 DEBUG    transport/_log MAC: hmac-sha2-256
2021-12-29 14:50:12 DEBUG    transport/_log Compression: none
2021-12-29 14:50:12 DEBUG    transport/_log === End of kex handshake ===
2021-12-29 14:50:12 DEBUG    transport/_log kex engine KexCurve25519 specified hash_algo <built-in function openssl_sha256>
2021-12-29 14:50:12 DEBUG    transport/_log Switch to new keys ...
2021-12-29 14:50:12 DEBUG    transport/_log Adding ssh-ed25519 host key for **.com: b'**'
2021-12-29 14:50:12 DEBUG    transport/_log Trying SSH agent key b'***'
2021-12-29 14:50:13 DEBUG    transport/_log userauth is OK
2021-12-29 14:50:13 DEBUG    transport/_log Finalizing pubkey algorithm for key of type 'ssh-rsa'
2021-12-29 14:50:13 DEBUG    transport/_log Our pubkey algorithm list: ['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa']
2021-12-29 14:50:13 DEBUG    transport/_log Server-side algorithm list: ['']
2021-12-29 14:50:13 DEBUG    transport/_log Agreed upon 'rsa-sha2-512' pubkey algorithm
2021-12-29 14:50:13 INFO     transport/_log Authentication (publickey) failed.

Server-side algorithm list: [''] looks bad ..

OK, this seems to be OpenSSH 6.7 (Debian Jessie) which I don't believe supports server-sig-algs, so this falls under the above (servers w/o either server-sig-algs result in Paramiko defaulting to sha2-512). Please call back if the disabled_algorithms workaround doesn't work in this case.

Tho I agree that log message is misleading so I'll patch it up.

@bitprophet
Copy link
Member

When doing that I found a minor enhancement where we can raise an exception earlier if the server does send a signature list and we happen to not have any overlap. I expect this will not be a common case but the new logic flow strongly suggested it, so I put it in. (Will be in changelog.)

Anyway, for the case most of y'all are having, this is the new log output:

paramiko.transport.transport._log: Finalizing pubkey algorithm for key of type 'ssh-rsa'
paramiko.transport.transport._log: Our pubkey algorithm list: ['rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa']
paramiko.transport.transport._log: Server did not send a server-sig-algs list; defaulting to our first preferred algo ('rsa-sha2-512')
paramiko.transport.transport._log: NOTE: you may use the 'disabled_algorithms' SSHClient/Transport init kwarg to disable that or other algorithms if your server does not support them!
paramiko.transport.transport._log: Authentication (publickey) failed.

It's not a perfect fix but hopefully it will guide more users to what's still the current suggested workaround. And when that workaround is in use, you'll get this (reckon I could try to do more state tracking so the 'NOTE' is only emitted on error, but Paramiko's current design would make that complex so it's not currently worth it):

paramiko.transport.transport._log: Finalizing pubkey algorithm for key of type 'ssh-rsa'
paramiko.transport.transport._log: Our pubkey algorithm list: ['ssh-rsa']
paramiko.transport.transport._log: Server did not send a server-sig-algs list; defaulting to our first preferred algo ('ssh-rsa')
paramiko.transport.transport._log: NOTE: you may use the 'disabled_algorithms' SSHClient/Transport init kwarg to disable that or other algorithms if your server does not support them!
paramiko.transport.transport._log: Authentication (publickey) successful!

(Also - update your servers! 😱 SHA1 is broken!)

@bitprophet
Copy link
Member

Another note, I expect in some post-3.0 release (which will be the next "feature level" release with a drop of Python 2 and probably a small number of related tweaks) I will get to the much needed auth and algorithm-configuration overhauls I've been planning basically since I took this over. When that happens it should be much easier to specify ENABLED algorithms in addition to DISABLED ones, which will smooth over this kind of problem a bit more as well.

bitprophet added a commit that referenced this issue Jan 8, 2022
@bitprophet
Copy link
Member

bitprophet commented Jan 8, 2022

2.9.2 now out with the above. I'm going to close this as "WONTFIX". If anyone can identify legit issues with the new feature that are not fixable by using disabled_algorithms (or pinning to 2.8.x until one's servers are upgraded - absolutely an acceptable solution!), please file a new ticket with details and proof - thanks!

bsiegel added a commit to bsiegel/remarkable_mouse that referenced this issue Jan 20, 2022
As documented in paramiko/paramiko#1961 there is an incompatibility between Paramiko 2.9+ and dropbear (the SSH server used on Remarkable) when negotiating the pubkey algorithm. This fixes the issue by disabling SHA-2 algorithms, causing Paramiko to offer a SHA-1 algorithm which Dropbear will accept.
Evidlo pushed a commit to Evidlo/remarkable_mouse that referenced this issue Jan 20, 2022
As documented in paramiko/paramiko#1961 there is an incompatibility between Paramiko 2.9+ and dropbear (the SSH server used on Remarkable) when negotiating the pubkey algorithm. This fixes the issue by disabling SHA-2 algorithms, causing Paramiko to offer a SHA-1 algorithm which Dropbear will accept.
@nemesifier
Copy link

nemesifier commented May 3, 2022

What about the following situation?

A system has devices running both newer and older versions of dropbear.
SSH connections via paramiko to newer versions of dropbear will work.
SSH connections via paramiko to older versions of dropbear will not work.

We can apply the fix disabled_algorithms=dict(pubkeys=["rsa-sha2-512", "rsa-sha2-256"]),, but wouldn't that mean that on new version of dropbear we'd be using the old sha1?

I think a good solution could be to change this code as follows:

paramiko/paramiko/transport.py

Lines 1338 to 1343 in f2b4be8

if isinstance(hostkey, RSAKey):
self._preferred_keys = [
"rsa-sha2-512",
"rsa-sha2-256",
"ssh-rsa",
]

if isinstance(hostkey, RSAKey):
    self._preferred_keys = self.preferred_rsa_keys

And allow us to pass a list of preferred_rsa_keys which is backward compatible with old systems.

Sometimes old routers cannot be upgraded because they're EOL, but customers may want to keep using them in their internal network which they consider secure for internal usage by their employees. It wouldn't be great to force them to throw those away just because of this, sooner or later these routers will die anyway but it's better to have an easier way to maintain backward compatibility and support them.

@bskinn
Copy link
Contributor

bskinn commented May 6, 2022

@nemesisdesign This seems like a sensible feature request to consider. Can you repost as a new issue on the repo, with a link back to your comment here? Thanks!

@nemesifier
Copy link

Thanks for the feedback @bskinn, I opened #2049.

FedericoRessi pushed a commit to FedericoRessi/tobiko that referenced this issue May 31, 2022
Most tobiko tests are failing on RHEL9 environments due to ssh auth
issues. It was found that with newer paramiko version the issues
do not occur.

There is an open issue[1] in Paramiko starting from 2.9.2 (which allows
to connect to new servers like the one of RHEL 9) but in the while it
brakes the support for the old CirrOS image server (image version 5.2).

To keep support for old version we now uses disable_algorithms[1] option
when creating an SSH connection to CirrOS images. The workaround has
been documented in the Paramiko project page [2]

[1] paramiko/paramiko#1961
[2] https://github.com/rhevm-qe-automation/python-rrmngmnt/pull/149/files#diff-7b3ed02bc73dc06b7db906cf97aa91dec2b2eb21f2d92bc5caa761df5bbc168f

Change-Id: I301c18a832a05ddfd331bddd7ad2bc839205ad2d
@mirceaulinic
Copy link

Is anyone aware if there's an option I could throw into the SSH config file instead of applying a patch?

@alexNMD
Copy link

alexNMD commented Mar 4, 2023

You can also, in order to avoid edge effect disable algorithms for all connections use that :

try:
    ssh.connect(ip, **args)
except paramiko.ssh_exception.AuthenticationException:
    ssh.connect(ip, disabled_algorithms=dict(pubkeys=["rsa-sha2-512", "rsa-sha2-256"]), **args)

vignesh9293 pushed a commit to vignesh9293/netconf-exercise that referenced this issue Nov 28, 2023
Tried to use ncclient package in python to establish a netconf connection with IOS XR programmability sandbox.

faced couple of issues with respect to RSA algoritm selection on ncclient and not able to establish connection.
Applied the patch provided to paramiko package in this link : paramiko/paramiko#1961 to resolve it.
Enabled logging in code and debugged the above problem.
Also, was able to connect to the device with ssh cli command and able to see the hello messages in ssh cli.

Still facing below bad authentication type error when trying to run the program, need to investigate and fix the same to run the application and establish a connection with device.

DEBUG:ncclient.transport.ssh:Authentication type (publickey) not permitted.
DEBUG:ncclient.transport.ssh:Allowed methods: ['keyboard-interactive', 'password']
DEBUG:ncclient.transport.ssh:[host sandbox-iosxr-1.cisco.com session 0x7fa398c863a0] Bad authentication type; allowed types: ['keyboard-interactive', 'password']
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests