add ed22519 support

[zzzeek]

Looking at issues like #651, #532, #583, it looks like pymysql does not wish to support any client side authentication modules, such as sha256, or ed25519 which is what MariaDB supports. Two of the issues seem to be closed without any explanation though in #583 I see statements like "I don't want to add dependency" "Please move your code to pymysql/auth/sha256password.py".

It doesn't seem like there is a "pymysql/auth" folder in the source tree.

I have a need to understand what the path is here, my immediate use case is the ed25519 plugin. So here are what I see as all the possible choices, you can check these off:

• PyMySQL supports pluggable authentication schemes via the auth_plugin_map parameter, and we welcome contributors to add these plugins to an as-yet-not-created folder pymysql/auth. PyMySQL will distribute the plugins but not support them unless their authors do
• PyMySQL supports pluggable authentication schemes, but does NOT want any of the source code in the PyMySQL repo. Please publish your plugins on pypi and maintain them separately. (are there any such plugins ? I couldn't find any)
• PyMySQL does not support authentication plugins at all, you've misunderstood what the auth_plugin_map argument does. You will need to use mysql-connector-python or mysqlclient (does the latter support them?)
• other

[methane]

I support sha256 and caching_sha256 already.
https://github.com/PyMySQL/PyMySQL/blob/master/pymysql/_auth.py

I never used pluggable auth. I merged PR for adding auth_plugin_map with real experience.
While implementing sha256 and caching_sha256, I found current plugin API is not so good.
So I implemented them without plugin API.

[methane]

BTW, I don't like cryptography dependency so I want to use pure Python solution.
I learned PKCS and sent pull request to python-rsa. But it is not merged yet.

sybrenstuvel/python-rsa#126

[zzzeek]

I support sha256 and caching_sha256 already.
https://github.com/PyMySQL/PyMySQL/blob/master/pymysql/_auth.py

I never used pluggable auth. I merged PR for adding auth_plugin_map with real experience.
While implementing sha256 and caching_sha256, I found current plugin API is not so good.
So I implemented them without plugin API.

right when I looked at the plugin API i had the impression it wasn't necessarily going to work :)

So what is my path forward for a customer that wants to use ed25519? do you want me to PR a scheme for _auth.py that uses the cryptography library as it seems like you are OK with that as a dependency ?

[methane]

Ah, yes. Please add it to _auth.py.

[zzzeek]

OK so there is a reference Python implementation that has no dependencies at https://ed25519.cr.yp.to/python/ed25519.py , intro is at https://ed25519.cr.yp.to/software.html and this is public domain. that's one approach where if we were to use that I'd likely keep ed25519 in its own file. another approach is to use the C bindings: https://pypi.org/project/ed25519/ which fortunately seems to be a standalone C implementation, last released in 2015 so might not be tested with newer python 3 versions.

would you want to break out _auth.py into a package so that things can be separated better?

[dciabrin]

After having discussed with @zzzeek , I evaluated what I believe were the possible options for supporting MariaDB's Ed25519-based authentication.

TL;DR: I have proposed #791 to add initial support for Ed25519 for passwords which are 32 bytes long, which would unblock us until another commit supports passwords of arbitrary size.

Detailed context:
MariaDB implements an authentication method that consists in signing a scramble with a cypher scheme inspired by Ed25519.

I'm saying "inspired", because in the reference Ed25519 implementation, the seed used to generate the signing key and the public key must be a 32 byte random input, whereas MariaDB expects the
seed to be the user password (without constraint on the password's length).

This has an impact on the crypto package we can depend upon. Below is a list of what I evaluated:

1. DJB's pure python implementation of Ed25519
   https://ed25519.cr.yp.to/software.html
   A pure-python, Public Domain implementation of Ed25519.
   Cons:

   • It is extremely slow, it doesn't work with python 3 [1], which is a no-go.
   • It is not immune to side channel attack (e.g. timing attack)

2. moneropy's derivative of ed25519
   https://github.com/bigreddmachine/MoneroPy/blob/master/moneropy/crypto/ed25519.py
   BSD licensed; essentially same as 1., but with python3 support.
   Cons:

   • It's as slow and insecure as 1., which is a no-go IMHO.
   • It is BSD licensed, which I don't know if it's an issue

3. pyca's derivative of ed25519
   https://github.com/pyca/ed25519
   MIT licensed; a reimplementation of 1. with some optimization
   Cons:

   • Per authors' README: "This code may be useful in cases where you absolutely cannot have any C code dependencies. Otherwise, PyNaCl provides a version of the original author's C implementation, which runs faster and is carefully engineered to avoid side-channel attacks."
   • Doesn't look actively maintained or production ready

4. pyca's PyNaCl
   https://github.com/pyca/pynacl
   Apache2 licensed; Python bindings to the C-library libsodium
   Pro:

   • Ed25519 based on djb's original C-based implementation
   • Actively maintained
   • Very fast, and works with python2 and python3
     Cons:
   • Currently, it can only handle 32 bytes long passwords

Note: "The Apache2 license + use of C library" seems reasonable to me,
as PyMySQL uses cryptography.hazmat which has the same properties
(licensing + native code).

5. warner's python-ed25519
   https://github.com/warner/python-ed25519
   MIT licensed; Python bindings to DJB's original C-based implementation
   Cons:
   • Same limitation as PyNaCl, but looks abandonned, which is a no-go IMHO

In https://github.com/dciabrin/PyMySQL/tree/ed25519-djb we started discussing a first implementation based on 1. with @zzzeek , but given the limitations, I dropped it. Based on our experience, it seems to me that the only viable Ed25519 implementation rely on native libraries, and all those libraries only support 32 bytes seed as input.

Based on that, I did two implementations based on PyNaCl:

• The first one is what I proposed in Add ed25519 auth support #791, and it only
  supports authentication password whose size is 32 bytes.

• The second one is just an experiment based on a modified PyNaCl that exposes the necessary crypto functions for me to implement the full mariadb auth protocol and support all passwords.

I'm planning to discuss the possible improvement with the PyNaCl folks upstream, but until then I believe #791 is ready to be reviewed and would allow us to at least partially support Ed25519 in MariaDB.

[zzzeek]

See also https://jira.mariadb.org/browse/MDEV-19217 in case it's not linked here yet.

[dciabrin]

So starting libsodium 1.0.17 we have all the low-level crypto function necessary to implement signature for arbitrary-length password. PyNaCl currently only provides bindings for libsodium 1.0.16, so I'll propose a PR to support to 1.0.17. Once this support lands I'll update #791 to fully support auth_ed25519 in PyMySQL.

[dciabrin]

Heads up, FTR dciabrin@b02098d implements full support for auth_ed25519, with arbitrary passwords length. I didn't update #791 yet because it requires on pyca/pynacl#528 to land first.

[dciabrin]

Heads up: PyMySQL 1.4.0 has just been released so I updated #791 to use it accordingly in CI so it should be green now.
@methane I believe it's now ready for consumption and review, if you have some time to look at it.
Thanks!