Can't import cryptography generated pkcs12 into macos keychain

[ojii]

Versions:

• Python: 3.10.1
• cryptography: 37.0.2
• cffi: 1.15.0

Cryptography installed with poetry 1.2.0b1

I'm trying to create a self-signed certificate with cryptography, which mostly works, but it fails to import to macos (12.1) keychain. If I export the same certificate & private key as a chained PEM file and then use the openssl command line tool to convert it into a p12, I can successfully import it.

This is the code I use to try to export the p12 using cryptography:

def export(path: Path, cert: x509.Certificate, key: rsa.RSAPrivateKeyWithSerialization, password: bytes):
    with path.open('wb') as fobj:
        fobj.write(
            pkcs12.serialize_key_and_certificates(
                name=None,
                cas=None,
                key=key,
                cert=cert,
                encryption_algorithm=serialization.BestAvailableEncryption(password),
            )
        )

However, if instead I do this, macos is happy to import it:

def export(
    path: Path,
    cert: x509.Certificate,
    key: rsa.RSAPrivateKeyWithSerialization,
    password: bytes,
):
    with path.with_suffix(".pem").open("wb") as fobj:
        fobj.write(
            key.private_bytes(
                encoding=serialization.Encoding.PEM,
                format=serialization.PrivateFormat.TraditionalOpenSSL,
                encryption_algorithm=serialization.BestAvailableEncryption(password),
            )
        )
        fobj.write(cert.public_bytes(encoding=serialization.Encoding.PEM))
    subprocess.check_call(
        [
            "openssl",
            "pkc12",
            "-export",
            "-in",
            str(path.with_suffix(".pem")),
            "-passin",
            f"pass:{password.decode()}",
            "-out",
            str(path),
            "-passout",
            f"pass:{password.decode()}",
        ]
    )

Am I using serialize_key_and_certificates wrong?

[reaperhulk]

What happens if you try this using 36.0.2? I wonder if this is related to #7043 (comment)

[ojii]

What happens if you try this using 36.0.2? I wonder if this is related to #7043 (comment)

with 36.0.2 it imports just fine.

[reaperhulk]

Okay, this looks like another manifestation of the OpenSSL 3 transition. Tagging this as a bug we need to get resolved for 38.

[alex]

We should figure out why these aren't loading in macOS, and figure out if there's some action we should be pushing Apple to take -- e.g. are they rejecting valid PKCS#12 files?

[TotallyNotASecret]

@alex I don't think the PKCS#12 files are valid, because the same thing also happened to me with some Windows machines.

[alex]

Unfortunately that's not sufficient to say whether they're valid. Different vendors may have the same bug, or implement the same limited set of OIDs, so we really need an analytic inquiry into the cause of the incompatibility.

[jobec]

#7043 has some openssl commands to troubleshoot those PKCS#12 files.

If a recent version of openSSL can't read them, I think you can be confident that it's a invalid file.

Most likely MacOS has the same issue as windows, where it doesn't support AES256 as encryption for the PKCS#12 file.
On windows 2016 and older, it must be 3DES with SHA1 MAC or it won't work.

[TotallyNotASecret]

At least for me OpenSSL can read the files.

[jobec]

And what's the output from command below?

openssl pkcs12 -noout -info -in some.pfx

With a OpenSSL v3, try converting the PFX to 3DES + SHA1 MAC. Does that work?

openssl pkcs12 -in some.pfx -out temp.pem -nodes
openssl pkcs12 -export -in temp.pem -out legacy_3DES.pfx -macalg SHA1 -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES
del /F /Q temp.pem
openssl pkcs12 -noout -info -in legacy_3DES.pfx

[schwabe]

So I did some testing/work on this and the same seems to be true for Android as well. I can import pkcs12 when I use SHA1 as macalg but it breaks as soon as I change it to SHA256:

openssl3  pkcs12 -passout pass:test123 -export -in etestcerts.pem -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -macalg sha1 -out etest-ossl3-3des-sha1.p12` 

works

openssl3  pkcs12 -passout pass:test123 -export -in etestcerts.pem -keypbe PBE-SHA1-3DES -certpbe PBE-SHA1-3DES -macalg sha256 -out etest-ossl3-3des-sha1.p12`

is broken.

Digging into the OpenSSL source itself suggests that the macalg function is calling

if (!PKCS12_set_mac(p12, mpass, -1, NULL, 0, maciter, macmd)) {`

with macmd being the EVP_MD that is the one specified by the -macalg option. The function seems to be first appeared in OpenSSL 3.0.