Issue imporitng certificates created using OpenSSL::PKCS12 on macOS devices

[ravinderrana]

Hi!

Am having issue with certificates created using OpenSSL::PKCS12 which need to be installed on macOS devices.

Ruby Version: 3.2.2
System OpenSSL Version: OpenSSL 3.0.2 15 Mar 2022
Ruby OpenSSL Gem Version: 3.1.0

Code:

private_key = OpenSSL::PKey::RSA.new 2048
x509_cert = OpenSSL::X509::Certificate.new
x509_cert.public_key = private_key.public_key
signed_cert = x509_cert.sign(private_key, OpenSSL::Digest::SHA256.new)
File.binwrite('/tmp/my-certificate.p12', OpenSSL::PKCS12.create('123456', 'My Cert', private_key, signed_cert).to_der)

Info dump from “openssl pkcs12 -info -in /tmp/my-certificate.p12” shows that the certificate is generated with:

MAC: sha256, Iteration 2048
MAC length: 32, salt length: 8

Seems macOS have issues with hash function HMAC and only SHA1 is supported. Using older version of either Ruby's OpenSSL gem (tested with 2.1.4) or older System's OpenSSL version (Tested with OpenSSL 1.1.1f 31 Mar 2020) seems to work as it generates certificate with:

MAC: sha1, Iteration 1
MAC length: 20, salt length: 8

Following didn't worked as well:

File.binwrite('/tmp/my-certificate.p12', OpenSSL::PKCS12.create('123456', 'My Cert', private_key, signed_cert, nil, "PBE-SHA1-3DES", "PBE-SHA1-RC2-40", nil, 1).to_der)

What should we use to generate the certificate with MAC as SHA1 and iteration 1. Any help will be highly appreciated.

[ravinderrana]

Found similar issue reported for one of Python's cryptography package

There they've added support to set different PBES choices as well as set KDF rounds and MAC algorithm. Here's the relevant PR

Not sure if we need similar approach here or if there is any alternative that we can go with.

[rhenium]

Following didn't worked as well:

File.binwrite('/tmp/my-certificate.p12', OpenSSL::PKCS12.create('123456', 'My Cert', private_key, signed_cert, nil, "PBE-SHA1-3DES", "PBE-SHA1-RC2-40", nil, 1).to_der)

The issue is with RC2. OpenSSL::PKCS12.create fails because OpenSSL >= 3.0 implements RC2 in its legacy provider and it's not enabled by default.

Please see #611.