[question] What is the recommended way to manage SSL certificate with conan for windows and linux

[MamaTierra]

What is your question?

Hi ! I'm try to setup jfrog artifactory CE as conan remote with https.
I'm a little confused on the way to handle a self certificate both on linux and windows platform.

Environment details

• Conan version: 2.0.17
• Python version: 3.11.6
• Requests version: 2.31.0

Here what is working for me now:

On Linux:

I managed to add my certificate (.crt format) to system certificates via update-ca-certificates command on debian based distrib and setup conan to use it via core.net.http:cacert_path=/etc/ssl/certs/ca-certificates.crt in .conan2/global.conf

On Windows:

I found no better way to use the python package pip-system-certs that is a runtime patch to python package requests to use system certificate.

Here what is not working for me:

core.net.http:cacert_path=/path/to/my_certificate.crt in .conan2/global.conf give me

ERROR: HTTPSConnectionPool(host='conan.xxx.com', port=443): Max retries exceeded with url: /artifactory/api/conan/xxx/v1/ping (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)')))

My question

What would be the recommended way to manage SSL certificate with conan for windows and linux ?

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[dhuantes]

@MamaTierra I'm dealing with a similar problem at the moment. In Conan 1.x I just placed my certs in the cacert.pem file in the ~/.conan directory and everything worked great. Meaning I was able to successfully authenticate with artifactory. With Conan 2.x this did not work. After a bit of research I got to where you are and updated ~/.conan2/global.conf as you show in your issue post. The difference for me is that I no longer get the HTTPSConnectionPool error.

When I execute this command:
$ conan remote login <my-remote-name> <my-username> -p <MY-ACCESS-TOKEN>

I get this:

ERROR: { "errors" : [ { "status" : 404, "message" : "File not found." } ] }. [Remote: <my-remote-name>]

If I don't provide the access token at the command line then I'm prompted but then the process never returns. I suspect it's giving the same error but it's not being processed so I end up having to kill the terminal session.

$ conan remote login <my-remote-name> <my-username>

Not sure what's going and I was going to post an issue but saw yours and figured I'd share my experience and hopefully this will help JFrog help us both... BTW... This is on Windows 11 and my command line is using git for windows' bash terminal.

[memsharded]

Thanks @MamaTierra and @dhuantes for the feedback.

Quick question, did you manage to use your own cert in Conan 1.X using the cacert_path configuration in conan.conf? Or only adding the certificate to the conan .pem file? It would be useful to know if the cacert_path approach did work in 1.X for your case or not.

To clarify what Conan is doing with this configuration: very little. It takes the cacert_path and passes it to the pyhon-requests library as kwargs["verify"] = self._cacert_path verify as documented in https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification

If we are talking about https://requests.readthedocs.io/en/latest/user/advanced/#client-side-certificates client side certificates, the same applies to the core.net.http:client_cert Conan conf, it is passed to the requests library.

If that is the case, maybe it might be easier and faster to reproduce by using requests library directly?

Regarding pip-system-certs there is a thread discussion about it in #3123

[dhuantes]

@memsharded Thanks for the quick response. I did manage to use my own cert with Conan 1.x by only adding the certificate to the cacert.pem file. Never tried the cacert_path in 1.x

I was able to get it to work using the python requests library and received this response:

<Response [200]>

when I passed the command as follows using the python interactive terminal:

>>> requests.get('https://artificatory.myserver.com/artifactory/api/conan/my-conan-repo', verify='C:/users/me/.conan2/cacert.pem', auth=('my.uid','MyTOKEN'))

But when I use:

conan remote login <remote-name> <my-uid> -p 'MyTOKEN'

I'm still getting

ERROR: { "errors" : [ { "status" : 404, "message" : "File not found." } ] }. [Remote: <remote-name>]

Interestingly enough if I don't pass the auth= argument as shown below:

>>> requests.get('https://artificatory.myserver.com/artifactory/api/conan/my-conan-repo', verify='C:/users/me/.conan2/cacert.pem')

The response I get is
<Response [401]>

I did this test thinking maybe Conan is not providing the credentials properly and I'd get a 404 as shown above but since I got a 401 and 200 when I provided the proper auth credentials there is something in the Conan 2.0 implementation that doesn't seem to be correct.

Hopefully that helps identify the problem. Let me know what else I can do. Thank you.

[dhuantes]

Ok... This is a tad embarrasing but I found that the reason I was getting the 404 was because my remote URL was not matching the URL I was using when calling requests.get outside of Conan.

Specifically, the requests.get was this:

>>> requests.get('https://artificatory.myserver.com/artifactory/api/conan/my-conan-repo', verify='C:/users/me/.conan2/cacert.pem', auth=('my.uid','MyTOKEN'))

and when calling conan remote list the URL for 'remote-name' was showing as seen below:

conancenter: https://center.conan.io [Verify SSL: True, Enabled: True]
remote-name: https://artifactory.myserver.com/artifactory/my-conan-repo [Verify SSL: True, Enabled: True]

With this corrected, I could then successfully execute:

conan remote login <remote-name> <my-uid> -p 'MyTOKEN'

Then I was able to call conan search mylib/1.0.0 where mylib/1.0.0 only existed on my instance of artifactory and it found the recipe/package on my remote and indicated it did not exist on conancenter.

I then proceeded to attempt to replicate this on another development machine. Using only my certificate and I ran into a

[SSL: CERTIFICATE_VERIFY_FAILED] certificate vter erify failed: unable to get local issuer certificate

After a bit of head scratching, I realized that on the machine that I was successful on I had copied the cacert.pem from the original Conan 1.x install and had appended my cert to the end of the file and placed it in the ~/.conan2 directory. But on the system that was failing, the cacert.pem only had my cert in it. When I copied the Conan 1.x cert with my cert appended to it to the system that was failing it was happy again.

So I believe part of the long-term solution to this is to start including the cacert.pem from the Conan 1.x install in the Conan 2.x install. I say this as I have no clue which cert in the cacert.pem from 1.x was the solution. I suspect it was one of the last ones appended to the file but really do not know. But I have to assume there is a cert in there that conancenter expects and that would have been a real challenge to figure out. Hope this helps someone else and the next release. Thanks @memsharded!!!

@MamaTierra look forward to seeing if you can replicate this on Windows and Linux. I'm primarily on Windows but our CI/CD pipeline does both Windows and LInux builds so will be tracking to see if this all applies equally to the Linux environment. If I don't see anything, I'll be sure to update this thread once our CI/CD pipeline is updated and I can confirm this also works on Linux but that probably won't be this week.

[memsharded]

Some feedback: maintaining the old cacert.pem by Conan was something problematic in the past, because it could go outdated. We had some major breaking and outages with all Conan users affected because some certificate expired, and because the cacert.pem was part of Conan, it was not automatically updated (as system certificates or certificates outside of Conan would be), so all users were broken until an update.

We learned that maintaining our own cacert.pem was actually a discouraged practice, so we dropped it in Conan 2.0.

Now it seems that we have a different new problem, which is the UX to add custom certificates. I think this is what we want to improve, lets see what is possible. But most likely going back to the old cacert.pem maintained by the Conan application is not the way to go.

[spectras]

Just posting here in case someone has a similar issue: the solution in the initial question actually works.

On top of that, conan's templating can be leveraged by setting core.net.http:cacert_path = {{conan_home_folder}}/cacert.pem in global.conf. Then dropping our own cacert.pem next to it, as in conan1, works again.

[MattLichter]

Using Conan 2.2.2, I was able to get the original commenter's solution to work on Linux. In my CI build script, I just added

    cp -f "path/to/my-certificate.crt" /usr/local/share/ca-certificates
    update-ca-certificates
    echo "core.net.http:cacert_path=/etc/ssl/certs/ca-certificates.crt" >> ${HOME}/.conan2/global.conf
    conan remote login ${CONAN_REMOTE} ${CONAN_USER} -p ${CONAN_PASS}

However I'm still having trouble on Windows. I can get the certificate installed on the build machine using

    CertUtil -f -addstore "CA" "path\to\my-certificate.crt"
    py -m pip install pip-system-certs
    conan remote login ${CONAN_REMOTE} ${CONAN_USER} -p ${CONAN_PASS}

but I'm still getting the OP's SSLCertVerificationError. It doesn't matter if I add to "CA" vs "Root" on the first line. I suspect the problem is that I'm using the standalone Conan installer (from GitHub release page), rather than pip installing conan. Thus pip-system-certs isn't making a difference. But I much prefer the standalone installer to ensure all our devs are using an identical toolchain.

Can anyone suggest another solution on Windows? The old cacert.pem solution was working for me in Conan 1.x, but I'd like to upgrade to Conan 2 now.

Edit: The Linux solution does not work in Windows, because Windows doesn't store its root certificates in a text file, but rather as a collection of entries in the Windows registry.

[MattLichter]

I implemented a temporary workaround using the old Conan1 solution, using a cacerts.pem file. See #3670 .

[elvisdukaj]

@MattLichter did you solve the problem? I am also struggling with it.

[elvisdukaj]

I solved it, I just put the certificate and renamed to cacert.pem in the conan home folder and this worked without changing the global.conf path.

[MattLichter]

@elvisdukaj I added this line to my global.conf file, verbatim, including the double braces:

core.net.http:cacert_path = {{conan_home_folder}}/cacert.pem

and put my certificate file in ~/.conan2/. This works in both Windows and Linux as long as Linux line endings are used in the cert file. For the cacert file, I just used the one from Conan 1.61.0, and appended my custom certs at the end of the file. I'm using Conan 2.2.2.

[memsharded]

Hi all,

Sorry this has been postponed, it is challenging to track everything...

I'd like to have a follow up from the commenters. It seems that the current approach from Conan 2, which is basically forward the core.net.http:cacert_path to the requests.get(..., verify=cacert_path) can work on both Windows and Linux, and the problem is to provide a cacert.pem file that works correctly, and there are some challenges in it, like making sure about the line endings, etc. It doesn't look like that Conan can do much more in this regard beyond adding some documentation to help?

Can you please comment what is your current status? Did you manage to make it work with that solution, have you tried and still failing?

[MattLichter]

Thanks for the follow-up.

I have a workable solution, described in the comment above #3732 . Works the same way on Linux and Windows, so it's easy to maintain for the team. The downside is that it uses a modified version of the cacerts.pem found in Conan 1.61, so I'm guessing it may go stale at some point.

[memsharded]

@dhuantes
@elvisdukaj
@spectras
@MamaTierra

Any updated feedback considering latest @MattLichter suggestions?

[elvisdukaj]

Last comment fixed my issue 👍

[MamaTierra]

Sorry for the delay, the suggestion from @MattLichter is working fine for me too on both linux and windows.

Here is what I done from scratch in case it can help others:

# Create a container
docker run -it --rm ubuntu:focal /bin/bash

# Install conan 1.61 in the container and add a remote to create the cacert.pem
apt update
apt install -y python python3-pip
pip install conan==1.61
conan remote add remote <my_remote_url>
ls /root/.conan/cacert.pem

# Export the cacert.pem from the container to the host
docker cp <container_id>:/root/.conan/cacert.pem ~/.conan2/cacert.pem

# Append the content of  my_certificate.crt file in the new cacert.pem

# Add the path for the cacert.pem file in global.conf
echo "core.net.http:cacert_path = {{conan_home_folder}}/cacert.pem" >> ${HOME}/.conan2/global.conf

# The conan login is now working fine for both linux and windows
conan remote add remote <my_remote_url>
conan remote login remote <login> -p <password>

Thank you so much for the support !
I close the issue.

[memsharded]

Thanks so much all for the feedback and the hints.

Actually I am going to re-open and move it to the docs repo, to add there some hints based on your advice, I think it can be useful for other users too.