Trying to connect two services inside GCP, need help. Setting tls programmatically.

[mpiorowski]

So i am trying to connect two services running as Google Cloud Run.

I have a valid connection NodeJS Server -> Rust Client, but cannot make a connection between Rust Server -> Rust Client.
I am getting error:
Error: status: Unknown, message: "transport error", details: [], metadata: MetadataMap { headers: {} }

So e few thing i think may be related:

1. I need to add authorization header, and right now this is a way i am doing it:

let request = Request::from_parts(
    metadata.clone(),
    Default::default(),
    UserId {
        user_id: note.user_id.to_owned(),
    },
);

Where metadata is:
MetadataMap { headers: {"authorization": "Bearer xxxx" } }

2. On nodejs server i needed to create ssl credentials, and i don't see anything related here in rust. Maybe that's the problem?

[mpiorowski]

I think i am missing tls/ssl setup, but what i did

let channel = channel_users
            .tls_config(tonic::transport::ClientTlsConfig::new())
            .context("Failed to create tls config to users service")?
            .connect()
            .await
            .context("Failed to connect to users service")?;

Throw me an error

error trying to connect: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

[mpiorowski]

So this is how gcp do is using go:

func NewConn(host string, insecure bool) (*grpc.ClientConn, error) {
        var opts []grpc.DialOption
        if host != "" {
                opts = append(opts, grpc.WithAuthority(host))
        }

        if insecure {
                opts = append(opts, grpc.WithInsecure())
        } else {
                // Note: On the Windows platform, use of x509.SystemCertPool() requires
                // go version 1.18 or higher.
                systemRoots, err := x509.SystemCertPool()
                if err != nil {
                        return nil, err
                }
                cred := credentials.NewTLS(&tls.Config{
                        RootCAs: systemRoots,
                })
                opts = append(opts, grpc.WithTransportCredentials(cred))
        }

        return grpc.Dial(host, opts...)
}

And i will try do something similiar using https://github.com/est31/rcgen

[mpiorowski]

Next step using rcgen, still not working :( Now i am getting: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

        let subject_alt_names = vec![
            "xxx".to_string(),
            "localhost".to_string(),
        ];
        let cert = generate_simple_self_signed(subject_alt_names)
            .unwrap()
            .serialize_pem()
            .unwrap();
        let server_cert = cert.as_bytes();
        let tonic_cert = Certificate::from_pem(server_cert);
        let tls = ClientTlsConfig::new()
            .ca_certificate(tonic_cert)
            .domain_name("xxxx");
        let channel = channel_users
            .tls_config(tls)
            .context("Failed to create tls config to users service")?
            .connect()
            .await
            .context("Failed to connect to users service")?;

[mpiorowski]

This is a perfect example of what i am struggling with ;p
https://jessitron.com/2022/11/02/make-https-work-on-grpc-in-rust-load-a-root-certificate-into-the-tls-config/

[mpiorowski]

All i need to do was to add "tls-roots" to features........omg, such a simple solution......
And everything works with one simple line:
let users_conn = UsersServiceClient::connect("xxxx").await?;

[wspeirs]

I have HTTP2 enabled on the gRPC server, disabled on the client. I have features = ["tls", "tls-roots"] for tonic on the client, and no features specified on the server.

I'm only using DataAccessClient::connect(BACKEND_URI.as_str()).await? to setup the client (ie, not setting up a channel), but I'm still getting the "invalid peer certificate: UnknownIssuer" problem. I'm assuming you don't have code/example on GitHub?

[pascualex]

It's a company repo. But I can confirm I'm connecting the same way on the client. I don't see any relevant snippet on the server either.

If you are using a minimal Docker image to deploy your client server, you might be missing the certificates (not related to Axum).

I solved that for a different task prior to this, on my Dockerfile (we are using Debian):

RUN apt update && apt install -y ca-certificates

[wspeirs]

I have that in my client Docker file as well:

FROM debian:12-slim

RUN apt update
RUN apt install -y ca-certificates

I have enabled my server to listen on the internet, and can hit it from my machine via grpc_health_probe, but still fails during the startup of my client container. I was going over a VPC, but have disabled all of that to just try and get this working. Def going to make a blog post somewhere when I finally crack this :-)

[pascualex]

Haha it's certainly very weird. I would try to run the client from my PC without Docker (enabling public access for the gRPC server) just to discard any error that can occur on the building and deployment stages.

[wspeirs]

I got it working with the gRPC server on the Internet, and adding tls-webpki-roots on the client side. However, I think my real issue was that gcloud run services replace service.yaml wasn't actually doing anything, just redeploying the same broken image. I notice that println messages I was putting in my client code weren't changing... it tough to get something working when you aren't actually changing the code :-( Thanks for the help!