Segfault on musl when running in alpine

[p-alik]

Current and expected behavior

This code

fn main() {
    tokio::runtime::Builder::new_current_thread()
        .enable_all()
        .build()
        .unwrap()
    .block_on(async {
    match kube::client::Client::try_default()
            .await {
              Ok(_) => (),
              Err(error) => eprintln!("{error:?}")
            }
            // .expect("Expected a valid KUBECONFIG environment variable.");
    });
}

compiles well, but the binary crashes. See attachet backtrace.txt

20:05 $ rustc --version
rustc 1.66.0-nightly (758f19645 2022-10-24)
20:01 $ OPENSSL_DIR="/home/apastuchov/git/openssl/openssl/build" RUSTFLAGS="-C target-feature=+crt-static" cargo run --bin seg-fault --target x86_64-unknown-linux-musl

Possible solution

No response

Additional context

No response

Environment

19:58 $ kubectl version --output json
{
  "clientVersion": {
    "major": "1",
    "minor": "25",
    "gitVersion": "v1.25.2",
    "gitCommit": "5835544ca568b757a8ecae5c153f317e5736700e",
    "gitTreeState": "clean",
    "buildDate": "2022-09-21T14:33:49Z",
    "goVersion": "go1.19.1",
    "compiler": "gc",
    "platform": "linux/amd64"
  },
  "kustomizeVersion": "v4.5.7",
  "serverVersion": {
    "major": "1",
    "minor": "23",
    "gitVersion": "v1.23.9",
    "gitCommit": "c1de2d70269039fe55efb98e737d9a29f9155246",
    "gitTreeState": "clean",
    "buildDate": "2022-07-13T14:19:57Z",
    "goVersion": "go1.17.11",
    "compiler": "gc",
    "platform": "linux/amd64"
  }
}

Configuration and features

k8s-openapi = { version = "0.16", default-features = false, features = ["v1_25"] }
kube = { version = "0.75.0", default-features = true, features = ["derive", "runtime", "client", "rustls-tls"] }

Affected crates

No response

Would you like to work on fixing this bug?

maybe

[p-alik]

default-features = true is the root cause of the issue because openssl-tls is default feature for [kube-client|https://github.com/kube-rs/kube/blob/da6b5e7b963bd6f72190a23b428abdf5e321141d/kube-client/Cargo.toml#L19].
Changing default-features = false resolves the issue.

Consider to check "mutually exclusive"

✔ ~/git/github/kube-rs/kube [main|✚ 1] 
21:34 $ git diff
diff --git a/kube-client/src/lib.rs b/kube-client/src/lib.rs
index 03382bd..99166aa 100644
--- a/kube-client/src/lib.rs
+++ b/kube-client/src/lib.rs
@@ -1,3 +1,5 @@
+#[cfg(all(feature = "rustls-tls", feature = "openssl-tls"))]
+compile_error!("Feature rustls-tls and openssl-tls are mutually exclusive and cannot be enabled together");
 //! Crate for interacting with the Kubernetes API
 //!
 //! This crate includes the tools for manipulating Kubernetes resources as

[clux]

Hm, we used to have a static_assertion that we had exactly one of these features selected, but we might have taken that out once we made the client optional.

We should probably reintroduce the XOR check when the client feature is set. Thanks for the report.

[kazk]

I think this is a duplicate of #869 (comment) . OpenSSL is used by the default client when both are enabled. It's also documented to disable the default features when using rustls.

we might have taken that out once we made the client optional.

We removed the requirement to have TLS feature enabled when client feature is enabled, and also allowed having multiple TLS features (#540).

I haven't tested the current version, but we have an example changing it at runtime that worked at the time.

kube/examples/custom_client_tls.rs

Lines 1 to 3 in a7861aa

	// Custom client supporting both openssl-tls and rustls-tls
	// Must enable `rustls-tls` feature to run this.
	// Run with `USE_RUSTLS=1` to pick rustls.

[t3hmrman]

Just ran into this -- taking a look at #869 (comment) and switching to a debian base build solved the issue for me.

The failure mode I was seeing was that Client::incluster() was causing a panic & crash when built from alpine but not when built from Debian.

Having both TLS methods turned on at the same time (openssl via default features, rust TLS via explicit feature addition) didn't cause a problem for me (and didn't break locally obviously), but when running a container the application would start up and fail.

[clux]

Yeah, alpine is sometimes problematic if you're not doing the musl build correctly. We do use distroless (static) in controller-rs from static exe and in version-rs using muslrust as a builder so people can follow those build setups for minimal images. Debian/official rust build is usually harder to get wrong though.

I'm going to close this as a duplicate of #869, but will start writing a bit about container builds on kube-rs/website.