embedded lvmd doesn't work for me

[jurim76]

Latest helm chart 12.1.0 allows to run topolvm-node with embedded lvmd

Setting values.custom.yaml

lvmd:                                                                                                                                        
  managed: false 
node:
  lvmdEmbedded: true
  lvmdSocket: /run/topolvm/lvmd.sock
  kubeletWorkDirectory: /var/lib/kubelet

Install helm chart
helm -n topolvm-system install topolvm . -f values.custom.yaml

Get ds status

k get ds topolvm-node 
NAME           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
topolvm-node   4         4         4       4            4           topolvm=true    14m

k get sc
NAME                     PROVISIONER   RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
topolvm-ext4 (default)   topolvm.io    Delete          WaitForFirstConsumer   true                   15m
topolvm-xfs              topolvm.io    Delete          WaitForFirstConsumer   true                   15m

Container run parameters

spec:
  containers:
  - command:
    - /topolvm-node
    - --csi-socket=/var/lib/kubelet/plugins/topolvm.io/node/csi-topolvm.sock
    - --embed-lvmd

Newly created PVCs are in pending state, /run/topolvm/lvmd.sock is absent on the node, topolvm-node pods throw an errors

{"level":"error","ts":"2023-11-18T18:35:36Z","msg":"error on grpc call","method":"/csi.v1.Identity/Probe","error":"rpc error: code = FailedPrecondition desc = not found","stacktrace":"github.com/topolvm/topolvm/pkg/topolvm-node/cmd.ErrorLoggingInterceptor\n\t/workdir/pkg/topolvm-node/cmd/run.go:174\ngithub.com/container-storage-interface/spec/lib/go/csi._Identity_Probe_Handler\n\t/go/pkg/mod/github.com/container-storage-interface/spec@v1.6.0/lib/go/csi/csi.pb.go:5432\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.58.3/server.go:1374\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.58.3/server.go:1751\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.58.3/server.go:986"}

[cupnes]

@jurim76 To reproduce your situation, please tell me the following.

• Are the five lines of yaml in "Setting values.custom.yaml" the full text of values.custom.yaml? If it is part of values.custom.yaml, please send the full text.
• Has the topolvm-node pod status changed to CrashLoopBackOff?

[jurim76]

values.custom.yaml
https://pastebin.com/TNBcj1K0

All topolvm pods are ok and in running state

[cupnes]

@jurim76

https://pastebin.com/TNBcj1K0

lvmdEmbedded is false in this yaml. Is this the file in which this issue occurred?

[jurim76]

Sorry, missed that
The config is the same, but I've changed lvmdEmbedded to true, during the test

[peng225]

@jurim76 Hello.
In your values.yaml file, useLegacy is set to true. In this case the argument of topolvm-node command should look like:

• --csi-socket=/var/lib/kubelet/plugins/topolvm.cybozu.com/node/csi-topolvm.sock

On the other hand, you said that the command is --csi-socket=/var/lib/kubelet/plugins/topolvm.io/node/csi-topolvm.sock. This is strange.

Did you manually edit the manifest of topolvm node DaemonSet?

[jurim76]

Hello @peng225

I will perform another test with topolvm 0.24.0 and topolvm chart 13.0.0

Could you please clarify how to apply these parameters for embedded lvmd?

topolvm_deviceclass:
  - name: encryptedssd
    volume-group: topolvm-vg
    default: true
    spare-gb: 10

[peng225]

@jurim76 Sorry for my late response.
(I had to take some days off due to my health issue.)
I will check and answer your question in a few days.

[peng225]

@jurim76

Could you please clarify how to apply these parameters for embedded lvmd?

You can set deviceclass just same as non-embedded lvmd in values.yaml:

lvmd:
  deviceClasses:
    - name: encryptedssd
      volume-group: topolvm-vg
      default: true
      spare-gb: 10

[jurim76]

Sorry, I've missed that lvmd configmap
Now it works as expected, thank you!