Host mode networking hardcoded to true in ceph-csi chart

[Starttoaster]

Describe the bug

If you look at the helm chart yaml template, it hardcodes an option that turns on host mode networking for the ceph csi daemonset: https://github.com/ceph/ceph-csi/blob/devel/charts/ceph-csi-rbd/templates/nodeplugin-daemonset.yaml#L32

I thought this was strange, so I checked the values file, and it would seem like the intention was for that to be configured in helm values: https://github.com/ceph/ceph-csi/blob/devel/charts/ceph-csi-rbd/values.yaml#L180

Environment details

• Image/version of Ceph CSI driver : latest
• Helm chart version : latest
• Kernel version : n/a
• Mounter used for mounting PVC (for cephFS its fuse or kernel. for rbd its
  krbd or rbd-nbd) : n/a
• Kubernetes cluster version : n/a
• Ceph cluster version : n/a

[Starttoaster]

Oh, I'm actually mistaken. That configuration option in the helm values file is given for the provisioner deployment. Is there a purpose for it to be hardcoded for host mode networking for the nodeplugin daemonset, then? Seems odd, because the only container port in there is for a metrics endpoint, which I'm assuming is for Prometheus (this is my first time getting started with this CSI provider.) Seems like a networking detail that should/could be left to the user, to my lay man eyes.

[Madhu-1]

CSI needs to use the hostnetworking to avoid problems with mounts when the node-plugin pod restarts. if we use pod networking for nodeplugin pod, when a CephFS/RBD volume is mounted in a pod using Ceph CSI and then the CSI CephFS/RBD plugin is restarted or terminated (e.g. by restarting or deleting its DaemonSet), all operations on the volume become blocked, even after restarting the CSI pods. The only workaround is to restart the node where the Ceph CSI plugin pod was restarted. we don't want the users to have this pain so default hardcoded value is hostnetworking for csi pods.