How to access the virt-launchers libvirt socket for third party applications?

[abbbi]

hi,

Im searching for a sane way[2] to access the virt-launchers libvirtd socket from a sidecar pod.
My use case here is to be able to use the libvirt/qemu incremental backup features[0]

As i understand there are sidecar hooks that allow for patching the created libvirt xml during launch, but i fail to
understand how i would be able to access the libvirt daemon via socket from a pod in the same deployment.

The scenario would be:

• launch virtual machine via kubevirt
• spin up an sidecar pod that has access to the libvirt daemons socket (as in: beeing able to "virsh -c qemu:///session" (by sharing (run/libvirtd?)
• the sidecar periodically creates backups using virtnbdbackup[1], storing them on a PVC

Im pretty new to kubevirt and k8 in general, any help how to archive this is highly appreciated!

[0] https://libvirt.org/kbase/live_full_disk_backup.html (https://libvirt.org/formatbackup.html)
[1] https://github.com/abbbi/virtnbdbackup
[2] abbbi/virtnbdbackup#178

[alicefr]

@abbbi the answer is simple, it is because we discourage any direct access to libvirt and it doesn't follow the KubeVirt API, and it might leads to the an unexpected behavior.

For your particular use-case, you should rely on the CSI snapshot, not on QEMU snapshot. This is fundamental design decision for KubeVirt. Your disk image should be expanded to fill the entire PVC, so you might not have enough space left on the PVC for the snapshot.
Doesn't your storage provider support snapshots?

[alicefr]

If you still want to access it. Sidecars have the pvc option with a path that can be shared between the sidecar and the compute container (unfortunately, the only example we have is this how-to-start-qemu-launched-by-a-debugging-tool-eg-strace). You could try to configure libvirt to start the socket in the shared path

[abbbi]

hi,

thanks for the reply,

For your particular use-case, you should rely on the CSI snapshot, not on QEMU snapshot. This is fundamental design decision for KubeVirt. Your disk image should be expanded to fill the entire PVC, so you might not have enough space left on the PVC for the snapshot. Doesn't your storage provider support snapshots?

The new libvirt backup API does not use snapshots, but uses changed block tracking in the qemus block storage layer (dirty bitmaps). These bitmaps can then be exposed via NBD. If the volumes are QCOW based, this allows for incremental backups without having to scan the storage for changes. In case the volumes are RAW (which seems to be the case with kubevirt) it would at least be possible to create full backups.

[alicefr]

hi,

thanks for the reply,

For your particular use-case, you should rely on the CSI snapshot, not on QEMU snapshot. This is fundamental design decision for KubeVirt. Your disk image should be expanded to fill the entire PVC, so you might not have enough space left on the PVC for the snapshot. Doesn't your storage provider support snapshots?

The new libvirt backup API does not use snapshots, but uses changed block tracking in the qemus block storage layer (dirty bitmaps). These bitmaps can then be exposed via NBD. If the volumes are QCOW based, this allows for incremental backups without having to scan the storage for changes. In case the volumes are RAW (which seems to be the case with kubevirt) it would at least be possible to create full backups.

Even with CBT, the main idea doesn't change. We don't use block jobs from QEMU. There's been some ideas to use the qemu-storage-daemon to separate the qemu storage feature and CSI. However, this is hard to integrate and there's only been some experimentations, not a real plan.
Unfortunately, CSI still doesn't support CBT (see the KEP)

Otherwise, yes, KubeVirt uses raw images on PVC.

[alicefr]

What I want to make clear is that accessing the libvirt daemon might just be the tiny visible part of the iceberg. You need to consider how you are going to manage the storage in k8s.