Storageclass to device class mapping understanding

[druesendieb]

Hi topolvm folks,

We're currently testing adding new nodes with a different device class to our already existing clusters. At first we thought about adding a second storage class as per our understanding, storage class and device class are a 1:1 mapping. That's also been stated in this discussion here: #750

But during our testing we found this not to be the case. Configuring 2 lmd daemon sets with each having 1 device class with default: true set, allowed us to use only one storage class (without additionalParameters set to the device-class) to provision on all nodes with both device classes.

Is this wanted behaviour or a bug?

PoC Setup:

• lvmd.managed: true
• scheduler.enabled: true
• storageCapacityTracking.enabled: false
• webhook.podMutatingWebhook: true
• webhook.pvcMutatingWebhook: true

4 baremetal nodes

• 3 (1-3) nodes with disktype: thick set as label in k8s, the device class is called vg1
• 1 (4) node with disktype: thin as label in k8s, the device class is called vg1-thin

2 storage classes:

• the "old" one called topolvm without additionalParameters set
• the additional one called topolvm-thin with additionalParameters set to

      additionalParameters:
        "topolvm.io/device-class": "vg1-thin"

lvmd config:

• lmd-0 daemonset to run on nodes with disktype: thick
• additionalConfig: for lmd-1 has been added to run on nodes with disktype: thin using device class vg1-thin
• Both device classes have default set to true
• as this initially crashed the pods (initially we tested on 0.20, updated to 0.26 for the PoC after, the problem with the crashes has then been resolved)

Outcome

Expected

• PVC for storageclass topolvm will be created on node 1-3
• PVC for storageclass topolvm-thin will be created on node 4 only

Reality

• PVC for storageclass topolvm will be created on node 1-4
• PVC for storageclass topolvm-thin will be created on node 4 only

Troubleshooting

Approach for testing

Cordoning a whole set of of nodes (thick/thin), then applying a pod consuming a volume from the same storageclass

Observations

• Setting additionalParameters and the correct device class in the storage class allows only 1:1 mappings
• Using a lvmd-1 config with default: false for the vg1-thin node disabled scheduling of topolvm volumes on node 4 as well
• Using a lvmd-1 config with default: true for both device classes allowed to use the topolvm storage class to schedule on all nodes.

[cupnes]

Since the deviceclass is not set in storageclass topolvm, it seems obvious. If you want the behaviour you described in "Expected", you can set the deviceclass using additionalParameters in the storageclass topolvm.

[druesendieb]

We have a quite specific use case where we schedule stateful sets based on nodeselectors. So if I don't care which topolvm backed device class serves me on those nodes, using only one storageclass is a safe option?

[cupnes]

Yes, topolvm behaves as such.