Possible way on Cephfs data clone & capacity adjustment

[ycsong1212]

Hi~ I want to ask for help. I need to provide the ability that adjusts the capacity of user's data. My csi version is 3.8.0. However, kubernetes pvc does not support reducing the capacity. Therefore, I consider several ways could be chose. First, I could make a pvc clone of an existed pvc, referring to the doc under examples dir, but it is still forbidden, the result is as follow

Besides, I think maybe I could create a PV specifying the subvolume & subvolumePath that already exists on Ceph. After this, I could create a pvc binding with pv statically instead of dynamically binding with storage pvc. However, I have found nothing docs guiding me how to do this.
Last, maybe I have to create a new pvc and make a data transfer from the old pvc to the new pvc.
Is there any good ways I could choose to support my requirement. I'll appreciate it if you could help~

[Madhu-1]

@ycsong1212 whatever you mention can be done, you can follow the below steps for it

• Copy one of the PV objects and remove the claimRef and also set the ReclaimPolicy to Retain (not sure it's required)
• Make the request adjustments related to subvolume
• Create the PV
• Create the PVC with the PV name as the volumeName
• Update the ReclaimPolicy back to Delete

The above steps should work, please let me know if you face any issues.

[ycsong1212]

@ycsong1212 whatever you mention can be done, you can follow the below steps for it

• Copy one of the PV objects and remove the claimRef and also set the ReclaimPolicy to Retain (not sure it's required)
• Make the request adjustments related to subvolume
• Create the PV
• Create the PVC with the PV name as the volumeName
• Update the ReclaimPolicy back to Delete

The above steps should work, please let me know if you face any issues.
Hi~ I have tried referring to the way you have provided, but it doesn't work. The PV & PVC could be bounded successfully but it cannot be mounted in the pod, the result is as follow

The volume have already exists but it could not be seen so it cannot be mounted on my pod.
My pv & pvc config file is as follow

Thanks for ur help～

[Madhu-1]

@ycsong1212 can you also paste the subvolume info output for this one?

[ycsong1212]

@ycsong1212 can you also paste the subvolume info output for this one?

Is this information provided by pv what you want?

[Madhu-1]

can you run ceph fs subvolume info in ceph for this subvolume and provide the details?

[ycsong1212]

Hi~ I have checked it and it has already been deleted. Another question has been found. If a cephfs dir is used by two or more kubernetes pv and I delete one pv, is it expected that the corresponding subvolume is deleted?

[Madhu-1]

Yes correct, if the PV/PVC is deleted the corresponding subvolume will also gets deleted unless PV is using RetainPolicy. if the subvolume is not present nothing can be done. you are getting expected error

[ycsong1212]

Text is here:
volumeAttributes:
clusterID: 3ea02dcb-9ced-40cd-b2b1-dc4b64ab61b5
fsName: cephfs_aip
storage.kubernetes.io/csiProvisionerIdentity: 1685354196389-8081-cephfs.csi.ceph.com
subvolumeName: csi-vol-b1348f25-afb4-4022-9680-ff4469ff8efa
subvolumePath: /volumes/csi/csi-vol-b1348f25-afb4-4022-9680-ff4469ff8efa/7e3f0869-b624-4801-8e5e-98069c17f897

[ycsong1212]

Hi~I have made some attempts but still encountered the same issue. I have changed the policy from 'Delete' to 'Retain', here is my pv info & pvc info


The same pod issue is as follow

This time I am sure the data in cephfs already exists, the subvolume info is as follow

Thanks for ur help～

[Madhu-1]

It looks like the volumeHandle is not updated to use the subvolume name, it should be as below

csi:
        clusterID: rook-ceph
        fsName: myfs
        pool: myfs-replicated
        storage.kubernetes.io/csiProvisionerIdentity: 1706537996583-9998-rook-ceph.cephfs.csi.ceph.com
        subvolumeName: csi-vol-b1eae220-3d52-4923-95ab-eca3a0e29784
        subvolumePath: /volumes/csi/csi-vol-b1eae220-3d52-4923-95ab-eca3a0e29784/c932dfbd-b0dd-4533-b2a8-0d36d1ea4928
      volumeHandle: 0001-0009-rook-ceph-0000000000000001-b1eae220-3d52-4923-95ab-eca3a0e29784

The last part of the volume Handle b1eae220-3d52-4923-95ab-eca3a0e29784 is the name of the subvolume without csi-vol-

[ycsong1212]

Yeah~ I have tried it and it works, thanks a lot~