You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our Gardener Cluster running on OpenStack runs in reconciliation issues because it is not detaching volumes from shoot workers.
/area storage
/topology shoot
/kind bug
What happened:
When an instance in a shoot is deleted, the volume detachment is not finalized before deletion of the shoot worker node. Thus, in some cases, the volume will still be counted as attached and the shoot cluster is thus running into reconciliation errors.
What you expected to happen:
An instance will only ever be deleted after all cinder volumes are functionally detached from the Instance.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
Gardener version: 1.76.3
Kubernetes version (use kubectl version): 1.24.5
Cloud provider or hardware configuration: OpenStack
The text was updated successfully, but these errors were encountered:
On instance (by instance I assume an OS server/node), the MCM should try to drain the node before attempting to delete it. That should take care of moving most workloads out of the node, including moving the volumes after their pods have been scheduled elsewhere. Did you see issues during this process ?
Were the volumeattachments not being deleted, or maybe where there issues with CSI preventing the detach ?
I am sorry, i cannot provide more context, as this ticket is already me describing a blackbox ;)
Given that the mechanism you described should take care of detaching/migrating all existing resources attached to and scheduled on that node I think this problem might be more on openstack side than on gardener side...
This is something that usually happens during and after Openstack upgrades.
How to categorize this issue?
Our Gardener Cluster running on OpenStack runs in reconciliation issues because it is not detaching volumes from shoot workers.
/area storage
/topology shoot
/kind bug
What happened:
When an instance in a shoot is deleted, the volume detachment is not finalized before deletion of the shoot worker node. Thus, in some cases, the volume will still be counted as attached and the shoot cluster is thus running into reconciliation errors.
What you expected to happen:
An instance will only ever be deleted after all cinder volumes are functionally detached from the Instance.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
kubectl version
): 1.24.5The text was updated successfully, but these errors were encountered: