pkg/drain: Provide hooks for intermediate eviction errors

[ahmetb]

What would you like to be added

Currently https://pkg.go.dev/k8s.io/kubectl/pkg/drain offers hooks that callers can use to get notified when a Pod eviction is starting or finished. For example:

// OnPodDeletionOrEvictionFinished is called when a pod is eviction/deletetion is failed;
// for printing progress output
OnPodDeletionOrEvictionFinished func(pod *corev1.Pod, usingEviction bool, err error)

This method signature (and Godoc) made me think my code will get intermediate errors in the err variable that I can use to make decisions, or print progress.

However, that did not turn out to be the case.

Currently, the OnPodDeletionOrEvictionFinished() is only called when:

1. Evict or Delete Pod call succeeded + polling for Pod to be deleted succeeds.
2. Evict or Delete Pod call succeeded + there was an error in polling for waiting Pod to be fully deleted.

Proposal

Extend the semantics around OnPodDeletionOrEvictionFinished() hook to include intermediate errors.

Why is this needed

1. If my code gets an err that is of code=429 (indicating PDB disallowing eviction), I can cancel() the ctx I passed to RunNodeDrain(ctx), because waiting for many more minutes and retrying in 5s likely won't fix the issue.

2. The documentation of OnPodDeletionOrEvictionFinished() hook says for printing progress output. However, if the drain operation is currently failing due to a single Pod refusing to Evict (say, due to PDB), this can provide the callers with an option to print progress

   Currently, this hook only works for printing progress if the eviction is successful, which makes it basically the same as OnPodDeletedOrEvicted func(pod *corev1.Pod, usingEviction bool) hook.

/kind feature
/sig cli
/cc @adilGhaffarDev
/cc soltysh

[ardaguclu]

After looking at the signature of OnPodDeletionOrEvictionFinished, I've inferred the same that function also errors out intermediate errors. In my opinion, it makes sense to extend it, unless there won't be any problem in terms of backwards-compatibility. As far as I understand, err error in hook does not return anything?.

[mpuckett159]

/triage accepted