Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Coscheduling] CalculateAssignedPods cost too much time #707

Open
KunWuLuan opened this issue Mar 28, 2024 · 1 comment · May be fixed by #708
Open

[Coscheduling] CalculateAssignedPods cost too much time #707

KunWuLuan opened this issue Mar 28, 2024 · 1 comment · May be fixed by #708
Assignees
@KunWuLuan

Comments

@KunWuLuan
Copy link
Contributor

PodGroup manager will try every node and every pod in CalculateAssignedPods()

scheduler-plugins/pkg/coscheduling/core/core.go

Lines 246 to 264 in cd3e4fb

// CalculateAssignedPods returns the number of pods that has been assigned nodes: assumed or bound.
func (pgMgr *PodGroupManager) CalculateAssignedPods(podGroupName, namespace string) int {
nodeInfos, err := pgMgr.snapshotSharedLister.NodeInfos().List()
if err != nil {
klog.ErrorS(err, "Cannot get nodeInfos from frameworkHandle")
return 0
}
var count int
for _, nodeInfo := range nodeInfos {
for _, podInfo := range nodeInfo.Pods {
pod := podInfo.Pod
if util.GetPodGroupLabel(pod) == podGroupName && pod.Namespace == namespace && pod.Spec.NodeName != "" {
count++
}
}
}
return count
}

When there are 5k or more nodes and 50000+ pods in cluster, CalculateAssignedPods() will cost 30+ms.
We should cache the number of assigned pods to save time in scheduling cycle.
@KunWuLuan
Copy link
Contributor Author

/assign

@KunWuLuan KunWuLuan linked a pull request Mar 28, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@KunWuLuan KunWuLuan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cache assigned pod count KunWuLuan/scheduler-plugins
1 participant
@KunWuLuan