Add Indexed completionMode to Job API #98441

alculquicondor · 2021-01-26T18:13:57Z

What type of PR is this?

/kind feature
/kind api-change

/sig apps
/area workload-api/job
/area batch

What this PR does / why we need it:

Add the .spec.completionMode field with accepted values NonIndexed (default) and Indexed.

Also requires a feature gate IndexedJob and the .status.completedIndexes field in Job.

Which issue(s) this PR fixes:

Ref #97169 #14188 kubernetes/enhancements#2214

Special notes for your reviewer:

This PR only contains API changes. Changes to Job controller will follow up in another PR.

Does this PR introduce a user-facing change?:

Added `.spec.completionMode` field to Job, with accepted values `NonIndexed` (default) and `Indexed`. This is an alpha field and is only honored by servers with the `IndexedJob` feature gate enabled.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: http://git.k8s.io/enhancements/keps/sig-apps/2214-indexed-job
- [Usage]: TODO

alculquicondor · 2021-01-26T18:14:30Z

/hold
for kubernetes/enhancements#2245 to be merged

alculquicondor · 2021-01-26T20:38:10Z

/retest

alculquicondor · 2021-01-27T14:28:44Z

/assign @soltysh
for sig-apps review

alculquicondor · 2021-01-27T14:29:21Z

Keeping the hold until implementation PR (TODO) is also approved

alculquicondor · 2021-01-27T16:48:10Z

cc @ahg-g

ahg-g · 2021-02-02T14:40:36Z

pkg/apis/batch/types.go

+// CompletionMode specifies how Pod completions of a Job are tracked.
+type CompletionMode string
+
+const (


Interesting that the comment describes completion from the pod perspective. Since the mode is a job level parameter, it might be clearer if we describe it from the job perspective?

Any suggested wording?

Perhaps:

NonIndexedMode: a job is completed when the number of successfully completed pods is equal to Completions, if completions is nil, then the success of any pod signals the success of the job.

IndexedMode: a job is completed when at least one pod for each unique index between 0 to Completions - 1 successfully completed.

When we start describing the mode from the perspective of the job, I think it becomes more important to document the restrictions on Completions/Parallelism values here.

Sadly, comments for constants don't make it to the reference documentation https://kubernetes.io/docs/reference/kubernetes-api/

So leaving the comments here lean and putting all details in the field.

oh, that is bad!

ahg-g · 2021-02-02T15:16:01Z

pkg/apis/batch/types.go

+	NonIndexedCompletion CompletionMode = "NonIndexed"
+
+	// IndexedCompletion means that each Pod completion of a Job is tracked
+	// individually, being associated to a completion index.


are there restrictions on what values Parallelism and Completions can take when using this mode?

Yes, but I'm keeping them in the field, as that's where people will take a look first.

may be not in this case, but a field can exist in multiple places, and so we wouldn't repeat the description of each value on each field. The documentation is for the type, not really for the field.

ahg-g · 2021-02-02T15:17:45Z

pkg/apis/batch/types.go

+	// more consecutive numbers are compressed and represented by the first and
+	// last element of the series, separated by a hyphen.
+	// For example, if the completed indexes are 1, 3, 4, 5 and 7, they are
+	// represented as "1,3-5,7".


mention the limit on this field if there is one.

There is no limit here. It's just bounded by parallelism.

ahg-g · 2021-02-02T15:18:58Z

pkg/apis/batch/types.go

+	//
+	// This field is alpha-level and is only honored by servers that enable the
+	// IndexedJob feature gate. More completion modes can be added in the future.
+	// If the Job controller observes a mode that it doesn't recognize, it manages


why not reject it at validation?

We do reject it at validation. But an apiserver can temporarily be at a newer version than the controller. Then the controller might see a mode that it doesn't recognize.

This note is a safeguard to allow the addition of new modes in the future. In production clusters, the situation described wouldn't happen, as we first release new modes with a disabled feture gate.

But actually, it's safer to skip the sync. Changing.

ahg-g · 2021-02-02T15:23:45Z

pkg/apis/batch/validation/validation.go

@@ -31,6 +31,8 @@ import (
 	apivalidation "k8s.io/kubernetes/pkg/apis/core/validation"
 )

+const maxParallelismForIndexedJob = 100000


add a comment explaining why parallelism is now capped.

ahg-g · 2021-02-05T14:18:28Z

looks good to me

alculquicondor · 2021-02-22T21:47:09Z

/retest

alculquicondor · 2021-02-23T22:30:54Z

/retest

ahg-g · 2021-03-02T18:31:36Z

/retest

ahg-g · 2021-03-02T19:41:34Z

/retest

lavalamp · 2021-03-02T22:51:18Z

/approve
for pkg/registry

k8s-ci-robot · 2021-03-02T22:51:49Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alculquicondor, erictune, lavalamp, soltysh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~api/OWNERS~~ [erictune,lavalamp]
~~pkg/apis/OWNERS~~ [erictune,lavalamp]
~~pkg/features/OWNERS~~ [lavalamp,soltysh]
~~pkg/registry/OWNERS~~ [lavalamp]
~~staging/src/k8s.io/api/OWNERS~~ [erictune,lavalamp]
~~staging/src/k8s.io/kubectl/OWNERS~~ [lavalamp,soltysh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ahg-g · 2021-03-03T02:13:59Z

/lgtm

fejta-bot · 2021-03-03T08:52:17Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

ahg-g · 2021-03-03T13:03:17Z

/retest

ahg-g · 2021-03-03T14:03:23Z

/retest

And IndexedJob feature gate, disabled by default. Update JobDescriber

alculquicondor · 2021-03-03T19:18:02Z

/retest

ahg-g · 2021-03-03T20:08:53Z

/lgtm

k8s-ci-robot requested review from dims and hongchaodeng January 26, 2021 18:14

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 26, 2021

alculquicondor force-pushed the array-job-api branch from 1ef63c6 to 778da29 Compare January 26, 2021 19:38

k8s-ci-robot assigned soltysh Jan 27, 2021

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 30, 2021

alculquicondor force-pushed the array-job-api branch 2 times, most recently from 447128f to 0ee7ce4 Compare February 1, 2021 21:41

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 1, 2021

ahg-g reviewed Feb 2, 2021

View reviewed changes

alculquicondor force-pushed the array-job-api branch from 89a0d09 to 5f364c5 Compare February 5, 2021 15:51

alculquicondor mentioned this pull request Feb 5, 2021

Indexed job implementation #98812

Merged

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 19, 2021

alculquicondor force-pushed the array-job-api branch from 45865c3 to 5ed5e6c Compare February 19, 2021 20:43

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 19, 2021

alculquicondor mentioned this pull request Feb 23, 2021

Add Job.status.uncountedTerminatedPods for Job tracking #98817

Merged

erictune approved these changes Feb 23, 2021

View reviewed changes

alculquicondor force-pushed the array-job-api branch from 5ed5e6c to fa66b11 Compare March 1, 2021 20:24

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 2, 2021

k8s-ci-robot assigned ahg-g Mar 3, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 3, 2021

Add Job.spec.completionMode and Job.status.completedIndexes

a1a5868

And IndexedJob feature gate, disabled by default. Update JobDescriber

alculquicondor force-pushed the array-job-api branch from fa66b11 to a1a5868 Compare March 3, 2021 15:16

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 3, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 3, 2021

k8s-ci-robot merged commit 6059a67 into kubernetes:master Mar 3, 2021

k8s-ci-robot added this to the v1.21 milestone Mar 3, 2021

liggitt moved this from Assigned to API review completed, 1.21 in API Reviews Mar 4, 2021

liggitt mentioned this pull request Mar 12, 2021

Only default Job fields when feature gates are enabled #100188

Merged

csuzhangxc mentioned this pull request Jul 13, 2023

bump K8s to v1.21.14 pingcap/tidb-operator#5170

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Indexed completionMode to Job API #98441

Add Indexed completionMode to Job API #98441

alculquicondor commented Jan 26, 2021 •

edited by liggitt

alculquicondor commented Jan 26, 2021

alculquicondor commented Jan 26, 2021

alculquicondor commented Jan 27, 2021

alculquicondor commented Jan 27, 2021

alculquicondor commented Jan 27, 2021

ahg-g Feb 2, 2021

alculquicondor Feb 2, 2021

ahg-g Feb 2, 2021

alculquicondor Feb 2, 2021

ahg-g Feb 2, 2021

ahg-g Feb 2, 2021

alculquicondor Feb 2, 2021

ahg-g Feb 2, 2021

ahg-g Feb 2, 2021

alculquicondor Feb 2, 2021

ahg-g Feb 2, 2021

alculquicondor Feb 2, 2021

ahg-g Feb 2, 2021

alculquicondor Feb 2, 2021

ahg-g commented Feb 5, 2021

alculquicondor commented Feb 22, 2021

alculquicondor commented Feb 23, 2021

ahg-g commented Mar 2, 2021

ahg-g commented Mar 2, 2021

lavalamp commented Mar 2, 2021

k8s-ci-robot commented Mar 2, 2021

ahg-g commented Mar 3, 2021

fejta-bot commented Mar 3, 2021

ahg-g commented Mar 3, 2021

ahg-g commented Mar 3, 2021

alculquicondor commented Mar 3, 2021

ahg-g commented Mar 3, 2021

Add Indexed completionMode to Job API #98441

Add Indexed completionMode to Job API #98441

Conversation

alculquicondor commented Jan 26, 2021 • edited by liggitt

alculquicondor commented Jan 26, 2021

alculquicondor commented Jan 26, 2021

alculquicondor commented Jan 27, 2021

alculquicondor commented Jan 27, 2021

alculquicondor commented Jan 27, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ahg-g commented Feb 5, 2021

alculquicondor commented Feb 22, 2021

alculquicondor commented Feb 23, 2021

ahg-g commented Mar 2, 2021

ahg-g commented Mar 2, 2021

lavalamp commented Mar 2, 2021

k8s-ci-robot commented Mar 2, 2021

ahg-g commented Mar 3, 2021

fejta-bot commented Mar 3, 2021

ahg-g commented Mar 3, 2021

ahg-g commented Mar 3, 2021

alculquicondor commented Mar 3, 2021

ahg-g commented Mar 3, 2021

alculquicondor commented Jan 26, 2021 •

edited by liggitt