fix bug with migrating kyaml reader path, index, and id annotations #4227

natasha41575 · 2021-10-06T21:40:25Z

In writing an e2e test for kpt, I discovered a couple issues with the migration of the path/index/id annotation code I'd written earlier. The legacy annotations were being overwritten by the internal ones after being processed by functions. This PR fixes these bugs.

copyAnnotations should copy the annotations from one annotation (internal or legacy) to the other only if the other is empty. That way, if a function changes one but not the other we still have both and kio.reconcileInternalAnnotations will detect that they are different and reconcile them. Additionally, I added a check to detect if either the legacy or internal index annotation is set, in which case it should be copied to the other and not overwritten by the reader.

I've also added some unit tests to verify that it now behaves correctly.

kyaml/kio/byteio_reader.go

natasha41575 · 2021-10-12T18:34:31Z

kyaml/kio/byteio_writer_test.go

@@ -299,7 +299,13 @@ g:
 `,
 			},

-			expectedOutput: `a: b #first
+			expectedOutput: `e: f


This reordering is undoing the reordering done by the original PR https://github.com/kubernetes-sigs/kustomize/pull/4190/files

Is this considered as a breaking change?

It wasn't considered a breaking change in #4190 when the reordering occurred, so I don't think it would be considered a breaking change to reverse it.

Why do these annotations cause an ordering change, and under what circumstances? Any reordering that surfaces in Kustomize itself should be considered breaking. But judging by this being the only test where it surfaces, I take it only direct kyaml use is affected?

Any reordering that surfaces in Kustomize itself should be considered breaking.

I agree and this change only surfaced directly in kyaml. The reordering occurred in the test case where the nodes were being ordered - specifically in this case, some of the resources had the path/index annotations while this one did not. My guess (though I haven't done any investigation to back this up) is that the bug (introduced in #4190) which caused the legacy annotations to be sometimes overwritten or ignored caused the reordering, and this PR fixed it along with its other fixes.

natasha41575 · 2021-10-12T18:37:33Z

/hold

Want to wait for some other manual tests to pass before merging this

natasha41575 · 2021-10-12T19:15:42Z

/unhold

mengqiy · 2021-10-12T20:37:57Z

kyaml/kio/kioutil/kioutil_test.go

+metadata:
+ name: foobar
+ annotations:
+   internal.config.kubernetes.io/path: 'a/b.yaml'


When both exist and don't match, should we error it out somewhere?
If we are handling it in another place, can you please help me recall it?

This is handled in reconcileInternalAnnotations, which detects when both exist and don't match and reconciles them accordingly. However this is only checked after running a function to account for functions that mutate one but not the other.

If the input resource to an orchestrator has both set but they differ, the orchestrator will prefer the internal annotation over the legacy one. Do you believe it should throw an error instead?

It should error when the input to the function has both new and legacy annotations, the function modify both annotations and they mismatch. In this case, we can't infer the user's intend, we should error. It should be fine in other cases as long as we can infer user's intend.

sounds good. I will update it to throw an error instead.

mengqiy · 2021-10-12T20:39:36Z

kyaml/kio/byteio_writer_test.go

@@ -299,7 +299,13 @@ g:
 `,
 			},

-			expectedOutput: `a: b #first
+			expectedOutput: `e: f


Is this considered as a breaking change?

mengqiy

/lgtm
/approve

k8s-ci-robot · 2021-10-13T01:36:13Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mengqiy, natasha41575

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~kyaml/OWNERS~~ [mengqiy,natasha41575]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

natasha41575 requested review from mengqiy and KnVerey October 6, 2021 21:40

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 6, 2021

natasha41575 changed the title ~~fix bug with migrating annotations~~ fix bug with migrating kyaml reader path, index, and id annotations Oct 6, 2021

k8s-ci-robot requested a review from phanimarupaka October 6, 2021 21:40

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 6, 2021

natasha41575 removed the request for review from phanimarupaka October 6, 2021 21:40

mengqiy reviewed Oct 8, 2021

View reviewed changes

kyaml/kio/byteio_reader.go Outdated Show resolved Hide resolved

mengqiy reviewed Oct 8, 2021

View reviewed changes

kyaml/kio/byteio_reader.go Outdated Show resolved Hide resolved

natasha41575 force-pushed the MigrateIndexPathIdAnnotations branch 2 times, most recently from 97115d9 to f38d44b Compare October 12, 2021 17:56

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 12, 2021

natasha41575 force-pushed the MigrateIndexPathIdAnnotations branch from f38d44b to 3feb55d Compare October 12, 2021 18:00

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 12, 2021

natasha41575 force-pushed the MigrateIndexPathIdAnnotations branch 2 times, most recently from f665898 to 28f1661 Compare October 12, 2021 18:04

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 12, 2021

natasha41575 force-pushed the MigrateIndexPathIdAnnotations branch 4 times, most recently from 43fd08a to ab32c94 Compare October 12, 2021 18:16

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 12, 2021

natasha41575 force-pushed the MigrateIndexPathIdAnnotations branch from ab32c94 to 00335de Compare October 12, 2021 18:33

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 12, 2021

natasha41575 commented Oct 12, 2021

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 12, 2021

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 12, 2021

natasha41575 requested a review from mengqiy October 12, 2021 19:15

mengqiy reviewed Oct 12, 2021

View reviewed changes

natasha41575 added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 12, 2021

natasha41575 force-pushed the MigrateIndexPathIdAnnotations branch 3 times, most recently from 2c637fb to 67d8499 Compare October 12, 2021 22:23

natasha41575 marked this pull request as draft October 12, 2021 22:32

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 12, 2021

fix bug with migrating annotations

55ac9ca

natasha41575 force-pushed the MigrateIndexPathIdAnnotations branch from 67d8499 to 55ac9ca Compare October 12, 2021 22:43

natasha41575 marked this pull request as ready for review October 12, 2021 22:43

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 12, 2021

natasha41575 removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 12, 2021

natasha41575 requested a review from mengqiy October 12, 2021 23:00

mengqiy approved these changes Oct 13, 2021

View reviewed changes

k8s-ci-robot assigned mengqiy Oct 13, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 13, 2021

k8s-ci-robot merged commit ed76399 into kubernetes-sigs:master Oct 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix bug with migrating kyaml reader path, index, and id annotations #4227

fix bug with migrating kyaml reader path, index, and id annotations #4227

natasha41575 commented Oct 6, 2021

natasha41575 Oct 12, 2021

mengqiy Oct 12, 2021

natasha41575 Oct 12, 2021

KnVerey Oct 19, 2021

natasha41575 Oct 19, 2021

natasha41575 commented Oct 12, 2021 •

edited

natasha41575 commented Oct 12, 2021

mengqiy Oct 12, 2021

natasha41575 Oct 12, 2021

mengqiy Oct 12, 2021

natasha41575 Oct 12, 2021

mengqiy Oct 12, 2021

mengqiy left a comment

k8s-ci-robot commented Oct 13, 2021

fix bug with migrating kyaml reader path, index, and id annotations #4227

fix bug with migrating kyaml reader path, index, and id annotations #4227

Conversation

natasha41575 commented Oct 6, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

natasha41575 commented Oct 12, 2021 • edited

natasha41575 commented Oct 12, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mengqiy left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Oct 13, 2021

natasha41575 commented Oct 12, 2021 •

edited