Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix 4124] Skip local resource until all transformations have completed. #4180

Merged
merged 2 commits into from Sep 24, 2021
Merged

[Fix 4124] Skip local resource until all transformations have completed. #4180

merged 2 commits into from Sep 24, 2021

Conversation

yuwenma
Copy link
Contributor

@yuwenma yuwenma commented Sep 14, 2021

Fix: #4124

Description

Resources annotated as local-config are expected to be filtered out from stdout. This filtering (in func accumulateResources ) happens before any transformation operations. This causes the issue since
the local resources are needed in transformations. See #4124

This PR removes the "drop local resource" logic from "accumulateResources", and filter out the local resource after running transforms.

Open to discuss:

None of the existing ResMap functions can drop the resource slice easily: Clear will ruin the resource order, AppendAll only adds non-existing resource, AbsorbAll only add or modify but not delete.
Thus, we introduce a new func "Intersection" in resourceAccumulator to specifically removes the resource by ID and keep the original order.

@k8s-ci-robot
Copy link
Contributor

@yuwenma: This PR has multiple commits, and the default merge method is: merge.
You can request commits to be squashed using the label: tide/merge-method-squash

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 14, 2021
@k8s-ci-robot
Copy link
Contributor

Hi @yuwenma. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 14, 2021
@yuwenma
Copy link
Contributor Author

yuwenma commented Sep 14, 2021 •
edited

@natasha41575 @monopole @KnVerey PTAL

The fix is a little bit complicated than I expect. Let me know if there's any better approach or existing functions we can use.

@natasha41575
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 14, 2021
@natasha41575
Copy link
Contributor

BTW, you can run make prow-presubmit-check to run the kustomize-presubmit-master check locally.

@natasha41575 natasha41575 added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Sep 14, 2021
@natasha41575
Copy link
Contributor

/cc @monopole
/cc @natasha41575
/uncc @mortent

@k8s-ci-robot k8s-ci-robot requested review from monopole and natasha41575 and removed request for mortent September 14, 2021 18:10
@yuwenma yuwenma mentioned this pull request Sep 15, 2021
Merged
natasha41575
natasha41575 reviewed Sep 16, 2021
View reviewed changes
api/internal/target/kusttarget.go Outdated Show resolved Hide resolved
api/resource/factory_test.go Show resolved Hide resolved
api/resource/factory_test.go Outdated Show resolved Hide resolved
api/internal/target/kusttarget.go Outdated Show resolved Hide resolved
api/resource/factory.go Show resolved Hide resolved
@natasha41575
Copy link
Contributor

BTW, for merge conflicts we usually rebase and force-push.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 16, 2021
@yuwenma
[Fix 4124] Add unittest with the given example.
a100dca
@yuwenma
[fix 4124] Skip local resource until all transformations have completed.
f438273 
Resources annotated as "local-config" are expected to be ignored. This skip local resource happens in "accumulateResources" which happens before any transformation operations.
However, the local resource may be needed in transformations.
Thus, this change removes the "drop local resource" logic from accumulateResources and removes these local resource after all transformation operations and var operations are done.

Note:
None of the existing ResMap functions can drop the resource slice easily: "Clear" will ruin the resource order, "AppendAll" only adds non-existing resource, "AbsorbAll" only add or modify but not delete.
Thus, we introduce a new func "Intersection" for resourceAccumulator that specificaly removes the resource by ID and keep the original order.
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 16, 2021
@yuwenma
Copy link
Contributor Author

yuwenma commented Sep 16, 2021

Ready for another look. :) @natasha41575 @monopole @KnVerey

@natasha41575
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 20, 2021
monopole
monopole approved these changes Sep 24, 2021
View reviewed changes
Copy link
Contributor

@monopole monopole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@monopole monopole merged commit dd579c9 into kubernetes-sigs:master Sep 24, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: monopole, yuwenma

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 24, 2021
This was referenced Nov 1, 2021
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@natasha41575 natasha41575 natasha41575 left review comments

@monopole monopole monopole approved these changes

@KnVerey KnVerey Awaiting requested review from KnVerey

Assignees

@monopole monopole

@natasha41575 natasha41575

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

The config.kubernetes.io/local-config: "true" is too strict
4 participants
@yuwenma @k8s-ci-robot @natasha41575 @monopole