Code freeze on remote loader

[KnVerey]

As described in Open letter to Kubernetes reviewers/approvers (and contributors!), the Kubernetes project is taking steps to improve the quality of our codebases for the long term. Notably:

We don’t want to ask contributors to jump through too many hoops to contribute, but given the current quality of Kubernetes, we’d like to change the bar for accepting a PR from “don’t make the code locally worse” to “also don’t build on shaky foundations”.

One area that this is particularly relevant for Kustomize is the code and tests that support the ability to load remote Kustomizations. We had two recent regressions showing the need for better coverage in this area: #4559 and #4455. #4615 attempted to fix this, but several issues remain.

• Many of the new tests are skipped on CI because of Server lacks SSH keys #4620. They can still be run locally (which we do as part of the release process), but this is not a state we consider acceptable. We need to get all of the skips out of that test suite.
• Many of the remote tests are flakey, some so badly that they are being skipped: Flaky integration tests #4640

Before we accept any changes to this area, we need all of these skips and flakes to be resolved so that we are building on a solid foundation. Going forward, we will require solid, stable testing for all new additions to this code. We acknowledge that this can be difficult given that many of the outstanding feature requests in this area require remote infrastructure we do not currently support. But it is not good for us or for our users if we cannot validate that the features they rely on will continue to work.

[annasong20]

+1 It would definitely be nice to have more confidence in the CI pipeline and avoid all the "/retest"s 😄.

[natasha41575]

/triage accepted

[KnVerey]

@natasha41575 @annasong20 and I have agreed that thanks to @mightyguava great work, we can lift this freeze! 🎉
In its place, we will be freezing a much more specific piece of code that is involved in all Kustomization loading (remote and local) and was discovered to have poor error coverage as part of our efforts here. The new issue for that is #4807.

/close

[k8s-ci-robot]

@KnVerey: Closing this issue.

In response to this:

@natasha41575 @annasong20 and I have agreed that thanks to @mightyguava great work, we can lift this freeze! 🎉
In its place, we will be freezing a much more specific piece of code that is involved in all Kustomization loading (remote and local) and was discovered to have poor error coverage as part of our efforts here. The new issue for that is #4807.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.