diff --git a/examples/remoteBuild.md b/examples/remoteBuild.md index c4ddc459f1..1e935962ce 100644 --- a/examples/remoteBuild.md +++ b/examples/remoteBuild.md @@ -10,7 +10,7 @@ some query string parameters. Kustomize does not currently support ports in the URL. The directory is specified by appending a `//` after the repo URL. The following query string parameters can also be specified: - * `ref` - a `git fetch`-able ref, typically a branch, tag, or full commit hash + * `ref` - a [`git fetch`-able ref](https://git-scm.com/docs/git-fetch), typically a branch, tag, or full commit hash (short hashes are not supported) * `timeout` (default `27s`) - a number in seconds, or a go duration. specifies the timeout for fetching the resource diff --git a/proposals/22-04-localize-command.md b/proposals/22-04-localize-command.md index 2bcb13cda8..530435637b 100644 --- a/proposals/22-04-localize-command.md +++ b/proposals/22-04-localize-command.md @@ -41,17 +41,19 @@ in [issue #4154](https://github.com/kubernetes-sigs/kustomize/issues/4154). The proposed command has the added benefit of increasing user confidence in the integrity of their kustomization builds. Locally downloaded files, unlike urls, give users full control of file content. At the same time, the command does this -without modifying the original kustomization so that users can always run the command on the original again to fetch -upstream changes. +while preserving the original kustomization, allowing users to further iterate on the original and to build and localize +the iterations. **Goals:** -1. This command should localize - * all remote files that a kustomization file directly references - * remote exec binaries of referenced KRM functions +This command should localize - This command achieves this goal if, in the absence of remote images and custom fields in KRM - functions, `kustomize build` can run on the localized copy without network access. +* all remote files that a kustomization file directly references +* remote exec binaries of referenced KRM functions + +This command achieves this goal if, in the absence of remote images and custom fields in KRM +functions, `kustomize build` can run on the localized copy without network access and produce the same output as when +run on the original. **Non-goals:** @@ -64,21 +66,23 @@ upstream changes. The command takes the following form:
-kustomize localize target newDir [-s scope] [-n]
+kustomize localize target newDir [--scope scope] [--no-verify]
 
where the arguments are: -* `target`: a directory with a top-level kustomization file that kustomize will localize; can be a path to a local - directory or a url to a remote directory +* `target`: [kustomization root](https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#kustomization-root) + that kustomize will localize; can be local path + or [remote directory with `ref` parameter](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md) * `newDir`: optional destination directory of the localized copy of `target`; if not specified, the destination is a - directory named `localized-{target}` in the same directory as `scope` + directory in the working directory named + * `localized-{target}` for local `target` + * `localized-{target}-{ref}` for remote `target` and the flags are: -* `-s`, `--scope` - `scope`: optional root directory, files outside which kustomize is not allowed to copy and localize; if not specified, - takes on value of `target` -* `-n`, `--no-verify`: do not verify that the outputs of `kustomize build` for `target` and `newDir` are the same after +* `--scope scope`: optional local directory, files outside which kustomize is not allowed to copy and localize; only + applicable if `target` is local, and if not specified, `scope` takes on value of `target` +* `--no-verify`: do not verify that the outputs of `kustomize build` for `target` and `newDir` are the same after localization The command creates a copy of the `target` kustomization and the local files that `target` references at `newDir`. We @@ -89,12 +93,13 @@ define the "files that `target` references" as: * exec binaries of referenced KRM functions Here, configuration file means a non-kustomization yaml file. The command cannot run on `target`s that need -the `--load-restrictor LoadRestrictionsNone` flag on `kustomize build`. The command only copies referenced files that -reside inside `scope`. +the `--load-restrictor LoadRestrictionsNone` flag for `kustomize build`. Additionally, note that for the localization to +occur on a local `target`, `scope` must contain `target`. The copied files sit under the same relative paths in `newDir` +that their counterparts sit under in `scope` and in the repo, for local and remote `target`s, respectively. -The command localizes the copy of `target` at `newDir` by downloading all remote files that `target` -references. Users do not have executable permission for downloaded exec binaries that KRM functions reference and for -each such exec binary, the command will print a warning message to that effect. +The command localizes the copy of `target` in `newDir` by downloading all remote files that `target` references. For +each downloaded exec binary that KRM functions reference, the command removes users' executable permissions and prints a +warning message to that effect. The command creates a new `localized-files` directory, next to the file that referenced the downloaded files, to hold said files. Inside `localized-files`, the downloads are located on path: @@ -103,33 +108,33 @@ said files. Inside `localized-files`, the downloads are located on path: domain / organization / repo / version / path/to/file/in/repo -where `version` corresponds to the `ref` query string parameter in the url, though ideally `version` ia a stable tag as -opposed to a branch. +where `version` corresponds to a [`git fetch ref`](https://git-scm.com/docs/git-fetch), the same entity that the command +looks for in a remote `target`. `ref`s are query string parameters in directory urls and embedded in the path of raw +GitHub file urls. Ideally though, `ref`s are stable tags as opposed to branches. -The command replaces remote references in `newDir` with the local paths of the downloaded files. To help ensure +The command replaces remote references in `newDir` with local relative paths to the downloaded files. To help ensure that `newDir` is a clean copy, the command additionally overwrites absolute path references into `target` to point -to `newDir`. +to the corresponding file in `newDir` instead. As a convenience to the user, in the absence of the `--no-verify` flag, the command automatically tries to -run `kustomize build`, without any flags, on `target` and the localized `newDir` to compare their outputs. The command -indicates success if the outputs match and throws an error with the diff summary otherwise. This check, however, is not -useful for certain `target`s, including those that need flags to build. In these cases, the command prints next steps -that users can follow to check the output themselves. For example, for `target`s that reference KRM functions with a -remote exec binary, the command suggests the user: +run `kustomize build`, without any flags, on the original `target` and the localized `target` in `newDir` to compare +their outputs. The command indicates success if the outputs match and throws an error with a diff summary otherwise. +This check, however, is not useful for certain `target`s, including those that need flags to build. In these cases, the +command prints next steps that users can follow to check the output themselves. For example, for `target`s that +reference KRM functions with a remote exec binary, the command suggests the user: 1. add executable permissions for the downloaded exec binaries in `newDir` **that the user trusts** 2. run `kustomize build` with flags `--enable-alpha-plugins --enable-exec` and self-verify the outputs **Error cases**: -* `target` does not have a top-level kustomization file * `kustomize build` needs `--load-restrictor LoadRestrictionsNone` to run on `target` * `newDir` already exists +* `scope` specified for remote `target` * `scope` does not contain `target` * `target` references a local path that traverses outside of `scope` -* remote reference does not have a `version` +* remote url does not have a `version` * `localized-files` directory already exists -* kustomization file is malformed * cycle of kustomization file references exists * `kustomize build` produces different output for `target` and `newDir` in the absence of `--no-verify` @@ -145,8 +150,8 @@ running the command again. #### Story 1 -My company’s CI/CD pipeline currently pulls an `example` directory from our internal package management site. I want the -CI/CD pipeline to additionally run `kustomize build example/overlay`. My setup looks like this: +My company’s CI/CD pipeline currently fetches an `example` directory from our internal package management site. I want +the CI/CD pipeline to additionally run `kustomize build example/overlay`. My setup looks like this: ```shell └── example ├── overlay @@ -163,7 +168,7 @@ resources: ```shell # example/base/kustomization.yaml resources: - - github.com/kubernetes-sigs/kustomize/examples/multibases?ref=v1.0.6 + - https://github.com/kubernetes-sigs/kustomize//examples/multibases?ref=v1.0.6 - https://raw.githubusercontent.com/kubernetes-sigs/kustomize/v1.0.6/examples/helloWorld/configMap.yaml ``` @@ -172,13 +177,12 @@ pipeline does not have external network access. Fortunately, I remember that I can run `kustomize localize` on `example/overlay` on my local machine. I can then upload the localized directory to my company’s internal package management site for the CI/CD pipeline to pull and build -instead. I run -`kustomize localize example/overlay -s example`, where my `target` is `example/overlay`, I accept the default location -and name of `newDir`, and I expand my `scope` to `example` because `example/overlay` references `example/base`. I get -the following output: +instead. I run `kustomize localize example/overlay --scope example`, where my `target` is `example/overlay`, I accept +the default location and name of `newDir`, and I expand my `scope` to `example` because `example/overlay` +references `example/base`. I get the following output: ```shell -$ kustomize localize example/overlay -s example +$ kustomize localize example/overlay --scope example SUCCESS: example/overlay, localized-overlay produce same kustomize build output ``` @@ -189,7 +193,6 @@ SUCCESS: example/overlay, localized-overlay produce same kustomize build output │ └── base │ └── kustomization.yaml └── localized-overlay # the new, localized kustomization directory - ├── kustomization.yaml ├── base │ ├── kustomization.yaml │ └── localized-files @@ -202,11 +205,15 @@ SUCCESS: example/overlay, localized-overlay produce same kustomize build output │ │ └── configMap.yaml │ └── multibases │ ├── base + │ │ │── kustomization.yaml │ │ └── pod.yaml │ ├── dev + │ │ └── kustomization.yaml │ ├── kustomization.yaml │ ├── production + │ │ └── kustomization.yaml │ └── staging + │ └── kustomization.yaml └── overlay ├── kustomization.yaml └── localized-files @@ -222,19 +229,69 @@ SUCCESS: example/overlay, localized-overlay produce same kustomize build output # localized-overlay/overlay/kustomization.yaml resources: - ../base - - ./localized-files/github.com/kubernetes-sigs/kustomize/examples/helloWorld/deployment.yaml + - localized-files/github.com/kubernetes-sigs/kustomize/v1.0.6/examples/helloWorld/deployment.yaml ``` ```shell # localized-overlay/base/kustomization.yaml resources: - - ./localized-files/github.com/kubernetes-sigs/kustomize/examples/multibases - - ./localized-files/github.com/kubernetes-sigs/kustomize/examples/helloWorld/configMap.yaml + - localized-files/github.com/kubernetes-sigs/kustomize/v1.0.6/examples/multibases + - localized-files/github.com/kubernetes-sigs/kustomize/v1.0.6/examples/helloWorld/configMap.yaml ``` Now, I upload `localized-overlay` from my local setup to my company’s internal package management site. I change the -commands in my CI/CD pipeline to pull `localized-overlay` before running `kustomize build localized-overlay`, and the +commands in my CI/CD pipeline to fetch `localized-overlay` before running `kustomize build localized-overlay`, and the command executes successfully! +#### Story 2 + +Like in [Story 1](#story-1), I need kustomize to `localize` a +root, "https://github.com/annasong20/kustomize-test.git?ref=1.0.0", so that my company's CI/CD pipeline can +run `kustomize build` on it given network constraints. However, the difference this time is that I don't have a local +copy of the target root. Fortunately for me, `kustomize localize` provides me the convenience of remote targets, so that +I don't have to first run `git fetch`. + +On my local machine, I run `kustomize localize https://github.com/annasong20/kustomize-test.git?ref=1.0.0`, where the +url is my `target` and I once again accept the default `newDir`. Note that the `--scope` flag is not applicable here. I +get the following output: + +```shell +$ kustomize localize https://github.com/annasong20/kustomize-test.git?ref=1.0.0 +SUCCESS: https://github.com/annasong20/kustomize-test.git?ref=1.0.0, localized-kustomize-test-v1.0.0 produce same kustomize build output +``` + +```shell +└── localized-kustomize-test-v1.0.0 + ├── kustomization.yaml + └── localized-files + └── github.com + └── kubernetes-sigs + └── kustomize + └── v1.0.6 + └── examples + └── multibases + ├── base + │ │── kustomization.yaml + │ └── pod.yaml + ├── dev + │ └── kustomization.yaml + ├── kustomization.yaml + ├── production + │ └── kustomization.yaml + └── staging + └── kustomization.yaml +``` + +```shell +# localized-kustomize-test-v1.0.0/kustomization.yaml +resources: + - localized-files/github.com/kubernetes-sigs/kustomize/v1.0.6/examples/multibases +``` + +Once again, I upload `localized-kustomize-test-v1.0.0` from my local machine to the internal package management site and +program the CI/CD pipeline to fetch `localized-kustomize-test-v1.0.0` before +running `kustomize build localized-kustomize-test-v1.0.0`! Note that the pipeline can also run `kustomize build` on the +url for my upload of `localized-kustomize-test-v1.0.0` to avoid explicitly calling `git fetch` beforehand. + ### Risks and Mitigations One could argue that while uploading the localized `newDir` to a repository, a user could accidentally leak Secrets that @@ -246,15 +303,18 @@ configurations are also not too difficult to identify. Exec binaries that KRM functions reference are a different story. `kustomize localize` downloads remote exec binaries that, if malicious, are capable of almost anything during subsequent `kustomize build` calls. The command mitigates this -risk by leaving these downloaded exec binaries without executable permissions and warning the user, as mentioned in +risk by removing executable permissions on these downloaded exec binaries and warning the user, as mentioned in **Proposal**. `kustomize build` can only run the exec binary after the user deems the binary safe and changes its permissions. -Still another risk may be that if a user's kustomization tree is large, `kustomize localize` may be copying files from -unexpected locations. The command mitigates this risk with the `scope` flag. If not set, `kustomize localize` only -copies files in `target`. Otherwise, the user specifies `scope`and understands that `kustomize localize` only copies -files in `scope`. The qualification that the command can only localize `target`s that follow load restrictions helps -mitigate this risk as well. +Still another risk may be that if a user's kustomization tree is large, `kustomize localize` has the potential to copy +files from unexpected local locations. The command mitigates this risk with the `scope` flag. If the user +specifies `scope`, they understand that `kustomize localize` only copies files in `scope`. +Otherwise, `kustomize localize` treats `target` as `scope`. In either case, `kustomize localize` aborts with a +descriptive error message if `target` references local files outside of `scope` that the command would copy. Note that +for remote `target` and recursively referenced remote kustomization roots, the repo in which the remote root resides +is the implicit `scope`. The qualification that the command can only localize `target`s that follow load restrictions +helps mitigate this risk as well. ### Dependencies @@ -262,26 +322,20 @@ N/A ### Scalability -Large kustomization trees slows the performance of `kustomize localize`. These trees can have large local subtrees, have +Large kustomization trees slow the performance of `kustomize localize`. These trees can have large local subtrees, have large remote subtrees, be deeply nested, or be wide, with each overlay referencing multiple bases. Regardless of the -cause, large kustomization trees inevitably takes longer to copy and download. Parts of the kustomize code are not +cause, large kustomization trees inevitably take longer to copy and download. Parts of the kustomize code are not thread-safe, which precludes parallel execution. -On a separate note, a chain of remote kustomization directories in which the current kustomization file references the -next remote kustomization directories could create a `newDir` with deeply nested `localized-files`. This directory -structure would impede users’ navigation of `newDir`. However, this scenario should be unlikely as most kustomizations -only consist of a few layers. - The creation of the `localized-files` directory local to the referencing kustomization file additionally prevents the different layers of kustomization files from sharing the same copy of the remote files. Following the same logic, different potential `target` directories cannot share copies either. ## Drawbacks -Users whose layered kustomizations form a complex directory tree structure may have a hard time finding an -appropriate `scope`. However, many kustomizations exist in repositories, allowing the user to easily choose the repo -root as a valid `scope`. The warning messages that `kustomize localize` outputs for reference paths that extend -beyond `scope` should also help. +Users whose layered local kustomizations form a complex directory tree structure may have a hard time finding an +appropriate `scope`. However, the error messages that `kustomize localize` outputs for reference paths that extend +beyond `scope` should help. ## Alternatives @@ -295,18 +349,16 @@ beyond `scope` should also help. Despite its advantages, the alternative design violates the self-contained nature of each kustomize layer. Users would be unable to upload a fully localized kustomization directory in version control. Furthermore, this alternative - complicates the existing kustomize workflow by requiring the setup of global environment variables. -

+ complicates the existing kustomize workflow by requiring the setup of global environment variables.

* The command could, instead of making a copy, modify `target` directly. However, users would not have an easy way to - undo the command, which is undesirable. -

+ undo the command, which is undesirable.

* Instead of requiring the user to specify a second argument `scope`, the command could by definition limit its copying - to `target`. However, in the case of **Story 1**, the command would force the user to set `target` to `example` in - order to include `example/base` in the localization of `example/overlay`. The user would then have to create a - kustomization file at `example` that points to `example/overlay` under the `resources` field. The creation of the - kustomization file solely for this purpose is messy and more work for the user. + to the local `target`. However, in the case of [Story 1](#story-1), the command would force the user to set `target` + to `example` in order to include `example/base` in the localization of `example/overlay`. The user would then have to + create a kustomization file at `example` that points to `example/overlay` under the `resources` field. The creation of + the kustomization file solely for this purpose is messy and more work for the user. ## Rollout Plan @@ -316,15 +368,15 @@ This command will have at least alpha and GA releases. Depending on user feedbac This release will not support -* load restrictions check * KRM functions * absolute paths -* `target` kustomization with reference cycles -* any verification in the form of `--no-verification` flag or automatically running `kustomize build` at the end of +* any verification in the form of the `--no-verify` flag or automatically running `kustomize build` at the end of localization The entire command will be new in the alpha release, and so will not require an alpha flag. The command will not be -available in `kubectl kustomize` either as kubectl only has `kustomize build` builtin. +available in `kubectl kustomize` either, as kubectl only has `kustomize build` builtin. Instead, upon execution, the +command will print a warning message, which declares its alpha status and includes instructions on how users can provide +feedback. ### Beta/GA