馃悰 Updating WriteYAML to not add empty line to start files #626
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
k8s-ci-robot
added
the
cncf-cla: yes
k8s-ci-robot
added
the
size/XS
robscott
force-pushed
the
yaml-delimiter-cleanup
branch
from
56e4327
to
e1520fc
Compare
k8s-ci-robot
added
size/S
robscott
force-pushed
the
yaml-delimiter-cleanup
branch
3 times, most recently
from
b3a1559
to
1fe1610
Compare
howardjohn
reviewed
Sep 29, 2021
robscott
force-pushed
the
yaml-delimiter-cleanup
branch
from
1fe1610
to
a5f3860
Compare
robscott
changed the title
|
@howardjohn that's a good point, I was on the fence with the approach here but ended up deciding to go back to a delimiter to start the file, just without the extra prepended new line, hopefully that's a reasonable balance here. |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: alvaroaleman, robscott The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
a-hilaly
added a commit
to a-hilaly/ack-code-generator
that referenced
this pull request
Sep 9, 2022
Recently we bumped `controller-tools` version to `v0.9.2` to support building controller with go1.19. This change sneakily impacted the `build-controller-release.sh` script, which caused our prow testing jobs to fail. To summarize the bug we can generate the `role.yaml` file using controller-gen `0.7.0` and `0.9.2` and inspect the differences: ```bash diff role-controller-gen0.9.2.yaml role-controller-gen0.7.0.yaml 0a1 > ``` We can observe that the new line that used it be generated at the top of each file is no longer there, due to the fix introduced in 0.8.0 more precisely in kubernetes-sigs/controller-tools#626 This change combined with the command `tail -n +8` in https://github.com/aws-controllers-k8s/code-generator/blob/v0.20.0/scripts/build-controller-release.sh#L237 causes the script to cut an extra line which happens to be declaration of rules section in a controller ClusterRole manifest. Hence the error observed in the prow logs: ```bash Error: INSTALLATION FAILED: YAML parse error on emrcontainers-chart/templates/cluster-role-controller.yaml: error converting YAML to JSON: yaml: line 8: did not find expected key ``` This patch fixes the `tail` command to properly cut the generated `role.yaml` file and avoid deleteng the `rules` section.
ack-bot
pushed a commit
to aws-controllers-k8s/code-generator
that referenced
this pull request
Sep 9, 2022
This patch fixes the `tail` command to properly cut the generated `role.yaml` file and avoid deleteng the `rules` section. Context: Recently we bumped `controller-tools` version to `v0.9.2` to support building controller with go1.19. This change sneakily impacted the `build-controller-release.sh` script, which caused our prow testing jobs to fail. To summarize the bug we can generate the `role.yaml` file using controller-gen `0.7.0` and `0.9.2` and inspect the differences: ```bash diff role-controller-gen0.9.2.yaml role-controller-gen0.7.0.yaml 0a1 > ``` We can observe that the new line that used it be generated at the top of each file is no longer there, due to the fix introduced in 0.8.0 more precisely in kubernetes-sigs/controller-tools#626 This change combined with the command `tail -n +8` in the [build controller release script](https://github.com/aws-controllers-k8s/code-generator/blob/v0.20.0/scripts/build-controller-release.sh#L237) causes the script to cut an extra line which happens to be declaration of rules section in a controller ClusterRole manifest. Hence the error observed in the prow logs: ```bash Error: INSTALLATION FAILED: YAML parse error on emrcontainers-chart/templates/cluster-role-controller.yaml: error converting YAML to JSON: yaml: line 8: did not find expected key ``` By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently generated YAML files all have
\n---\nprepended to them. Although that's not an issue when usingkubectl applywith those files, it requires extra work when trying to do something simple like parse CRD yaml from a controller. Although not a significant issue, it's caused us (Gateway API + associated controller authors) to have to workaround it in several places. When parsing YAML files like this, the leading new line is parsed as an empty yaml struct, so we need to write code that can handle this empty/invalid object that starts all our generated YAML CRDs.It looks like this code originated from #208, and this particular part of the change does not seem like it was intentional.
Initially I wrote this to simply not add the delimiter to the start of the file, but as @howardjohn pointed out, that would make the concatenated files invalid YAML. This change does mean that concatenating YAML files that don't have a trailing new line will not work. Of course all files generated by this tool do have a trailing new line, and that seems like a reasonable expectation of other YAML files.