[lang] anchors are not supported

[jorgemoralespou]

I've seen #172 and the limitations and this prevents processing/installing knative resources with ytt.

https://github.com/knative/serving/blob/6c3eb0433d235d5ba9768ada4e7be90ddeb74458/config/core/resources/configuration.yaml#L26-L55

[cppforlife]

we do want them to work since they may be used within third party configuration.

[cppforlife]

minimal reproduction: https://get-ytt.io/#gist:https://gist.github.com/cppforlife/f03ac62b09da71bd8549e372fb921bd6

[cppforlife]

this open PR to go-yaml fixes issue: go-yaml/yaml#364; however, after the fix cases like knative config:

  - << : *version
    name: v1beta1

present a challenge since they introduce duplicate keys.

[dprotaso]

this open PR to go-yaml fixes issue: go-yaml/yaml#364; however, after the fix cases like knative config:

  - << : *version
    name: v1beta1

present a challenge since they introduce duplicate keys.

but the last key listed should just override no? I'm guessing there's something in the spec regarding the behaviour

[aaronshurley]

Some additional conversation/context in slack.

[pivotaljohn]

Adding to list of issues to review during pre-IPM ==> recommended

[groundnuty]

any progress on that? I'm attempting to port anchor heavy configurations to ytt and lack support for anchors is making it increasingly complicated

[pivotaljohn]

@groundnuty asked

any progress on that? I'm attempting to port anchor heavy configurations to ytt and lack support for anchors is making it increasingly complicated

Unfortunately not, so far.

We've been in a "wait and hope for upstream fix" for some time now. So, while technically "in progress" is correct, we're really blocked on this upstream fix... (or we decide to find a way to land it in our own fork).

It be clear, it's in the Carvel maintainer's backlog 👍🏻 , but it's currently sitting below the OpenAPI Schema export feature 🤷🏻 .

[groundnuty]

@pivotaljohn maybe it's somehow connected with the memorable moment, that go-yaml stopped supporting flexible/forward anchors? It broke a lot of anchoring features... here is the discussion, where you can find a commit that broke that: go-yaml/yaml#558 if that's the case you can just try to build with older version of go-yaml and see if it works, and if it does remove the anchor-breaking changes form your fork.

Just guessing here/trying to helpful :-)

[pivotaljohn]

Appreciate that, @groundnuty. I'm looking into this.

I'll update this comment with my "research", as I go.

Facts:

• Add large document benchmarks, tune alias heuristic, add max depth limits go-yaml/yaml#515 was introduced in v2.2.4 of go-yaml/yaml; ytt forked at v2.2.2
• fix for merge @cppforlife mentioned above was translated in 237b7e2 from handle unmarshalling merges into a MapSlice go-yaml/yaml#364 (~Aug 2020)
• also noted by @cppforlife, merged result includes duplicate keys. When the file contains templating and --implicit-map-key-overrides is provided, the merge works as desired.
• along the way to intending to improve how plain YAML is processed by ytt (i.e. YAML files that do not contain any templating), we short-circuit the handling of --implicit-map-key-overrides
  • introduced with Automatically detect when a YAML file is a template or plain YAML (obviating the --ignore-unknown-comments flag) #232
  • defect reported in duplicate key check in plain yamls #310 (among others)
• a workaround could be to prefix any plain YAML with some inert templating (e.g. #@ True would do it).
• a fix is to ensure that --implicit-map-key-overrides applies to plain YAML as well (not just templates).

[hasufell]

This also stops me from migrating my yamls to ytt.

[pivotaljohn]

FWIW, this issue was addressed on the main branch: #550 and will be included in the next release (coming soon).

[pivotaljohn]

Released in v0.39.0.

[dprotaso]

woot! thanks!