Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yq crashes when piping a yaml file with another yaml file as input #1200

Closed
hickersonj opened this issue Apr 27, 2022 · 3 comments
Closed

yq crashes when piping a yaml file with another yaml file as input #1200

hickersonj opened this issue Apr 27, 2022 · 3 comments
Labels

Comments

@hickersonj
Copy link

Describe the bug
When piping a file to yq as one input with another file as the other input there is a runtime error. However, providing the two files as arguments works.

Version of yq: 4.24.5
Operating system: Linux (Fedora 33)
Installed via: binary release

Input Yaml
Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
namespace.yml

---
apiVersion: v1
kind: Namespace
metadata:
  name: namespace

app.yml

---
apiVersion: v1
kind: Deployment
metadata:
  name: deploy-app
  labels:
    app: deploy-label
---
apiVersion: v1
kind: Daemonset
metadata:
  name: daemonset-app
  labels:
    app: daemonset-label

Command

cat namespace.yml | yq eval-all 'with(select(fi == 0); .metadata.name = env(Namespace)) | with(select(fi != 0); .metadata += {"namespace": env(Namespace)})' app.yml -

Actual behavior

panic: runtime error: index out of range [2] with length 2

goroutine 1 [running]:
github.com/mikefarah/yq/v4/pkg/yqlib.addMaps(0xc0003eb9d0, 0xc0003feba0?, 0xc0003e1a80?)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:193 +0x3af
github.com/mikefarah/yq/v4/pkg/yqlib.add(0xc0000a1800?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc0003eb500, 0xc0003eb960)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:63 +0x539
github.com/mikefarah/yq/v4/pkg/yqlib.resultsForRHS(0x19?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x203000?, {0x0, 0x0, 0x7d7740}, ...)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:88 +0x503
github.com/mikefarah/yq/v4/pkg/yqlib.doCrossFunc(0x22?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7c8, {0x0, 0x0, 0x7d7740})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:125 +0x413
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunctionWithPrefs(0x1?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc000397f50?, {0x0, 0x0, 0x7d7740})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:152 +0x212
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunction(0xc0000a1800?, {0xc0004020c0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x0?, 0xaeb540?, 0x0?)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:136 +0x9f
github.com/mikefarah/yq/v4/pkg/yqlib.addOperator(0xc0000a1800?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x0?)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:46 +0x1c5
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc000397860?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7c8)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.resultsForRHS(0x19?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x203000?, {0x0, 0x0, 0xc0003e1980}, ...)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:69 +0x1fc
github.com/mikefarah/yq/v4/pkg/yqlib.doCrossFunc(0x22?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7e0, {0x0, 0x0, 0xc0003e1980})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:125 +0x413
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunctionWithPrefs(0x1?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0x7f82cdaf95b8?, {0x0, 0x0, 0xc0003e1980})
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:152 +0x212
github.com/mikefarah/yq/v4/pkg/yqlib.crossFunction(0xc0003d2df8?, {0xc000397da0, 0xc000397860, 0x1, {0x0, 0x0}}, 0xc00037e7b0?, 0xc0003d2e60?, 0xf?)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:136 +0x9f
github.com/mikefarah/yq/v4/pkg/yqlib.assignUpdateOperator(0xc0000a1800?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e7e0)
	/home/runner/work/yq/yq/pkg/yqlib/operator_assign.go:31 +0x305
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc00037e798?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e7e0)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.compoundAssignFunction(0x4cf825?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e5d0, 0x7d7820)
	/home/runner/work/yq/yq/pkg/yqlib/operators.go:35 +0x3ba
github.com/mikefarah/yq/v4/pkg/yqlib.addAssignOperator(0xc0000a1800?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0x0?)
	/home/runner/work/yq/yq/pkg/yqlib/operator_add.go:19 +0x94
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc0000a1800?, {0xc0003977d0, 0xc000397860, 0x0, {0x0, 0x0}}, 0xc00037e5d0)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.withOperator(0xc0000a1800?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e600)
	/home/runner/work/yq/yq/pkg/yqlib/operator_with.go:25 +0x34c
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0x46af9e?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e600)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.pipeOperator(0xc0000a1800?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e618)
	/home/runner/work/yq/yq/pkg/yqlib/operator_pipe.go:13 +0x147
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc0003902a0?, {0xc0002f7080, 0x0, 0x0, {0x0, 0x0}}, 0xc00037e618)
	/home/runner/work/yq/yq/pkg/yqlib/data_tree_navigator.go:37 +0x25c
github.com/mikefarah/yq/v4/pkg/yqlib.(*allAtOnceEvaluator).EvaluateCandidateNodes(0xc0003d3ca8, {0x7fffa10a18cf?, 0x7fffa10a1952?}, 0xc0002f7080)
	/home/runner/work/yq/yq/pkg/yqlib/all_at_once_evaluator.go:42 +0xba
github.com/mikefarah/yq/v4/pkg/yqlib.(*allAtOnceEvaluator).EvaluateFiles(0x7b1cdb?, {0x7fffa10a18cf, 0x7a}, {0xc0000a1cc0, 0x2, 0xc0000c4f28?}, {0x838cb0, 0xc00038e180}, 0x1, {0x838220, ...})
	/home/runner/work/yq/yq/pkg/yqlib/all_at_once_evaluator.go:85 +0x35d
github.com/mikefarah/yq/v4/cmd.evaluateAll(0xc0000cb680?, {0xc0000a1cb0?, 0x3, 0x3})
	/home/runner/work/yq/yq/cmd/evaluate_all_command.go:144 +0x137e
github.com/spf13/cobra.(*Command).execute(0xc0000cb680, {0xc0000a1c80, 0x3, 0x3})
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:856 +0x67c
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000cb180)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
	/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902
main.main()
	/home/runner/work/yq/yq/yq.go:22 +0x1f1

Expected behavior

---
apiVersion: v1
kind: Namespace
metadata:
  name: app-test
---
apiVersion: v1
kind: Deployment
metadata:
  name: deploy-app
  labels:
    app: deploy-label
  namespace: app-test
---
apiVersion: v1
kind: Daemonset
metadata:
  name: daemonset-app
  labels:
    app: daemonset-label
  namespace: app-test

Additional context
The goal is to append the Namespace resource to the app.yml without needing to have seperate files, i.e. the namespace.yml would be piped into yq inside of a bash script.

@mikefarah
Copy link
Owner

Ok I found the bug - it's with appending to a map when the new key entry matches a value in an existing map (in this case namespace). I'll have a fix in the next release, till then, as a workaround, instead of .metadata += {"namespace": env(Namespace)} just do .metadata.namespace = env(Namespace)

@hickersonj
Copy link
Author

Glad you found the bug quickly! Also I didn't know that syntax .metadata.namespace = env(Namespace) would add a key value pair, glad to have learned that.

It is now working with the suggestions thanks.

@mikefarah
Copy link
Owner

Fixed in 4.25.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants