match() captures are malformed

[bencehornak]

Describe the bug

The match() function returns a corrupt captures array, when there are multiple capture groups.

Update: depending on the output-format, yq crashes in other cases.

Version of yq: 4.20.1
Operating system: mac
Installed via: binary homebrew

Input Yaml

abc

Command

The command you ran:

echo abc | yq 'match("(a)(b)(c)")' --output-format yaml
echo abc | yq 'match("(a)(b)(c)")' --output-format json
echo abc | yq 'match("(a)(b)(c)")' --output-format xml
echo abc | yq 'match("(a)(b)(c)")' --output-format props

Actual behavior

--output-format yaml

string: abc
offset: 0
length: 3
captures:
  - string: a
    offset: 0
    length: 1
  -? string: a
      offset: 0
      length: 1
    : string
    b: offset
    1: length
  -? string: a
      offset: 0
      length: 1
    :? string: a
        offset: 0
        length: 1
      : string
      b: offset
      1: length
    string: c
    offset: 2
    length: 1

The captures list looks totally unparsable.

--output-format <not yaml>

Crash:

panic: runtime error: index out of range [7] with length 7

goroutine 1 [running]:
github.com/mikefarah/yq/v4/pkg/yqlib.explodeNode(0xc0001a15e0, {0xc00031ac60, 0x0, 0x0, {0x0, 0x0}})
        github.com/mikefarah/yq/v4/pkg/yqlib/operator_anchors_aliases.go:227 +0x3fb
github.com/mikefarah/yq/v4/pkg/yqlib.explodeNode(0xc000342320, {0xc00031ac60, 0x0, 0x0, {0x0, 0x0}})
        github.com/mikefarah/yq/v4/pkg/yqlib/operator_anchors_aliases.go:192 +0x277
github.com/mikefarah/yq/v4/pkg/yqlib.explodeNode(0xc0001a1ea0, {0xc00031ac60, 0x0, 0x0, {0x0, 0x0}})
        github.com/mikefarah/yq/v4/pkg/yqlib/operator_anchors_aliases.go:232 +0x3c5
github.com/mikefarah/yq/v4/pkg/yqlib.explodeOperator(0xc00009d800, {0xc00031ac60, 0x0, 0x0, {0x0, 0x0}}, 0xc0002bd620)
        github.com/mikefarah/yq/v4/pkg/yqlib/operator_anchors_aliases.go:133 +0x31e
github.com/mikefarah/yq/v4/pkg/yqlib.(*dataTreeNavigator).GetMatchingNodes(0xc00009d800, {0xc00031ac60, 0x0, 0x0, {0x0, 0x0}}, 0xc0002bd620)
        github.com/mikefarah/yq/v4/pkg/yqlib/data_tree_navigator.go:37 +0x244
github.com/mikefarah/yq/v4/pkg/yqlib.(*resultsPrinter).PrintResults(0xc00033a900, 0xc00031ac60)
        github.com/mikefarah/yq/v4/pkg/yqlib/printer.go:95 +0x217
github.com/mikefarah/yq/v4/pkg/yqlib.(*streamEvaluator).Evaluate(0xc00036fd08, {0x137922e, 0xc000323b01}, {0x1400f60, 0xc00033a9c0}, 0xc00009d800, {0x1406430, 0xc00033a900}, {0x0, 0x0}, ...)
        github.com/mikefarah/yq/v4/pkg/yqlib/stream_evaluator.go:122 +0x386
github.com/mikefarah/yq/v4/pkg/yqlib.(*streamEvaluator).EvaluateFiles(0xc0000a2008, {0x7ff7bfeff7c3, 0x12}, {0xc00009dc90, 0x1, 0xc0000a6ec4}, {0x1406430, 0xc00033a900}, 0xc0, {0x1405958, ...})
        github.com/mikefarah/yq/v4/pkg/yqlib/stream_evaluator.go:72 +0x22c
github.com/mikefarah/yq/v4/cmd.evaluateSequence(0xc0000d1400, {0xc00009dc80, 0x3, 0x3})
        github.com/mikefarah/yq/v4/cmd/evalute_sequence_command.go:157 +0x138b
github.com/spf13/cobra.(*Command).execute(0xc0000d1400, {0xc00009dc50, 0x3, 0x3})
        github.com/spf13/cobra@v1.3.0/command.go:856 +0x60e
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000d1180)
        github.com/spf13/cobra@v1.3.0/command.go:974 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/cobra@v1.3.0/command.go:902
main.main()
        github.com/mikefarah/yq/v4/yq.go:22 +0x1f1

Expected behavior

--output-format yaml

string: abc
offset: 0
length: 3
captures:
  - string: a
    offset: 0
    length: 1
  - string: b
    offset: 1
    length: 1
  - string: c
    offset: 2
    length: 1

--output-format <not yaml>

Something equivalent.

[bencehornak]

I created an extended version for an existing test case to cover this case too.

Before:

yq/pkg/yqlib/operator_strings_test.go

Lines 32 to 39 in df32bae

	{
	description: "Match with capture groups",
	document: `abc abc`,
	expression: `[match("(abc)+"; "g")]`,
	expected: []string{
	"D0, P[], (!!seq)::- string: abc\n offset: 0\n length: 3\n captures:\n - string: abc\n offset: 0\n length: 3\n- string: abc\n offset: 4\n length: 3\n captures:\n - string: abc\n offset: 4\n length: 3\n",
	},
	},

After:

	{
		description: "Match with global capture group",
		document:    `abc abc`,
		expression:  `[match("(ab)(c)"; "g")]`,
		expected: []string{
			"D0, P[], (!!seq)::- string: abc\n  offset: 0\n  length: 3\n  captures:\n    - string: ab\n      offset: 0\n      length: 2\n    - string: c\n      offset: 2\n      length: 1\n- string: abc\n  offset: 4\n  length: 3\n  captures:\n    - string: ab\n      offset: 4\n      length: 2\n    - string: c\n      offset: 6\n      length: 1\n",
		},
	},

Hope it helps 👍

[mikefarah]

Fixed in 4.20.2