Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recursive struct type causes stack overflow / Handling of overlapping field tagged name is not aligned to std #657

Open
ngicks opened this issue Feb 14, 2023 · 1 comment · May be fixed by #659 or #660

Comments

@ngicks
Copy link

ngicks commented Feb 14, 2023

  • Using jsoniter.Marshal against struct types which contain recursive embedded type causes stack overflow.
    • std's encoding/json prevents this by these lines
    • Jsoniter seemingly does not have that visited map by quick read through of source code.
  • Also handling of overlapping tagged field names are not aligned to what is of std lib.
    • std's encoding/json describes this behavior in its source code but is not documented.

Searching through issues with "recursive" does not give me any similar issues.
Are these behaviors intentionally dropped?

> go version
go version go1.20 linux/amd64
require github.com/json-iterator/go v1.1.12

I've uploaded test code here.

https://github.com/ngicks/test-jsoniter

package testjsoniter_test

import (
	"encoding/json"
	"testing"

	jsoniter "github.com/json-iterator/go"
)

type OverlappingKey1 struct {
	Foo string
	Bar string `json:"Baz"`
	Baz string
}

type OverlappingKey2 struct {
	Foo string
	Bar string `json:"Bar"`
	Baz string `json:"Bar"`
}

type OverlappingKey3 struct {
	Foo string
	Bar string `json:"Baz"`
	Baz string
	Qux string `json:"Baz"`
}

type Sub1 struct {
	Foo string
	Bar string `json:"Bar"`
}

type OverlappingKey4 struct {
	Foo string
	Bar string
	Baz string
	Sub1
}

type Recursive1 struct {
	R string `json:"r"`
	Recursive2
}

type Recursive2 struct {
	R  string `json:"r"`
	RR string `json:"rr"`
	*OverlappingKey5
}

type OverlappingKey5 struct {
	Foo string
	Recursive1
}

func TestJsonIter_behavioral_deference(t *testing.T) {
	for _, config := range []jsoniter.API{
		jsoniter.ConfigCompatibleWithStandardLibrary,
		jsoniter.ConfigDefault,
	} {
		for _, v := range []any{
			OverlappingKey1{"foo", "bar", "baz"},
			OverlappingKey2{"foo", "bar", "baz"},
			OverlappingKey3{"foo", "bar", "baz", "qux"},
			OverlappingKey4{Foo: "foo", Bar: "bar", Baz: "baz", Sub1: Sub1{Foo: "foofoo", Bar: "barbar"}},
			// These cause stack overflow
			// OverlappingKey5{Foo: "foo", Recursive1: Recursive1{R: "r", Recursive2: Recursive2{R: "rr"}}},
			// OverlappingKey5{Foo: "foo", Recursive1: Recursive1{R: "r", Recursive2: Recursive2{R: "rr", OverlappingKey5: &OverlappingKey5{Foo: "foo"}}}},
		} {
			binOrg, err := json.Marshal(v)
			if err != nil {
				panic(err)
			}
			binMimic, err := config.Marshal(v)
			if err != nil {
				panic(err)
			}

			str1, str2 := string(binOrg), string(binMimic)
			if str1 != str2 {
				t.Errorf("not same, type = %T. org = %s, jsoniter = %s\n", v, str1, str2)
			}

		}
	}
}

This gives result of

> go test -v ./...
=== RUN   TestJsonIter_behavioral_deference
    recursive_test.go:82: not same, type = testjsoniter_test.OverlappingKey1. org = {"Foo":"foo","Baz":"bar"}, jsoniter = {"Foo":"foo","Baz":"baz"}
    recursive_test.go:82: not same, type = testjsoniter_test.OverlappingKey3. org = {"Foo":"foo"}, jsoniter = {"Foo":"foo","Baz":"qux"}
    recursive_test.go:82: not same, type = testjsoniter_test.OverlappingKey4. org = {"Foo":"foo","Bar":"bar","Baz":"baz"}, jsoniter = {"Foo":"foo","Baz":"baz","Bar":"barbar"}
    recursive_test.go:82: not same, type = testjsoniter_test.OverlappingKey1. org = {"Foo":"foo","Baz":"bar"}, jsoniter = {"Foo":"foo","Baz":"baz"}
    recursive_test.go:82: not same, type = testjsoniter_test.OverlappingKey3. org = {"Foo":"foo"}, jsoniter = {"Foo":"foo","Baz":"qux"}
    recursive_test.go:82: not same, type = testjsoniter_test.OverlappingKey4. org = {"Foo":"foo","Bar":"bar","Baz":"baz"}, jsoniter = {"Foo":"foo","Baz":"baz","Bar":"barbar"}
--- FAIL: TestJsonIter_behavioral_deference (0.00s)
FAIL
FAIL    github.com/ngicks/test-jsoniter.git     0.002s
FAIL
@ngicks
Copy link
Author

ngicks commented Feb 14, 2023

Oh it doesn't also align its behavior to std where ,string option should not apply to null or non-applicable type.
The former one is poorly documented. The latter is explicit.

type Quote struct {
	Foo int  `json:",string"`
	Bar *int `json:",string"`
	Baz Sub  `json:",string"`
}

type Sub struct {
	Qux string
}

func TestQuote_behavior(t *testing.T) {
	v := Quote{
		Foo: 10,
		Bar: nil,
		Baz: Sub{
			Qux: "qux",
		},
	}

	binOrg, err := json.Marshal(v)
	if err != nil {
		panic(err)
	}
	binMimic, err := jsoniter.Marshal(v)
	if err != nil {
		panic(err)
	}

	str1, str2 := string(binOrg), string(binMimic)
	if str1 != str2 {
		t.Errorf("not same, type = %T. org = %s, jsoniter = %s\n", v, str1, str2)
	}
}
--- FAIL: TestQuote_behavior (0.00s)
    string_opt_test.go:40: not same, type = testjsoniter_test.Quote. org = {"Foo":"10","Bar":null,"Baz":{"Qux":"qux"}}, jsoniter = {"Foo":"10","Bar":"null","Baz":"{"Qux":"qux"}"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant