Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encode a struct with ",remain" back to a map? #279

Open
kszafran opened this issue Apr 7, 2022 · 5 comments · May be fixed by #307
Open

Encode a struct with ",remain" back to a map? #279

kszafran opened this issue Apr 7, 2022 · 5 comments · May be fixed by #307
Labels

Comments

@kszafran
Copy link

kszafran commented Apr 7, 2022

I have a use case where I need to decode a JSON, where I care only about a few fields, change it here and there, and then serialize it back to JSON. However, the remainder values seem to get serialized into an empty "" JSON field. This is my code:

func TestDecodeEncode(t *testing.T) {
	var stuff struct {
		Field string                 `mapstructure:"field"`
		Other map[string]interface{} `mapstructure:",remain"`
	}
	err := Decode(strings.NewReader(`{
		"field": "hello",
		"anotherField": "world"
	}`), &stuff)
	if err != nil {
		panic(err)
	}
	bs, err := Encode(stuff)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s\n", bs)
}

func Decode(r io.Reader, result interface{}) error {
	var m interface{}
	if err := json.NewDecoder(r).Decode(&m); err != nil {
		return err
	}
	return mapstructure.Decode(m, result)
}

func Encode(v interface{}) ([]byte, error) {
	var m map[string]interface{}
	if err := mapstructure.Decode(v, &m); err != nil {
		return nil, err
	}
	return json.Marshal(m)
}

And the output is:

{"":{"anotherField":"world"},"field":"hello"}

Am I missing some configuration option or is remain just not supported when encoding a struct into a map?

I'm using mapstructure version v1.4.3.

@kszafran
Copy link
Author

kszafran commented Apr 7, 2022

For now I'm using this simple workaround:

func Encode(v interface{}) ([]byte, error) {
	var m map[string]interface{}
	if err := mapstructure.Decode(v, &m); err != nil {
		return nil, err
	}
	expandRemainderValues(m)
	return json.Marshal(m)
}

func expandRemainderValues(m map[string]interface{}) {
	for k, v := range m {
		v, ok := v.(map[string]interface{})
		if !ok {
			continue
		}
		if k == "" {
			for remainderK, remainderV := range v {
				m[remainderK] = remainderV
			}
			delete(m, "")
		} else {
			expandRemainderValues(v)
		}
	}
}

@mitchellh mitchellh added the bug label Apr 20, 2022
@kszafran
Copy link
Author

Looks like the behavior has changed in 1.5.0. Now my Other fields are serialized not as "", but as "Other". (It doesn't solve the problem of course, but requires changing the workaround.)

@mitchellh
Copy link
Owner

Yeah, there was a bug fixed in 1.5.0 that caused some struct fields to get an empty string in the map. So you're seeing that fix, while still retaining your bug. We just need to turn your repro into a test case, and get a fix on top of it.

vaguecoder added a commit to vaguecoder/mapstructure that referenced this issue Oct 2, 2022
Signed-off-by: Bhargav Ravuri <vaguecoder0to.n@gmail.com>
@vaguecoder vaguecoder linked a pull request Oct 2, 2022 that will close this issue
@tschaub
Copy link

tschaub commented Mar 7, 2023

#307 looks like a nice fix for this issue.

@Arsen204
Copy link

@mitchellh can we merge #307? it's a tested by prod fix :)

sagikazarmark added a commit to go-viper/mapstructure that referenced this issue Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants