Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding.TextUnmarshaler not supported on koanf.Unmarshaled #163

Closed
genofire opened this issue Aug 12, 2022 · 1 comment
Closed

encoding.TextUnmarshaler not supported on koanf.Unmarshaled #163

genofire opened this issue Aug 12, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@genofire
Copy link

genofire commented Aug 12, 2022

Describe the bug
Custom Datetypes (like for time, data unit) are not Unmarshaled (on yaml).
Therefore is https://pkg.go.dev/encoding#TextUnmarshaler defined and

To Reproduce

type DataUnit int64

func (d *DataUnit) UnmarshalText(text []byte) error {
	var magnitude float64
	s := strings.ToLower(string(text))
	switch {
	case strings.HasSuffix(s, "tb"):
		s, magnitude = s[:len(s)-2], 1024*1024*1024*1024
	case strings.HasSuffix(s, "gb"):
		s, magnitude = s[:len(s)-2], 1024*1024*1024
	case strings.HasSuffix(s, "mb"):
		s, magnitude = s[:len(s)-2], 1024*1024
	case strings.HasSuffix(s, "kb"):
		s, magnitude = s[:len(s)-2], 1024
	default:
		magnitude = 1
	}
	v, err := strconv.ParseFloat(s, 64)
	if err != nil {
		return err
	}
	*d = DataUnit(v * magnitude)
	return nil
}

func TestUnmarshalDataUnit(t *testing.T) {
	target := struct {
		Got DataUnit `config:"value"`
	}{}
	for input, expect := range map[string]DataUnit{
		"value: 0.6tb": 659706976665,
		"value: 1.2gb": 1288490188,
		"value: 256mb": 268435456,
		"value: 128kb": 131072,
		"value: 128":   128,
	} {
		k := koanf.New("/")
		if err := k.Load(rawbytes.Provider([]byte(input)), yaml.Parser()); err != nil {
			t.Fatal(err)
		}
		if err := k.UnmarshalWithConf("", &target, koanf.UnmarshalConf{Tag: "config"}); err != nil {
			t.Fatal(err)
		} else if target.Got != expect {
			t.Fatalf("expected value %d but got %d", expect, target.Got)
		}
	}
}

Test failed:

--- FAIL: TestUnmarshalDataUnit (0.00s)
    config_test.go:295: 1 error(s) decoding:
        
        * cannot parse 'value' as int: strconv.ParseInt: parsing "256mb": invalid syntax
FAIL
FAIL	github.com/matrix-org/dendrite/setup/config	0.012s
FAIL

Expected behavior
Defined Test works

@genofire
Copy link
Author

genofire commented Aug 12, 2022

default decoder should contain more Hooks, like:

mapstructure.ComposeDecodeHookFunc(
				mapstructure.StringToTimeDurationHookFunc(),
				mapstructure.StringToSliceHookFunc(","),
				mapstructure.TextUnmarshallerHookFunc())

in

koanf/koanf.go

Line 242 in ed3461c

mapstructure.StringToTimeDurationHookFunc()),


mainly mapstructure.TextUnmarshallerHookFunc() would solve the problem.

@genofire genofire changed the title enconfing.TextUnmarshaler not supported on koanf.Unmarshaled encoding.TextUnmarshaler not supported on koanf.Unmarshaled Aug 12, 2022
@knadh knadh closed this as completed in 0e9e5e9 Aug 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant