Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sonic.Marshal Incidental panic msg "runtime error: growslice: cap out of range" #536

Open
GoneWithWind1 opened this issue Oct 8, 2023 · 5 comments

Comments

@GoneWithWind1
Copy link

GoneWithWind1 commented Oct 8, 2023

高并发场景下使用sonic出现growslice: cap out of range的panic,详细报错

github.com/bytedance/sonic/internal/encoder.encodeTypedPointer(0xc0872996b0, 0x18da580, 0xc087ed3eb8, 0x7fba581efa68?, 0x0)
	/root/go/pkg/mod/github.com/bytedance/sonic@v1.9.1/internal/encoder/primitives.go:83 +0x1c5
github.com/bytedance/sonic/internal/encoder.encodeInto(0xc0872996b0, {0x18da580, 0xc087edd680}, 0xc0867ac320?)
	/root/go/pkg/mod/github.com/bytedance/sonic@v1.9.1/internal/encoder/encoder.go:211 +0xdf
github.com/bytedance/sonic/internal/encoder.Encode({0x18da580, 0xc087edd680}, 0xc087edd680?)
	/root/go/pkg/mod/github.com/bytedance/sonic@v1.9.1/internal/encoder/encoder.go:168 +0xcc
github.com/bytedance/sonic.frozenConfig.Marshal(...)
	/root/go/pkg/mod/github.com/bytedance/sonic@v1.9.1/sonic.go:83
github.com/bytedance/sonic.Marshal(...)
	/root/go/pkg/mod/github.com/bytedance/sonic@v1.9.1/api.go:148
@liuq19
Copy link
Collaborator

liuq19 commented Oct 9, 2023

能否提供可以复现的完整代码和环境参数

@GoneWithWind1 GoneWithWind1 reopened this Oct 9, 2023
@GoneWithWind1 GoneWithWind1 closed this as not planned Won't fix, can't repro, duplicate, stale Oct 9, 2023
@GoneWithWind1 GoneWithWind1 reopened this Oct 9, 2023
@GoneWithWind1 GoneWithWind1 reopened this Oct 9, 2023
@GoneWithWind1
Copy link
Author

能否提供可以复现的完整代码和环境参数
@liuq19 复现场景是高并发和超大结构体转换时出现问题,完整代码不太方便贴出来给出部分代码如下:
func Map2Struct(mapData interface{}, data interface{}) error { if tools.IsEmpty(mapData) { return errors.New("mapData empty") } if reflect.Ptr != reflect.TypeOf(data).Kind() { return errors.New("data is not ptr") } strInfo, err := sonic.Marshal(mapData) if err != nil { return err } err = sonic.Unmarshal(strInfo, data) return err },panic报错为sonic.Marshal这一行,定位到底层包encodeTypedPointer函数的err := fn(buf, unsafe.Pointer(vp), sb, fv)这行,
go env参数:
GO111MODULE="on" GOARCH="amd64" GOBIN="/home/work/go/bin" GOCACHE="/home/work/.cache/go-build" GOENV="/home/work/.config/go/env" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/work/go/pkg/mod" GONOSUMDB="*" GOOS="linux" GOPATH="/home/work/go" GOROOT="/home/work/sdk/go1.20.7" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/home/work/sdk/go1.20.7/pkg/tool/linux_amd64" GOVCS="" GOVERSION="go1.20.7" GCCGO="gccgo" GOAMD64="v1" AR="ar" CC="/opt/compiler/gcc-8.2/bin/gcc" CXX="/opt/compiler/gcc-8.2/bin/g++" CGO_ENABLED="1" GOMOD="/dev/null" GOWORK="" CGO_CFLAGS="-O2 -g" CGO_CPPFLAGS="" CGO_CXXFLAGS="-O2 -g" CGO_FFLAGS="-O2 -g" CGO_LDFLAGS="-O2 -g" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3757990476=/tmp/go-build -gno-record-gcc-switches"
个人猜测可能和内存管理有关,在结构体小的高并发场景下不复现

@AsterDY
Copy link
Collaborator

AsterDY commented Oct 11, 2023

更新到1.10.2试试

@GoneWithWind1
Copy link
Author

更新到1.10.2试试

@AsterDY 请问1.9.1->1.10.2是有做这部分代码的优化么?

@AsterDY
Copy link
Collaborator

AsterDY commented Oct 31, 2023

可能有gc相关的问题。请问有解决吗?@GoneWithWind1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants