Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decode fails when input stream ends with a newline #698

Open
lev112 opened this issue Apr 19, 2024 · 0 comments
Open

Decode fails when input stream ends with a newline #698

lev112 opened this issue Apr 19, 2024 · 0 comments

Comments

@lev112
Copy link

lev112 commented Apr 19, 2024

when using the Encode of the standard package, it adds a \n after every object,
but when json-iterator gets it as an input, it fails,
see code for an example:

import (
	"bytes"
	"encoding/json"
	"io"
	"testing"

	jsoniter "github.com/json-iterator/go"

	"github.com/stretchr/testify/require"
)

func TestJsonIterator(t *testing.T) {
	// encode jsons using the built in package
	buf := new(bytes.Buffer)
	enc := json.NewEncoder(buf)
	err := enc.Encode(map[string]string{"key": "value"})
	require.NoError(t, err)
	err = enc.Encode(map[string]string{"key2": "value2"})
	require.NoError(t, err)

	// after the last value, \n is added
	var nl byte = '\n'
	require.Equal(t, nl, buf.Bytes()[buf.Len()-1])

	// the build in package can decode the values as expected
	reader1 := bytes.NewReader(buf.Bytes())
	dec1 := json.NewDecoder(reader1)

	var res map[string]string

	err = dec1.Decode(&res)
	require.NoError(t, err)
	require.Equal(t, map[string]string{"key": "value"}, res)

	res = nil
	err = dec1.Decode(&res)
	require.NoError(t, err)
	require.Equal(t, map[string]string{"key2": "value2"}, res)

	res = nil
	err = dec1.Decode(&res)
	require.Equal(t, io.EOF, err)

	reader2 := bytes.NewReader(buf.Bytes())
	// if the last \n is removed from the input, everything works as expected
	//reader2 := bytes.NewReader(buf.Bytes()[:buf.Len()-1])

	dec2 := jsoniter.ConfigDefault.NewDecoder(reader2)

	err = dec2.Decode(&res)
	require.NoError(t, err)
	require.Equal(t, map[string]string{"key": "value"}, res)

	res = nil
	err = dec2.Decode(&res)
	require.NoError(t, err)
	require.Equal(t, map[string]string{"key2": "value2"}, res)

	res = nil
	err = dec2.Decode(&res)
	// this check fails
	// the actual error is:
	// &errors.errorString{s:"ReadMapCB: expect { or n, but found \x00, error found in #10 byte of ...|:\"value2\"}\n|..., bigger context ...|{\"key\":\"value\"}\n{\"key2\":\"value2\"}\n|..."}
	require.Equal(t, io.EOF, err)
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant