Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ValidParser #68

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Conversation

kevburnsjr
Copy link
Contributor

@kevburnsjr kevburnsjr commented May 16, 2021

Presently, users that want to validate their json before parsing need to run Validate and Parse separately.
This is inefficient because each json string must be scanned twice.

This PR introduces a ValidParser which validates as it parses.

It is slightly less efficient than the normal parser, but much more efficient than running Validate and Parse separately.

BenchmarkParse/small/fastjson-12                        10067671      111.0 ns/op  1712.14 MB/s    0 B/op  0 allocs/op
BenchmarkParse/small/fastjson-valid-parser-12           10503465      109.8 ns/op  1729.86 MB/s    0 B/op  0 allocs/op
BenchmarkParse/small/fastjson-validate-and-parse-12      6553486      176.5 ns/op  1076.23 MB/s    0 B/op  0 allocs/op

BenchmarkParse/medium/fastjson-12                        1797662      639.2 ns/op  3643.68 MB/s    0 B/op  0 allocs/op
BenchmarkParse/medium/fastjson-valid-parser-12           1635453      795.7 ns/op  2927.14 MB/s    0 B/op  0 allocs/op
BenchmarkParse/medium/fastjson-validate-and-parse-12      945916     1400 ns/op    1663.97 MB/s    0 B/op  0 allocs/op

BenchmarkParse/large/fastjson-12                          121093     9001 ns/op    3123.79 MB/s    0 B/op  0 allocs/op
BenchmarkParse/large/fastjson-valid-parser-12             116598    10946 ns/op    2568.69 MB/s    0 B/op  0 allocs/op
BenchmarkParse/large/fastjson-validate-and-parse-12        57669    20630 ns/op    1362.95 MB/s    0 B/op  0 allocs/op

BenchmarkParse/canada/fastjson-12                           1072  1205458 ns/op    1867.39 MB/s    1 B/op  0 allocs/op
BenchmarkParse/canada/fastjson-valid-parser-12              1022  1151216 ns/op    1955.38 MB/s    1 B/op  0 allocs/op
BenchmarkParse/canada/fastjson-validate-and-parse-12         678  1781026 ns/op    1263.91 MB/s    2 B/op  0 allocs/op

BenchmarkParse/citm/fastjson-12                             2584   476059 ns/op    3628.13 MB/s    0 B/op  0 allocs/op
BenchmarkParse/citm/fastjson-valid-parser-12                2528   481104 ns/op    3590.09 MB/s    0 B/op  0 allocs/op
BenchmarkParse/citm/fastjson-validate-and-parse-12          1443   844153 ns/op    2046.08 MB/s    1 B/op  0 allocs/op

BenchmarkParse/twitter/fastjson-12                          6711   164653 ns/op    3835.42 MB/s  756 B/op  0 allocs/op
BenchmarkParse/twitter/fastjson-valid-parser-12             5918   192253 ns/op    3284.80 MB/s    0 B/op  0 allocs/op
BenchmarkParse/twitter/fastjson-validate-and-parse-12       3758   317174 ns/op    1991.06 MB/s    0 B/op  0 allocs/op

Adds 7 new symbols to the API:

type ValidScanner
type ValidParser 
type ValidParserPool
func ValidParse(s string) (*Value, error)
func ValidParseBytes(b []byte) (*Value, error)
func MustValidParse(s string) *Value
func MustValidParseBytes(b []byte) *Value

See #58

@codecov
Copy link

codecov bot commented May 16, 2021

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6dae91c) 97.11% compared to head (80b9537) 97.77%.
Report is 2 commits behind head on master.

❗ Current head 80b9537 differs from pull request most recent head f240cbf. Consider uploading reports for the commit f240cbf to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #68      +/-   ##
==========================================
+ Coverage   97.11%   97.77%   +0.66%     
==========================================
  Files           9       11       +2     
  Lines        1073     1306     +233     
==========================================
+ Hits         1042     1277     +235     
+ Misses         22       21       -1     
+ Partials        9        8       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@flowchartsman
Copy link

flowchartsman commented Mar 22, 2023

I, too, do not enjoy needing two calls for peace-of-mind that my JSON will not break once I start processing it. This would be super handy to have merged into the library, and the PR looks well-tested and benchmarked.

@valyala, is there some hesitation with this feature which has left it in limbo for so long? If not, I think a lot of people would appreciate being able to avoid separate invocations. 🙏

@kevburnsjr kevburnsjr changed the title Add ValidateParse, ValidateParser and ValidateScanner Add ValidParse, ValidParser and ValidScanner Feb 17, 2024
@kevburnsjr kevburnsjr changed the title Add ValidParse, ValidParser and ValidScanner Add ValidParser Feb 17, 2024
@kevburnsjr
Copy link
Contributor Author

I've updated the symbol names. ValidateParser is now ValidParser for brevity.
I will maintain a fork with this PR merged until this PR is merged for anyone who wants to use it.
https://github.com/logbn/fastjson-valid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants