Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use expanded Analyzer for some validators for speed #149

Merged
merged 1 commit into from May 20, 2022

Conversation

koron
Copy link
Contributor

@koron koron commented May 19, 2022

this may fix a part of #146

the slow validation is caused by many $refs in a YAML
See also an example spec file ref-1000.zip

the cause is that defaultValidator and exampleValidator don't use a common resolve cache.
so each $ref loads YAML file and it takes about 20msec in per a load.
therefore it tooks 20sec if YAML has 1000 $ref

in this patch, it uses s.expanded.Analyzer instead of s.analyer if it is available.
s.expanded and Analyzer are expected that be filled by validateReferencesValid which called by SpecValidator.Validate.

@koron
Copy link
Contributor Author

koron commented May 19, 2022

i'm not sure that this is right to fix.
but in my case, significantly improved speed in my case: 28sec -> 4sec

@koron koron changed the title use expaneded Analyzer for some validators for speed use expanded Analyzer for some validators for speed May 19, 2022
@koron koron force-pushed the validation-extremely-slow-146 branch from b68537f to cfc09bf Compare May 19, 2022 08:41
@koron
Copy link
Contributor Author

koron commented May 19, 2022

i modify more validators to use expanded analyzer.

@youyuanwu youyuanwu requested a review from casualjim May 20, 2022 02:47
@codecov
Copy link

codecov bot commented May 20, 2022

Codecov Report

Merging #149 (cfc09bf) into master (c5b9514) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #149      +/-   ##
==========================================
- Coverage   90.70%   90.65%   -0.06%     
==========================================
  Files          22       22              
  Lines        2970     2975       +5     
==========================================
+ Hits         2694     2697       +3     
- Misses        224      225       +1     
- Partials       52       53       +1     
Impacted Files Coverage Δ
default_validator.go 87.62% <100.00%> (ø)
example_validator.go 83.06% <100.00%> (ø)
helpers.go 82.24% <100.00%> (ø)
spec.go 87.08% <100.00%> (-0.24%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5b9514...cfc09bf. Read the comment docs.

@casualjim casualjim merged commit ae5ca97 into go-openapi:master May 20, 2022
@youyuanwu
Copy link
Member

@casualjim Do we have some regression, or this is just a improvement?

@casualjim
Copy link
Member

I think this was just an improvement.

@koron koron deleted the validation-extremely-slow-146 branch May 22, 2022 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants