Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added BufferedStream for reading from DeflateStream - reduces loading… #5924

Merged
merged 1 commit into from Oct 5, 2021

Conversation

martintomasek
Copy link
Contributor

Fixes issue #5913
Issue from .NET runtime: dotnet/runtime#39233 (comment)

Buffered reader around the binary reader will significantly improve loading time of bigger ML models using .NET/.NET Core runtime.

@codecov
Copy link

codecov bot commented Sep 3, 2021

Codecov Report

Merging #5924 (39c066c) into main (6ae2cff) will increase coverage by 0.15%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #5924      +/-   ##
==========================================
+ Coverage   68.19%   68.35%   +0.15%     
==========================================
  Files        1142     1142              
  Lines      242367   244071    +1704     
  Branches    25355    25861     +506     
==========================================
+ Hits       165287   166835    +1548     
- Misses      70400    70549     +149     
- Partials     6680     6687       +7     
Flag Coverage Δ
Debug 68.35% <100.00%> (+0.15%) ⬆️
production 62.98% <100.00%> (+0.10%) ⬆️
test 88.72% <ø> (+0.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...rosoft.ML.Data/DataLoadSave/Binary/BinaryLoader.cs 63.17% <100.00%> (+0.03%) ⬆️
...ansforms/PermutationFeatureImportanceExtensions.cs 92.17% <0.00%> (-3.71%) ⬇️
.../Microsoft.ML.Data/Prediction/CalibratorCatalog.cs 92.73% <0.00%> (-1.08%) ⬇️
...rosoft.Data.Analysis/ArrowStringDataFrameColumn.cs 63.54% <0.00%> (-0.47%) ⬇️
...c/Microsoft.ML.FastTree/Dataset/SegmentIntArray.cs 0.00% <0.00%> (ø)
...soft.ML.Transforms/Text/WordEmbeddingsExtractor.cs 86.87% <0.00%> (ø)
...osoft.Data.Analysis.Tests/DataFrameGroupByTests.cs 100.00% <0.00%> (ø)
...soft.ML.Tests/PermutationFeatureImportanceTests.cs 100.00% <0.00%> (ø)
...L.FastTree/TreeEnsembleFeaturizationTransformer.cs 100.00% <0.00%> (ø)
...st/Microsoft.Data.Analysis.Tests/DataFrameTests.cs 99.45% <0.00%> (+0.01%) ⬆️
... and 16 more

Copy link
Member

@michaelgsharp michaelgsharp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for my delay. Looks good to me. A couple test projects had some intermittent failures, so I have retriggered those. Will merge this in as soon as they complete.

@michaelgsharp
Copy link
Member

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@michaelgsharp michaelgsharp merged commit 51c4984 into dotnet:main Oct 5, 2021
@dotnet dotnet locked as resolved and limited conversation to collaborators Mar 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants