Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BinaryFormater serialization of on-disk RAR cache is slow #6057

Closed
Tracked by #5873
rokonec opened this issue Jan 21, 2021 · 1 comment
Closed
Tracked by #5873

BinaryFormater serialization of on-disk RAR cache is slow #6057

rokonec opened this issue Jan 21, 2021 · 1 comment
Assignees
@rokonec
Labels
performance triaged
Projects
16.10 Perf sprint Feb 1st - Feb 14th 2021
Milestone
MSBuild 16.10

Comments

@rokonec
Copy link
Contributor

rokonec commented Jan 21, 2021 •
edited

Issue Description

BinaryFormater serialization of on-disk RAR cache is slow.
Optimizing deserialization time will likely be synergic with RAR pre-caching goals.

Steps to Reproduce

Incremental rebuild or Roslyn repo.

Data

For Roslyn incremental build reading and parsing the RAR cache took almost 40% of RAR wall-clock time.
image

Analysis

When looking for optimal solution benchmark was create to measure serialization and deserialization of real project RAR cached data.
SerializationBenchmarks

After evaluating pros and cons decision has been made to implement serialization as custom binary format, hand written code.

During this decision following was considered:

  • CPU performance
  • memory performance
  • on-disk size
  • maintainability
  • security
  • estimated effort

Versions & Configurations

Measured at latest production dotnet 5.0.102 SDK.
@rokonec rokonec self-assigned this Jan 21, 2021
@rokonec rokonec added this to To Do in 16.10 via automation Jan 21, 2021
@rokonec rokonec moved this from To Do to In Progress in 16.10 Jan 21, 2021
This was referenced Jan 22, 2021
Closed
Merged
@ladipro ladipro added this to To Do in Perf sprint Feb 1st - Feb 14th 2021 via automation Feb 1, 2021
@ladipro ladipro moved this from To Do to In Progress in Perf sprint Feb 1st - Feb 14th 2021 Feb 1, 2021
@ladipro ladipro mentioned this issue Feb 1, 2021
Closed
4 tasks
@ladipro ladipro moved this from In Progress to Done in Perf sprint Feb 1st - Feb 14th 2021 Feb 15, 2021
ladipro pushed a commit that referenced this issue Feb 25, 2021
@rokonec
On disk cache serialization (#6094)
6819f7a 
### Context
BinaryFormater serialization of on-disk RAR cache is slow and unsecure. 
Related issue: #6057

### Changes Made
Serialization changed to use custom binary format by using existing `ITranslatable` 

### Testing
Unit testing
Manual testing
- Roslyn repo rebuild + incremental build
@marcpopMSFT
Copy link
Member

@rokonec #6094 is in so closing this out. Reactivate if you were using this to track any additional changes.

16.10 automation moved this from In Progress to Done Mar 2, 2021
@ladipro ladipro added this to the MSBuild 16.10 milestone Dec 9, 2021
@AR-May AR-May added the triaged label Feb 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rokonec rokonec

Labels
performance triaged
Projects
16.10
Done
Perf sprint Feb 1st - Feb 14th 2021
Done
Milestone
MSBuild 16.10
Development

No branches or pull requests
4 participants
@ladipro @marcpopMSFT @rokonec @AR-May