Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize EngineFileUtilities.GetFileList #6227

Merged
merged 6 commits into from Mar 11, 2021

Conversation

ladipro
Copy link
Member

@ladipro ladipro commented Mar 4, 2021

Fixes #6061

Context

EngineFileUtilities.GetFileList has been identified as one of the performance bottlenecks when building .NET Core projects. Since this code is called as part of project evaluation, it directly impacts Visual Studio performance as well, especially solution load.

Changes Made

Tweaked the code under GetFileList:

  • Smarter order of wildcard checks to optimize for the common case.
  • Optimized hex number parsing and wildcard detection.

Testing

Existing unit tests for correctness, ETL traces for performance.

This change together with #6151 is showing about 30% less time spent in this particular function when building a Hello World .NET Core project with the Framework version of MSBuild. It's an OK win for project evaluation perf but translates to less than 1 ms of total command line build time.

Notes

This PR is small but still recommended to be reviewed commit by commit.

@ladipro ladipro force-pushed the 6061-optimize-GetFileListEscaped branch 2 times, most recently from a1f33c2 to 1117dac Compare March 5, 2021 09:11
Copy link
Member

@Forgind Forgind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

{
return false;
}

// If the item's Include has both escaped wildcards and real wildcards, then it's
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You took out the color from the original! But that's ok 😉

Copy link
Member

@benvillalobos benvillalobos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM

string hexString = escapedString.Substring(indexOfPercent + 1, 2);
char unescapedCharacter = (char)int.Parse(hexString, System.Globalization.NumberStyles.HexNumber,
CultureInfo.InvariantCulture);
char unescapedCharacter = (char)((digit1 << 4) + digit2);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

}
index = escapedString.IndexOf('%', index + 1, escapedString.Length - index - 3);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why subtract 3 here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want something that looks like %3f or whatever, and that takes three characters. This means no need for verifying that there are enough characters afterwards.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, index + 1 means 1 less character to look at ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we also know that index is the index of a %, so using index instead would result in saying index = index; ad infinitum.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tweaked the expression to be more readable and added comments.

@Forgind Forgind added the merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now. label Mar 8, 2021
@ladipro ladipro force-pushed the 6061-optimize-GetFileListEscaped branch from 1117dac to 5f4f7c2 Compare March 9, 2021 10:41
@benvillalobos benvillalobos merged commit e06c993 into dotnet:master Mar 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Microsoft.Build.Internal.EngineFileUtilities.GetFileListEscaped is 3x slower for netcore projects
5 participants