You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, .NET runtime CI tests run on "any platform/any device," but they do not evaluate the performance of the generated code. When developers make changes to the code, they typically rely on autofilling performance issues to detect performance-related issues. This approach can lead to a delay in identifying the actual cause of the issue and proposing a fix. Additionally, these issues are often reported on a weekly basis, which can further prolong the delay in addressing performance issues.
Proposal
The proposed solution is to introduce performance templates for hot code paths and intrinsics in the form of functional tests. Running microbenchmarks for every PR can be resource-intensive. This is where these templates can be beneficial. We can create a functional application that runs as part of our functional tests and measures frequently used code paths. The local script could help here, but it requires additional manual verification which is sometimes skipped, or not tested on all architectures/OS.
These tests should focus on measuring both code size and execution speed. We should ensure that any changes made to the code do not result in significant performance regressions in the covered code paths. While a local script could assist in this regard, it often requires additional manual verification, which is sometimes skipped or not tested on all architectures and operating systems.
The text was updated successfully, but these errors were encountered:
Motivation and background
Currently, .NET runtime CI tests run on "any platform/any device," but they do not evaluate the performance of the generated code. When developers make changes to the code, they typically rely on autofilling performance issues to detect performance-related issues. This approach can lead to a delay in identifying the actual cause of the issue and proposing a fix. Additionally, these issues are often reported on a weekly basis, which can further prolong the delay in addressing performance issues.
Proposal
The proposed solution is to introduce performance templates for hot code paths and intrinsics in the form of functional tests. Running microbenchmarks for every PR can be resource-intensive. This is where these templates can be beneficial. We can create a functional application that runs as part of our functional tests and measures frequently used code paths. The local script could help here, but it requires additional manual verification which is sometimes skipped, or not tested on all architectures/OS.
These tests should focus on measuring both code size and execution speed. We should ensure that any changes made to the code do not result in significant performance regressions in the covered code paths. While a local script could assist in this regard, it often requires additional manual verification, which is sometimes skipped or not tested on all architectures and operating systems.
The text was updated successfully, but these errors were encountered: