Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Call setSustainedPerformanceMode for better benchmarks #2329

Merged
merged 1 commit into from Nov 4, 2022

Conversation

stefanosiano
Copy link
Member

📜 Description

Added window.setSustainedPerformanceMode(true) on benchmark activity, to get better results.
Reduced espresso default timeout to 10 seconds
Updated list of devices for benchmarks

#skip-changelog

💡 Motivation and Context

setSustainedPerformanceMode is one of the things the android benchmark runner (which we cannot use due to problems on Saucelabs) does. It tries to limit the thermal throttling by reducing the overall performance. https://developer.android.com/reference/kotlin/androidx/benchmark/junit4/AndroidBenchmarkRunner#clock-stability
This will result in more predictable benchmarks, as currently there are few runs failing.
Saucelabs devices for benchmarks were updated: the Google Pixel 3 was replaced because consistently timing out

💚 How did you test it?

📝 Checklist

  • I reviewed the submitted code
  • I added tests to verify the changes
  • I updated the docs if needed
  • No breaking changes

🔮 Next steps

… to get better results

reduced espresso default timeout to 10 seconds
updated list of devices for benchmarks
@github-actions
Copy link
Contributor

github-actions bot commented Nov 3, 2022

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 369.58 ms 405.46 ms 35.88 ms
Size 1.73 MiB 2.32 MiB 608.53 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
f809aac 301.51 ms 346.60 ms 45.09 ms
90e9745 314.68 ms 357.28 ms 42.60 ms

App size

Revision Plain With Sentry Diff
f809aac 1.73 MiB 2.32 MiB 608.63 KiB
90e9745 1.73 MiB 2.32 MiB 608.63 KiB

@stefanosiano stefanosiano marked this pull request as ready for review November 3, 2022 17:38
Copy link
Member

@markushi markushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@stefanosiano stefanosiano merged commit 34811f4 into main Nov 4, 2022
@stefanosiano stefanosiano deleted the tests/benchmark-sustained-mode branch November 4, 2022 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants