Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Evaluate Profile-Guided Optimization (PGO) #2627

Open
zamazan4ik opened this issue Jul 16, 2023 · 3 comments
Open

Evaluate Profile-Guided Optimization (PGO) #2627

zamazan4ik opened this issue Jul 16, 2023 · 3 comments
Labels
A-frontend Area: Frontends for interpreters, like the `ruby` or `irb` binaries. A-performance Area: Performance improvements and optimizations. C-enhancement Category: New feature or request. E-help-wanted Call for participation: Help is requested to fix this issue. S-speculative Status: This is just an idea.

Comments

@zamazan4ik
Copy link

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are here. LLVM-related results are here.

Since PGO showed measurable improvements in compiler-like loads (CPython, Clang, Clangd, clang-format, GCC, Rustc, etc.) I think it could be useful to check PGO on Artichoke as well.

We need to perform PGO benchmarks on Artichoke. And if it shows improvements - add a note about possible improvements in Artichoke performance with PGO. Providing an easier way (e.g. a build option) to build scripts with PGO can be useful for the end-users too. Maybe testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too but I recommend starting from the usual PGO.

If you want to test PGO, I recommend starting with https://github.com/Kobzol/cargo-pgo .
@lopopolo lopopolo added C-enhancement Category: New feature or request. E-help-wanted Call for participation: Help is requested to fix this issue. A-build Area: CI build infrastructure. A-frontend Area: Frontends for interpreters, like the `ruby` or `irb` binaries. S-speculative Status: This is just an idea. labels Jul 16, 2023
@lopopolo
Copy link
Member

Interesting. Is this something you'd be interested in looking into @zamazan4ik?

@zamazan4ik
Copy link
Author

Maybe one day :) Do you have a good bench quite on which I can profile and bench Artichoke?

@lopopolo
Copy link
Member

lopopolo commented Jul 16, 2023 •
edited

The benches here using the spec-runner are a good place to start:

artichoke/.github/workflows/crash.yaml

Lines 53 to 71 in 58efd61

- name: Run specs
run: |
./target/release/spec-runner --quiet --format artichoke all-core-specs.toml > /dev/null
./target/release/spec-runner --quiet --format summary all-core-specs.toml > /dev/null
./target/release/spec-runner --quiet --format tagger all-core-specs.toml > /dev/null
./target/release/spec-runner --quiet --format yaml all-core-specs.toml > /dev/null
working-directory: "spec-runner"
env:
PERNOSCO_ENABLE: 1
- name: Run specs
run: |
./target/release/spec-runner --quiet --format artichoke spec-state.toml > /dev/null
./target/release/spec-runner --quiet --format summary spec-state.toml > /dev/null
./target/release/spec-runner --quiet --format tagger spec-state.toml > /dev/null
./target/release/spec-runner --quiet --format yaml spec-state.toml > /dev/null
working-directory: "spec-runner"
env:
PERNOSCO_ENABLE: 1

@lopopolo lopopolo added A-performance Area: Performance improvements and optimizations. and removed A-build Area: CI build infrastructure. labels Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-frontend Area: Frontends for interpreters, like the `ruby` or `irb` binaries. A-performance Area: Performance improvements and optimizations. C-enhancement Category: New feature or request. E-help-wanted Call for participation: Help is requested to fix this issue. S-speculative Status: This is just an idea.
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lopopolo @zamazan4ik