Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-1897: Upload benchmark results to S3 #4464

Merged
merged 10 commits into from
May 16, 2024

Conversation

TimDiekmann
Copy link
Member

@TimDiekmann TimDiekmann commented May 14, 2024

🌟 What is the purpose of this PR?

To create a time series of benchmark results they are uploaded to S3.

🔍 What does this change?

  • Add interface for repo-chores-cli to upload data to S3
  • Call upload function to upload benchmarks after running

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

@TimDiekmann TimDiekmann self-assigned this May 14, 2024
@github-actions github-actions bot added area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team labels May 14, 2024
Copy link

codecov bot commented May 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 21.10%. Comparing base (2f96a29) to head (4a7d0e7).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4464      +/-   ##
==========================================
- Coverage   21.21%   21.10%   -0.11%     
==========================================
  Files         446      449       +3     
  Lines       15170    15247      +77     
  Branches     2255     2275      +20     
==========================================
  Hits         3218     3218              
- Misses      11911    11988      +77     
  Partials       41       41              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@TimDiekmann TimDiekmann requested a review from CiaranMn May 15, 2024 06:53
@TimDiekmann TimDiekmann marked this pull request as ready for review May 15, 2024 06:53
…esults-in-s3' into t/h-1897-save-graph-benchmarks-results-in-s3
Copy link
Contributor

Benchmark results

@rust/graph-benches – Integrations

scaling_read_entity_complete_one_depth

Function Value Mean
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 5 $$24.2 \mathrm{ms} \pm 299 \mathrm{μs}\left({\color{gray}0.744 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 50 $$259 \mathrm{ms} \pm 2.25 \mathrm{ms}\left({\color{lightgreen}-82.969 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 10 $$45.1 \mathrm{ms} \pm 2.47 \mathrm{ms}\left({\color{red}51.3 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 25 $$69.3 \mathrm{ms} \pm 307 \mathrm{μs}\left({\color{gray}0.290 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 1 $$20.2 \mathrm{ms} \pm 84.8 \mathrm{μs}\left({\color{gray}0.685 \mathrm{\%}}\right) $$

representative_read_entity

Function Value Mean
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$16.7 \mathrm{ms} \pm 209 \mathrm{μs}\left({\color{gray}3.37 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$16.5 \mathrm{ms} \pm 174 \mathrm{μs}\left({\color{lightgreen}-10.694 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$16.0 \mathrm{ms} \pm 193 \mathrm{μs}\left({\color{gray}1.03 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$16.7 \mathrm{ms} \pm 232 \mathrm{μs}\left({\color{gray}2.05 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$16.7 \mathrm{ms} \pm 166 \mathrm{μs}\left({\color{lightgreen}-35.466 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$18.1 \mathrm{ms} \pm 192 \mathrm{μs}\left({\color{red}12.1 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$15.4 \mathrm{ms} \pm 145 \mathrm{μs}\left({\color{lightgreen}-5.473 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$16.6 \mathrm{ms} \pm 196 \mathrm{μs}\left({\color{gray}-3.156 \mathrm{\%}}\right) $$
entity_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579, Entity Type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$16.0 \mathrm{ms} \pm 177 \mathrm{μs}\left({\color{gray}1.36 \mathrm{\%}}\right) $$

representative_read_multiple_entities

Function Value Mean
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$1.99 \mathrm{s} \pm 7.66 \mathrm{ms}\left({\color{gray}0.735 \mathrm{\%}}\right) $$
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$1.05 \mathrm{s} \pm 3.58 \mathrm{ms}\left({\color{gray}0.002 \mathrm{\%}}\right) $$
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$1.04 \mathrm{s} \pm 9.30 \mathrm{ms}\left({\color{gray}0.030 \mathrm{\%}}\right) $$
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$94.3 \mathrm{ms} \pm 544 \mathrm{μs}\left({\color{gray}-1.152 \mathrm{\%}}\right) $$
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$420 \mathrm{ms} \pm 1.47 \mathrm{ms}\left({\color{gray}0.939 \mathrm{\%}}\right) $$
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$59.4 \mathrm{ms} \pm 489 \mathrm{μs}\left({\color{gray}-1.119 \mathrm{\%}}\right) $$
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$2.86 \mathrm{s} \pm 10.8 \mathrm{ms}\left({\color{gray}0.610 \mathrm{\%}}\right) $$
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$979 \mathrm{ms} \pm 5.57 \mathrm{ms}\left({\color{gray}0.254 \mathrm{\%}}\right) $$
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$965 \mathrm{ms} \pm 2.26 \mathrm{ms}\left({\color{gray}-2.219 \mathrm{\%}}\right) $$
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$39.6 \mathrm{ms} \pm 292 \mathrm{μs}\left({\color{gray}0.717 \mathrm{\%}}\right) $$
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$357 \mathrm{ms} \pm 1.41 \mathrm{ms}\left({\color{gray}-1.167 \mathrm{\%}}\right) $$
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$35.9 \mathrm{ms} \pm 340 \mathrm{μs}\left({\color{gray}0.981 \mathrm{\%}}\right) $$

representative_read_entity_type

Function Value Mean
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.36 \mathrm{ms} \pm 9.29 \mathrm{μs}\left({\color{gray}0.475 \mathrm{\%}}\right) $$

scaling_read_entity_linkless

Function Value Mean
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 10 $$2.38 \mathrm{ms} \pm 11.6 \mathrm{μs}\left({\color{gray}0.587 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 10000 $$13.2 \mathrm{ms} \pm 135 \mathrm{μs}\left({\color{gray}-0.015 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 100 $$2.52 \mathrm{ms} \pm 12.6 \mathrm{μs}\left({\color{gray}0.260 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 1000 $$3.37 \mathrm{ms} \pm 36.1 \mathrm{μs}\left({\color{lightgreen}-16.602 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 1 $$2.39 \mathrm{ms} \pm 11.5 \mathrm{μs}\left({\color{gray}1.43 \mathrm{\%}}\right) $$

scaling_read_entity_complete_zero_depth

Function Value Mean
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 5 $$2.41 \mathrm{ms} \pm 10.8 \mathrm{μs}\left({\color{gray}-0.704 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 50 $$4.51 \mathrm{ms} \pm 20.6 \mathrm{μs}\left({\color{gray}-0.735 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 10 $$2.61 \mathrm{ms} \pm 16.8 \mathrm{μs}\left({\color{gray}0.167 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 25 $$3.04 \mathrm{ms} \pm 17.6 \mathrm{μs}\left({\color{lightgreen}-16.199 \mathrm{\%}}\right) $$
get_entity_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba, Number Of Entities: 1 $$2.39 \mathrm{ms} \pm 8.46 \mathrm{μs}\left({\color{gray}0.185 \mathrm{\%}}\right) $$

@TimDiekmann TimDiekmann added this pull request to the merge queue May 16, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 16, 2024
@TimDiekmann TimDiekmann added this pull request to the merge queue May 16, 2024
Merged via the queue into main with commit eea99a3 May 16, 2024
139 checks passed
@TimDiekmann TimDiekmann deleted the t/h-1897-save-graph-benchmarks-results-in-s3 branch May 16, 2024 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/deps Relates to third-party dependencies (area) area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team
Development

Successfully merging this pull request may close these issues.

None yet

2 participants