Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(launch): improved git fetch time by specifying a refspec and depth=1 #4459

Merged
merged 3 commits into from Nov 8, 2022

Conversation

gtarpenning
Copy link
Contributor

https://wandb.atlassian.net/browse/WB-11392

Description

Changes the origin.fetch command to only look for specific head object. master, main, and the optional passed in version.

This leads to 2-10x improvement in clone speed, depending on how many branches the target remote repo has and if the version is specified.

For example, using the wandb/examples repo had the following results.

branch command origin.fetch duration (s)
master wandb launch https://github.com/wandb/examples 85.6
this PR wandb launch https://github.com/wandb/examples 22.5
this PR wandb launch https://github.com/wandb/examples --git-version=xyz 10.5

Testing

How was this PR tested?

Checklist

  • Include reference to internal ticket "Fixes WB-NNNN" and/or GitHub issue "Fixes #NNNN" (if applicable)
  • Ensure PR title compliance with the conventional commits standards

@codecov
Copy link

codecov bot commented Nov 8, 2022

Codecov Report

Merging #4459 (e56b68b) into main (63941ea) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #4459   +/-   ##
=======================================
  Coverage   83.03%   83.04%           
=======================================
  Files         258      258           
  Lines       32865    32870    +5     
=======================================
+ Hits        27290    27297    +7     
+ Misses       5575     5573    -2     
Flag Coverage Δ
functest 56.88% <83.33%> (+0.02%) ⬆️
unittest 73.14% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
wandb/sdk/launch/utils.py 92.59% <100.00%> (+0.10%) ⬆️
wandb/filesync/step_prepare.py 93.67% <0.00%> (-1.27%) ⬇️
wandb/sdk/lib/sock_client.py 93.12% <0.00%> (-1.06%) ⬇️
wandb/sdk/wandb_run.py 90.91% <0.00%> (-0.12%) ⬇️
wandb/cli/cli.py 69.10% <0.00%> (-0.10%) ⬇️
wandb/sdk/lib/git.py 79.01% <0.00%> (ø)
wandb/sdk/internal/system/system_info.py 91.55% <0.00%> (+3.24%) ⬆️
wandb/errors/term.py 93.47% <0.00%> (+6.52%) ⬆️

wandb/sdk/launch/utils.py Outdated Show resolved Hide resolved
@gtarpenning gtarpenning changed the title chore(launch): Vastly improved git fetch time by specifying a refspec and depth=1 chore(launch): improved git fetch time by specifying a refspec and depth=1 Nov 8, 2022
@github-actions github-actions bot added cc-chore and removed cc-chore labels Nov 8, 2022
@gtarpenning gtarpenning marked this pull request as ready for review November 8, 2022 18:09
Copy link
Contributor

@KyleGoyette KyleGoyette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥

@gtarpenning gtarpenning merged commit eddc2ac into main Nov 8, 2022
@gtarpenning gtarpenning deleted the griffin/launch/git-depth branch November 8, 2022 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants