Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cli): deflake wandb verify #4438

Merged
merged 2 commits into from Nov 5, 2022
Merged

fix(cli): deflake wandb verify #4438

merged 2 commits into from Nov 5, 2022

Conversation

vanpelt
Copy link
Contributor

@vanpelt vanpelt commented Nov 4, 2022

Description

We're using wandb verify in CI and I noticed flakiness in one of the checks. There's a race where a file has been uploaded but it hasn't yet been processed from the filemeta queue. This wraps those download calls in our regular retry logic.

As a bonus, I also made the ids of the runs human readable for better debugging when they flake.

Testing

How was this PR tested?

Checklist

  • Include reference to internal ticket "Fixes WB-NNNN" and/or GitHub issue "Fixes #NNNN" (if applicable)
  • Ensure PR title compliance with the conventional commits standards

@github-actions github-actions bot added the cc-fix label Nov 4, 2022
@codecov
Copy link

codecov bot commented Nov 4, 2022

Codecov Report

Merging #4438 (e28f9cc) into main (a3eec65) will increase coverage by 9.00%.
The diff coverage is 33.33%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4438      +/-   ##
==========================================
+ Coverage   74.06%   83.06%   +9.00%     
==========================================
  Files         258      258              
  Lines       32863    32866       +3     
==========================================
+ Hits        24339    27300    +2961     
+ Misses       8524     5566    -2958     
Flag Coverage Δ
functest 56.87% <33.33%> (+0.01%) ⬆️
unittest 73.16% <33.33%> (+15.89%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
wandb/sdk/verify/verify.py 33.69% <33.33%> (+21.60%) ⬆️
wandb/sdk/service/server_sock.py 91.87% <0.00%> (-1.02%) ⬇️
wandb/env.py 74.89% <0.00%> (+0.86%) ⬆️
wandb/sdk/internal/tb_watcher.py 88.03% <0.00%> (+0.99%) ⬆️
wandb/sdk/lib/sock_client.py 94.17% <0.00%> (+1.05%) ⬆️
wandb/sdk/lib/ipython.py 41.89% <0.00%> (+1.35%) ⬆️
wandb/sdk/lib/mailbox.py 93.49% <0.00%> (+1.36%) ⬆️
wandb/sdk/wandb_manager.py 94.07% <0.00%> (+1.48%) ⬆️
wandb/sdk/data_types/helper_types/image_mask.py 90.62% <0.00%> (+1.56%) ⬆️
wandb/sdk/data_types/base_types/media.py 96.52% <0.00%> (+2.08%) ⬆️
... and 88 more

Copy link
Member

@dmitryduev dmitryduev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏🏻

@dmitryduev dmitryduev merged commit a6d6e2d into main Nov 5, 2022
@dmitryduev dmitryduev deleted the task/verify-better branch November 5, 2022 03:14
andrewtruong pushed a commit that referenced this pull request Dec 2, 2022
fix(cli): Deflake wandb verify
@kptkin kptkin changed the title fix(cli): Deflake wandb verify fix(cli): deflake wandb verify Dec 6, 2022
@github-actions github-actions bot added cc-fix and removed cc-fix labels Dec 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants