New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
push: Assume 0 bytes pushed if git-annex does not provide bytesize #7049
Conversation
Otherwise we can get KeyError and underlying code can provide records without bytesize
Inspired by a fix we needed for push: datalad#7049 I looked into status to realize that we do not even consider those files for which git-annex would not report bytesize. This commit makes them considered in count but would assume 0 bytesize for "recorded" etc. It would not address the issue that we would still not consider/interrogate sizes for those files even if their content available: $> datalad status --annex=all 2 annex'd files (0.0 B/0.0 B present/total size) $> ls -lL * -r-------- 1 yoh yoh 0 Sep 20 15:52 123.dat -r-------- 1 yoh yoh 723 Sep 20 15:57 sample3.csv where sample3.csv is the relaxed one.
Codecov ReportBase: 45.45% // Head: 88.36% // Increases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## maint #7049 +/- ##
===========================================
+ Coverage 45.45% 88.36% +42.90%
===========================================
Files 354 354
Lines 58559 46336 -12223
Branches 6615 0 -6615
===========================================
+ Hits 26618 40943 +14325
+ Misses 31779 5393 -26386
+ Partials 162 0 -162
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
uff all travis fails seems to be due to unrelated
|
no objections, the fix is trivial, let's proceed and retry releasing again! |
@jwodder -- note that here as well we didn't have github actions ran and changelog entry was pushed by github-actions bot although I thought we should have switched to that yarikoptic-gitmate token/account... will need to check/keep an eye... |
PR released in |
Inspired by a fix we needed for push: datalad#7049 I looked into status to realize that we do not even consider those files for which git-annex would not report bytesize. This commit makes them considered in count but would assume 0 bytesize for "recorded" etc. It would not address the issue that we would still not consider/interrogate sizes for those files even if their content available: $> datalad status --annex=all 2 annex'd files (0.0 B/0.0 B present/total size) $> ls -lL * -r-------- 1 yoh yoh 0 Sep 20 15:52 123.dat -r-------- 1 yoh yoh 723 Sep 20 15:57 sample3.csv where sample3.csv is the relaxed one.
Otherwise we can get KeyError and underlying code can provide records without bytesize. Reported in https://neurostars.org/t/datalad-push-causes-keyerror-bytesize-keyerror/23570 with
datalad==0.15.4
git==2.35.0
git-annex==8.20210903
first I failed to reproduce with the following script following user report with --fast
but then reproduced nicely with --relaxed
and verified that with this PR we do not crash. Given that git-annex unlikely to even provide progress report if not enough time passed, I think I would not bother crafting a unittest which would trigger this condition atm.