ui: fix missing progress & tidy #3335
Conversation
casperdcl
added
enhancement
dvc/remote/base.py
Outdated
| self.cache.upload(from_info, to_info, no_progress_bar=True) | ||
| self.cache.upload(from_info, to_info, no_progress_bar=False) |
There was a problem hiding this comment.
@efiop wanted to ask if this change is OK
-
P.S.: ignore the other changes in this PR for now (they'll disappear after ui: make progress bars respect--quiet#3316 is merged).
There was a problem hiding this comment.
just to add to this - if this change is OK then there's a lot of nice tidying/refactoring possible since upload() would never require a no_progress_bar argument
There was a problem hiding this comment.
@casperdcl Btw, how does the pbar look for this operation? I'm worried about it having some random looking paths in it.
There was a problem hiding this comment.
this would occur on dvc add <dir>. It gets short cache names but is cleared away upon completion. Padding it with some persistent logging messages (and padding fs.copyfile with some logging & sleep(5)) gives this:
There was a problem hiding this comment.
@casperdcl Yeah, not sure about that. That random name is not very helpful. How about we put some desc instead? There is a name param for that upload method, so should be pretty simple. Though, not sure how to describe this in a user-friendly way. Btw, did this particular part of code cause any issues in your research? It works with tiny files and usually uploading them is very quick.
As to the name here, maybe something like "Computing effective hash for directory '{}'" would be good enough.
There was a problem hiding this comment.
@casperdcl Got it. So the plan is to only refactor the defaults in those remotes, right?
There was a problem hiding this comment.
If we allow no_progress_bar=False then we can refactor away. But perhaps for now we should keep no_progress_bar=True and not refactor.
There was a problem hiding this comment.
@casperdcl Sorry, looks like I'm not following. So gdrive/gs had no_progress_bar=True, which contradicts the other remotes. So we will fix those, right? Or are we going to do something else as well? 🙂 Just trying to understand the plan for this PR.
There was a problem hiding this comment.
I was (and have always been) only talking about this specific cache.upload() line. The PR is now fine to merge.
gdrive/gs things elsewhere in this PR are no-brainers I think.
There was a problem hiding this comment.
Just to clarify further - if we change the cache.upload() line to say no_progress_bar=False, then it would mean every single instance of upload() in the code base would only ever have no_progress_bar=False and thus we can drop that argument from all upload() functions.
Codecov Report
@@ Coverage Diff @@
## master #3335 +/- ##
==========================================
+ Coverage 92.4% 92.89% +0.49%
==========================================
Files 140 141 +1
Lines 8474 8647 +173
==========================================
+ Hits 7830 8033 +203
+ Misses 644 614 -30
Continue to review full report at Codecov.
|
casperdcl
force-pushed
the
missing-progress
branch
from
19e9de2
to
37850cb
Compare
casperdcl
changed the title
make
no_progress_bar=Falserather thanTrueremote.gdrive.RemoteGDrive.gdrive_upload_file()remote.base.RemoteBASE._get_dir_info_checksum():self.cache.upload()remote.gs.RemoteGDrive.RemoteGS._upload()and_download()--quiet#3316