Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API to support tracking progress while reading from a file #1759

Merged
merged 28 commits into from Mar 30, 2022
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
cf91bd4
Add `Progress.read` and `rich.progress.read` to track progress readin…
althonos Dec 15, 2021
5b2a4a7
Update documentation of `rich.progress.Progress.read`
althonos Dec 16, 2021
a5fa231
Update `Progress.read` to close inner file only if given a path
althonos Dec 16, 2021
d6f61d8
Add tests to `tests.test_progress` to make sure files are closed prop…
althonos Dec 16, 2021
5cfa203
Add missing type annotations to `rich.progress._Reader`
althonos Dec 16, 2021
1432457
Add an example showing how to copy a file with a progress bar
althonos Dec 16, 2021
46576a9
Ignore type hint in `rich.progress._Reader.readall`
althonos Dec 16, 2021
97da885
Reformat new `progress` code with `black`
althonos Dec 16, 2021
5e8da24
Fix additional typing issues in `rich.progress`
althonos Dec 16, 2021
f3f1bca
Reformat `rich.progress` and document `Progress.read` in `progress.rst`
althonos Dec 16, 2021
a270e60
Fix `os.PathLike` type annotations in `rich.progress`
althonos Dec 16, 2021
3008780
Remove unused import from `rich.progress`
althonos Dec 18, 2021
1320046
Split `Progress.read` into `Progress.open` and `Progress.wrap_file` m…
althonos Mar 13, 2022
fc341f7
Add test to ensure `Progress.wrap_file` can extract the total from a …
althonos Mar 13, 2022
8663073
Make `Progress.open` have a signature compatible with `open`
althonos Mar 13, 2022
2b8ec3b
Add overloaded annotations for `Progress.open` and `rich.progress.open`
althonos Mar 13, 2022
0406578
Update `CHANGELOG.md` to show the `Progress.open` and `Progress.wrap_…
althonos Mar 13, 2022
2e61862
Merge branch 'master' into progress-reader
althonos Mar 14, 2022
b736f8a
Merge branch 'master' into progress-reader
willmcgugan Mar 15, 2022
2e77302
Remove unused `update_period` argument from `rich.progress`
althonos Mar 21, 2022
aa5ca60
Fix failing tests in `tests.test_progress`
althonos Mar 21, 2022
130f5b4
Reformat code in `rich.progress`
althonos Mar 21, 2022
5ea3032
Force keyword arguments for `Progress.open` options not in builtin `o…
althonos Mar 30, 2022
54d660d
Update example for `wrap_file` in `rich.progress` API documentation
althonos Mar 30, 2022
bb9790e
Force keyword arguments for `Progress.wrap_file` arguments after `total`
althonos Mar 30, 2022
76953e2
Remove `_Reader.readall` method since wrapped object may not all have…
althonos Mar 30, 2022
7a870e4
Merge branch 'master' into progress-reader
darrenburns Mar 30, 2022
5717d8d
Add ignore codes for `mypy` in `rich.progress`
althonos Mar 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 5 additions & 3 deletions CHANGELOG.md
Expand Up @@ -5,14 +5,16 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
## 12.1.0

### Added

- Progress.open and Progress.wrap_file method to track the progress while reading from a file or file-like object https://github.com/willmcgugan/rich/pull/1759

### Changed

- Improve performance of cell_length https://github.com/Textualize/rich/pull/2061

## [12.0.1] - 2022-03-14

### Fixed

- Fix capturing stdout on legacy Windows https://github.com/Textualize/rich/pull/2055
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTORS.md
Expand Up @@ -18,6 +18,7 @@ The following people have contributed to the development of Rich:
- [Finn Hughes](https://github.com/finnhughes)
- [Josh Karpel](https://github.com/JoshKarpel)
- [Andrew Kettmann](https://github.com/akettmann)
- [Martin Larralde](https://github.com/althonos)
- [Hedy Li](https://github.com/hedythedev)
- [Alexander Mancevice](https://github.com/amancevice)
- [Will McGugan](https://github.com/willmcgugan)
Expand Down
54 changes: 50 additions & 4 deletions docs/source/progress.rst
Expand Up @@ -26,6 +26,16 @@ For basic usage call the :func:`~rich.progress.track` function, which accepts a
for n in track(range(n), description="Processing..."):
do_work(n)


To get a progress bar while reading from a file, you may consider using the :func:`~rich.progress.read` function, which accepts a path, or a *file-like* object. It will return a *file-like* object in *binary mode* that will update the progress information as it's being read from. Here's an example, tracking the progresses made by :func:`json.load` to load a file::

import json
from rich.progress import read

with read("data.json", description="Loading data...") as f:
data = json.load(f)


Advanced usage
--------------

Expand All @@ -34,9 +44,9 @@ If you require multiple tasks in the display, or wish to configure the columns i
The Progress class is designed to be used as a *context manager* which will start and stop the progress display automatically.

Here's a simple example::

import time

from rich.progress import Progress

with Progress() as progress:
Expand Down Expand Up @@ -179,7 +189,7 @@ If you have another Console object you want to use, pass it in to the :class:`~r
with Progress(console=my_console) as progress:
my_console.print("[bold blue]Starting work!")
do_work(progress)


Redirecting stdout / stderr
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand All @@ -199,6 +209,43 @@ If the :class:`~rich.progress.Progress` class doesn't offer exactly what you nee
def get_renderables(self):
yield Panel(self.make_tasks_table(self.tasks))

Reading from a file
~~~~~~~~~~~~~~~~~~~

You can obtain a progress-tracking reader using the :meth:`~rich.progress.Progress.open` method by giving it a path. You can specify the number of bytes to be read, but by default :meth:`~rich.progress.Progress.open` will query the size of the file with :func:`os.stat`. You are responsible for closing the file, and you should consider using a *context* to make sure it is closed ::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the Progress.open API 😎


import json
from rich.progress import Progress

with Progress() as progress:
with progress.open("file.json", "rb") as file:
json.load(file)


Note that in the above snippet we use the `"rb"` mode, because we needed the file to be opened in binary mode to pass it to :func:`json.load`. If the API consuming the file is expecting an object in *text mode* (for instance, :func:`csv.reader`), you can open the file with the `"r"` mode, which happens to be the default ::

from rich.progress import Progress

with Progress() as progress:
with progress.open("README.md") as file:
for line in file:
print(line)


Reading from a file-like object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can obtain a progress-tracking reader wrapping a file-like object using the :meth:`~rich.progress.Progress.wrap_file` method. The file-like object must be in *binary mode*, and a total must be provided, unless it was provided to a :class:`~rich.progress.Task` created beforehand. The returned reader may be used in a context, but will not take care of closing the wrapped file ::

import io
import json
from rich.progress import Progress

with Progress() as progress:
file = io.BytesIO("...")
json.load(progress.read(file, total=2048))
althonos marked this conversation as resolved.
Show resolved Hide resolved


Multiple Progress
-----------------

Expand All @@ -208,4 +255,3 @@ Example
-------

See `downloader.py <https://github.com/willmcgugan/rich/blob/master/examples/downloader.py>`_ for a realistic application of a progress display. This script can download multiple concurrent files with a progress bar, transfer speed and file size.

39 changes: 39 additions & 0 deletions examples/cp_progress.py
@@ -0,0 +1,39 @@
"""
A very minimal `cp` clone that displays a progress bar.
"""
import os
import shutil
import sys

from rich.progress import (
BarColumn,
DownloadColumn,
Progress,
TaskID,
TextColumn,
TimeRemainingColumn,
TransferSpeedColumn,
)

progress = Progress(
TextColumn("[bold blue]{task.description}", justify="right"),
BarColumn(bar_width=None),
"[progress.percentage]{task.percentage:>3.1f}%",
"•",
DownloadColumn(),
"•",
TransferSpeedColumn(),
"•",
TimeRemainingColumn(),
)

if __name__ == "__main__":
if len(sys.argv) == 3:

with progress:
desc = os.path.basename(sys.argv[1])
with progress.read(sys.argv[1], description=desc) as src:
with open(sys.argv[2], "wb") as dst:
shutil.copyfileobj(src, dst)
else:
print("Usage:\n\tpython cp_progress.py SRC DST")