Option to log progess and duration estimations

[felix-ht]

First check

• I added a descriptive title to this issue.
• I used the GitHub search to find a similar request and didn't find it.
• I searched the Prefect documentation for this feature.

Prefect Version

2.x

Describe the current behavior

Some long running tasks have a clear estimation of their progess as well as their remaining duration. Curretnly the only way to track this is to conntect the logger to say tqdm. This tends to spam the logs, as long as one doesn't throttle the logging.

Describe the proposed behavior

It would be great if the progess could be logged to prefect so the gui could show how long the current task will still take.

@task
def foo():
    logger =  get_run_progress_logger()
    for i in logger(range(100), total=100):
        sleep(1)

Example Use

No response

Additional context

No response

[billpalombi]

Thanks for submitting this issue @felix-ht! We have a feature in the works that will support this use case. It should ship in a few weeks. I'll circle back once its released.

[hubsbendang]

Any update on this?

[billpalombi]

@hubsbendang, this proved to be a greater design challenge than we anticipated. While we do want to address this need, we don't have a timeline for doing so.