Log artifacts to an existing run ID in MLflow

[pedrety]

🚀 Feature

Right now you can't log artifact to an existing run in MLflow because you can only specify the experiment name and the desired run name you want to log artifacts to. This creates an issue where a new run is created instead of logging the data into an existing one. Also if you use mlflow.pytorch.autolog() and you want to log extra items into the current run it will create a new one because the MLflowLogger (MLflowLogger.__init__) can't be created specifying a run id.

Motivation

I'm trying to log extra items into an existing run during the training and it will create a new MLflow run instead of logging them into an existing run (1).

Pitch

I just want to log new items into my existing run by specifying the current run id without creating new runs and having the information divided into 2 runs.

Alternatives

Additional context

(1) As you can see in this image

cc @Borda

[awaelchli]

Hey @Kr4is @pedrety

Great contribution. Just a heads up: Lightning 1.6 will soon get released. It's possible #12290 won't land in time. Thanks for the patience <3