Consider kedro run -v verbose mode

[noklam]

Description

The current logging configuration is too granular. We should use root logger only. The exception would be info_logger or error_logger because they are mean to capture that specific log level.

The most common case is that I want to see all the logs during debugging. In production case (packaged project), how do I even change it to see the log if I want to? Editing the configuration file doesn't seem to be a good idea.

Inspired by pytest -v pytest -vv
Proposal:

1. kedro run -v verbose mode - The Project logging level will be set to DEBUG because this should be the most common case, I want to see the log message that I wrote in the project
2. kedro run -vv more verbose mode - Everything should be shown now, including any plugin logging or kedro logs

Context

It's hard to get logging right, in most case I either expect seeing just the default level or I want to see everything during debugging. a -v flag comes in handy and is a common convention among many CLI tools. i.e. pytest.

Possible Implementation

Add new argument in the CLI, and override the level of logging in LOGGING.

Possible Alternatives

Edited 2024-03-07
I have splited this ticket to two tickets

• Change the default_logging.yml and logging.yml to have more sensible default #3687

[noklam]

Putting a note here to remind myself to do a demo when we groom this.

[astrojuanlu]

I've said it in #3446 (comment), but for the record, +1 to this. From https://12factor.net/logs:

A twelve-factor app never concerns itself with routing or storage of its output stream. It should not attempt to write to or manage logfiles. Instead, each running process writes its event stream, unbuffered, to stdout.

[astrojuanlu]

Another point in favour of having kedro -vv: sometimes you just want to know what kedro is doing without having to configure the whole stdlib logging. This is how most other tools work.

Here is an example of a user that's puzzled because kedro just hangs https://linen-slack.kedro.org/t/16630828/i-m-encountering-issues-when-running-the-load-ext-kedro-ipyt#3758284a-34fb-435c-b054-dd3713cd78c9

[noklam]

It was moved to discussion due to #3755 (comment)

[astrojuanlu]

To be continued