[rust] Include mixed output (INFO, WARN, DEBUG, etc. to stderr and minimal JSON to stdout)

[bonigarcia]

Description

This PR includes a new type of output for Selenium Manager -called mixed- for the --output argument . This output makes SM to write the regular logs (INFO, WARN, DEBUG, etc.) to stderr while they are generated, and a minimal JSON output (composed by driver_path and browser_path) to the stdout and the end of the SM execution. This feature has been tested on Windows and Linux:

user@ubuntu-vm:~/dev/selenium/rust$ cargo run -- --browser chrome --debug --output mixed > stdout.txt 2> stderr.txt

user@ubuntu-vm:~/dev/selenium/rust$ cat stdout.txt 
{
  "driver_path": "/home/user/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver",
  "browser_path": "/usr/bin/google-chrome"
}

user@ubuntu-vm:~/dev/selenium/rust$ cat stderr.txt 
    Finished dev [unoptimized + debuginfo] target(s) in 1.02s
     Running `target/debug/selenium-manager --browser chrome --debug --output mixed`
DEBUG	chromedriver not found in PATH
DEBUG	chrome detected at /usr/bin/google-chrome
DEBUG	Running command: /usr/bin/google-chrome --version
DEBUG	Output: "Google Chrome 120.0.6099.199 "
DEBUG	Detected browser: chrome 120.0.6099.199
DEBUG	Required driver: chromedriver 120.0.6099.109
DEBUG	chromedriver 120.0.6099.109 already in the cache
INFO	Driver path: /home/user/.cache/selenium/chromedriver/linux64/120.0.6099.109/chromedriver
INFO	Browser path: /usr/bin/google-chrome

C:\Users\boni\Documents\dev\selenium\rust>cargo run -- --browser chrome --debug --output mixed > stdout.txt 2> stderr.txt

C:\Users\boni\Documents\dev\selenium\rust>type stdout.txt
{
  "driver_path": "C:\\Users\\boni\\.cache\\selenium\\chromedriver\\win64\\120.0.6099.109\\chromedriver.exe",
  "browser_path": "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
}

C:\Users\boni\Documents\dev\selenium\rust>type stderr.txt
   Compiling selenium-manager v0.4.17-nightly (C:\Users\boni\Documents\dev\selenium\rust)
    Finished dev [unoptimized + debuginfo] target(s) in 5.76s
     Running `target\debug\selenium-manager.exe --browser chrome --debug --output mixed`
DEBUG   chromedriver not found in PATH
DEBUG   chrome detected at C:\Program Files\Google\Chrome\Application\chrome.exe
DEBUG   Running command: wmic datafile where name='C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe' get Version /value
DEBUG   Output: "\r\r\n\r\r\nVersion=120.0.6099.199\r\r\n\r\r\n\r\r\n\r"
DEBUG   Detected browser: chrome 120.0.6099.199
DEBUG   Required driver: chromedriver 120.0.6099.109
DEBUG   chromedriver 120.0.6099.109 already in the cache
INFO    Driver path: C:\Users\boni\.cache\selenium\chromedriver\win64\120.0.6099.109\chromedriver.exe
INFO    Browser path: C:\Program Files\Google\Chrome\Application\chrome.exe

NOTE 1: This PR also stops writing the value of the driver path in result.message of the old JSON format. This feature has been supported to provide compatibility with the initial versions of SM, but I believe this is not required anymore. Example:

C:\Users\boni\Documents\dev\selenium\rust>cargo run -- --browser chrome --output json
    Finished dev [unoptimized + debuginfo] target(s) in 0.27s
     Running `target\debug\selenium-manager.exe --browser chrome --output json`
{
  "logs": [
    {
      "level": "INFO",
      "timestamp": 1704797456,
      "message": "Driver path: C:\\Users\\boni\\.cache\\selenium\\chromedriver\\win64\\120.0.6099.109\\chromedriver.exe"
    },
    {
      "level": "INFO",
      "timestamp": 1704797456,
      "message": "Browser path: C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
    }
  ],
  "result": {
    "code": 0,
    "message": "",
    "driver_path": "C:\\Users\\boni\\.cache\\selenium\\chromedriver\\win64\\120.0.6099.109\\chromedriver.exe",
    "browser_path": "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe"
  }
}

NOTE 2: The value of driver_path is also used for the grid path. Examples:

C:\Users\boni\Documents\dev\selenium\rust>cargo run -- --grid --output mixed --debug
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `target\debug\selenium-manager.exe --grid --output mixed --debug`
DEBUG   grid not found in the system
DEBUG   Required driver: selenium-server 4.16.1
DEBUG   selenium-server 4.16.1 already in the cache
INFO    Driver path: C:\Users\boni\.cache\selenium\grid\4.16.1\selenium-server-4.16.1.jar
{
  "driver_path": "C:\\Users\\boni\\.cache\\selenium\\grid\\4.16.1\\selenium-server-4.16.1.jar",
  "browser_path": ""
}

C:\Users\boni\Documents\dev\selenium\rust>cargo run -- --grid --output json
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `target\debug\selenium-manager.exe --grid --output json`
{
  "logs": [
    {
      "level": "INFO",
      "timestamp": 1704797731,
      "message": "Driver path: C:\\Users\\boni\\.cache\\selenium\\grid\\4.16.1\\selenium-server-4.16.1.jar"
    }
  ],
  "result": {
    "code": 0,
    "message": "",
    "driver_path": "C:\\Users\\boni\\.cache\\selenium\\grid\\4.16.1\\selenium-server-4.16.1.jar",
    "browser_path": ""
  }
}

Motivation and Context

This feature has been requested in the selenium_manager Slack channel.
Fixes #13260

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• I have read the contributing document.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.

[diemol]

I prefer to avoid a value like mixed because it can be confusing. Why don't we leave the output as it is and log JSON to stderr? I cannot imagine someone needing stdout in one format and stderr in another.

If that looks strange on the console. Then let's have a flag for the output/result, and a a flag for the log output. What do you think?

[titusfortner]

I think we want to get rid of --output as a toggle all together, because I *think mixed provides what each use case wants to see?

• "SHELL" — the current output no longer makes sense. The original idea is you could grab the driver path directly from stdout and use it. but now that there are multiple paths and it is unformatted, it isn't useful
• "JSON" if we don't need to be able to parse logs, we only need the results JSON from this to format browser/driver/grid path info
• "LOGGER" the point was to see what it is doing by default, which in MIXED will always be the case

[bonigarcia]

I've done that way for different reasons. First, because writing in the stderr something that is not an error message seems weird to me. But at least the name mixed suggests hybrid behavior. Also, because this mixed type combines JSON and standard log in the same output.

And more important, creating a new output type (called mixed or another name) is not a breaking change. But using the old value json for the new output (logs to stderr and minimal JSON to stdout) would be a breaking change, which requires an immediate change in the bindings (to adapt to the new JSON parsing etc.).

Having said that, I can update the PR if you still consider a new json output as a better approach (i.e., logs to stderr and minimal JSON to stdout). In that case, currently, maybe the build will be broken, since now (although it should be temporal), the latest SM binaries are constructed with the PR.

[diemol]

I guess it makes sense to return the results by default in a json structure because it is easier to consume, and ideally the ones who use this binary need to do that.

Now, my doubt with this PR, is how can we show a good error message in the client bindings when an error happens?

[titusfortner]

If status code > 0, then the stdout needs to be the error message.

[bonigarcia]

the stdout needs to be the error message

That, for me, is a bit against nature.

Maybe the "minimal JSON", as I said in the PR, should contain the field already available in the former result field of the former JSON. I mean the following:

{
    "code": number,
    "message": "Error meesage. Only available for code != 0. For code 0, this message will be empty",
    "driver_path": "/path/to/driver (or /path/to/grid)",
    "browser_path": "path/to/browser"
  }

And if you prefer, I can remove the mixed output, and the json output will be like that (i.e., regular traces to stderr and minimal JSON to stdout).

[titusfortner]

That, for me, is a bit against nature.

You're right, my suggestion is not standard; disregard. 😁

We should be able to parse ERROR messages from stderr and put them in the exception, still?

[bonigarcia]

We should be able to parse ERROR messages from stderr and put them in the exception, still?

Using the approach I proposed in my last message, the error message should be in the message field (in the the stdout). And if that output is not parseable, it would mean that an uncontrolled panic error happened. In that case, I would attach the whole stdout/stderr to the exception (although I believe everything will go to stderr).

Then. Should I implemented the last approach? If so, remember that it is a breaking change and all bindings should be updated to parse the new JSON etc.

[titusfortner]

let's keep "mixed" for this release so we have time to update the implementation in all the bindings. Whether we replace json output (Diego's suggestion) or remove the toggle all together (mine), we can do that in next release.

And sure, let's add "error" to stdout json. I'm not sure it's necessary, but we can remove it later if we need to.

[diemol]

Sounds good.

[bonigarcia]

Ok then. I have just committed another patch to this PR to include the error message (i.e., a field called message) in the minimal JSON (i.e., for the mixed output). I finally did not include the code since it should be redundant to the process exit code. Here it is some examples:

./selenium-manager --browser chrome --debug --output mixed > out.txt 2> err.txt

cat out.txt
{
  "driver_path": "C:\\Users\\boni\\.cache\\selenium\\chromedriver\\win64\\120.0.6099.109\\chromedriver.exe",
  "browser_path": "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
  "message": ""
}

cat err.txt
DEBUG   chromedriver not found in PATH
DEBUG   chrome detected at C:\Program Files\Google\Chrome\Application\chrome.exe
DEBUG   Running command: wmic datafile where name='C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe' get Version /value
DEBUG   Output: "\r\r\n\r\r\nVersion=120.0.6099.224\r\r\n\r\r\n\r\r\n\r"
DEBUG   Detected browser: chrome 120.0.6099.224
DEBUG   Required driver: chromedriver 120.0.6099.109
DEBUG   chromedriver 120.0.6099.109 already in the cache
INFO    Driver path: C:\Users\boni\.cache\selenium\chromedriver\win64\120.0.6099.109\chromedriver.exe
INFO    Browser path: C:\Program Files\Google\Chrome\Application\chrome.exe

./selenium-manager --browser bad --debug --output mixed > out.txt 2> err.txt

cat out.txt
{
  "driver_path": "",
  "browser_path": "",
  "message": "Invalid browser name: bad"
}

cat type err.txt
ERROR   Invalid browser name: bad

[bonigarcia]

Update: I have changed the field message in the minimal JSON to error, which is more descriptive. For instance:

./selenium-manager --browser aaa --debug --output mixed
ERROR   Invalid browser name: aaa
{
  "driver_path": "",
  "browser_path": "",
  "error": "Invalid browser name: aaa"
}

[diemol]

Should we merge this after releasing 4.18 to have time to update the bindings?

[titusfortner]

Since the changes are only for --output mixed then we should be able to merge so long as we keep --output json in the release. I'd actually like to be able to implement the changes in the bindings for 4.19 with the features already in trunk.

The one important piece that is missing is --log-level. @bonigarcia is that something we could get before 4.18? Thanks for your work on this!

[bonigarcia]

is that something we could get before 4.18?

Sure, I'll make a PR about it soon.

[bonigarcia]

--log-level is implemented in #13566.