Start-Process: allow redirecting std output/error to console stream

[aetonsi]

Summary of the new feature / enhancement

Hello.
I am trying to start a process in a new window with Start-Process and i'm -waiting for its execution.

Then i need to process its output. I would like to do it all in memory, without writing to disk, as the output is confidential. Unfortunately at the moment this doesn't seem to be possible (correct me if i'm wrong) without resorting to .NET implementations like this...

Currently, this is the best you can do:

$output = & {start-process powershell "read-host 'Enter string: '" -Wait -RedirectStandardOutput output.txt ; get-content output.txt}

In this example,

• powershell "read-host 'Enter string: '" is my command with confidential output.
• -Wait waits for my command to exit
• -RedirectStandardOutput permits the redirection to a file
• get-content simply reads back that file.

Proposed technical implementation details (optional)

A possible solution would be a parameter like -RedirectStandardOutputToStream / -RedirectStandardErrorToStream

$output = & {start-process powershell "read-host 'Enter string: '" -wait -RedirectStandardOutputToStream 1}

Where 1 is the ID of the success stream.

---

Or, more simply:

$output = & {start-process powershell "read-host 'Enter string: '" -wait -GetStandardOutput}

... with the corresponding -GetStandardError

[aetonsi]

Here i posted a workaround leveraging a temp file: aetonsi/pwsh__StartProcessWaitRedirectToStream

[jhoneill]

What are you using start-process at all ?
$output = powershell "read-host 'I should not ask for parameters to be typed' " or
$output = powershell "read host 'hyphen missing will cause error' " 2> null if you want to remove error output.

If it is a command line program start-process will (by default) run in a new window, not receive output and not wait for the program it has started to finish. You can specify Wait or NoNewWindow but you still need to work around the output not coming back

$output = Invoke-Command -ScriptBlock {powershell "read-host 'I should not ask for parameters to be typed' " }
Also works the way you want, I think.

[aetonsi]

I need my command to be run in a separate window, that's why i resorted to start-process. Is there some other way?

[jborean93]

Not without using dotnet but that’s the beauty of PowerShell, you can wrap all that in your own function to achieve what you want.

[mklement0]

Note that there's a previous discussion about generally allowing targeting variables in contexts where file paths are expected, so that Start-Process ... -RedirectStandardOutput variable:out would capture stdout in variable $out.

This is not only more flexible than -RedirectStandardOutput 1, it also prevents the output from instantly mixing with the caller's streams, which in asynchronous Start-Process calls (which are typical) you wouldn't want.

See:

• Suggestion: Add redirection syntax that allows in-memory collection of stderr lines output by external utilities, akin to the -ErrorVariable common parameter #4332 (original discussion, in the context of >)
• Start-Process should support redirecting to variable names or stream objects #5184 (later discussion, in the context of Start-Process)

[aetonsi]

@mklement0 Mmh i see, that would be cool. Not the same as what i was picturing (-RedirectStandardOutputToStream 1 would print to the success stream, and you could capture it in a variable)... but ultimately that's what i would like to end up to do: save the output to a variable to reuse it.

[SteveL-MSFT]

@JamesWTruher I thought you had a proof-of-concept for supporting redirection to the variable provider?

[JamesWTruher]

@SteveL-MSFT Indeed I do: #16497

[jborean93]

That doesn’t really help here for Start-Process if I understand it correctly? You can already capture output with just invoking the exe today. Supporting Start-Process redirection to a var would be quite tricky unless it meant -Wait was always implicit as it would require a background task to be running which could mutate the var as it’s being read.

[jhoneill]

That doesn’t really help here for Start-Process if I understand it correctly?

I think in this case there is no output from Start-Process itself that can be redirected. It would need start-process to allow
-RedirectStandardOutput variable:foo but that reports "Cannot open file because the current provider (Microsoft.PowerShell.Core\Variable) cannot open a file."

[aetonsi]

You can already capture output with just invoking the exe today

Yes but AFAIK you cannot start a process in a new window, wait for it and capture the output

[jborean93]

Why would you want to, that would mean you have a blank window with no output. When you redirect a std pipe in a process you cause that output to no longer appear. You would have to somehow split the output to go to a pipe you can capture but also have it output to the console at the same time which isn’t really easy to do at all.

[mklement0]

@jborean93, it's a good point that redirecting to variables could only work meaningfully with (possibly implied) -Wait

You can already capture output with just invoking the exe today

The original issue, #4332, was focused on 2> redirections to allow you to capture stderr output in a variable, as the native analog to -ErrorVariable - direct invocation of an external program currently doesn't support that.

In the context of (synchronous use of) Start-Process:

Why would you want to [....]?

They're certainly not common uses cases, but two come to mind (in both cases, -WindowStyle Hidden can be used to hide the new window):

• If you want to run a process as another user, with -Credential (on Windows), which cannot be done without Start-Process and invariably involves a new window.

• If you need to "sandbox" invocation of a PowerShell script that may interfere with the state of the calling session if invoked directly. (Arguably, though, Start-Job could be used instead and is the better choice, but requires more ceremony) (use pwsh { ... } - see below)

[jhoneill]

Why would you want to, that would mean you have a blank window with no output. When you redirect a std pipe in a process you cause that output to no longer appear.

@aetonsi I didn't think of this when you said to my suggestion before that you wanted it its own window. Why do do you want a blank window?

[mklement0]

@jhoneill:

• The uses cases in my previous comment imply hidden execution, in which case the question of a blank window is moot.

• Even if the launched window runs visibly, the window is only invariably blank if you redirect both stdout and stderr.

The bottom line is:

Are there legitimate use cases for synchronous use of Start-Process where capturing stdout and/or stderr output in-memory is useful?

[jborean93]

The original issue, #4332, was focused on 2> redirections to allow you to capture stderr output in a variable, as the native analog to -ErrorVariable - direct invocation of an external program currently doesn't support that.

It’s not pretty but you can do this today

$stdout = $null
$stderr = . { my.exe args | Set-Variable stdout } 2>&1 | ForEach-Object ToString

The ForEach-Object ToString` is only needed for WinPS because on the older versions these objects are error records and you want the string value instead.

If you want to run a process as another user, with -Credential (on Windows), which cannot be done without Start-Process and invariably involves a new window.

I will concede this is a valid point, thank you I didn’t think off this.

If you need to "sandbox" invocation of a PowerShell script that may interfere with the state of the calling session if invoked directly. (Arguably, though, Start-Job could be used instead and is the better choice.)

I’m not sure why this needs Start-Process. You can certainly just do pwsh { … } or as you said Start-Job to do this in such a nicer way. PowerShell will even deal with complex objects with both of these cases.

Still I feel like the better option is to add NoteProperties to the return object if -PassThru -Wait is set and redirection needed. The UX here is pretty poor but it ensures that there’s no concurrency problem, i.e. the output is only accessible once nothing else will add to it.

[mklement0]

@jborean93

I’m not sure why this needs Start-Process. You can certainly just do pwsh { … }

Good point: pwsh { … } is definitely the way to go here, which again calls for a simple 2> solution (see below).

It’s not pretty

That's precisely the point: let's make it pretty (easy):

# Capture stderr output in $stderr
$stdout = my.exe args 2>variable:stderr

As for the least ugly version right now:

$stdout, $stderr = (my.exe args).Where({ $_ -is [string] }, 'Split')

But all of that relates to direct invocation, which I think is the primary use case for the proposed enhancement.

I don't feel strongly about the Start-Process case.

I'm unclear on what you mean by adding NoteProperties.

[aetonsi]

Why would you want to, that would mean you have a blank window with no output. When you redirect a std pipe in a process you cause that output to no longer appear. You would have to somehow split the output to go to a pipe you can capture but also have it output to the console at the same time which isn’t really easy to do at all.

@mklement0 @jhoneill
Afaik (i'm new to PS so i might be wrong), the window would be blank only if all of the streams with any output are redirected.
But for example i have a simple program that writes to streams # 1, 2, 3 and 6 (write-host) and i need just one of those streams redirected to a variable. ATM i can do it by using a support file via -RedirectStandardOutput or -RedirectStandardError...
Again, if you're wondering why i need start-process, it is simply to open a new window. Why? for example because the original process has no window (because it has been started by another c# program with .StartInfo.WindowStyle = ProcessWindowStyle.Hidden)

And now that i think about it, this can work only for streams 1/2, number 3-6 are unmanageable. Even with a syntactic sugar like $stdout = my.exe args 6>variable:stdinfo, you couldnt apply this to start-process.

[mklement0]

@aetonsi, yes, when you call a script via the PowerShell CLI, the stdout and stderr streams are the only ones available, as with any child process. By default, PowerShell only sends its error stream to stderr (curiously, only when stderr is redirected), all others (success output, warnings, verbose messages, ...) are output to stdout (which is problematic - see #7989)

The only way to distinguish between PowerShell's output streams via a CLI call would be to use -OutputFormat xml in order to receive CLIXML data via stdout (CLIXML is an XML-based object-serialization format). Deserializing this data with [System.Management.Automation.PSSerializer]::Deserialize(), would allow you to separate the deserialized objects into their streams of origin by their type.

In other words: this approach indeed would leave your window invariably blank, because all output would be communicated via stdout.

[ImportTaste]

Redirecting to variables would mean that the process would have to completely finish before you printed to console. Being able to redirect to the current runspace's streams would mean you could have streaming output.