Escaping external command arguments

[savornicesei]

I have a powershell script that is run on linux (pwsh v7.2.latest) and windows (pwsh v7.3.latest) and just builds a project:

$VSWHERE_EXE = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
$MSBUILD_EXE = & $VSWHERE_EXE -latest -products * -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\MSBuild.exe | Select-Object -first 1
$MSBUILD_PARAMS = @("/p:Configuration=$BuildConfiguration",
                    """/p:DefineExtraConstants=$CompilationFlags""", 
                    '/p:WarningLevel=0')

Write-Output "
    vswhere.exe path: $VSWHERE_EXE,
    msbuild.exe path: $MSBUILD_EXE,
    msbuild.exe additional params: $MSBUILD_PARAMS"

& $MSBUILD_EXE (Join-Path $BuildRoot "SomeProject.sln") /t:CreateLocalBuildPropsFile $MSBUILD_PARAMS

The command executed should be:

"C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Current\Bin\MSBuild.exe" SomeProject.sln /t:CreateLocalBuildPropsFile /p:Configuration=Debug "/p:DefineExtraConstants=PUBLISH_APPDATA;" /p:WarningLevel=0

But the required quotes for DefineExtraConstants are not passed as they are but they're re-escaped, the command line being

"C:\Program Files\Microsoft Visual Studio\2022\Professional\MSBuild\Current\Bin\MSBuild.exe" SomeProject.sln /t:CreateLocalBuildPropsFile /p:Configuration=Debug "\"/p:DefineExtraConstants=PUBLISH_APPDATA;\"" /p:WarningLevel=0

I've tried all kinds of escaping sequences in MSBUILD_PARAMS but the escaping in command line still happens.
What is the correct solution here? I'm not sure if it's related to #15239

Thank you and Happy Winter Holidays! ☃️ 🎅

[lpatalas]

Hello @savornicesei, I think that you're calling the msbuild command correctly. If your problem is that you're getting MSB1005: Specify a property and its value error then it's caused by semicolon (;) at the end of /p:DefineExtraConstants argument. You didn't mention what is the actual error so let me know if I guessed incorrectly. :)

In general I don't think you even have to put any additional quotes in $MSBUILD_PARAMS because when you call & $MSBUILD_EXE $MSBUILD_PARAMS then every element of the $MSBUILD_PARAMS array should be passed as a single argument to msbuild.exe.

[alex4998]

I tried those variants as well. None of them worked.

msbuild.exe "/property:SubstituteTargets="Something1;Something2""

gives

error MSB4057: The target ""Something1" does not exist in the project, as if the quote symbol is the actual beginning of the string, in other words as if the string's first symbol is the quote symbol instead of the S for Something1.

msbuild.exe --% /property:SubstituteTargets="Something1;Something2"

gives a different error not related to this situation but I'm also curious how to fix it. I, also, have this argument in my full test case /property:Platform="Any CPU" and I didn't find a way to escape this. I, also, tried many variants. The error is

MSBUILD : error MSB1008: Only one project can be specified.
Full command line: '"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\MSBuild\Current\Bin\amd64\MSBuild.exe" Solution.sln /target:BeforeBuild /property:Configuration=Debug "/property:Platform="Any" "CPU"" "/property:SubstituteTargets="Something1;Something2"" /maxcpucount /verbosity:minimal'

Using this

$tempParams = @('/property:SubstituteTargets="Something1;Something2"')

gives the same error as in the first suggestion

error MSB4057: The target ""Something1" does not exist in the project.

[lpatalas]

Is the code you're running available somewhere so I could take a look? I'm pretty sure all those variants should work correctly so there may be something else we're missing. Also which PS version you're using? Is it on Windows or Linux?

[alex4998]

Like I said it started happening in the latest version when the new command line parsing became mainstream. In my case I upgraded to 7.3.2. I'm running it on Windows. Giving a better description or an example in the context of my complete solution might be a problem. There are many components which are part of the final build script I was talking about.

Let me try to explain the problem from different perspective. Let's say this is my proj file.

<?xml version="1.0" encoding="utf-8" ?> <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <Target Name="Test"> <Message Importance="high" Text="$(SubstituteTargets)" /> </Target> </Project>

Now, if I use the legacy powershell argument parsing code and the way I escaped quotes I have this result

MSBuild .\Test.proj /target:Test /property:SubstituteTargets=""One;Two""
MSBuild version 17.4.1+9a89d02ff for .NET Framework
Build started 2/5/2023 0:14:25.
Project "C:\Test.proj" on node 1 (Test target(s)).
Test:
One;Two

You see, it displays One;Two without any quotes. This is what I need. Now, if switch to the new powershell argument parsing code I'll have this result

MSBuild .\Test.proj /target:Test /property:SubstituteTargets=""One;Two""
MSBuild version 17.4.1+9a89d02ff for .NET Framework
Build started 2/5/2023 0:21:19.
Project "C:\Test.proj" on node 1 (Test target(s)).
Test:
"One;Two"

You see, it displays "One;Two", the string actually starts and ends with the quote character, and it breaks the rest of the build logic. From this point onward I either have a problem with the string starting from the quote character or the command line apparently breaks at the semicolon ( ; ).

MSBuild .\Test.proj /target:Test /property:SubstituteTargets="One;Two"
MSBuild version 17.4.1+9a89d02ff for .NET Framework
MSBUILD : error MSB1006: Property is not valid.
Switch: Two

MSBuild .\Test.proj /target:Test "/property:SubstituteTargets="One;Two""
MSBuild version 17.4.1+9a89d02ff for .NET Framework
Build started 2/5/2023 0:13:14.
Project "C:\Test.proj" on node 1 (Test target(s)).
Test:
"One;Two"

[lpatalas]

Thanks, I get it now. I didn't notice that the quotes are included in the property value. I think this issue is tracked here: #18660. Sorry for the initial confusion :)

[iamjonny]

I've just landed here whilst reading on here-strings
here (comments now closed)
https://devblogs.microsoft.com/powershell/variable-expansion-in-strings-and-here-strings/?nsl_bypass_cache=a0136d9eb032801ae1edfd4bea192856#comments

whilst trying to understand issue with here-string used here
https://stackoverflow.com/questions/72436452/using-powershell-to-execute-github-cli-graphql-query-parse-error-on-lbra/73129290?noredirect=1#comment129269545_73129290

I suspect it's related to the call of the external program
gh api --hostname "$Hostname" graphql -F query="$graphQl"

and wondered if there's a better fix than that mentioned

issue was related to escaping the double quotes inside the here-string and this is the fix:

$graphQl = @"
mutation createBranchProtectionRule {
createBranchProtectionRule(input: {
repositoryId: "$repoNodeId"
pattern: "$BranchPattern"
#... rest omitted for brevity ...
"@

I thought if anyone can explain this it's the PS Possee! (plus I think this org was linked by article by superJeff : )

proposed GPT solution

[cmdletbinding()]
param(
    [parameter(Mandatory=$true)]
    [string]$Organisation,
    [parameter(Mandatory=$true)]
    [string]$Repository,    
    [hashtable]$Properties = [ordered]@{
        allowsDeletions='false'
        allowsForcePushes='false'
        blocksCreations='false'
        dismissesStaleReviews='true'
        isAdminEnforced='true'
        requiredApprovingReviewCount=1 
        requiresApprovingReviews='true'
        requiresCodeOwnerReviews='false'
        requiresCommitSignatures='false'
        requiresConversationResolution='true'
        requiresLinearHistory='false'
        requiresStatusChecks='true'
        requiresStrictStatusChecks='true'
        restrictsPushes='false'
        restrictsReviewDismissals='false'
    },
    [parameter(Mandatory=$false)]
    [string]$BranchPattern = '[mp][ar][ise][ntp]*',
    [string]$Hostname = 'github.com'
)

$repo = "repos/$Organisation/$Repository"
$repoNodeId = $(gh api --hostname "$hostname" $repo --jq .node_id)
Write-Debug "`$repo = '$repo'; `$repoNodeId = $repoNodeId"

$graphqlInput = @{
    repositoryId = $repoNodeId
    pattern = $BranchPattern
    allowsDeletions = 'false'
    allowsForcePushes = 'false'
    blocksCreations = 'false'
    dismissesStaleReviews = 'true'
    isAdminEnforced = 'true'
    requiredApprovingReviewCount = 1 
    requiresApprovingReviews = 'true'
    requiresCodeOwnerReviews = 'false'
    requiresCommitSignatures = 'false'
    requiresConversationResolution = 'true'
    requiresLinearHistory = 'false'
    requiresStatusChecks = 'true'
    requiresStrictStatusChecks = 'true'
    restrictsPushes = 'false'
    restrictsReviewDismissals = 'false'
}

$graphqlParams = @{
    input = $graphqlInput | ConvertTo-Json -Depth 10
}

$ghApiCommand = @{
    hostname = $Hostname
    command = 'graphql'
    query = $graphQl
    input = $graphqlParams
}

gh api @ghApiCommand

parameters are stored in a hashtable ($graphqlInput), which is then converted to JSON using ConvertTo-Json. The resulting JSON is assigned to the input parameter in another hashtable ($graphqlParams). Finally, splatting is used to pass the parameters to the gh api command. This way, you avoid issues with variable expansion in the here-string

Just curious! Thanks!!

[rhubarb-geek-nz]

I have a powershell script that is run on linux (pwsh v7.2.latest) and windows (pwsh v7.3.latest) and just builds a project:

I am wondering how you run Visual Studio on Linux.

If you want cross platform builds have you considered using "dotnet" to do the builds rather than building using Visual Studio? You should be able to build just with the dotnet sdk.

[savornicesei]

@rhubarb-geek-nz Hi!
I use Rider on linux with mono and/or dotnet and the build script detects the location of msbuild based on the platform it runs.

All projects targeting .NET 4 or higher can be converted to SDK-style projects and can be built with dotnet (a conversion I strongly suggest to all in .NET world, see try-convert and upgrade assistant ).

My pwsh build script uses InvokeBuild module which offers an exec command and the solution that finally works for me in regards to passing parameters to MSBuild is to pass them as array

param (
[Parameter(HelpMessage = 'Additional compilation flags like LOG_CONFIG - to log app configuration')]
    [string[]] $CompilationFlags = @()
) 
# ..... other stuff...
$EXTRA_BUILD_CONSTANTS = $CompilationFlags -join ";"
# ..... other stuff...
$rebuildParams = $Rebuild ? @('--force', '--no-incremental') : @()
$optimizeParams = $IS_CI_BUILD ? @('--property:EnableNETAnalyzers=false') : @()

    exec { & dotnet build `
        --configuration $BuildConfiguration `
        --nologo --verbosity:$verbosity `
        $rebuildParams `
        --property:ExtraConstants="""$EXTRA_BUILD_CONSTANTS""" `
        --property:DebugMode=$DebugMode `
        --property:EnvironmentName=$RUN_ENVIRONMENT `
        $optimizeParams `
    }

Best regards,
Simo