securestring broken in Test-AzResourceGroupDeployment/New-AzResourceGroupDeployment

[DOMZE]

Description

When passing a secure string to either Test-AzResourceGroupDeployment or New-AzResourceGroupDeployment the cmdlet returns

InvalidTemplate - Deployment template validation failed: 'Template parameter JToken type is not valid. Expected
'String, Uri'. Actual 'Object'. Please see https://aka.ms/resource-manager-parameter-files for usage details.'.

Occurs in 2.5.0
Doesn't occur in 2.4.0

Steps to reproduce

Arm template:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "_artifactsLocation": {
           "type": "string",
           "metadata": {
                "description": "The storage account location"
            }
        },
        "_artifactsLocationSasToken": {
           "type": "securestring",
           "metadata": {
                "description": "Storage SAS Token"
            }
        }
    },
    "variables": {
        "url": "[uri(parameters('_artifactsLocation'), concat('container/myFile.ps1', parameters('_artifactsLocationSasToken')))]",
    },
    "resources": [],
    "outputs": {}
}

$ResourceGroupName = "rg-myresourcegroup"
$TemplateFile = ".\test-arm.jsonc"
$location = "https://mystorage.blob.core.windows.net"
$sasTokenSecure = ConvertTo-SecureString "my_amazing_token!" -AsPlainText -Force

Test-AzResourceGroupDeployment -ResourceGroupName $ResourceGroupName `
								-TemplateFile $TemplateFile `
								-_artifactsLocation $location `
								-_artifactsLocationSasToken $sasTokenSecure

Environment data

Name                           Value
----                           -----
PSVersion                      7.0.3
PSEdition                      Core
GitCommitId                    7.0.3
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

Get-InstalledModule -Name Az.Resources

2.5.0 is problematic. Uninstalling 2.5.0 and installing 2.4.0 doesn't generate the error

Error output

InvalidTemplate - Deployment template validation failed: 'Template parameter JToken type is not valid. Expected
'String, Uri'. Actual 'Object'. Please see https://aka.ms/resource-manager-parameter-files for usage details.'.

[krishjag]

I ran into the same problem with our ARM deployments from Azure DevOps using Microsoft hosted agents (VS-2017).

Currently, I worked around the problem by forcing a specific version of the Az module.

https://www.powershellgallery.com/packages/Az/4.5.0

Install-Module -Name Az -RequiredVersion 4.5.0 -AllowClobber -AcceptLicense -Force

[isra-fel]

Hi @DOMZE we have acknowledged this issue. Fix will be available in the next release. Thanks for reporting!

[mangatram1]

Install-Module -Name Az -RequiredVersion 4.5.0 -AllowClobber -AcceptLicense -Force

This definetly resolves the issue as suggested by @krishjag

[cveld]

Is this fixed in 4.6.1?

[isra-fel]

Is this fixed in 4.6.1?

No, sorry, 4.6.1 fixed an issue in Az.Compute.
Resources team is working hard to fix issues and providing better experience ragarding deployment in our next relaese this month.

[ghost]

Was this closed in favor of #12819?

[nevergonnagityouup]

Having the same issue trying to deploy an ARM template with "New-AzResourceGroupDeployment" in Azure Automation Runbooks.

[ckiefer]

Thanks for sharing your findings. Spent 4 hours debugging my script and template until I found this post.
@isra-fel: When do you expect this fix to be rolled out?

[isra-fel]

@ckiefer we are targeting next Tuesday, 9/22

[pravin-g-otk]

I am also facing this issue while using securestring to be passed in New-AzResourceGroupDeployment using powershell in AzureDevOps release pipeline using VS2017-win2016 agent but not with windows-2019 agent.
Is this issue is specific with hosted agent for vs2017-win2016 ?
And when we are expecting to fix this issue ?

[isra-fel]

Hi @pravin-g-otk this issue is fixed in Az 4.7.0 / Az.Resources 2.5.1 (just released). Could you please try and verify?

[PaulVrugt]

omg, lost half a day to find out why the deployment using azure pipelines stopped working. Please update the hosted pipeline images!

[edfreeman]

@isra-fel This still doesn't seem to work when using the TemplateParameterObject hashtable parameter of Test-AzResourceGroupDeployment (or New-AzResourceGroupDeployment).

The only way we got it to work using the TemplateParameterObject parameter was to pass in the secure string in a roundabout way:

$templateParams = @{}
$StagingSasToken = "xxxxxxxxxx"

$sasToken = (ConvertTo-SecureString -AsPlainText -Force $StagingSasToken)
$sasCred = New-Object -TypeName PSCredential -ArgumentList 'sas', $sasToken

$templateParams["_artifactsLocation"] = $location
$templateParams["_artifactsLocationSasToken"] = $sasCred.Password      ## <- This seems to work, rather than passing $sasToken directly

Test-AzResourceGroupDeployment `
                -ResourceGroupName $ResourceGroupName `
                -TemplateFile $TemplateFile `
                -TemplateParameterObject $templateParams

We have no idea why $sasCred.Password works, but $sasToken itself doesn't - they're both Secure Strings.

Anyway, if you don't do that workaround, the only way we found that worked (using just ConvertTo-SecureString "xxxxxxxx" -AsPlainText -Force) was by listing the template parameters individually:

$StagingSasToken = "xxxxxxxxxx"
$sasToken = (ConvertTo-SecureString -AsPlainText -Force $StagingSasToken)

Test-AzResourceGroupDeployment `
                -ResourceGroupName $ResourceGroupName `
                -TemplateFile $TemplateFile `
                -_artifactsLocation $location `
                -_artifactsLocationSasToken $sasToken

like the OP. Or, equivalently, by splatting the parameters:

$templateParams = @{}

$StagingSasToken = "xxxxxxxxxx"
$sasToken = (ConvertTo-SecureString -AsPlainText -Force $StagingSasToken)

$templateParams["_artifactsLocation"] = $location
$templateParams["_artifactsLocationSasToken"] = $sasToken

Test-AzResourceGroupDeployment `
                -ResourceGroupName $ResourceGroupName `
                -TemplateFile $TemplateFile `
                @templateParams

So I think there's still the same bug when applying template parameters through the TemplateParameterObject parameter.

[jberezanski]

I can confirm the problem still appears when using Az.Resources 6.6.0 (we haven't been able to upgrade to newer due to #21647).

However, I found a slightly terser workaround:

$sasToken = (ConvertTo-SecureString -AsPlainText -Force $StagingSasToken)

$templateParams["_artifactsLocationSasToken"] = $sasToken.PSObject.BaseObject

We have no idea why $sasCred.Password works, but $sasToken itself doesn't - they're both Secure Strings.

I think this is caused by the fact that many/most objects in PowerShell are automatically wrapped in PSObject instances. Those wrappers are usually transparent to the code that operates on those objects; unfortunately, template deployment seems to choke on them. My workaround reaches through the wrapper PSObject and retrieves the real SecureString instance. I guess the value returned from the Password property of PSCredential also avoids being wrapped.