AzureRm V2: Cannot create keyVault without Subscription Level Permissions

[FlagshipApps]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

azurerm version 2.0
terraform version v0.12.21

Affected Resource(s)

• azurerm_key_vault

Terraform Configuration Files

provider "azurerm" {
    version = "2.0"
    features {}
    skip_provider_registration = "true"
}
 
resource "azurerm_key_vault" "AKVDea01" {
    name                            = "mykvname"
    location                        = "westeurope"
    resource_group_name             = "myrgname"
    tenant_id                       = "my-tenant-id"

    sku_name = "standard" 
}

Expected Behavior

I expect the keyVault to be created

Actual Behavior

I can't create the Keyvault and i get the following error:

Error: Error checking for the presence of an existing Soft-Deleted Key Vault "mykvname" (Location "westeurope"): keyvault.VaultsClient#GetDeleted: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'b.xxx@xxx.de' with object id 'xxxxx' does not have authorization to perform action 'Microsoft.KeyVault/locations/deletedVaults/read' over scope '/subscriptions/xxxxx' or the scope is invalid. If access was recently granted, please refresh your credentials."

Steps to Reproduce

create a new terraform project.
Copy my code from above and try:

terraform init
terraform apply

Important Factoids

I do not have access on the subscription level, just on the resource group.
Of course this should not be a problem (and it wasn't).
Im azurerm version 1.28.0 i can create a keyVault without the same permissions i have right know.
the following script works for me:

provider "azurerm" {
    version = "1.28.0"
    skip_provider_registration = "true"
}
 
resource "azurerm_key_vault" "AKVDea01" {
    name                            = "mykvname"
    location                        = "westeurope"
    resource_group_name             = "myrgname"
    tenant_id                       = "my-tenant-id"

    sku {
        name = "standard"
    }
}

[sai-ns]

@KSLiquid can you try adding below in features block and see if it really skips not checking for soft-deleted keyvaults. I still had issues with it, just wanted to make sure I am not the only one
features {
key_vault {
recover_soft_deleted_key_vaults = false
}

[FlagshipApps]

@sai-ns thanks for sharing the idea! I tried it, but it does not work :(
Does anyone else has a fix here?

[jasong1978]

I have the same issue, this might be an issue with Terraform security. There are plenty of environments that allow for automation accounts like service principals to only have access to their resource groups so that if a service principal has been compromised, it does not have affect outside of that. There is a workaround you can do that worked for us, but this is not the final solution, Terraform needs to allow for the subscription/resource group scope instead. But for now you need to assign a custom role to your service principal....

{
"Name": "Key Vault Soft Delete Read Access",
"Id": null,
"IsCustom": true,
"Description": "Allows for read access list soft deleted vaults",
"Actions": [
"Microsoft.KeyVault/locations/deletedVaults/read"
],
"NotActions": [],
"AssignableScopes": [
"/subscriptions/{subscriptionId}",
]

New-AzRoleDefinition -InputFile "C:\CustomRoles\KeyVault-Soft-Delete-Read01.json"

I recommend using Custom RBAC role with special permission to list, a Key Vault in Soft delete state
https://docs.microsoft.com/en-us/azure/key-vault/key-vault-ovw-soft-delete#soft-delete-retention-period

The special permissions specified in the doc below will allow to list a Key Vault in soft delete state
https://docs.microsoft.com/en-us/azure/key-vault/key-vault-soft-delete-powershell#required-permissions
Required permissions
Key Vault operations are separately managed via role-based access control (RBAC) permissions as follows:
Operation Description User permission
List Lists deleted key vaults. Microsoft.KeyVault/deletedVaults/read
Recover Restores a deleted key vault. Microsoft.KeyVault/vaults/write
Purge Permanently removes a deleted key vault and all its contents. Microsoft.KeyVault/locations/deletedVaults/purge/action

For information on creating a custom RBAC role kindly refer to the link mentioned below:
https://docs.microsoft.com/en-us/azure/role-based-access-control/custom-roles-powershell#create-a-custom-role

[IntoTheNature]

same issue here. The org we work with limit permissions on resource group level even for Service Principals.

[FlagshipApps]

@jasong1978 sadly this requires permissions that i do not have :(

[Jawvig]

I know no-one has suggested it yet, but to pre-empt, setting the purge_soft_delete_on_destroy flag to false as well does not get around the problem.

provider "azurerm" {
    version = "=2.2"
    # ....
    features {
        key_vault {
            # Setting these options does _not_ stop permission needed at the subscription level 
            # on "Microsoft.KeyVault/locations/deletedVaults/read"
            recover_soft_deleted_key_vaults = false
            purge_soft_delete_on_destroy = false
        }
    }
}

[jemag]

Same problem here, setting

      recover_soft_deleted_key_vaults = false

does not help, it still requires subscription wide permissions. Using provider version 2.2.0

[Jawvig]

Okay, having attempted a fix (that did work and allowed me to create the key vault without subscription level permissions) I think this issue is unresolvable without adding another feature flag or dropping part of the functionality.

Assuming there is already a soft deleted key vault of the same name as the one you're trying to create, the functionality is:

If the RecoverSoftDeletedKeyVaults flag is true, recover the existing one.
Else if it is false, return an error stating that it already exists.

In order to reach this logic it must already have checked whether there is a soft deleted key vault of the same name, needing subscription level permissions. If we alter the code (as I did) to skip checking for a soft deleted key vault if RecoverSoftDeletedKeyVaults is false, then it is never possible to reach the statement that returns an error stating that it already exists.

There ought to be an Azure error if that's the case, but the test TestAccAzureRMKeyVault_softDeleteRecoveryDisabled is checking for the specific error returned from the provider rather than from Azure.

Two potential fixes:

1. Add another feature flag such as `SkipCheckingDeletedKeyVaults'
2. Skip the code that returns the error if there's a soft deleted key vault of the same name.

Opinions?

[Jawvig]

I've created pull request #6260 which I think resolves this issue. With reference to my previous comment, I have gone for option 2 which is to skip the code that returns the error if there's a soft deleted key vault of the same name. I feel this is the right thing to do because Azure already performs this check itself. And, in fact, it performs it more subtly. It allows for creation if the soft deleted key vault had no content (secrets, keys, etc.) and only errors if it had content.

I've adjusted the acceptance test to expect the error from Azure rather than the error text that I removed.

[FlagshipApps]

Thanks alot Jawvig for tackling that.
I wanted to do a similar thing myself but failed to do the development setup with go, make and stuff :D

[sai-ns]

@Jawvig Been waiting for your PR to be merged or Terraform to come up with a fix. Anyway we can bump this up making it a priority?

[ttjackott]

Any news on the release of this?

[jasong1978]

Hello, any news on this release? Can you please bump up the priority? This is a security risk to have to assign a service principal custom roles to the whole subscription in order to proceed with automation using Terraform and AzureRM.

[Jawvig]

The PR #6260 has been tagged with release 2.7.0, so I think it will be soon.

[ghost]

This has been released in version 2.8.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.8.0"
}
# ... other configuration ...

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!