Support generic OIDC authentication for AzureRM backend

[karlschriek]

Terraform Version

1.2.9

Use Cases

We want to set up workflows that run terraform using Azure Workload Identities. The workload identity approach works by treating an AKS cluster as an OIDC provider, and a specific ServiceAccount within a specific Namespace on that cluster as an identity, which can be federated to an Azure AD Service Principal. (https://azure.github.io/azure-workload-identity/docs/)

A similar (but specific to GitHub) solution was introduced in #30936. Similarly, this was introduced for GitHub on the AzureRM provider here: hashicorp/terraform-provider-azurerm#16555 (the naming of both PRs is very generic, but the actual implementation is GitHub specific).

Generic OIDC: AzureRM Provider

A generic OIDC implementation (which works for Azure Workload Identities) was added in the AzureRM provider here: hashicorp/terraform-provider-azurerm#18118. It uses github.com/hashicorp/go-azure-helpers v0.40.0 and introduces an oidc_token variable

This implementation allows us to configure the provider using values populated by the AWI webhook.

export ARM_CLIENT_ID=$AZURE_CLIENT_ID
export ARM_SUBSCRIPTION_ID=xxx-xxx-xxx-xxx-xxx #this isn't populated by the AWI webhook
export ARM_TENANT_ID=$AZURE_TENANT_ID
export ARM_OIDC_TOKEN=$(cat $AZURE_FEDERATED_TOKEN_FILE)

In conjunction with:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.20.0"
    }
  }
}

and

provider "azurerm" {
  use_oidc = true
  features {}
}

Generic OIDC: AzureRM Backend

A generic OIDC implementation does not exist for the AzureRM Backend yet. I would like to be able to configure the backend as follows, and have it use the same authentication approach as what is in the AzureRM Provider, i.e:

export ARM_CLIENT_ID=$AZURE_CLIENT_ID
export ARM_SUBSCRIPTION_ID=xxx-xxx-xxx-xxx-xxx #this isn't populated by the AWI webhook
export ARM_TENANT_ID=$AZURE_TENANT_ID
export ARM_OIDC_TOKEN=$(cat $AZURE_FEDERATED_TOKEN_FILE)

terraform {
  backend "azurerm" {
    use_oidc=true             
    resource_group_name=myrg
    storage_account_name=mysa
    container_name=myctr
  }
}

or, if setting everything explicitly:

terraform {
  backend "azurerm" {
    use_oidc=true             
    resource_group_name=myrg
    storage_account_name=mysa
    container_name=myctr
    client_id=xxxxxxxxxxxxxxxxx
    subscription_id=xxxxxxxxxxxxxxxxx
    tenant_id=xxxxxxxxxxxxxxxxx
    oidc_token=xxxxxxxxxxxxxxxxx
  }
}

Attempted Solutions

For this specific situation there is no real workaround other than to use a different authentication method altogether.

Proposal

I would propose looking into the changes introduced in hashicorp/terraform-provider-azurerm#18118 and following the same approach to add an oidc_token / ARM_OIDC_TOKEN parameter to the AzureRM Backend implementation.

References

No response

[karlschriek]

I have started looking at how this could be implemented. It looks like it should be quite straightforward, namely:

1. In go.mod, set github.com/hashicorp/go-azure-helpers v0.40.0
2. In internal/backend/remote-state/azure/arm_client.go, add the following

	builder := authentication.Builder{
               ...
		IDToken: config.OIDCToken,  
	}

authentication.Builder refers to a struct in github.com/hashicorp/go-azure-helpers, to which IDToken was added in v0.40.0

3. In internal/backend/remote-state/azure/arm_client.go, add the following

func New() backend.Backend {
	s := &schema.Backend{
		Schema: map[string]*schema.Schema{
                        ...

			"oidc_token": {
				Type:        schema.TypeString,
				Optional:    true,
				DefaultFunc: schema.EnvDefaultFunc("ARM_OIDC_TOKEN", ""),
				Description: "The OIDC ID token for use when authenticating as a Service Principal using OpenID Connect.",
			},
			...
		},
	}
}

and

type BackendConfig struct {
    ....
    OIDCToken                     string
    ....
}

and

func (b *Backend) configure(ctx context.Context) error {
         ...
	config := BackendConfig{
                ...
		OIDCToken:                     data.Get("oidc_token").(string),
                ...
	}

---

UPDATE:

I have now tested it with the changes above and it works.

[karlschriek]

Note, the above is based on branching from tag v1.2.9.

I am happy to have a go at submitting a PR, but since the first RC for 1.3.0 has just been cut, the timing is probably not that great. Should I try to get it in for the next RC or do you have feature freeze until the stable version is out?

[crw]

Thanks for this request and possible solution! I notified the Azure Provider team. Thanks again!

[matt3621]

Thanks for this request and possible solution! I notified the Azure Provider team. Thanks again!

Hi, I've run into issues assuming the functionality in the backend mirrored the azurerm provider, while trying to use circleci OIDC to authenticate to Azure. Details here hashicorp/terraform-provider-azurerm#18641, I opened this issue today and then searched the correct place and found this issue (and thankfully the reason terraform had been driving me crazy for the last few days),

Is this likely to be incorporated in a release any time soon ?

Many thanks

Matthew

[Amier3]

@karlschriek

We'd definitely welcome a PR on this if you're still up for it. Your described approach is exactly what the team would do 😄 .

@matt3621 Thanks for reviving this conversation, if Karl is willing to take this up i'll let him give some context into when he'd be able to get the PR in. After that comes in we should be able to merge the PR before the following release.

[karlschriek]

Sure I'm happy to have a go at a PR. Anything I should know? Otherwise I'll just branch from HEAD and add the changes as described above.

[11lein]

The fix from #31966 works for me with Gitlab and Azure Federated Identities (OIDC).

[adamrushuk]

Thought I'd share a blog post with example code of this working: https://adamrushuk.github.io/configure-terraform-openid-connect-oidc-authentication-from-gitlab-ci-to-azure/

[mattrawles]

Thought I'd share a blog post with example code of this working: https://adamrushuk.github.io/configure-terraform-openid-connect-oidc-authentication-from-gitlab-ci-to-azure/

Thanks @adamrushuk for this and thanks to @karlschriek for getting this fix in, I can also confirm it works nicely on circleci, i've posted some examples https://github.com/CodeInAVan/cicd-oidc-authentication/tree/main/pipeline-examples/azure and also have referenced your blog (more working examples you can find on this the easier it is to adapt it to your use case!).

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.