AuthenticationFailedException when consuming storage account queues using User Managed Identity

[damianvandoom]

When using a user-managed identity, a function app cannot consume a storage account queue. However, when changing to a system-managed identity, the function app was able to consume the storage queue.

The error (when using a user-managed identity) is:

2024-04-30T18:17:34.308 [Error] An unhandled exception has occurred. Host is shutting down.
Azure.Identity.AuthenticationFailedException : ManagedIdentityCredential authentication failed: Service request failed.
Status: 400 (Bad Request)

Content:


Headers:
Date: Tue, 30 Apr 2024 18:17:33 GMT
Server: Kestrel
Transfer-Encoding: chunked
X-CORRELATION-ID: REDACTED
Content-Type: application/json; charset=utf-8

See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot ---> Azure.RequestFailedException : Service request failed.
Status: 400 (Bad Request)

Content:


Headers:
Date: Tue, 30 Apr 2024 18:17:33 GMT
Server: Kestrel
Transfer-Encoding: chunked
X-CORRELATION-ID: REDACTED
Content-Type: application/json; charset=utf-8

   at async Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async,TokenRequestContext context,Response response,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async,TokenRequestContext context,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async,TokenRequestContext context,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async,TokenRequestContext requestContext,CancellationToken cancellationToken)
   End of inner exception
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex,String additionalMessage)
   at async Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async,TokenRequestContext requestContext,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Identity.ManagedIdentityCredential.GetTokenAsync(TokenRequestContext requestContext,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources,TokenRequestContext requestContext,Boolean async,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async,TokenRequestContext requestContext,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex,String additionalMessage)
   at async Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async,TokenRequestContext requestContext,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(TokenRequestContext context,Boolean async,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message,TokenRequestContext context,Boolean async)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message,TokenRequestContext context,Boolean async)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AuthenticateAndAuthorizeRequestAsync(HttpMessage message,TokenRequestContext context)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Storage.StorageBearerTokenChallengeAuthorizationPolicy.AuthorizeRequestInternal(HttpMessage message,Boolean async)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(??)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.InnerProcessAsync(HttpMessage message,ReadOnlyMemory`1 pipeline)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(??)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.RetryPolicy.ProcessAsync(??)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Core.Pipeline.RetryPolicy.ProcessAsync(??)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Storage.Queues.QueueRestClient.GetPropertiesAsync(Nullable`1 timeout,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Storage.Queues.QueueClient.GetPropertiesInternal(Boolean async,CancellationToken cancellationToken,String operationName)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Storage.Queues.QueueClient.ExistsInternal(Boolean async,CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Azure.Storage.Queues.QueueClient.ExistsAsync(CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Extensions.Storage.Common.Listeners.QueueListener.ExecuteAsync(CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Extensions.Storage.Common.Timers.TaskSeriesTimer.RunAsync(CancellationToken cancellationToken)
2024-04-30T18:17:35.512 [Information] Host Status: {
  "id": "ddmstorageapptest",
  "state": "Running",
  "version": "4.33.1.22394",
  "versionDetails": "4.33.1+3a214f2665e01b267f01f6d5c7cc49f79c118642",
  "platformVersion": "102.0.7.131",
  "instanceId": "bbca31cf6f23378b6946b204461c2591a64068f9c9d27ce34bf1b5503c85fe17",
  "computerName": "dw1sdwk0004HN",
  "processUptime": 19315,
  "functionAppContentEditingState": "Unknown"
}

Investigative information

Please provide the following:

• Timestamp: 2024-04-30T18:17:35.512
• Function App version: 4
• Function App name: ddmstorageapptest
• Function name(s) (as appropriate): Function1
• Invocation ID:
• Region: North Europe

Repro steps

1. Create a Windows 6.0 .NET function app in VS
2. Create Azure function app (s1)
3. Create an Azure storage account with a queue
4. Class and settings as below in Related information
5. Publish
6. Create User-Manager Identity
7. Assign User-Manager Identity to Azure function app
8. Assign User-Manager Identity to store account as role Storage Queue Data Contributor
9. Create a queue item in Storage account
10. Review host logs for Function app - observe error in logs
11. Remove the User-Manager Identity from storage account roles
12. Remove the User-Manager Identity from the user managed identities in the function app
13. Enable system managed identity for the function app
14. Assign User-Manager Identity to store account as role Storage Queue Data Contributor
15. Add queue item
16. Review host logs for Function app - observe that queue is processed.

Expected behavior

The function app should be able to authenticate to the storage account and interrogate the queues using a user-managed identity.

Actual behavior

When using a user-managed identity, the function app errors as above and it will not process the storage account queues.

Known workarounds

Use the function system managed identity

Related information

Class

public class Function1
{
    [FunctionName("Function1")]
    public void Run([QueueTrigger("thequeue", Connection = "QueueConection")]string myQueueItem, ILogger log)
    {
        log.LogInformation($"C# Queue trigger function processed: {myQueueItem}");
    }
}

QueueConection

"QueueConection__queueServiceUri": "https://xxxxxx.queue.core.windows.net/"