AWS Go SDK v1.28.0 Code Generated Error Types Missing Error Message

[bflad]

Please fill out the sections below to help us address your issue.

Version of AWS SDK for Go?

Version 1.28.5

Version of Go (go version)?

Go 1.13

What issue did you see?

Changes in v1.28.0 of the AWS Go SDK (imported and released by Terraform AWS Provider version 2.45.0) introduced code generated types for certain service error shapes, which can be used to extract additional information provided by the API response. Prior to these code generated types, most (if not all) of these errors returned the correct error messaging via Message() via the message protocol response field. However after this change, since some of these error shapes in the service API models are incorrectly missing the Message structure member, this causes the Message_ (Go struct type) to be missing, which causes the error message via Message() to always be empty.

An example of this missing error messaging can be seen with codebuild service InvalidInputException shape:

aws-sdk-go/models/apis/codebuild/2016-10-06/api-2.json

Lines 998 to 1003 in 6627e89

	"InvalidInputException":{
	"type":"structure",
	"members":{
	},
	"exception":true
	},

aws-sdk-go/service/codebuild/api.go

Lines 5333 to 5336 in 6627e89

	// Message returns the exception's message.
	func (s InvalidInputException) Message() string {
	return ""
	}

Debug output from the application shows the error message being returned, but it does not seem possible to access given the above (note missing error message after error code and colon on last line):

2020/01/20 12:59:15 [DEBUG] [aws-sdk-go] DEBUG: Response codebuild/CreateProject Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 400 Bad Request
Connection: close
Content-Length: 167
Content-Type: application/x-amz-json-1.1
Date: Mon, 20 Jan 2020 17:59:14 GMT
X-Amzn-Requestid: 2f589026-d531-458d-b851-86bd4afb5cd0


-----------------------------------------------------
2020/01/20 12:59:15 [DEBUG] [aws-sdk-go] {"__type":"InvalidInputException","message":"CodeBuild is not authorized to perform: sts:AssumeRole on arn:aws:iam::--OMITTED--:role/tf-acc-test-4387558838057379294"}
2020/01/20 12:59:15 [DEBUG] [aws-sdk-go] DEBUG: Validate Response codebuild/CreateProject failed, attempt 0/25, error InvalidInputException:

Full SDK analysis of this new behavior was performed here: hashicorp/terraform-provider-aws#11642 (comment)

In terms of the AWS Go SDK, this change potentially affected the following services:

• apigatewaymanagementapi
• cloud9
• cloudhsm
• cloudtrail
• cloudwatchevents
• cloudwatchlogs
• codebuild
• codecommit
• codedeploy
• codepipeline
• codestar
• configservice
• dax
• directconnect
• ecs
• elasticsearchservice
• elastictranscoder
• emr
• eventbridge
• lexmodelbuildingservice
• managedblockchain
• pinpointemail
• rdsdataservice
• rekognition
• servicecatalog
• sesv2
• sms
• ssm
• ssooidc
• textract
• transfer
• waf
• wafregional
• workdocs

In terms of the Terraform AWS Provider, this change broke the following behaviors which were dependent on checking the previously available error messages:

• In aws_cloud9_environment_ec2 resource, unable to retry for IAM eventual consistency on NotFoundException during creation (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_cloud9_environment_ec2.go#L96-L98)
• In aws_cloudtrail resource, unable to retry for CloudWatch Logs eventual consistency on InvalidCloudWatchLogsRoleArnException and InvalidCloudWatchLogsLogGroupArnException during creation or update (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_cloudtrail.go#L186-L191)
• In aws_cloudwatch_log_destination resource, unable to retry for InvalidParameterException with test message delivery during creation or update (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_cloudwatch_log_destination.go#L70-L72)
• In aws_cloudwatch_log_subscription resource, unable to retry for InvalidParameterException with test message delivery and Lambda Function execution during creation (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_cloudwatch_log_subscription_filter.go#L71-L76)
• In aws_codebuild_project resource, unable to retry for IAM eventual consistency on InvalidInputException during creation and update (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_codebuild_project.go#L665-L669)
• In aws_codedeploy_deployment_group resource, unable to retry for SNS Topic eventual consistency on InvalidTriggerConfigException during creation and update (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_codedeploy_deployment_group.go#L783-L788)
• In aws_config_delivery_channel resource, unable to retry for running Config Configuration Recorder on LastDeliveryChannelDeleteFailedException during deletion (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_config_delivery_channel.go#L172-L174)
• In aws_ecs_service resource, unable to retry for cross-service eventual consistency on InvalidParameterException during creation and update (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_ecs_service.go#L524-L533)
• In aws_elasticsearch_domain resource, unable to retry for various eventual consistency errors on InvalidTypeException / ValidationException during creation (https://github.com/terraform-providers/terraform-provider-aws/blob/96840caea28e3abf72d0d994550d2ee7d4b7fc45/aws/resource_aws_elasticsearch_domain.go#L459-L474)

Historically, we have preferred to sometimes check specific error messaging from the service API responses since the error code itself may not be descriptive enough to determine if the error could/should be retried. While the service APIs should likely introduce more granular error codes and fix their API models, the unexpected removal of the error messaging represents a fairly problematic change in the AWS Go SDK since it was previously supported by many services.

Is it possible for these errors to try and return a message, if available, even if the Message member is undefined in the API model?

References:

• private/mode/api: Add codegen typed Errors for RESTJSON and JSONRPC APIs #3062
• Terraform AWS Provider Version 2.45.0: Missing Error Messages and Not Retrying hashicorp/terraform-provider-aws#11682
• Resource CodeBuild project broken in 2.45 hashicorp/terraform-provider-aws#11642
• aws_elasticsearch_domain failed with ValidationException: Authentication error status code: 400 hashicorp/terraform-provider-aws#7725

Steps to reproduce

From https://github.com/terraform-providers/terraform-provider-aws:

$ TF_ACC=1 go test ./aws -v -count 1 -timeout 120m -parallel 20 -run='TestAccAWSCodeBuildProject_basic'

[jasdel]

Thanks for reporting this bug, and the in depth description of the problem. It looks like the SDK's generation needs to ensure all errors have a Message_ member regardless if the error is modeled as having the value.

We'll work on getting a fix out for this bug.