fix(s3-deployment): updating memoryLimit or vpc results in stack update failure

[corymhall]

This fixes the issue where updating the memoryLimit or vpc of the
BucketDeployment resource would result in a stack update failure. In
order to fix this the ID of the BucketDeployment CustomResource includes
info on the memoryLimit and vpc in the same way that the Lambda function
does. This means that when either of these values are updated, the
BucketDeployment CustomResource will be recreated along with the Lambda
function.

If anyone is setting retainOnDelete=false (default is true) then
this change would result in the data in the bucket being deleted. In
order to avoid this scenario, this PR introduces a bucket tag that
controls whether or not a BucketDeployment resource can delete data from
the bucket.

The BucketDeployment resource will now add a tag to the deployment
bucket with a format like aws-cdk:cr-owned:{keyPrefix}:{uniqueHash}.
For example:

{
  Key: 'aws-cdk:cr-owned:deploy/here/:240D17B3',
  Value: 'true',
}

Each bucket+keyPrefix can be "owned" by 1 or more BucketDeployment
resources. Since there are some scenarios where multiple BucketDeployment
resources can deploy to the same bucket and key prefix
(e.g. using include/exclude) we also append part of the id to
make the key unique.

As long as a bucket+keyPrefix is "owned" by a BucketDeployment
resource, another CR cannot delete data. There are a couple of
scenarios where this comes into play.

1. If the LogicalResourceId of the CustomResource changes
   (e.g. memoryLimit is updated) CloudFormation will first issue a 'Create'
   to create the new CustomResource and will update the Tag on the bucket.
   CloudFormation will then issue a 'Delete' on the old CustomResource
   and since the new CR "owns" the Bucket+keyPrefix (indicated by the
   presence of the tag), the old CR will not delete the contents of the bucket

2. If the BucketDeployment resource is deleted and it is the only CR
   for that bucket+keyPrefix then CloudFormation will first remove the tag
   from the bucket and then issue a "Delete" to the CR. Since there are no
   tags indicating that this bucket+keyPrefix is "owned" then it will delete
   the contents.

3. If the BucketDeployment resource is deleted and it is not the only
   CR for that bucket:keyPrefix then CloudFormation will first remove the tag
   from the bucket and then issue a "Delete" to the CR.
   Since there are other CRs that also "own" that bucket+keyPrefix
   (indicated by the presence of tags), there will
   still be a tag on the bucket and the contents will not be removed. The
   contents will only be removed after all BucketDeployment resource that
   "own" the bucket+keyPrefix have been removed.

4. If the BucketDeployment resource and the S3 Bucket are both removed,
   then CloudFormation will first issue a "Delete" to the CR and since there
   is a tag on the bucket the contents will not be removed. If you want the
   contents of the bucket to be removed on bucket deletion, then
   autoDeleteObjects property should be set to true on the Bucket.

Scenario 3 & 4 are changes to the existing functionality in that they
now do not delete data in some scenarios when they did previously.

fixes #7128

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[gitpod-io]

[otaviomacedo]

We should probably also log the error to make it easier to investigate issues in the future.

[otaviomacedo]

But this is part of the key, which is limited to 128 characters. In any case, how do you enforce this limit, given that the destinationKeyPrefix property doesn't seem to have any length constraint?

[corymhall]

Good call. I added some validation which will throw an error if the tag key is > 128 characters (destinationKeyPrefix can't be > 104 characters). That is a very long key prefix so I don't think we will ever hit that limit, but I also can't think of a better way of handling this.

[mergify]

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[mergify]

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
• Commit ID: b761d3b
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).