aws-s3-deployment: BucketDeployment executes in stack that is not being deployed

[Kilowhisky]

Describe the bug

I often break up my CDK apps into multiple stacks, i've found that if i place a BucketDeployment in stack A and then ask to deploy stack B with npx cdk deploy B stack A will execute the bucket deployment even when i haven't asked it to.

Expected Behavior

Don't execute the BucketDeployment when not being asked to?

Current Behavior

It executed the BucketDeployment incorrectly and then stopped.

Reproduction Steps

Build a stack like below then attempt to deploy prod with npx cdk deploy prod-web. It will synth the dev assets first and then immediately start the deployment of the DEV asset to the prod bucket.

const app = new cdk.App();

createAppForStage(app, { env: 'dev' });
createAppForStage(app, { env: 'prod' });

function createAppForStage(app: cdk.App, options: Options) {
     new WebStack(app, `${options.env}-web`, options, {
         stackName: `${options.env}-web`,
         tags,
         env
  })
}

class WebStack extends cdk.Stack {
  constructor(scope: Construct, id: string, options: Options, props?: cdk.StackProps) {
    super(scope, id, props);

    const bucket = new Bucket(this, 'AngularBucket', {
      bucketName: `client-ui-${options.env}`,
      removalPolicy: cdk.RemovalPolicy.RETAIN,
    })

    const policy = new cdk.aws_cloudfront.OriginAccessIdentity(this, 'AngularOriginAccess', {
      comment: "client-access-identity"
    });

    const origin = new S3Origin(bucket, {
      originId: `${bucket.bucketName}.s3.us-west-2.amazonaws.com`,
      originAccessIdentity: policy
    });

    const distro = new Distribution(this, 'AngularDistro', {
      defaultRootObject: 'index.html',
      enableLogging: false,
      httpVersion: HttpVersion.HTTP2,
      defaultBehavior: {
        origin,
        viewerProtocolPolicy: ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
      },
    });

    // Based on this: https://medium.com/@john_andreas/build-deploy-angular-app-on-aws-using-aws-cdk-v2-f960187d5409
     new BucketDeployment(this, 'AngularAppDeployment', {
        destinationBucket: bucket,
        distribution: distro,
        sources: [
          Source.asset('stacks/web', {
            bundling: {
              image: cdk.DockerImage.fromRegistry("public.ecr.aws/docker/library/node:lts"),
              local: {
                tryBundle(outputDir) {
                  try {
                    spawnSync("npm --version");
                    spawnSync(
                      `cd stacks/web && npm run build:${options.env} -- --output-path ${outputDir}`,
                      {
                        shell: true, // for debugging
                        windowsHide: false,
                        stdio: "inherit",
                      }
                    );
                    return true;
                  } catch {
                    return false;
                  }
                },
              }
            },
          })
        ]
      })
  }
}

Possible Solution

I've tried to see if the code knows which app is being deployed but its not present in the process.env or args.

Additional Information/Context

No response

CDK CLI Version

2.137.0 (build bb90b4c)

Framework Version

No response

Node.js Version

v20.12.0

OS

windows 11

Language

TypeScript

Language Version

5.4.3

Other information

No response

[pahud]

Let me simplify the provided code as below:

#!/usr/bin/env node
import 'source-map-support/register';
import { Construct } from 'constructs';
import * as cdk from 'aws-cdk-lib';
import {
    aws_s3 as s3,
    aws_s3_deployment as s3d,
} from 'aws-cdk-lib';

const app = new cdk.App();

interface Options {
    env: string;
}

class WebStack extends cdk.Stack {
    constructor(scope: Construct, id: string, options: Options, props?: cdk.StackProps) {
      super(scope, id, props);
  
      const stackName = cdk.Stack.of(this).stackName;
      const bucket = new s3.Bucket(this, `AngularBucket${options.env}`, {
        bucketName: `client-ui-${options.env}-${stackName}`,
        removalPolicy: cdk.RemovalPolicy.RETAIN,
      })
  
       new s3d.BucketDeployment(this, `AngularAppDeployment${options.env}`, {
          destinationBucket: bucket,
          sources: [ s3d.Source.jsonData(`object-key-${options.env}.json`, { json: 'object' }) ],
        })
    }
}

function createAppForStage(app: cdk.App, options: Options) {
    new WebStack(app, `${options.env}-web`, options, {
        stackName: `${options.env}-web`,
 })
}

createAppForStage(app, { env: 'dev' });
createAppForStage(app, { env: 'prod' });

Now if I run cdk ls I get:

dev-web
prod-web

so I run cdk diff prod-web I see this

Resources
[+] AWS::S3::Bucket AngularBucketprod AngularBucketprod277B4E0F 
[+] AWS::Lambda::LayerVersion AngularAppDeploymentprod/AwsCliLayer AngularAppDeploymentprodAwsCliLayer0A51C6C0 
[+] Custom::CDKBucketDeployment AngularAppDeploymentprod/CustomResource AngularAppDeploymentprodCustomResourceC94342E5 
[+] AWS::IAM::Role Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265 
[+] AWS::IAM::Policy Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF 
[+] AWS::Lambda::Function Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536 

Only Custom::CDKBucketDeployment for prod will be deployed.

Now I run cdk deploy prod-web, I only get 1 bucket created for prod and I can see the object in that bucket is

indicating this is for production only.

I didn't see the dev deployed.

Are you able to simplify your app and verify again?

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[Kilowhisky]

I think i've figured out exactly what is happening now.

It has to do with CDK caching the output of the BucketDeployment Assets and thinking it doesn't need to rebuild, so it just deploys what it has cached, which happens to be the 'dev' build, not the 'prod' build like i instructed.

I see above that you were not able to replicate using the Source.JsonData. Are you able to using the Source.asset with tryBundle() like i have above?

[Kilowhisky]

I played around with it a little bit and it appears that even if i manage to get the Source.asset to recompile, its output is not considered when determining if the stack needs to push. The CDK just returns (no changes).

So the real bug here is that CDK is not rebuilding the Source.Asset() for the BucketDeployment and even if it does, the hash of the assets is not being considered when pushing changes.

To replicate this,

1. Push a 'dev' build.
2. Delete the cdk.out folder
3. Push a 'prod' build.
4. See that the CDK decides to not publish because it doesn't detect any changes.

[Kilowhisky]

Alright, i don't know how i didn't see this setting but setting the

Source.asset('stacks/web', {
          assetHashType: cdk.AssetHashType.OUTPUT,
          .... 

Solves the problem.

By default it is set to SOURCE which means its checking for changes in the source files in order to decide if it needs to recompile or redeploy. Problem is that the source is an angular/react project that COMPILES the output. So of course the source wouldn't have changed.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.