DeploymentBucket parameters not being replaced

[mcshiz]

I've noticed that when I try and do parameter replacement for the provider.deploymentBucket or provider.deploymentBucket.name property I am getting errors.

I receive these error with and without the serverless-psuedo-parameters plugin.
Configuration: (Same error when nesting name or omitting it or using Fn::Sub)

  deploymentBucket: 
    name: !Sub 'development-team-deployment-bucket-${AWS::AccountId}'

Result:
Configuration warning at 'provider.deploymentBucket.name': should be string

Configuration:

  deploymentBucket: ${self:provider.stage}-brandzooka-deployment-bucket-#{AWS::AccountId}

Result:
Configuration warning at 'provider.deploymentBucket': should match pattern "(?!^(\d{1,3}\.){3}\d{1,3}$)(^(([a-z0-9]|[a-z0-9][a-z0-9-]*[a-z0-9])\.)*([a-z0-9]|[a-z0-9][a-z0-9-]*[a-z0-9])$)"

Configuration Where ${self:custom.accountId} = ${AWS::AccountId}

  deploymentBucket: 
    name: !Sub ${self:provider.stage}-brandzooka-deployment-bucket-${self:custom.accountId}

Result
This is what wound up in the .serverless/serverless-state.json file which looks like it should work BUT I get a fatal error from serverless Could not locate deployment bucket. Error: str.indexOf is not a function

      "deploymentBucket": {
        "Fn::Sub": "development-brandzooka-deployment-bucket-${AWS::AccountId}"
      },

However this works, so I believe the issue lies in the recent integration of serverless pseudo parameters.

${self:provider.stage}-brandzooka-deployment-bucket-413466xxxxxx

Framework Core: 2.23.0 (local)
Plugin: 4.4.2
SDK: 2.3.2
Components: 3.6.2

Command used to run sls deploy --stage development --region us-east-1 --aws-profile me

[pgrzesik]

Hello @mcshiz 👋 Thanks a lot for reporting and sorry to hear that you've run into trouble with it. At the moment, we're not supporting Sub for deploymentBucket as we're relying on resolved name during the deployment process and Sub will be resolved by CloudFormation. So for this particular case, you'll need to supply AccountID in some other way.

[jwthomp]

I spun my wheels quite a bit today on the same issue. When I restore the deprecated serverless-pseudo-parameters plugin then it works again.

[pgrzesik]

Hello @jwthomp - did you have issue with deploymentBucket setting as well? What version of the Framework are you using?

[jwthomp]

Mine was trying to use !Sub in the custom parameters for serverless-plugin-datadog plugin. Specifically:
custom:
datadog:
forwarder: !Sub... (or #{AWS::AccountId})

I wound up having to re-enable serverless-pseudo-parameters to get #{AWS::AccountId} to work.

[pgrzesik]

Hello @jwthomp - thanks for the explanation, the built-in notation for pseudo-params is a bit different: https://www.serverless.com/framework/docs/providers/aws/guide/variables/#pseudo-parameters-reference did you try that one as well? Additionally, what error did you experience? Did it come from the plugin directly?

[HaaLeo]

I experience a similar issue when I try to set the deploymentBucket.name via an environment variable:

  deploymentBucket: 
    name: ${env:S3_BUCKET}

Serverless then simply ignores this setting and tries to create its own ServerlessDeploymentBucket instead.
Using serverless 2.34.0

[pgrzesik]

Hello @HaaLeo, what version of Framework are you using?

[HaaLeo]

v2.34.0 Updated my previous comment

[pgrzesik]

Thank you @HaaLeo - I've just tried to reproduce your issue but I'm observing correct behavior on my side. Could you provide a reproducible example with minimal serverless.yml and output of sls deploy? Maybe that will make it easier to reproduce the problem and diagnose the potential bug.

[HaaLeo]

My setup is the following
I run the following two commands in two separate CodeBuild steps:

1. sls package args...
2. sls deploy --package .serverless/

The deployment was failing. Therefore, I only checked those logs 🤦🏼‍♂️. However, taking a closer look at the packaging step it indeed warns me about the missing parameter:

Serverless Warning --------------------------------------
 
A valid environment variable to satisfy the declaration 'env:S3_BUCKET' could not be found.

So my mistake was to set the environment variable during the deploy step but not for the packaging step where it is actually needed. Changing this fixed my issue.

Sorry @pgrzesik for over eagerly commenting on this issue.

[pgrzesik]

No worries @HaaLeo, happy to hear you've been able to resolve your issue 👍

[mahendraramesh]

I'm having the same issue. Trying to use the AWS account id in the deployment bucket name so that the bucket is unique across accounts. Is there any way to achieve this?

@pgrzesik you mentioned "you'll need to supply AccountID in some other way", can you suggest any other way to do it?

[pgrzesik]

@mahendraramesh you can for example pass it via env variable and use that variable during resolution

[lludovic]

I have the same issue:
Framework Core: 2.52.0 Plugin: 5.4.3 SDK: 4.2.4 Components: 3.13.4

• In my env.dev file I have var

DEPLOYMENT_BUCKET=${cf:dev-artifacts.ArtifactBucketName}

• serveless.yml

deploymentBucket: ${env:DEPLOYMENT_BUCKET}

Then I Got:
`
Serverless Error ----------------------------------------

Could not locate deployment bucket. Error: The specified bucket is not valid.
`

NB: Everything was fine until I update serverless version

[pgrzesik]

Hey @lludovic - what version of the Framework were you using previously?

[lludovic]

Hey @lludovic - what version of the Framework were you using previously?
The previous version was
Framework Core: 2.39.1
Plugin: 4.5.3
SDK: 4.2.2
Components: 3.9.1

[pgrzesik]

Thanks for sharing, could you also share to which value is deploymentBucket resolved for you at the moment in comparison to previously resolved value?

[lludovic]

The previous resolved value was dev-artifacts-storage the now it's ${cf:dev-artifacts.ArtifactBucketName}

[pgrzesik]

Thanks for sharing all that additional information - it looks like it's specific to the use case where variable is defined as a part of ENV VAR - during dotenv resolution, it tries to expand the value but fails to do so which alters the original value. Expansion change has been introduced recently as a part of this PR #9615

I believe the only way to support this would be to either make the variable expansion via dotenv-expand optional, or you would have to use a different way to pass it than the current approach you're using.

[pgrzesik]

I'm going to close this ticket as original issue has been clarified and there's a way to achieve that now with aws variable source. As for your problem @lludovic - if you haven't been able to resolve it, please feel free to open a dedicated issue. Thanks in advance 🙇