Variables syntax collides with AWS params syntax #3184
Comments
|
It would be great if serverless could either output yaml and preserve the tags, or render the tags properly into the long form for the json output. |
|
I tried to do this very thing yesterday to discover I couldn't. |
|
Sub clashed with the Serverless "${}" variable declaration syntax as AWS uses exactly the same syntax with Fn::Sub |
|
Exactly. There are probably many ways to support both. Ideally, serverless recognizes the Another idea is to make it explicit with a prefix much like |
|
Introducing a separate CF: prefix for vars sounds more irritating because it just would skip the variable substitution. More intuitive would be to use double-$ to escape variable substitution. So How does that sound? |
|
That works too. So Fn::Sub:
- "$${NotAServerlessVar}-abcd-${ServerlessVar}-$${NotAServerlessVar2}"
- NotAServerlessVar: 123
NotAServerlessVar2: xyz |
Yes, that sounds like a good idea. Since Here's another, old discussion about escaping etc. #1728 (comment) Furthermore you can use the |
|
What was ultimately done to address this? Or is escaping a $ to a CloudFormation object not supported yet? |
|
here is my workaround for now, exclude 'AWS::' as these are the variables that I needed for Fn::Sub. Example serverless.yml |
|
This plugin does this pretty well, for me anyway. |
|
Any updates on this being implemented natively? |
|
@sam3d not sure but we solve this by using the serverless.js instead of serverless.yml. I’m not sure why this is not mentioned in a docs as a recommended way how to define your stack 😉 |
|
Would definitely also appreciate an update on how to support this without adding additional plugins, I have run into this issue multiple times in the past week and would appreciate a native solution. Like the purposed double escape $$ |
|
I believe the workaround is to just use You may have to nest a For example: |
|
Everyday those people saying serverless just causes more problems than it solves, just use raw CloudFormation, make more and more sense to me. |
|
I have this issue: An error occurred (ValidationError) when calling the ValidateTemplate operation: Template error: variable names in Fn::Sub syntax must contain only alphanumeric characters, underscores, periods, and colons Any solution? |
|
How is this not solved yet? It's been almost 3 years. |
|
ping |
|
Seriously, can we just do like a double $$ and be done with it? This is getting ridiculous. I feel so dirty having to use Fn::Join with serverless. |
It really is a bit frustrating... I've solved my problem using this plugin: https://www.npmjs.com/package/serverless-pseudo-parameters Variables declared with ${} will be handled by serverless, and variables declared with #{} will be subbed for ${} on the final template, letting CF do its work. |
|
I couldn't end up using So instead this works: # serverless.yml
provider:
# serverless ignores ${AWS...} and ${_...}
variableSyntax: "\\${((?!AWS|_)[ ~:a-zA-Z0-9._@'\",\\-\\/\\(\\)]+?)}"
# later on this works as expected.
foos:
- !Sub
- some-${AWS::AcountId}-${_specialVar}-string
- _specialVar: barNot claiming to be the best solution, but I like it. And one can change the prefix ( |
|
Edit: works with native CF template You can escape ${} in your templates using ${!} like this: |
|
@Ownmarc if that's working for you it's because you've defined a variableSyntax value that states that that serverless shouldn't interpret ${!var_name} as a serverless var. That's no different than the solution above your post. |
|
Hi all, I think I have found a possible solution. The serverless-pseudo-parameters plugin has been mentioned, and it has the disadvantage of having a custom/weird syntax ( However, I managed to make it work with the serverless variable syntax. Here are a few examples: functions:
hello:
handler: handler.hello
environment:
# We can reference `AWS::` variables
REGION: ${AWS::Region}
# We can use !Ref inline via the `ref:` prefix:
BUCKET_NAME: ${ref:MyBucket}
# We can use !GetAtt inline via the `getatt:` prefix:
BUCKET_URL: 'https://${getatt:MyBucket.DomainName}'
# We can use mix all that together in a string:
REGIONAL_URL: 'https://${ref:MyBucket}.s3.${AWS::Region}.amazonaws.com'
# We can also mix all of that with actual serverless variables (e.g. `${self:provider.stage}`)
resources:
Resources:
MyBucket:
Type: AWS::S3::BucketIn my example above, although I don't explicitly write And all of those strings are compiled to How it worksNothing clever. For my proof of concept, I defined the following variables via a plugin:
Then, the serverless-pseudo-parameters plugin transforms these variables to (here is the PR: svdgraaf/serverless-pseudo-parameters#66) Now, while it works, is this something that we could eventually integrate into Serverless? (this is a question for maintainers) I will be opening the same issue in the plugin (to see if we can add support more quickly), but if we can have that officially, I'm up for giving it a try (maybe with some advice/help?). I don't like the 2 step process, maybe we can do better. Also, all those CloudFormation variables do not work locally, but in any case I don't see how we could make things work locally when we reference cloud resources (which is what Edit: since the plugin also supports functions:
hello:
handler: handler.hello
environment:
REGION: !Sub ${AWS::Region}
BUCKET_NAME: !Sub ${ref:MyBucket}
BUCKET_URL: !Sub 'https://${getatt:MyBucket.DomainName}'
REGIONAL_URL: !Sub 'https://${ref:MyBucket}.s3.${AWS::Region}.amazonaws.com' |
|
I want to add an alternative. CloudFormation syntax usually begins with upper case so: provider:
# Allow using CloudFormation variable syntax without changing the serverless syntax
# The differentiating factor is the capitalization of the first character
# Upper case character means CloudFormation syntax, eg: ${AWS::Region}, ${DatabaseSecret}
# Everything else means serverless syntax, eg: ${ssm:...}, ${self:...}
# https://www.serverless.com/framework/docs/providers/aws/guide/variables#using-custom-variable-syntax
variableSyntax: "\\${((?![A-Z])[ ~:a-zA-Z0-9._@'\",\\-\\/\\(\\)]+?)}"With that configuration you can then do stuff like this: provider:
# [...]
environment:
STAGE: ${opt:stage, self:provider.stage} # use serverless syntax as usual
MAILER_DSN: !Sub 'ses://default?region=${AWS::Region}' # use CloudFormation !Sub as usual
DATABASE_URL: !Sub '//${AWS::Region}/${Database}' # reference other resource as long as they use the UpperCamelCase naming convention |
|
@medikoo that's too bad, this issue contains 3 years of discussion about this, with plenty of solutions and workarounds discussed. It's also very easy to find when users search |
|
@mnapoli that's a very valid point. Let me reopen this one and close the newer one |
medikoo
added
bug/design
medikoo
changed the title
|
I've also proposed a solution in top description, which will most likely solve it for AWS case. |
|
@medikoo sounds really really good!
That means that both AWS native variables and Will you implement it or would a PR help? |
|
@mnapoli PR will definitely help 🙇 Thank you! |
|
The regex update doesn't cover situations where a variable variable may be used. eg. Not sure support for this is intended in Serverless, but the above syntax did resolve prior to the regex update. |
|
@logicalor can you explain what the problem is? I tested your string and it seems to be covered by the regex, like the rest:
|
|
Hi @mnapoli , Your example shows that the regex only matches the inner variable within the string. My understanding is that it should match the entire string. In v2.2.0 of serverless (ie prior to the release with the regex update), I could do something like this: and serverless would interpolate the variable placeholders correctly, resulting in 'ap-southeast-2'. v2.3.0 of serverless and after produce an error with the above. If I modify v2.8.0 and replace the regex with the version of the regex from v2.2.0 and below, the interpolation works again. |
|
@logicalor can you open a new bug report with all the details? (this issue was closed and it addressed other issue) |
and others
I was attempting to use !Sub as shown in the EC2 metadata file section
This example is exactly reflected on the Fn::Sub Page
Yet when I place this in the serverless.yml I get the following exception:
Similar issue: #8116
Proposed solution
Improve variables format regex into
'\\${([^{}:]+?:[^:{}][^{}]*?)}'- This should ensure that notations${StepFunctionArn}(as reported at #8116) or${AWS::Whatever}are not picked as SLS variablesAdditionally to ensure a solution for eventual more difficult cases (not necessary AWS related) we may consider to introduce an escape notation (so e.g.
\${is not recognized as variables start bracket)The text was updated successfully, but these errors were encountered: