AWS API Gateway: Lack of support for CF intrinsic functions at functions[].events[].http.authorizer.arn

[rochdev]

• What was the config you used?

functions:
  myFunction:
    handler: handler.myFunction
    events:
      - http:
          ...
          authorizer:
            arn: { "Fn::Join" : [ ":", [ "arn:aws:lambda", { "Ref" : "AWS::Region" }, { "Ref" : "AWS::AccountId" }, "function:myAuthorizer" ] ] }
            ...

• What stacktrace or error message from your provider did you see?

Serverless: Packaging service...

  Type Error ---------------------------------------------

     functionArn.split is not a function

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues

     Please report this error. We think it might be a bug.

  Your Environment Information -----------------------------
     OS:                 darwin
     Node Version:       4.3.2
     Serverless Version: 1.6.1

Similar or dependent issues:

• Bug Report : Not be able to use Fn::ImportValue or Ref in the Authorizer section of a File in *.YAML File #3129

Additional Data

• Serverless Framework Version you're using: 1.6.1
• Operating System: macOS Sierra 10.12.3

---

Proposed solution:

This report signals two issues:

1. Messed up error reporting (functionArn.split is not a function)

No need to do anything here, as such errors will be handled neatly, once schema is configured for this event (see: #8018)

2. No support for CF intrinsic functions at authorizer.arn.

See: #3212 (comment)

1. When arn is passed in object form, require name property (validate it's existence inline, at least it might be hard to validate it with schema, and receive human friendly error message).
2. When arn is passed in string form and no name is passed. deprecate name resolution with message that it won't be auto-resolved in next major and to have authorizer named, name has to be explicitly set in config

[jatsrt]

Similar issue:
One project that is the global authorizer, and exports lambda function ARNs

Other project that want to use that authorizer

authorizer:
  apn:
    Fn::ImportValue: ${self:authService}:${opt:stage, self:provider.stage}:AuthLambdaARN

This yields the same problem as the naming cannot be determined from this. Only other option is to hardcode the ARN for now.

[HyperBrain]

@eahefnawy @pmuens This is a similar issue as the arn parsing in the stream arns. I think we need a general arn parser function somewhere that is capable to
(1) read string formatted arns
(2) read arbitrary dynamic arns (Fn::Join, Fn::GetAttr, Ref)
And return the reference resource logical id.
It makes no sense to fix the problem in various places separately with different implementations.

I suggest to have one parseArn() function in serverless.utils that would be used from everywhere.

In my plugin I used a very generic function for that:

	static findAllReferences(root) {
		const resourceRefs = [];
		const stack = [ { parent: null, value: root, path: '' } ];

		while (!_.isEmpty(stack)) {
			const property = stack.pop();

			_.forOwn(property.value, (value, key) => {
				if (key === 'Ref') {
					resourceRefs.push({ ref: value, path: property.path });
				} else if (key === 'Fn::GetAtt') {
					resourceRefs.push({ ref: value[0], path: property.path });
				} else if (_.isObject(value)) {
					key = _.isArray(property.value) ? `[${key}]` : (_.isEmpty(property.path) ? `${key}` : `.${key}`);
					stack.push({ parent: property, value, path: `${property.path}${key}` });
				}
			});
		}

		return resourceRefs;
	}

Given a root object (in this case the arn: object) it will return an array of all contained references (including those contained within GetAtt) with their path. You can use _.get(root, refPath) to access the ref object or just use the returned refs (which are the logical resource ids).

Can someone link the other issue as related here?

[pmuens]

@HyperBrain love the idea of having a central ARN parser!

What do you think about opening up a separate issue for this. Or should we rename this one?

This implementation will also be affected by the ARN parser: #3111 (comment)

[HyperBrain]

@pmuens I think we should open a separate issue and link all existing arn parser exception issues there. Then it is clear for every single issue that it will be handled centrally.

[HyperBrain]

Link to #3212 (comment) as the s3 event bucket specification could also be a generic ARN as an alternative to create the bucket implicitly. This would make the s3 events much more flexible.

#3309

[pmuens]

Adding #3162 here (as brought up by @HyperBrain)

[HyperBrain]

Adding a good example brought up by @erikerikson in the context of step functions:

Resource: [lambda name | activity name | lambda ARN | activity ARN | Fn::GetAtt (lambda) | Fn::Ref (activity) | Fn::ImportValue]

So the Arn parser should support the following declarations and parse them correctly:

• lambda name
• Fully qualified ARN (arn::....)
• Fn::Ref (resource id)
• Fn::GetAtt (resource id),(identifier)
• Fn::Join (containing a split arn possibly with Fn::Ref, Fn::GetAtt or Fn::ImportValue as parts)
• Fn::ImportValue
• Fn::Sub

Any additions are welcome.

[pmuens]

Thanks for updating @HyperBrain

Here's the link to the comment mentioned above 🔝 --> #3024 (comment)

[jliebrand]

Perhaps I'm missing something; but isn't this issue also about referencing resources from within the same stack?

I have a cognito user pool in my Resources section, and I would like to reference it as an authorizer using Fn::GetAtt but the way the code is written today, this is not possible, right?

ie this is no just for cross stack referencing, is it? Or is there a way of doing what i want that I'm missing?

[HyperBrain]

This task is for a general universal ARN parser that allows specifying references as ARN string, Fn::GetAtt, Ref or whatever notations there are to reference resources. It should unify the references, may it be event declarations or something else.
So you're completely right - the initial reason for the task was, that multiple places where in-stack resources had been referenced where/are implemented differently in the different locations with the effect that e.g. some notations work in one place but not in other places.

[jliebrand]

Ok; and to be clear; right now there's no way to define a Cognito UserPool in the Resources section and then have the functions use events that auth against that newly created user pool, since there's no way to get to the ARN of that created pool; correct?

[erikerikson]

As HyperBrain notes, ARN parser was the original thought this was created under (the stream attribute was used for both Kinesis and Dynamo streams) but it is definitely intended to be a generalized resource reference implementation.

[erikerikson]

The reason for this is current inconsistency in the implementation. That sometimes you can use Ref or ImportValue, et cetera but not others. For a specific case, try it and see.

[pmuens]

As HyperBrain notes, ARN parser was the original thought this was created under (the stream attribute was used for both Kinesis and Dynamo streams) but it is definitely intended to be a generalized resource reference implementation.

The reason for this is current inconsistency in the implementation. That sometimes you can use Ref or ImportValue, et cetera but not others. For a specific case, try it and see.

@erikerikson comments are a pretty good summary of the purpose for the arn parser.

@jliebrand have you used the new cognitoUserPool event source (released in v1.15) to achieve that? Or are you using another solution right now?

[jliebrand]

@pmuens no, I wasn't aware of any cognito event source... What I do now is first deploy my stack with the functions having simply no authorizer at all... Then once the stack is deployed for each of my stages, I add these custom vars to the top of my yml:

custom:
  stagingUserPool: arn:aws:cognito-idp:xxxOne
  masterUserPool: arn:aws:cognito-idp:xxxTwo
  devUserPool: arn:aws:cognito-idp:xxxThree

And then add these authorizers:

          authorizer:
            arn: ${self:custom.${opt:stage}UserPool}

This will obviously break if I ever remove and re-deploy a stack, so it's far from ideal

[pmuens]

Interesting solution. Thanks for sharing 👍

We've just added the cognitoUserPool event source. Only supports triggers right now though...

[hassankhan]

Just chiming in to say a global ARN parser would be incredibly useful.

With #3657 we can define user pools, but we can't reference their ARNs as Lambda authorizers. This unfortunately means either having to split a stack or deploy once without authorizers and then again after retrieving and setting the ARN 😞

[horike37]

I'm also happy it would implement this.
With my https://github.com/horike37/serverless-step-functions plugin, you need describe Lambda ARN directly, so not userfriendly..
Therefore I'd like to use a global ARN parser and would also like to reference Lambda functions ARN

[pmuens]

Another interesting suggestion / feedback can be found here.

[brettstack]

Any reason not to add checks for authorization.type === 'COGNITO_USER_POOLS' || in these places:

1. https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/compile/events/apiGateway/lib/method/authorization.js#L39
2. https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/compile/events/apiGateway/lib/authorizers.js#L28
3. https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/compile/events/apiGateway/lib/permissions.js#L53

It works for my template, but not sure if there are edge cases that it may break. Happy to send a PR. @pmuens

Doing a search on cognitoIdpArnExpr https://github.com/serverless/serverless/search?q=cognitoIdpArnExpr&unscoped_q=cognitoIdpArnExpr I see there may be other places to patch.

[rodrigogs]

After a lot of try and error, I figured that if you add a name prop the whole thing works. I didn't try to find why it works, to be honest. I'll leave that for @rochdev ;)

hello-world:
  handler: hello-world.handler
  events:
    - http:
        path: /hello-world
        authorizer:
          name: SharedAuthorizer
          arn:
            'Fn::ImportValue': ${self:provider.stage}-SharedAuthorizer

It doesn't work if you have several services where you wish to use the same authorizer. It gives me Authorizer name must be unique. Authorizer authorizer already exists in this RestApi.

Is there a really simple way to share an authorizer? Cuz, for God's sake, it should be pretty basic!

[rodrigogs]

After a lot of try and error, I figured that if you add a name prop the whole thing works. I didn't try to find why it works, to be honest. I'll leave that for @rochdev ;)

hello-world:
  handler: hello-world.handler
  events:
    - http:
        path: /hello-world
        authorizer:
          name: SharedAuthorizer
          arn:
            'Fn::ImportValue': ${self:provider.stage}-SharedAuthorizer

It doesn't work if you have several services where you wish to use the same authorizer. It gives me Authorizer name must be unique. Authorizer authorizer already exists in this RestApi.

Is there a really simple way to share an authorizer? Cuz, for God's sake, it should be pretty basic!

It works if you give a unique authorizer name to each service... which looks strange.

[medikoo]

If I see correctly, this report signals two issues:

1. Messed up error reporting, functionArn.split is not a function signals lack of support for non-string format for authorizer.arn -> This will be handled neatly, once schema is configured for this event (see: Config schema: Define AWS "http" function event properties #8018)

2. No support for CF intrinsic functions at authorizer.arn.

To solve latter, there's a call to introduce a global arn parser, and implenting such doesn't seem to be trivial. In other functionalities we handle any resolution from CF intrinsic functions case by case, and I think similar approach should be taken (if needed) here.

However, when I looked closer, I've noticed that the only reason we try to inspect arn is to resolve a referenced resource name, so it's used as a name of an APIGW authorizer. This approach, appears to derive from initial implementation, where just authorizers that referenced functions in a same service were supported (then we accepted only name property, which was used for authorizer name, and to construct authorizer uri.

When support for custom authorizers was added, name was unconditionally assumed from passed arn, and that seems problematic (later it was improved, so we do not attempt to resolve if authorizer.name is passed directly)

If I read correctly name is not a required property on AWS side: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-authorizer.html#cfn-apigateway-authorizer-name

I believe right fix, is to drop the resolution of name whenever arn is passed, and by doing that we can support any format on authorizer.arn.

While, if for some reason we want to have name unconditional, then imo right fix would be to unconditionally require name with arn.

I've re-titled issue and updated top description. PR with a fix is welcome

[rodrigogs]

After a lot of try and error, I figured that if you add a name prop the whole thing works. I didn't try to find why it works, to be honest. I'll leave that for @rochdev ;)

hello-world:
  handler: hello-world.handler
  events:
    - http:
        path: /hello-world
        authorizer:
          name: SharedAuthorizer
          arn:
            'Fn::ImportValue': ${self:provider.stage}-SharedAuthorizer

It doesn't work if you have several services where you wish to use the same authorizer. It gives me Authorizer name must be unique. Authorizer authorizer already exists in this RestApi.
Is there a really simple way to share an authorizer? Cuz, for God's sake, it should be pretty basic!

It works if you give a unique authorizer name to each service... which looks strange.

And the worst part is: if you name each authorizer you will reach AWS authorizers limit pretty soon

[rodrigogs]

After reaching AWS authorizers limit I had to make it work. Turns out that sls creates a CF resource for the authorizer lambda, but not for the authorizer itself(I guess it just creates the authorizer via API). This way I cant really export an actual authorizer from a service unless I create a CF for it.

So... in the end I have two considerations about this whole situation:

1. Don't go with the approach of creating a named authorizer for each service. AWS only gives you 10 authorizer slots by default.
2. The way to go is to provision your own authorizer and exporting its authorizer id. This example really helped me with that: https://github.com/medwig/serverless-shared-authorizer/blob/master/authorizer_and_api/serverless.yml

[OndeVai]

This is a nasty issue!
I was able to hack around it by doing this

  create:
    handler: src/create.main
    events:
      - http:
          path: some-resources
          method: post
          cors: true
          authorizer:
            type: COGNITO_USER_POOLS #undocumented type :(
            name: cognitoAuthorizer #needs a name
            arn: !ImportValue outworkit-${self:custom.stage}-user-pool-arn

When I add the type and the name, it only creates one authorizer for me --even if I provide the same name in multiple lamdba authorizer sections.

I had to figure this out by diving into the source code :(
https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/compile/events/apiGateway/lib/authorizers.js#L27

[pgrzesik]

I believe right fix, is to drop the resolution of name whenever arn is passed, and by doing that we can support any format on authorizer.arn.

While, if for some reason we want to have name unconditional, then imo right fix would be to unconditionally require name with arn.

I've investigated it a little bit and it seems like the authorizer.name is also needed to resolve logicalId that will be used in CloudFormation template for AWS::ApiGateway::Authorizer. I was thinking about alternative approach to resolving that logicalId, but I feel like the name-based generation seems like the best choice. Given that, I think that while arn is passed as a string, we should try to resolve name from it if it's not provided explicitly - otherwise, if arn is passed as an object (with CF intrinsic function), we should require name to be defined as well.

@medikoo What do you think?

About the issues that e.g. @OndeVai mentioned with his workaround - in order to support sharing authorizers cleanly between multiple functions, we would have to allow an option to define authorizer e.g. on provider.apiGateway level and allow to reference it in specific functions, but that seems like a separate task.

[medikoo]

Given that, I think that while arn is passed as a string, we should try to resolve name from it if it's not provided explicitly - otherwise, if arn is passed as an object (with CF intrinsic function), we should require name to be defined as well.

@pgrzesik very good point. I totally agree that's the way to go.

we would have to allow an option to define authorizer e.g. on provider.apiGateway level and allow to reference it in specific functions, but that seems like a separate task.

Yes, and we have a dedicated issue for that: #2887 (I've also added it now to our project, as I think it's worth addressing at some point)

[medikoo]

I've updated spec in top description with proposal, where we require name with object configuration for authorizer arn.

[pgrzesik]

Thank you - I'll continue with that approach 💯