Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Don't duplicate PolicyDocument statement for stream events (#12313) #12320

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tibbe
Copy link

@tibbe tibbe commented Jan 4, 2024

Each function consuming a stream event would emit its own PolicyDocument statement. This statement would contain a list of actions that doesn't change between functions. For DynamoDB streams the list is:

"Action": [
    "dynamodb:GetRecords",
    "dynamodb:GetShardIterator",
    "dynamodb:DescribeStream",
    "dynamodb:ListStreams"
],

Duplicating these for each function causes the IAM policy to exceed the AWS limit after about 30 functions.

The resource names are still duplicated, if they happen to be the same.

Closes: #12313

…erless#12313)

Each function consuming a stream event would emit its own PolicyDocument
statement. This statement would contain a list of actions that doesn't
change between functions. For DynamoDB streams the list is:

```
"Action": [
    "dynamodb:GetRecords",
    "dynamodb:GetShardIterator",
    "dynamodb:DescribeStream",
    "dynamodb:ListStreams"
],
```

Duplicating these for each function causes the IAM policy to exceed the
AWS limit after about 30 functions.

The resource names are still duplicated, if they happen to be the same.
@tibbe
Copy link
Author

tibbe commented Jan 4, 2024

I ran test/integration/aws/stream.test.js, which passes.

@tibbe
Copy link
Author

tibbe commented Jan 4, 2024

I didn't implement de-duplication of resource names, as I'm not sure how to do so correctly in the preference of functions like Fn::GetAtt in the arn property of the stream object in the Serverless config.

In my case resource name de-duplication would be helpful, as even with this fix the DynamoDB table name is duplicated 30 times in the Resource array. This fix makes our prod backend deployable again so it's still a (large) improvement.

@tibbe
Copy link
Author

tibbe commented Jan 13, 2024

Anything else I can do to shepherd this along?

@tibbe
Copy link
Author

tibbe commented Feb 21, 2024

Could someone please take look at this? This is rather problematic issue for large users of serverless.

@tibbe
Copy link
Author

tibbe commented Mar 8, 2024

Ping

@tibbe
Copy link
Author

tibbe commented Apr 2, 2024

Does this project accept external contributions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Each DynamoDB stream event handler duplicates the IAM policy, causing "Maximum policy size exceeded"
1 participant