Supporting SASL/SCRAM auth with MSK events

[pedrocava]

Use case description

There's an MSK cluster running with SASL auth. There's a backend producing events and a Serverless application listening to certain topics, using messages to trigger parametrized reactions, such as calling other lambda functions or uploading data to a DB.

Proposed solution

AWS has recently started supporting SASL auth for MSK events. Unfortunately, this feature's documentation in CloudFormation is rather scarce.

Extending the convention @pgrzesik provided here, we should have something like:

functions:
  compute:
    handler: handler.compute
    events:
      - msk:
          arn: arn:aws:kafka:region:XXXXXX:Cluster/xxx
          topic: kafka-topic
          startingPosition: LATEST
          batchSize: 1000
          saslScram512: arn:aws:secrets_manager:xxxxxxxxx

The ARN provided to saslScram512 should be associated with the secret created during the SASL setup, described in more detail here. Much like the accessConfigurations field in kafka events works.

[medikoo]

@pedrocava great thanks for that request. Proposal looks good enough, and I think we can jump to PR directly (which is welcome!).

[pgrzesik]

Thanks @pedrocava - it looks great 👍 It will be a bit different than for kafka, where these configurations are nested under accessConfigurations, however, I see that in case of msk the saslScram512 is the only possible configuration which suggests that in this case, nesting under accessConfigurations doesn't make much sense. 👍

[renetta96]

hello, is this supported yet?

[pgrzesik]

Hello @renetta96 👋 At the moment it's not supported, but we'd be more than happy to accept a community contribution for it 💯