Appconfig middleware throws error when trying to fetch configuration of type "feature flag"

[mju-spyrosoft]

Describe the bug
Not sure if this is a bug or a feature request but when trying to fetch Appconfig configuration of type "Feature flag" middleware throws Failed to resolve internal value error.

To Reproduce
How to reproduce the behaviour:

1. Prepare configuration in appconfig of type feature flag
   

2. Sample code

import middy from '@middy/core';
import appConfig from '@middy/appconfig';

async function baseHandler(event: any, context: any) {
  console.log(context.appconfig);
}

export const handler = middy(baseHandler).use(
  appConfig({
    fetchData: {
      config: {
        Application: 'xkhkg5h',
        ClientId: 'userSpecifiedId',
        Configuration: '2cqyhl',
        Environment: 'p5oyugh'
      }
    },
    setToContext: true
  })
);

3. Thrown error

{
    "errorType": "Error",
    "errorMessage": "Failed to resolve internal values",
    "stack": [
        "Error: Failed to resolve internal values",
        "    at getInternal (/var/task/dist/index.js:26269:11)",
        "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
        "    at async appConfigMiddlewareBefore (/var/task/dist/index.js:26409:20)",
        "    at async runMiddlewares (/var/task/dist/index.js:26124:17)",
        "    at async runRequest (/var/task/dist/index.js:26081:5)"
    ]
}

Expected behaviour
Middleware should fetch configuration of type "feature flag" without throwing error

Environment (please complete the following information):

• Node.js: [e.g. 18.13.0]
• Middy: [e.g. ^4.2.3]
• AWS SDK - "@aws-sdk/client-appconfigdata": "^3.267.0"

Additional context
I called GetConfigurationCommand used by appconfig middleware directly in index.js. Underlying error i got is:

{
    "errorType": "BadRequestException",
    "errorMessage": "Feature flag configurations must be accessed via AWS AppConfig Data's GetLatestConfiguration API.",
    "name": "BadRequestException",
    ...
}

It seems that GetConfigurationCommand is deprecated and GetLatestConfiguration from @aws-sdk/client-appconfig should be used instead to retrive configs. API is bit different.
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-appconfig/classes/getconfigurationcommand.html
https://docs.aws.amazon.com/appconfig/latest/userguide/appconfig-retrieving-the-configuration.html

[willfarrell]

Thanks for reporting! I think you might be the first to use this middleware, so sorry you're running into these issues. Would you like to put a PR together to resolve this?

[mju-spyrosoft]

Sure I could try putting something together soon-ish but I'd need help with deciding what should be done with appconfig NextPollIntervalInSeconds attribute which i'm not sure how it should work with middy cache TTL.

Fetching app config flow is.

1. Start new config session, You get token in a response
2. Fetch config using token. You get next config token, and NextPollIntervalInSeconds
3. You use latest token to fetch any config updates

According to docs

NextPollIntervalInSeconds
The amount of time the client should wait before polling for configuration updates again. Use
RequiredMinimumPollIntervalInSeconds to set the desired poll interval.

So options could be something like

1. middleware should automatically fetch latest configuration on execution of lambda after NextPollIntervalInSeconds time passed since previous one. This would mean that middy cache TTL should be set dynamically based on NextPollIntervalInSeconds.
2. alternatively NextPollIntervalInSeconds could be ignored and middy cache TTL should be used to decide when config should be fetched

Option 2 seems valid as AWS docs say:

We recommend tuning the polling frequency of your GetLatestConfiguration API calls based on your budget, the expected frequency of your configuration deployments, and the number of targets for a configuration.

Seems like option 2 stays consistent with middy design in other middlewares? but not really sure what design should be.

[mju-spyrosoft]

One more thing that i'll point out is AWS provides Lambda extension that already handles polling for appconfig. Seems like it's doing same thing as this middleware would.

[willfarrell]

Sorry on the delay, been a busy week.

Because this is the only AWS SDK that has this polling feature that I know of let's go with option 2

2. alternatively NextPollIntervalInSeconds could be ignored and middy cache TTL should be used to decide when config should be fetched

and include in the docs that cacheExpiry is equivalent to NextPollIntervalInSeconds.

[mju-spyrosoft]

Alright. I'll try to put something together on a weekend

[willfarrell]

Hey, just wanted to follow up on this.

[mju-spyrosoft]

Yeah sorry for being quiet, been busy with preparation for bunch of presentations internally in company. I'd like to try to do this on weekend but im not sure if i'll manage to. If its not going happen on this weekend I don't want to keep this stalled forever

[willfarrell]

Thanks for all of your work on this. I'm published 5.0.0-alpha.0 with just your PR.

[dobeerman]

The same issue with ssm middleware

export const handler = middy(lambda)
  .use(ssm({
    fetchData: {
      chatProxy: `/${process.env.ENVIRONMENT}/chat-proxy/api-key`,
    },
    setToContext: true,
  }));

throws an error

{
    "errorType": "Error",
    "errorMessage": "Failed to resolve internal values",
    "stack": [
        "Error: Failed to resolve internal values",
        "    at getInternal (/var/task/index.js:54653:11)",
        "    at processTicksAndRejections (node:internal/process/task_queues:96:5)",
        "    at async ssmMiddlewareBefore (/var/task/index.js:54867:20)",
        "    at async runMiddlewares (/var/task/index.js:54580:17)",
        "    at async runRequest (/var/task/index.js:54537:5)"
    ]
}

[dobeerman]

The same issue with ssm middleware

Solved.
The issue was with policies that didn't contain "ssm:..." actions.