bedrock: L1 constructs error #29966

Samir-AWS · 2024-04-26T00:12:44Z

Describe the bug

Bedrock L1 constructs have been added via #29911 in version 2.139.0

Issue1
Unable to use cdk migrate to generate it from a CloudFormation stack

$ cdk migrate --stack-name Test2 --from-path ./test2.yml --language csharp

This command is an experimental feature.
 ❌  Migrate failed for `Test2`: Test2Stack could not be generated because Name is not a valid property for resource BedrockKnowledgeBase of type AWS::Bedrock::KnowledgeBase

Test2Stack could not be generated because Name is not a valid property for resource BedrockKnowledgeBase of type AWS::Bedrock::KnowledgeBase

Issue2
CfnKnowledgeBase class is missing a required property StorageConfiguration in CfnKnowledgeBaseProps

Issue3
I tried to deploy without this property anyway and ran into another issue-

$ cdk synth

<successfully synthesises - output omitted>

$ cdk deploy

Test: deploying... [1/1]
Test: creating CloudFormation changeset...

 ❌  Test failed: Error [ValidationError]: Template format error: Unrecognized resource types: [AWS::Bedrock::DataSource, AWS::Bedrock::KnowledgeBase]
    at Request.extractError (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:46692)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:91559)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:91007)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:199533)
    at Request.transition (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:193085)
    at AcceptorStateMachine.runTo (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:157957)
    at /usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:158287
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:193377)
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:199608)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:91727) {
  code: 'ValidationError',
  time: 2024-04-25T23:56:02.578Z,
  requestId: 'f5515800-5335-434b-9df6-d930bcf788c4',
  statusCode: 400,
  retryable: false,
  retryDelay: 236.99598614253415
}

 ❌ Deployment failed: Error [ValidationError]: Template format error: Unrecognized resource types: [AWS::Bedrock::DataSource, AWS::Bedrock::KnowledgeBase]
    at Request.extractError (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:46692)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:91559)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:91007)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:199533)
    at Request.transition (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:193085)
    at AcceptorStateMachine.runTo (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:157957)
    at /usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:158287
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:193377)
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:199608)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.139.0/libexec/lib/node_modules/aws-cdk/lib/index.js:385:91727) {
  code: 'ValidationError',
  time: 2024-04-25T23:56:02.578Z,
  requestId: 'f5515800-5335-434b-9df6-d930bcf788c4',
  statusCode: 400,
  retryable: false,
  retryDelay: 236.99598614253415
}

Template format error: Unrecognized resource types: [AWS::Bedrock::DataSource, AWS::Bedrock::KnowledgeBase]

Expected Behavior

Support for cdk migration
Addition of StorageConfiguration
Successful deployment without any validation error

Current Behavior

Migration is not possible for Bedrock resources
AWS::Bedrock::KnowledgeBase is invalid without StorageConfiguration property
Deployment error- not recognising AWS::Bedrock::DataSource, AWS::Bedrock::KnowledgeBase resources

Reproduction Steps

C#

        var bedrockKnowledgeBase = new CfnKnowledgeBase(this, "BedrockKnowledgeBase", new CfnKnowledgeBaseProps
        {
            Name = "bedrock-dotnet-demo-kb",
            KnowledgeBaseConfiguration = new KnowledgeBaseConfigurationProperty
            {
                Type = "VECTOR",
                VectorKnowledgeBaseConfiguration = new VectorKnowledgeBaseConfigurationProperty
                {
                    EmbeddingModelArn = $"arn:aws:bedrock:{Region}::foundation-model/amazon.titan-embed-text-v1"
                }
            },
            RoleArn = bedrockExecutionRoleForKnowledgeBase.AttrArn
        });

        var bedrockKnowledgeBaseDataStore = new CfnDataSource(this, "BedrockKnowledgeBaseDataStore", new CfnDataSourceProps
        {
            Name = "bedrock-dotnet-demo-kb-data-source",
            KnowledgeBaseId = bedrockKnowledgeBase.Ref,
            DataSourceConfiguration = new DataSourceConfigurationProperty
            {
                Type = "S3",
                S3Configuration = new S3DataSourceConfigurationProperty
                {
                    BucketArn = bedrockRagDataBucket.AttrArn,
                    InclusionPrefixes = new[] { "kb-" }
                }
            }
        });

Possible Solution

No response

Additional Information/Context

$ cdk version
2.139.0 (build 2699ffd)

DotNet Amazon.CDK.Lib NuGet version- 2.139.0

OS- MacOS Sonoma

CDK CLI Version

2.139.0

Framework Version

No response

Node.js Version

18.17.0

OS

MacOS Sonoma 14.4.1

Language

.NET

Language Version

8.0.201

Other information

No response

The text was updated successfully, but these errors were encountered:

ssinghgai · 2024-04-26T14:36:09Z

+1 - Observed the same behavior - "#: required key [StorageConfiguration] not found ""

khushail · 2024-04-26T19:35:18Z

+1 - Observed the same behavior - "#: required key [StorageConfiguration] not found ""

@Samir-AWS , Thanks for reporting this issue. I am also facing the same deployment issue as shared by @ssinghgai.

I see there is some mismatch in the Cfn Service spec mentioning the StorageConfiguration -

and the CDK Documentation for CfnKnowledgeBaseProps.

CDK imports servicespec using spec2cdk and based on the package.json, current latest CDK release is using 0.0.64 . I also see some discrepancies in documentation as well - Here is AWS Documentation and this is CDK documentation. Marking the issue as appropriate.

pahud · 2024-04-26T21:16:18Z

I can confirmed StorageConfiguration is available in CFN spec 171.0.0 at us-east-1.

The aws-bedrock-knowledgebase.json of awscdk-service-spec@v0.0.65 is having StorageConfiguration now. This means StorageConfiguration is available in awscdk-service-spec@v0.0.65

Looks like the most recent aws-cdk-lib@2.139.0 is still using awscdk-service-spec@0.0.64

I believe the next aws-cdk release should include awscdk-service-spec@0.0.65 that has the StorageConfiguration support.

TheRealAmazonKendra · 2024-04-29T19:54:06Z

The addition of resources in cdk migrate (in cdk-from-cfn) is independent of their addition into aws-cdk-lib. We had been blocked from releasing in that repo due to an unrelated issue so no updates made it into the last cdk release. We can now release again so these updates should make it into the next release.

colifran · 2024-05-02T14:21:51Z

Hi @Samir-AWS. This PR has added StorageConfiguration into our generated bedrock L1s.

github-actions · 2024-05-02T18:22:23Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Samir-AWS · 2024-05-05T23:38:36Z

Thank you team for fixing this issue. StorageConfiguration is now available and I could deploy successfully in us-east-1 and us-west-2

However, the deployment issue still exists in ap-southeast-2 region. I haven't tried any other regions, but assuming those might have this issue as well.

$ cdk deploy

❌  bedrock-dotnet-demo failed: Error [ValidationError]: Template format error: Unrecognized resource types: [AWS::Bedrock::DataSource, AWS::Bedrock::KnowledgeBase]
    at Request.extractError (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:46692)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:91600)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:91048)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:199651)
    at Request.transition (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:193203)
    at AcceptorStateMachine.runTo (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:158075)
    at /usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:158405
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:193495)
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:199726)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:91768) {
  code: 'ValidationError',
  time: 2024-05-05T23:30:51.682Z,
  requestId: '3f9ec639-1b0f-4eee-8258-1a60cfc7396c',
  statusCode: 400,
  retryable: false,
  retryDelay: 8.307862464785432
}

 ❌ Deployment failed: Error [ValidationError]: Template format error: Unrecognized resource types: [AWS::Bedrock::DataSource, AWS::Bedrock::KnowledgeBase]
    at Request.extractError (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:46692)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:91600)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:91048)
    at Request.emit (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:199651)
    at Request.transition (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:193203)
    at AcceptorStateMachine.runTo (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:158075)
    at /usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:158405
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:193495)
    at Request.<anonymous> (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:199726)
    at Request.callListeners (/usr/local/Cellar/aws-cdk/2.140.0/libexec/lib/node_modules/aws-cdk/lib/index.js:387:91768) {
  code: 'ValidationError',
  time: 2024-05-05T23:30:51.682Z,
  requestId: '3f9ec639-1b0f-4eee-8258-1a60cfc7396c',
  statusCode: 400,
  retryable: false,
  retryDelay: 8.307862464785432
}

Template format error: Unrecognized resource types: [AWS::Bedrock::DataSource, AWS::Bedrock::KnowledgeBase]

cc: @colifran @TheRealAmazonKendra @pahud

colifran · 2024-05-06T00:00:22Z

@Samir-AWS It doesn't look like AWS::Bedrock::Datasource or AWS::Bedrock::KnowledgeBase are available in ap-southeast-2 - https://d2stg8d246z9di.cloudfront.net/latest/gzip/CloudFormationResourceSpecification.json. I don't actually see any bedrock resources available in ap-southeast-2. Here is the AWS CloudFormation resource specification page - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html.

Samir-AWS added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Apr 26, 2024

github-actions bot added the @aws-cdk/aws-cloudformation Related to AWS CloudFormation label Apr 26, 2024

khushail added investigating This issue is being investigated and/or work is in progress to resolve the issue. p2 and removed needs-triage This issue or PR still needs to be triaged. labels Apr 26, 2024

khushail added effort/medium Medium work item – several days of effort and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Apr 26, 2024

github-actions bot mentioned this issue Apr 29, 2024

Weekly issue metrics report #29993

Closed

TheRealAmazonKendra added toolkit/migrate Related to cdk migrate and removed @aws-cdk/aws-cloudformation Related to AWS CloudFormation effort/medium Medium work item – several days of effort labels Apr 29, 2024

TheRealAmazonKendra self-assigned this Apr 29, 2024

github-actions bot mentioned this issue May 1, 2024

Monthly issue metrics report #30025

Closed

tbrand mentioned this issue May 1, 2024

[DNM] Knowledge Base RAG の構築テストブランチ aws-samples/generative-ai-use-cases-jp#479

Open

piradeepk assigned colifran and unassigned TheRealAmazonKendra May 1, 2024

piradeepk added p1 and removed p2 labels May 1, 2024

colifran closed this as completed May 2, 2024

josebagar mentioned this issue May 8, 2024

bedrock: L1 construct for KnowledgeBase is missing the StorageConfiguration parameter in python #30116

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bedrock: L1 constructs error #29966

bedrock: L1 constructs error #29966

Samir-AWS commented Apr 26, 2024

ssinghgai commented Apr 26, 2024

khushail commented Apr 26, 2024 •

edited

pahud commented Apr 26, 2024

TheRealAmazonKendra commented Apr 29, 2024

colifran commented May 2, 2024 •

edited

github-actions bot commented May 2, 2024

Samir-AWS commented May 5, 2024

colifran commented May 6, 2024 •

edited

bedrock: L1 constructs error #29966

bedrock: L1 constructs error #29966

Comments

Samir-AWS commented Apr 26, 2024

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

CDK CLI Version

Framework Version

Node.js Version

OS

Language

Language Version

Other information

ssinghgai commented Apr 26, 2024

khushail commented Apr 26, 2024 • edited

pahud commented Apr 26, 2024

TheRealAmazonKendra commented Apr 29, 2024

colifran commented May 2, 2024 • edited

github-actions bot commented May 2, 2024

⚠️COMMENT VISIBILITY WARNING⚠️

Samir-AWS commented May 5, 2024

colifran commented May 6, 2024 • edited

khushail commented Apr 26, 2024 •

edited

colifran commented May 2, 2024 •

edited

colifran commented May 6, 2024 •

edited