Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cli): template created during import should be written to assets folder #29830

Merged
merged 28 commits into from May 10, 2024
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
c332aff
fix: import should write overrideTemplate to assembly.outdir
nburtsev Mar 15, 2024
8d73b83
Merge branch 'main' into main
nburtsev Mar 15, 2024
b91e06a
Merge branch 'main' into fix/cdk-import-template-path
nburtsev Mar 18, 2024
8b5e5cb
fix: update cli import test to ensure template tiss larger than 50kb
nburtsev Mar 23, 2024
16f3529
Merge branch 'main' into fix/cdk-import-template-path
nburtsev Mar 23, 2024
b798c93
Merge branch 'main' into fix/cdk-import-template-path
nburtsev Mar 26, 2024
3f3f730
Merge branch 'main' into fix/cdk-import-template-path
nburtsev Apr 6, 2024
de6d0da
Merge branch 'main' into fix/cdk-import-template-path
nburtsev Apr 15, 2024
de09c72
Merge branch 'main' into fix/cdk-import-template-path
nburtsev Apr 15, 2024
72af4a2
Merge branch 'main' into fix/cdk-import-template-path
nburtsev Apr 16, 2024
8578e9f
Merge branch 'main' of github.com:aws/aws-cdk into fix/cdk-import-tem…
bergjaak May 8, 2024
84a9409
Merge branch 'main' into fix/cdk-import-template-path
bergjaak May 8, 2024
75a5dcb
in progress
bergjaak May 8, 2024
c520676
made test actually work
bergjaak May 9, 2024
ca4607e
Merge branch 'main' of github.com:aws/aws-cdk into fix/cdk-import-tem…
bergjaak May 9, 2024
69cbad7
Merge branch 'main' of github.com:aws/aws-cdk into fix/cdk-import-tem…
bergjaak May 9, 2024
06a62be
Merge branch 'main' of github.com:aws/aws-cdk into fix/cdk-import-tem…
bergjaak May 9, 2024
a6f2c82
made test pass
bergjaak May 9, 2024
d6f4308
test
colifran May 9, 2024
7157944
remove test comment
colifran May 9, 2024
02fd882
Merge branch 'fix/cdk-import-template-path' of https://github.com/nbu…
bergjaak May 9, 2024
0261c0a
Merge branch 'main' of github.com:aws/aws-cdk into fix/cdk-import-tem…
bergjaak May 10, 2024
bd0eb88
fix failing tests
bergjaak May 10, 2024
fa109b3
use cfn fixture
bergjaak May 10, 2024
75918ac
fix test again
bergjaak May 10, 2024
7e3b24e
remove duplicate line
bergjaak May 10, 2024
2cb9614
improve test comments
bergjaak May 10, 2024
f5cc882
Merge branch 'main' into fix/cdk-import-template-path
mergify[bot] May 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 4 additions & 3 deletions packages/@aws-cdk-testing/cli-integ/package.json
Expand Up @@ -30,14 +30,15 @@
"license": "Apache-2.0",
"devDependencies": {
"@aws-cdk/cdk-build-tools": "0.0.0",
"@types/semver": "^7.5.8",
"@types/yargs": "^15.0.19",
"@aws-cdk/pkglint": "0.0.0",
"@types/fs-extra": "^9.0.13",
"@types/glob": "^7.2.0",
"@types/npm": "^7.19.3",
"@aws-cdk/pkglint": "0.0.0"
"@types/semver": "^7.5.8",
"@types/yargs": "^15.0.19"
},
"dependencies": {
"@aws-sdk/client-cloudformation": "^3.572.0",
"@octokit/rest": "^18.12.0",
"aws-sdk": "^2.1610.0",
"axios": "^1.6.8",
Expand Down
Expand Up @@ -232,10 +232,37 @@ class MigrateStack extends cdk.Stack {
}
}

class ImportableStack extends MigrateStack {
class ImportableStack extends cdk.Stack {
constructor(parent, id, props) {
super(parent, id, props);
new cdk.CfnWaitConditionHandle(this, 'Handle');

if (process.env.INCLUDE_SINGLE_QUEUE === '1') {
const queue = new sqs.Queue(this, 'Queue', {
removalPolicy: (process.env.RETAIN_SINGLE_QUEUE === '1') ? cdk.RemovalPolicy.RETAIN : cdk.RemovalPolicy.DESTROY,
});

new cdk.CfnOutput(this, 'QueueName', {
value: queue.queueName,
});

new cdk.CfnOutput(this, 'QueueUrl', {
value: queue.queueUrl,
});

new cdk.CfnOutput(this, 'QueueLogicalId', {
value: queue.node.defaultChild.logicalId,
});
}

if (process.env.LARGE_TEMPLATE === '1') {
for (let i = 1; i <= 70; i++) {
new sqs.Queue(this, `cdk-import-queue-test${i}`, {
enforceSSL: true,
removalPolicy: cdk.RemovalPolicy.DESTROY,
});
}
}
}
}

Expand Down
@@ -1,6 +1,7 @@
import { promises as fs, existsSync } from 'fs';
import * as os from 'os';
import * as path from 'path';
import * as CFN from '@aws-sdk/client-cloudformation';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that fixture has its own CFN client, so I should use that instead of this, probably.

import { integTest, cloneDirectory, shell, withDefaultFixture, retry, sleep, randomInteger, withSamIntegrationFixture, RESOURCES_DIR, withCDKMigrateFixture, withExtendedTimeoutFixture } from '../../lib';

jest.setTimeout(2 * 60 * 60_000); // Includes the time to acquire locks, worst-case single-threaded runtime
Expand Down Expand Up @@ -1575,37 +1576,60 @@ integTest('skips notice refresh', withDefaultFixture(async (fixture) => {
}));

/**
* Create a queue with a fresh name, redeploy orphaning the queue, then import it again
* Create a queue, orphan that queue, then import the queue.
*
* We want to test with a large template to make sure large templates can work with import.
*/
integTest('test resource import', withDefaultFixture(async (fixture) => {
// GIVEN
const outputsFile = path.join(fixture.integTestDir, 'outputs', 'outputs.json');
await fs.mkdir(path.dirname(outputsFile), { recursive: true });

// Initial deploy
await fixture.cdkDeploy('importable-stack', {
modEnv: { ORPHAN_TOPIC: '1' },
modEnv: { LARGE_TEMPLATE: '1', INCLUDE_SINGLE_QUEUE: '1', RETAIN_SINGLE_QUEUE: '1' },
options: ['--outputs-file', outputsFile],
});

const outputs = JSON.parse((await fs.readFile(outputsFile, { encoding: 'utf-8' })).toString());
const queueName = outputs.QueueName;
const queueLogicalId = outputs.QueueLogicalId;
fixture.log(`Setup complete, created queue ${queueName}`);
fixture.log('Setup complete');

const cfnClient = new CFN.CloudFormationClient();
let queues: { [queueLogicalId: string]: { QueueUrl: string } } = {};
try {
// Deploy again, orphaning the queue
// Write a resource mapping file based on the ID from step one, then run an import
const mappingFile = path.join(fixture.integTestDir, 'outputs', 'mapping.json');
const fullStackName = fixture.fullStackName('importable-stack');
const queueUrl = outputs[fullStackName].QueueUrl;
const queueLogicalId = outputs[fullStackName].QueueLogicalId;
queues[queueLogicalId] = { QueueUrl: queueUrl };

// Remove the queue from the stack but don't delete the queue from AWS
await fixture.cdkDeploy('importable-stack', {
modEnv: { OMIT_TOPIC: '1' },
modEnv: { LARGE_TEMPLATE: '1', INCLUDE_SINGLE_QUEUE: '0', RETAIN_SINGLE_QUEUE: '0' },
});
const cfnTemplateBeforeImport = await cfnClient.send(new CFN.GetTemplateCommand({ StackName: fullStackName }));
expect(cfnTemplateBeforeImport.TemplateBody).not.toContain(queueLogicalId);

// Write a resource mapping file based on the ID from step one, then run an import
const mappingFile = path.join(fixture.integTestDir, 'outputs', 'mapping.json');
await fs.writeFile(mappingFile, JSON.stringify({ [queueLogicalId]: { QueueName: queueName } }), { encoding: 'utf-8' });
await fs.writeFile(
mappingFile,
JSON.stringify(queues),
{ encoding: 'utf-8' },
);

// WHEN
await fixture.cdk(
['import', '-m', mappingFile, fixture.fullStackName('importable-stack')],
{ modEnv: { LARGE_TEMPLATE: '1', INCLUDE_SINGLE_QUEUE: '1', RETAIN_SINGLE_QUEUE: '0' } },
);

await fixture.cdk(['import',
'--resource-mapping', mappingFile,
fixture.fullStackName('importable-stack')]);
// THEN
const describeStacksResponse = await cfnClient.send(new CFN.DescribeStacksCommand({ StackName: fullStackName }));
const cfnTemplateAfterImport = await cfnClient.send(new CFN.GetTemplateCommand({ StackName: fullStackName }));
expect(cfnTemplateAfterImport.TemplateBody).toContain(queueLogicalId);
expect(describeStacksResponse.Stacks![0].StackStatus).toEqual('IMPORT_COMPLETE');
expect(cfnTemplateAfterImport.TemplateBody).toContain(queueLogicalId);
} finally {
// Cleanup
// Clean up
await fixture.cdkDestroy('importable-stack');
}
}));
Expand Down
4 changes: 3 additions & 1 deletion packages/aws-cdk/lib/api/util/template-body-parameter.ts
@@ -1,3 +1,4 @@
import * as path from 'path';
import * as cxapi from '@aws-cdk/cx-api';
import * as chalk from 'chalk';
import * as fs from 'fs-extra';
Expand Down Expand Up @@ -68,7 +69,8 @@ export async function makeBodyParameter(
if (overrideTemplate) {
// Add a variant of this template
templateFile = `${stack.templateFile}-${templateHash}.yaml`;
await fs.writeFile(templateFile, templateJson, { encoding: 'utf-8' });
const templateFilePath = path.join(stack.assembly.directory, templateFile);
await fs.writeFile(templateFilePath, templateJson, { encoding: 'utf-8' });
}

assetManifest.addFileAsset(templateHash, {
Expand Down