Skip to content

Improved management of context caching #19797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 tasks
gillisandrew opened this issue Apr 7, 2022 · 3 comments
Open
2 tasks

Improved management of context caching #19797

gillisandrew opened this issue Apr 7, 2022 · 3 comments
Labels
@aws-cdk/core Related to core CDK functionality @aws-cdk/custom-resources Related to AWS CDK Custom Resources effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2

Comments

@gillisandrew
Copy link
Contributor

Describe the feature

Users need better tooling for managing context caching behaviors and for maintaining cdk.context.json cached values.

Use Case

Users regularly open issues regarding stale values being returned by ContextProvider-enabled features (e.g. Parameter.valueFromLookup). As AWS itself recommends commiting cdk.context.json to source control to ensure reproducible builds, additional tooling should support maintaining it as well as control caching behaviors of lookups in general.

Proposed Solution

  1. Accept a validity duration as part of the lookup, after which the value will be refreshed by the context provider.
export declare class StringParameter extends ParameterBase implements IStringParameter {
  static valueFromLookup(scope: Construct, parameterName: string, validity?: Duration): string;
}
  1. Accept filtering options for cdk context --reset <key or number> cli command. Allow wildcard key matching and environment-based filtering so unused values can be removed without impacting determinism of the synthesis.

If plugin-registered context-providers do eventually leave alpha, consider calling a staleness check with the cached values on context providers to support more complex cache-busting behaviors.

export interface ContextProviderPlugin {
    shouldRefresh(oldValue: any): Promise<boolean>;
    getValue(args: {
        [key: string]: any;
    }): Promise<any>;
}

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.19.0

Environment details (OS name and version, etc.)

Windows 11 (WSL2 Ubuntu 20.04)

@gillisandrew gillisandrew added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Apr 7, 2022
@github-actions github-actions bot added the @aws-cdk/custom-resources Related to AWS CDK Custom Resources label Apr 7, 2022
@rix0rrr
Copy link
Contributor

rix0rrr commented Apr 7, 2022

Interesting ideas, will keep them in mind, thanks.

In the mean time, the cdk context command was intended to be the primary method for interacting with those. I'm not opposed to changes that will make that command more powerful.

@rix0rrr rix0rrr added effort/medium Medium work item – several days of effort p2 and removed needs-triage This issue or PR still needs to be triaged. labels Apr 7, 2022
@rix0rrr rix0rrr removed their assignment Apr 7, 2022
mergify bot pushed a commit that referenced this issue Apr 11, 2022

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Implementation of one of the solutions I proposed in #19797 using glob-style expressions to match keys. Uses minimatch which already existed as a dependency to match stack names in the synth and deploy commands. 

Makes the --reset command throw on no-ops  i.e when trying to reset context defined in cdk.json or ~/.cdk.json

Adds tests and prints messages to clarify previously undocumented behavior.

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
otaviomacedo pushed a commit that referenced this issue Apr 11, 2022
Implementation of one of the solutions I proposed in #19797 using glob-style expressions to match keys. Uses minimatch which already existed as a dependency to match stack names in the synth and deploy commands. 

Makes the --reset command throw on no-ops  i.e when trying to reset context defined in cdk.json or ~/.cdk.json

Adds tests and prints messages to clarify previously undocumented behavior.

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
StevePotter pushed a commit to StevePotter/aws-cdk that referenced this issue Apr 27, 2022
Implementation of one of the solutions I proposed in aws#19797 using glob-style expressions to match keys. Uses minimatch which already existed as a dependency to match stack names in the synth and deploy commands. 

Makes the --reset command throw on no-ops  i.e when trying to reset context defined in cdk.json or ~/.cdk.json

Adds tests and prints messages to clarify previously undocumented behavior.

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link
Contributor

github-actions bot commented Apr 7, 2023

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

@github-actions github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label Apr 7, 2023
@RichiCoder1
Copy link
Contributor

Bump.

@github-actions github-actions bot removed the closing-soon This issue will automatically close in 4 days unless further comments are made. label Apr 7, 2023
@khushail khushail added the @aws-cdk/core Related to core CDK functionality label Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/core Related to core CDK functionality @aws-cdk/custom-resources Related to AWS CDK Custom Resources effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2
Projects
None yet
Development

No branches or pull requests

4 participants