[Epic] Code generation quality #7804
Labels
area/codegen
SDK-gen, program-gen, convert
kind/engineering
Work that is not visible to an external user
kind/epic
Large new features or investments
resolution/fixed
This issue was fixed
Milestone
Code generation quality spans (currently) four languages, SDK generation, and examples generation. We have substantial gaps in coverage that have led to some painful regressions in recent history. We have opportunities to make targeted testing infrastructure improvements to make it safer to make changes in this area, and to discover bugs sooner.
This becomes more important as we think about community and multi-language authors consuming this surface area, not just core pulumi team members.
While there are some advanced techniques we could use such as generative property testing, we have some low hanging fruit we should tackle first:
Adding an output compilation step in our tests will bring more confidence when making changes, and prevent future regressions. A campaign to bring more examples under the suite will simultaneously expand coverage and improve the quality of our documentation.
If we make it this far, we may want to consider closing the loop by running a preview on some of the generated programs. However, there is more complexity there (acquiring a provider binary, cloud credentials, etc) which is not well suited to the current pulumi/pulumi test infrastructure.
The text was updated successfully, but these errors were encountered: