Performance degradation of AttachResourceConfigTransformer on big graphs

[alexott]

Terraform Version

1.7.0

Use Cases

We need to evaluate big graphs, and this part of the code has N^2 complexity, leading to significant performance degradation during plan/apply

Attempted Solutions

Code changes are made and PR will be submitted

Proposal

No response

References

No response

[crw]

Thanks for this feature request! If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!

[arnaudlh]

hi folks we are seeing dramatic performance decrease since 1.8.x and 1.9.x, on a typical aztfmod/caf run for a plan, we got:

• 1.7.5: 2:07.17 total plan time
• 1.8.0: 2:05:10 total plan time
• 1.8.2/1.8.1: 15:37.49 total plan time
• 1.9.0-alpha20240501: 16:51.03 total plan time

[alexott]

@arnaudlh Out of curiosity, how many objects are in your TF plan and state? For me, performance degradation was in 1.4.5 and 1.5.7 (I didn't test the other versions), so I made this fix.

We have other places where we have N^2 operations:

• DestroyEdgeTransformer performs iteration over all vertices and then for each of them iterates over the changes

Potential places with similar performance profile:

• AttachStateTransformer
• ReferenceTransformer

[crw]

@arnaudlh If the 1.9 alpha didn't fix the issue, then you should open a new issue -- it will be unrelated to this one.