fix: tf plan truncated on PR comments and plan_validate_all error logs in github actions #1129

nbugden · 2024-03-01T20:14:05Z

Context:
The terraform plan generated in GitHub Actions is often truncated when the comment is posted to a PR. This is because the comment is posting the stdout for all environments not just the plan generated by terraform. This PR logs the terraform plan as a separate text file for each environment and then uses those files to create 1 comment per environment on the PR from GitHub Actions.

This also removes the need for #1113 as stdout from the plan_validate_all stage is no longer written to a file.

Issue(s) Resolved:
#1109
#1130

google-cla · 2024-03-01T20:14:09Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

nbugden · 2024-04-17T18:06:12Z

@rjerrems @eeaton @gtsorbo can one of you folks take a look. PR needs /gcbrun and an approving review.

nbugden · 2024-04-24T12:38:56Z

@rjerrems @eeaton @gtsorbo can one of you folks take a look. PR needs /gcbrun and an approving review.

Nudge, can I get a review. If there is something else required before review, please let me know.

fmichaelobrien · 2024-04-24T12:59:01Z

build/tf-wrapper.sh

@@ -139,7 +139,7 @@ convert_path() {
 ## Terraform apply for single environment.
 tf_apply() {
  local path=$1
-  local tf_env="${path#$base_dir/}"
+  local tf_env="${path#"$base_dir"/}"


These are good - usually the pr lint flags these - I have had to wrap all vars before in other repos

Thanks
Seems the linter has been Expected — Waiting for status to be reported for a number of weeks (possibly waiting for an approval to run). I validated locally using shellcheck. If this is likely to get flagged by the linter I can change it back as it is not functional change that is part of the PR.

For reference here is the issue flagged by shellcheck: https://www.shellcheck.net/wiki/SC2295

/gcbrun to get the linter going

apeabody · 2024-04-25T15:52:47Z

/gcbrun

(unsure if it "works" in a nested comment)

nbugden · 2024-04-26T01:09:12Z

@eeaton @apeabody thanks for triggering the checks!
terraform-example-foundation-int-trigger-HubAndSpoke (cloud-foundation-cicd) has failed, but I'm unable to see the logs. Can either of you help me with the error so I can address any changes needed in the PR?

apeabody · 2024-04-26T01:23:46Z

@eeaton @apeabody thanks for triggering the checks! terraform-example-foundation-int-trigger-HubAndSpoke (cloud-foundation-cicd) has failed, but I'm unable to see the logs. Can either of you help me with the error so I can address any changes needed in the PR?

Might be a flake:

Step #30 - "destroy-networks": === NAME  TestNetworks/production
Step #30 - "destroy-networks":     destroy.go:11: 
Step #30 - "destroy-networks":         	Error Trace:	/builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.11/modules/terraform/destroy.go:11
Step #30 - "destroy-networks":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:456
Step #30 - "destroy-networks":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:543
Step #30 - "destroy-networks":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:556
Step #30 - "destroy-networks":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/utils/stages.go:31
Step #30 - "destroy-networks":         	            				/builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:559
Step #30 - "destroy-networks":         	Error:      	Received unexpected error:
Step #30 - "destroy-networks":         	            	'terraform [destroy -auto-approve -input=false -var access_context_manager_policy_id=727108865530 -var remote_state_bucket=bkt-d5y-b-seed-tfstate-1323 -var ingress_policies=[{"from" = {"sources" = {"access_levels" = ["*"]}, "identity_type" = "ANY_IDENTITY"}, "to" = {"resources" = ["*"], "operations" = {"storage.googleapis.com" = {"methods" = ["google.storage.objects.get", "google.storage.objects.list"]}}}}] -var egress_policies=[{"from" = {"identity_type" = "ANY_IDENTITY"}, "to" = {"resources" = ["*"], "operations" = {"storage.googleapis.com" = {"methods" = ["google.storage.objects.get", "google.storage.objects.list"]}}}}] -var perimeter_additional_members=[] -no-color -lock=false]' unsuccessful after 1 retries
Step #30 - "destroy-networks":         	Test:       	TestNetworks/production

apeabody · 2024-04-26T01:24:16Z

/gcbrun

nbugden · 2024-04-26T13:06:33Z

Looks like this time both cloud build checks failed. Possibly another flake? I also had to pull in the latest changes. Can you help me with the error so I can address any changes needed in the PR?

apeabody · 2024-04-26T15:33:23Z

/gcbrun

nbugden · 2024-04-26T19:09:51Z

Checks have passed. Can a get a review and merge if this is ready to go?

nbugden · 2024-04-30T17:46:56Z

Pulled in the latest from master, can I get a review as well?

nbugden · 2024-05-03T15:08:30Z

/gcbrun

daniel-cit · 2024-05-03T19:01:38Z

@nbugden I will try a deploy on github for the review

…s in github actions

nbugden · 2024-05-03T20:58:06Z

FYI @daniel-cit I just fixed the named branches in the pipeline triggers. Still had non-production from a bad rebase.

nbugden · 2024-05-07T12:46:59Z

@daniel-cit any questions about the PR? Did you get a chance to test it?

daniel-cit · 2024-05-10T02:26:11Z

@nbugden tested

daniel-cit · 2024-05-10T02:27:10Z

/gcbrun

daniel-cit

LGTM

daniel-cit · 2024-05-10T03:39:16Z

/gcbrun

daniel-cit · 2024-05-10T18:51:23Z

/gcbrun

daniel-cit · 2024-05-10T22:33:37Z

/gcbrun

daniel-cit · 2024-05-10T23:56:06Z

/gcbrun

daniel-cit · 2024-05-11T03:08:37Z

/gcbrun

nbugden · 2024-05-13T12:46:02Z

@daniel-cit one of the integration test pipelines failed

daniel-cit · 2024-05-13T14:27:59Z

/gcbrun

daniel-cit · 2024-05-14T12:33:58Z

/gcbrun

daniel-cit · 2024-05-14T19:02:59Z

/gcbrun

daniel-cit · 2024-05-16T03:48:32Z

/gcbrun

daniel-cit · 2024-05-18T03:11:31Z

/gcbrun

daniel-cit · 2024-05-19T18:59:43Z

@apeabody the build is green :)

apeabody

already approved and no change to module code

nbugden requested review from rjerrems, gtsorbo and a team as code owners March 1, 2024 20:14

nbugden force-pushed the fix/gh-comments branch from 92ff0a4 to fb13d7a Compare March 26, 2024 15:09

nbugden requested a review from eeaton as a code owner April 3, 2024 20:22

nbugden force-pushed the fix/gh-comments branch 2 times, most recently from 38779ca to 60b9950 Compare April 15, 2024 13:43

fmichaelobrien reviewed Apr 24, 2024

View reviewed changes

nbugden force-pushed the fix/gh-comments branch from 60b9950 to 1163fee Compare April 25, 2024 14:43

nbugden force-pushed the fix/gh-comments branch from a6ffcbb to b236605 Compare April 26, 2024 13:03

nbugden force-pushed the fix/gh-comments branch 3 times, most recently from 525b3ce to b7db511 Compare May 2, 2024 12:54

nbugden added 2 commits May 3, 2024 16:55

fix: tf plan truncated on PR comments and plan_validate_all error log…

f101d50

…s in github actions

fix: non-prod to nonprod

e6e99c0

nbugden force-pushed the fix/gh-comments branch from 2526248 to e6e99c0 Compare May 3, 2024 20:57

Merge branch 'master' into fix/gh-comments

7c21729

nbugden added 3 commits May 7, 2024 08:47

Merge branch 'master' into fix/gh-comments

8b94426

Merge branch 'master' into fix/gh-comments

ef7d9e7

Merge branch 'master' into fix/gh-comments

60b17e9

daniel-cit approved these changes May 10, 2024

View reviewed changes

Merge branch 'master' into fix/gh-comments

936702b

Merge branch 'master' into fix/gh-comments

cf7bdee

Merge branch 'master' into fix/gh-comments

ae2ec47

Merge branch 'master' into fix/gh-comments

c357695

apeabody approved these changes May 20, 2024

View reviewed changes

apeabody merged commit aee08a4 into terraform-google-modules:master May 20, 2024
5 checks passed

nbugden deleted the fix/gh-comments branch May 20, 2024 20:57

eeaton mentioned this pull request May 29, 2024

terraform plan truncated in github actions comments #1130

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: tf plan truncated on PR comments and plan_validate_all error logs in github actions #1129

fix: tf plan truncated on PR comments and plan_validate_all error logs in github actions #1129

nbugden commented Mar 1, 2024 •

edited

google-cla bot commented Mar 1, 2024

nbugden commented Apr 17, 2024

nbugden commented Apr 24, 2024

fmichaelobrien Apr 24, 2024

nbugden Apr 24, 2024

eeaton Apr 25, 2024

apeabody commented Apr 25, 2024

nbugden commented Apr 26, 2024

apeabody commented Apr 26, 2024

apeabody commented Apr 26, 2024

nbugden commented Apr 26, 2024

apeabody commented Apr 26, 2024

nbugden commented Apr 26, 2024

nbugden commented Apr 30, 2024

nbugden commented May 3, 2024

daniel-cit commented May 3, 2024

nbugden commented May 3, 2024

nbugden commented May 7, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit left a comment

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 11, 2024

nbugden commented May 13, 2024

daniel-cit commented May 13, 2024

daniel-cit commented May 14, 2024

daniel-cit commented May 14, 2024

daniel-cit commented May 16, 2024

daniel-cit commented May 18, 2024

daniel-cit commented May 19, 2024

apeabody left a comment

fix: tf plan truncated on PR comments and plan_validate_all error logs in github actions #1129

fix: tf plan truncated on PR comments and plan_validate_all error logs in github actions #1129

Conversation

nbugden commented Mar 1, 2024 • edited

google-cla bot commented Mar 1, 2024

nbugden commented Apr 17, 2024

nbugden commented Apr 24, 2024

fmichaelobrien Apr 24, 2024

Choose a reason for hiding this comment

nbugden Apr 24, 2024

Choose a reason for hiding this comment

eeaton Apr 25, 2024

Choose a reason for hiding this comment

apeabody commented Apr 25, 2024

nbugden commented Apr 26, 2024

apeabody commented Apr 26, 2024

apeabody commented Apr 26, 2024

nbugden commented Apr 26, 2024

apeabody commented Apr 26, 2024

nbugden commented Apr 26, 2024

nbugden commented Apr 30, 2024

nbugden commented May 3, 2024

daniel-cit commented May 3, 2024

nbugden commented May 3, 2024

nbugden commented May 7, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit left a comment

Choose a reason for hiding this comment

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 10, 2024

daniel-cit commented May 11, 2024

nbugden commented May 13, 2024

daniel-cit commented May 13, 2024

daniel-cit commented May 14, 2024

daniel-cit commented May 14, 2024

daniel-cit commented May 16, 2024

daniel-cit commented May 18, 2024

daniel-cit commented May 19, 2024

apeabody left a comment

Choose a reason for hiding this comment

nbugden commented Mar 1, 2024 •

edited