Failing test(s): TestAccCloudfunctions2functionIam*

[SarahFrench]

Impacted tests

• TestAccCloudfunctions2functionIamBindingGenerated
• TestAccCloudfunctions2functionIamMemberGenerated
• TestAccCloudfunctions2functionIamPolicyGenerated

Affected Resource(s)

• google_XXXXX

Failure rates

• 100% when run with Terraform versions >1.2.5

Message(s)

=== RUN   TestAccCloudfunctions2functionIamBindingGenerated
=== PAUSE TestAccCloudfunctions2functionIamBindingGenerated
=== CONT  TestAccCloudfunctions2functionIamBindingGenerated
    testing_new_import_state.go:125: unexpected output type: <nil>
--- FAIL: TestAccCloudfunctions2functionIamBindingGenerated (83.87s)
FAIL

This failure happens during the import steps of the test. For TestAccCloudfunctions2functionIamBindingGenerated it fails on the second step, the first import test after the binding resource is created.

Nightly build test history

• TestAccCloudfunctions2functionIamBindingGenerated
• TestAccCloudfunctions2functionIamMemberGenerated
• TestAccCloudfunctions2functionIamPolicyGenerated

[SarahFrench]

Context:

After releasing provider-defined functions in the provider we bumped up TeamCity to use the release candidate 1.8.0-rc1, in anticipation of the GA 1.8.0 release a week after then.

After updating TeamCity to use 1.8.0-rc1 we identified several tests failing unexpectedly with a 'non-empty plan' error and a plan that didn't indicate what field was triggering the diff. This bug we identified helped the Terraform core team address the bug and include the fix in the 1.8.0 release.

We then updated TeamCity to use 1.8.0, as the bug we identified was addressed. However after this we spotted some test failures continued.

In the case of the testing_new_import_state.go:125: unexpected output type: <nil> failures described in this ticket, the cause isn't 1.8.0. The error is observed if you run the test with Terraform version 1.4.0, 1.5.0, 1.6.0, 1.7.0, and 1.8.0. If you run the tests with 1.3.0 the binding and member tests pass but the policy resource test fails with a different error:

------- Stdout: -------
=== RUN   TestAccCloudfunctions2functionIamPolicyGenerated
=== PAUSE TestAccCloudfunctions2functionIamPolicyGenerated
=== CONT  TestAccCloudfunctions2functionIamPolicyGenerated
    vcr_utils.go:152: Step 2/4 error running import: exit status 1
        Error writing state file: Failed to serialize resource instance in state: Instance data.google_cloudfunctions2_function_iam_policy.foo has status ObjectPlanned, which cannot be saved in state.
--- FAIL: TestAccCloudfunctions2functionIamPolicyGenerated (80.06s)
FAIL

[SarahFrench]

Note: you can start custom builds in TeamCity to download different versions of Terraform. When the modal appears to launch your custom build look for the TERRAFORM_CORE_VERSION environment variable and edit its value.

When running tests locally on your laptop you can use the TF_ACC_TERRAFORM_PATH environment variable to make the tests use different versions of Terraform present on your machine. You can use this in combination with cloning the hashicorp/terraform repo, checking out a version tag, building the binary locally, and then repeating for all versions you want to use to run the test.

[SarahFrench]

I've found a way to make these tests avoid the issue - I'm going to open a PR. Assigning this issue to me