[BUG] btp_subaccount_service_instance: "AttributeName("context"): unsupported type json.Delim sent as tftypes.String" after upgrading rc1 -> rc2

[SeanKilleen]

Is there an existing issue for this?

• I have searched the existing issues

What version of the Terraform provider are you using?

1.0.0-rc2

What version of the Terraform CLI are you using?

1.7.0

What type of issue are you facing

bug report

Describe the bug

I've got a service instance that has been in place for a while long time:

resource "btp_subaccount_service_instance" "service_manager_instance_kyma" {
  subaccount_id  = btp_subaccount.products_kyma.id
  serviceplan_id = data.btp_subaccount_service_plan.service_manager_service_plan_kyma.id
  name           = "products-service-manager"
}

I upgraded from 1.0.0-rc1 to 1.0.0-rc2 of the provider.

After doing so, when applying the TF, I see:

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Unable to Read Previously Saved State for UpgradeResourceState
│
│   with module.product_environment.module.product_environment_btp.btp_subaccount_service_instance.service_manager_instance_kyma,
│   on modules\product_environment_btp\main.tf line 316, in resource "btp_subaccount_service_instance" "service_manager_instance_kyma":
│  316: resource "btp_subaccount_service_instance" "service_manager_instance_kyma" {
│
│ There was an error reading the saved resource state using the current resource schema.
│
│ If this resource state was last refreshed with Terraform CLI 0.11 and earlier, it must be refreshed or applied with an older provider version first. If you manually modified the resource state, you will need to manually modify it to     
│ match the current resource schema. Otherwise, please report this to the provider developer:
│
│ AttributeName("context"): unsupported type json.Delim sent as tftypes.String

This was done in a branch where the only change was to upgrade the provider.

Downgrading back to 1.0.0-rc1 removes the issue.

Expected Behavior

Since my Terraform has not changed and my environment has not changed, I'd expect to see that there are no changes.

Steps To Reproduce

No response

User's Role Collections

No response

Add screenshots to help explain your problem

No response

Additional context

No response

[lechnerc77]

@SeanKilleen : The issue you are facing is due to the breaking change we had to introduce in the RC2 as some backend API made incompatible schema changes of the response object. This is mentioned in the release notes and the linked PR #609

As we are not GA yet we do not provide migration options or implement the state-upgrade function. After GA situations like this will be avoided or if we are forced to do it, handled on our end.

In order to overcome this issue for now, you can give the -refresh-only parameter for plan and apply a try, However, I am afraid that this won't do the trick. If this doesn't work it depends on your setup and constraints:

• You can taint the resource and force a recreate
• You can remove the resource from the state and try an import. However there are some restrictions wrt to service instances with parameters due to configurations of the platform
• You can manually adjust the state, which would be the most dangerous approach.

[SeanKilleen]

@lechnerc77 ahhh, sorry for missing that in the release notes; I forgot that I was using that resource and I skimmed way too quickly.

As you predicted, -refresh-only did not do the trick.

Is this something that will break from me from this point onward? Or something that will be resolved when things go GA? Trying to understand my path forward.

[SeanKilleen]

@lechnerc77 FWIW when I attempt to taint the resource, I see an error:

Error writing state file: string is required

Is that related to this as well? I'd expect to be able to taint something still.

[lechnerc77]

@SeanKilleen concerning the way forward: there are no breaking changes to expect from RC2 to GA release, at least as of today we do not have any breaking change in our backlog. Of course, I cannot exclude the possibility if a bug is that leads to such a change, that this might happen, but chances of another breaking change are low.

From GA onwards we will take care that breaking changes will be avoided or properly mitigated so that there is no disruption on your end.

With respect to the taint error - yes that is the same origin, the change of the data type for the context seems to cause this.

[SeanKilleen]

@lechnerc77 My question was: there's not going to be some upgrade resource added to solve this in GA from where I am now, right?

I'm trying to understand my options. It sounds like no matter what my only option is to delete this resource to recreate it, since taint also isn't working. And regardless of any future releases, this will be a step I need to take. Correct?

[lechnerc77]

I understand. So from my perspective you have two options:

Option 1 is as you write deleting the resource and recreating it. However, there might be side effects depending on your overall configuration. This is something I cannot make any comment on.

Option 2: You try to manually correct the state file be changing the content of the context entry to an empty string and re-trying the -refresh-only approach. Manually manipulating the state is of course always risky

[SeanKilleen]

Thanks! I'll take a backup of my state and try the manual update tomorrow. I'll let you know the results.

[SeanKilleen]

@lechnerc77 This worked! Took the following steps in case anyone else is wondering.

• Checked out the branch with RC2 in it
• Selected the applicable Terraform workspace (terraform workspace select)
• Ran terraform init -upgrade
• Ran terraform plan -refresh-only
• Ran terraform state pull >> dev.tfstate to get the JSON file locally
• Opened the JSON, found the btp_subaccount_service_instance's attributes.context entry
• Edited context to turn it into an empty string
• Incremented the "serial" value so I could push it.
• Saved the local JSON file
• Attempted to push with terraform state push dev.tfstate

At this point it actually failed with an error Failed to persist state: string is required -- but that makes sense, because there was another resource this was affected by and I needed to update the context for it as well.

Once that was complete, terraform apply worked as expected.

Just need to perform that step for each environment and I'll be good to go. Thanks for the guidance!

[lechnerc77]

@SeanKilleen Great to hear that it worked. Sorry for the inconveniences that you had. As mentioned once we go GA hiccups like this will be avoided from our end as far as possible.

[SeanKilleen]

@lechnerc77 no problem! Being part of the road to production is bound to have these bumps. Your great guidance has made all the difference.

[tcasteli]

@lechnerc77 This worked! Took the following steps in case anyone else is wondering.

• Checked out the branch with RC2 in it
• Selected the applicable Terraform workspace (terraform workspace select)
• Ran terraform init -upgrade
• Ran terraform plan -refresh-only
• Ran terraform state pull >> dev.tfstate to get the JSON file locally
• Opened the JSON, found the btp_subaccount_service_instance's attributes.context entry
• Edited context to turn it into an empty string
• Incremented the "serial" value so I could push it.
• Saved the local JSON file
• Attempted to push with terraform state push dev.tfstate

At this point it actually failed with an error Failed to persist state: string is required -- but that makes sense, because there was another resource this was affected by and I needed to update the context for it as well.

Once that was complete, terraform apply worked as expected.

Just need to perform that step for each environment and I'll be good to go. Thanks for the guidance!

This worked like a charm! Thanks for sharing.