Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"The graph node for ... has no configuration attached to it" while destroying #31850

Closed
marcincuber opened this issue Sep 23, 2022 · 9 comments · Fixed by #31990
Closed

"The graph node for ... has no configuration attached to it" while destroying #31850

marcincuber opened this issue Sep 23, 2022 · 9 comments · Fixed by #31990
Assignees
Labels
bug v1.3 Issues (primarily bugs) reported against v1.3 releases

Comments

@marcincuber
Copy link
Contributor

marcincuber commented Sep 23, 2022

Terraform Version

1.3.0

Terraform Configuration Files

resource "aws_ssoadmin_managed_policy_attachment" "org_admin_service_catalog" {
  instance_arn       = tolist(data.aws_ssoadmin_instances.master.arns)[0]
  managed_policy_arn = "arn:aws:iam::aws:policy/AWSServiceCatalogAdminFullAccess"
  permission_set_arn = aws_ssoadmin_permission_set.org_admin.arn
}

Debug Output

╷
│ Error: error waiting for SSO Permission Set (arn:aws:sso:::permissionSet/ssoins-753534c2fad[62](https://github.com/global/tf-aws-master-setup/actions/runs/3105903088/jobs/5032315406#step:11:63)e05/ps-92ba2c6d0e956dcb) to provision: unexpected state 'FAILED', wanted target 'SUCCEEDED'. last error: %!s(<nil>)

Later when I remove the resource I get:

Error: Resource node has no configuration attached
│ 
│ The graph node for
│ aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog has no
│ configuration attached to it. This suggests a bug in Terraform's apply
│ graph builder; please report it!

Expected Behavior

It should create the resource

Actual Behavior

failing to apply

Steps to Reproduce

terraform init
terraform apply

Additional Context

In case you think this is aws provider issue, please let me know.

References

No response

@marcincuber marcincuber added bug new new issue not yet triaged labels Sep 23, 2022
@jbardin
Copy link
Member

jbardin commented Sep 23, 2022

Hi @marcincuber,

Thanks for filing the issue. We have seen this once before a very long time ago, so I don't know if it's specifically v1.3 related, but it would be great to finally solve how this happened! I'm not sure how you arrived here, but the fact that you saw an error from the resource is a good clue.

Is this reproducible, and is it possible to share the complete trace log output?

Thanks!

@jbardin jbardin added waiting-response An issue/pull request is waiting for a response from the community waiting for reproduction unable to reproduce issue without further information v1.3 Issues (primarily bugs) reported against v1.3 releases and removed new new issue not yet triaged labels Sep 23, 2022
@marcincuber
Copy link
Contributor Author

Hi @jbardin. I managed to reproduce the error on ubuntu. Please see the trace below, it is actually stating that IAM Role is missing required permission. However, Terraform should definitely handle such issue better.

2022-09-23T13:23:19.487Z [DEBUG] provider.terraform-provider-aws_v4.31.0_x5: [DEBUG] [aws-sdk-go] {"PermissionSetProvisioningStatus":{"CreatedDate":1.663939394208E9,"FailureReason":"Received a 403 status error: Access denied by IAM. Please check your policy, or wait for role propagation to complete. IAM Error: User: arn:aws:sts::***:assumed-role/github-actions-deploy-role/GitHubActions is not authorized to perform: iam:AttachRolePolicy on resource: role AWSReservedSSO_AWSOrgAdmin_f84c064fb37df788 because no identity-based policy allows the iam:AttachRolePolicy action (Service: AmazonIdentityManagement; Status Code: 403; Error Code: AccessDenied; Request ID: b0946534-7985-4009-935d-27324ea4a524; Proxy: null)","PermissionSetArn":"arn:aws:sso:::permissionSet/ssoins-753534c2fad62e05/ps-92ba2c6d0e956dcb","RequestId":"93bc510d-2855-4404-af10-3a7c56a6df63","Status":"FAILED"}}
2022-09-23T13:23:19.487Z [TRACE] provider.terraform-provider-aws_v4.31.0_x5: Called downstream: tf_mux_provider=*schema.GRPCProviderServer tf_resource_type=aws_ssoadmin_managed_policy_attachment tf_req_id=4c894934-0610-4ce7-6033-bf9bcabd5b72 tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.22.0/helper/schema/resource.go:838 @module=sdk.helper_schema tf_provider_addr=registry.terraform.io/hashicorp/aws timestamp=2022-09-23T13:23:19.487Z
2022-09-23T13:23:19.487Z [TRACE] maybeTainted: aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog encountered an error during creation, so it is now marked as tainted
2022-09-23T13:23:19.487Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog
2022-09-23T13:23:19.487Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: writing state object for aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog
2022-09-23T13:23:19.487Z [TRACE] evalApplyProvisioners: aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog is tainted, so skipping provisioning
2022-09-23T13:23:19.487Z [TRACE] maybeTainted: aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog was already tainted, so nothing to do
2022-09-23T13:23:19.487Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog
2022-09-23T13:23:19.487Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: writing state object for aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog
Error: -23T13:23:19.487Z [ERROR] vertex "aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog" error: error waiting for SSO Permission Set (arn:aws:sso:::permissionSet/ssoins-753534c2fad62e05/ps-92ba2c6d0e956dcb) to provision: unexpected state 'FAILED', wanted target 'SUCCEEDED'. last error: %!s(<nil>)
2022-09-23T13:23:19.487Z [TRACE] vertex "aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog": visit complete, with errors
2022-09-23T13:23:19.487Z [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" errored, so skipping
2022-09-23T13:23:19.488Z [TRACE] dag/walk: upstream of "root" errored, so skipping
2022-09-23T13:23:19.488Z [TRACE] provider.terraform-provider-aws_v4.31.0_x5: Received downstream response: diagnostic_error_count=1 tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_duration_ms=5964 tf_req_id=4c894934-0610-4ce7-6033-bf9bcabd5b72 tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:37 @module=sdk.proto diagnostic_warning_count=0 tf_resource_type=aws_ssoadmin_managed_policy_attachment timestamp=2022-09-23T13:23:19.487Z
Error: -23T13:23:19.488Z [ERROR] provider.terraform-provider-aws_v4.31.0_x5: Response contains error diagnostic: tf_proto_version=5.3 tf_req_id=4c894934-0610-4ce7-6033-bf9bcabd5b72 tf_resource_type=aws_ssoadmin_managed_policy_attachment tf_rpc=ApplyResourceChange diagnostic_detail= diagnostic_summary="error waiting for SSO Permission Set (arn:aws:sso:::permissionSet/ssoins-753534c2fad62e05/ps-92ba2c6d0e956dcb) to provision: unexpected state 'FAILED', wanted target 'SUCCEEDED'. last error: %!s(<nil>)" @module=sdk.proto diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/hashicorp/aws @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/diag/diagnostics.go:55 timestamp=2022-09-23T13:23:19.487Z
2022-09-23T13:23:19.488Z [TRACE] provider.terraform-provider-aws_v4.31.0_x5: Served request: tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=4c894934-0610-4ce7-6033-bf9bcabd5b72 @module=sdk.proto tf_resource_type=aws_ssoadmin_managed_policy_attachment tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/tf5server/server.go:831 timestamp=2022-09-23T13:23:19.487Z
╷
│ Error: error waiting for SSO Permission Set (arn:aws:sso:::permissionSet/ssoins-753534c2fad62e05/ps-92ba2c6d0e956dcb) to provision: unexpected state 'FAILED', wanted target 'SUCCEEDED'. last error: %!s(<nil>)
│ 
│   with aws_ssoadmin_managed_policy_attachment.org_admin_service_catalog,
│   on iam-sso-permission-sets.tf line 18, in resource "aws_ssoadmin_managed_policy_attachment" "org_admin_service_catalog":
│   18: resource "aws_ssoadmin_managed_policy_attachment" "org_admin_service_catalog" {
│ 

@marcincuber
Copy link
Contributor Author

Note that fixing IAM permission issue also allowed terraform to apply successfully.

@apparentlymart
Copy link
Member

Hi @marcincuber! Thanks for the additional context.

The apply-time error you shared here does seem to be a provider error, because Terraform Core itself doesn't know anything about AWS IAM.

The destroy-time error is curious, though. As the error message mentions, this is not something we expect should happen so it seems like you bumped into a Terraform Core bug there which may or may not be related to the provider error you saw first.

Do you also have a trace log for that destroy error? It would be helpful to see what exactly Terraform Core was doing when it encountered that error.

@marcincuber
Copy link
Contributor Author

Hi @apparentlymart
I am unable to replicated the error from terraform destroy action.

@petercharleston
Copy link

Further issues for: he graph node has no configuration attached

I had further issues with this today:

**m│
**m│ The graph node for azurerm_public_ip.VNGWPublicIP has no configuration
**m│ attached to it. This suggests a bug in Terraform's apply graph
builder;
**m│ please report it!
**m╵
**m╷
**m│ mmError: **mResource node has no configuration attached
**m│
**m│ The graph node for azurerm_backup_protected_vm.AzureVMBackup[0] has no
**m│ configuration attached to it. This suggests a bug in Terraform's apply
**m│ graph builder; please report it!

On Fri, Sep 30, 2022 at 5:34 PM Peter Charleston @.***>

@apparentlymart apparentlymart changed the title Resource fails to apply and fails to destroy "The graph node for ... has no configuration attached to it" while destroying Oct 5, 2022
@weeniearms
Copy link

weeniearms commented Oct 11, 2022

We were able to repro this issue with the following manifest and steps:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
    }
  }
}

resource "aws_sqs_queue" "test" {
  name = "test-queue"
}
  1. Run terraform apply with the manifest above
  2. Remove the created queue outside of terraform (e.g. through the AWS web console, API, or CLI)
  3. Remove the queue from the manifest
  4. Run terraform apply

The result of this is exit code equal to 1 (which seriously impacts any script that relies on this to succeed) and the following error message:

╷
│ Error: Resource node has no configuration attached
│ 
│ The graph node for aws_sqs_queue.test has no configuration attached to it.
│ This suggests a bug in Terraform's apply graph builder; please report it!
╵

See tf.log for TRACE logs.

The error is slightly different if the same scenario is run with custom providers published to a private registry.
Assuming that I have a provider defined as:

terraform {
  required_providers {
    myprovider = {
      source = "myregistry.com/mygroup/myprovider"
    }
  }
}

the TRACE is as follows (note that for some reason it's looking for registry.terraform.io/hashicorp/myprovider instead of myregistry.com/mygroup/myprovider):

2022-10-11T08:28:58.856+0200 [INFO]  backend/local: apply calling Apply
2022-10-11T08:28:58.856+0200 [DEBUG] Building and walking apply graph for NormalMode plan
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.ConfigTransformer
2022-10-11T08:28:58.856+0200 [TRACE] ConfigTransformer: Starting for path:
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.ConfigTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.RootVariableTransformer
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.RootVariableTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.ModuleVariableTransformer
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.ModuleVariableTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.LocalTransformer
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.LocalTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.OutputTransformer
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.OutputTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.DiffTransformer
2022-10-11T08:28:58.856+0200 [TRACE] DiffTransformer starting
2022-10-11T08:28:58.856+0200 [TRACE] DiffTransformer: found NoOp change for myprovider_res5.test
2022-10-11T08:28:58.856+0200 [TRACE] DiffTransformer: myprovider_res5.test will be represented by myprovider_res5.test
2022-10-11T08:28:58.856+0200 [TRACE] DiffTransformer complete
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.DiffTransformer with new graph:
  myprovider_res5.test - *terraform.NodeApplyableResourceInstance
  ------
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.AttachStateTransformer
2022-10-11T08:28:58.856+0200 [DEBUG] Resource state not found for node "myprovider_res5.test", instance myprovider_res5.test
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.AttachStateTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.OrphanOutputTransformer
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.OrphanOutputTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.AttachResourceConfigTransformer
2022-10-11T08:28:58.856+0200 [TRACE] Completed graph transform *terraform.AttachResourceConfigTransformer (no changes)
2022-10-11T08:28:58.856+0200 [TRACE] Executing graph transform *terraform.graphTransformerMulti
2022-10-11T08:28:58.856+0200 [TRACE] (graphTransformerMulti) Executing graph transform *terraform.ProviderConfigTransformer
2022-10-11T08:28:58.856+0200 [TRACE] (graphTransformerMulti) Completed graph transform *terraform.ProviderConfigTransformer with new graph:
  myprovider_res5.test - *terraform.NodeApplyableResourceInstance
  provider["myregistry.com/mygroup/myprovider"] - *terraform.NodeApplyableProvider
  ------
2022-10-11T08:28:58.856+0200 [TRACE] (graphTransformerMulti) Executing graph transform *terraform.MissingProviderTransformer
2022-10-11T08:28:58.857+0200 [DEBUG] adding implicit provider configuration provider["registry.terraform.io/hashicorp/myprovider"], implied first by myprovider_res5.test
2022-10-11T08:28:58.857+0200 [TRACE] (graphTransformerMulti) Completed graph transform *terraform.MissingProviderTransformer with new graph:
  myprovider_res5.test - *terraform.NodeApplyableResourceInstance
  provider["registry.terraform.io/hashicorp/myprovider"] - *terraform.NodeApplyableProvider
  provider["myregistry.com/mygroup/myprovider"] - *terraform.NodeApplyableProvider
  ------
2022-10-11T08:28:58.857+0200 [TRACE] (graphTransformerMulti) Executing graph transform *terraform.ProviderTransformer
2022-10-11T08:28:58.857+0200 [TRACE] ProviderTransformer: exact match for provider["registry.terraform.io/hashicorp/myprovider"] serving myprovider_res5.test
2022-10-11T08:28:58.857+0200 [DEBUG] ProviderTransformer: "myprovider_res5.test" (*terraform.NodeApplyableResourceInstance) needs provider["registry.terraform.io/hashicorp/myprovider"]
2022-10-11T08:28:58.857+0200 [TRACE] (graphTransformerMulti) Completed graph transform *terraform.ProviderTransformer with new graph:
  myprovider_res5.test - *terraform.NodeApplyableResourceInstance
    provider["registry.terraform.io/hashicorp/myprovider"] - *terraform.NodeApplyableProvider
  provider["registry.terraform.io/hashicorp/myprovider"] - *terraform.NodeApplyableProvider
  provider["myregistry.com/mygroup/myprovider"] - *terraform.NodeApplyableProvider
  ------
2022-10-11T08:28:58.857+0200 [TRACE] (graphTransformerMulti) Executing graph transform *terraform.PruneProviderTransformer
2022-10-11T08:28:58.857+0200 [DEBUG] pruning unused provider["myregistry.com/mygroup/myprovider"]
2022-10-11T08:28:58.857+0200 [TRACE] (graphTransformerMulti) Completed graph transform *terraform.PruneProviderTransformer with new graph:
  myprovider_res5.test - *terraform.NodeApplyableResourceInstance
    provider["registry.terraform.io/hashicorp/myprovider"] - *terraform.NodeApplyableProvider
  provider["registry.terraform.io/hashicorp/myprovider"] - *terraform.NodeApplyableProvider
  ------
2022-10-11T08:28:58.857+0200 [TRACE] Completed graph transform *terraform.graphTransformerMulti with new graph:
  myprovider_res5.test - *terraform.NodeApplyableResourceInstance
    provider["registry.terraform.io/hashicorp/myprovider"] - *terraform.NodeApplyableProvider
  provider["registry.terraform.io/hashicorp/myprovider"] - *terraform.NodeApplyableProvider
  ------
2022-10-11T08:28:58.857+0200 [TRACE] Executing graph transform *terraform.RemovedModuleTransformer
2022-10-11T08:28:58.857+0200 [TRACE] Completed graph transform *terraform.RemovedModuleTransformer (no changes)
2022-10-11T08:28:58.857+0200 [TRACE] Executing graph transform *terraform.AttachSchemaTransformer
2022-10-11T08:28:58.857+0200 [TRACE] terraform.contextPlugins: Initializing provider "registry.terraform.io/hashicorp/myprovider" to read its schema
2022-10-11T08:28:58.857+0200 [TRACE] Completed graph transform *terraform.AttachSchemaTransformer (no changes)
2022-10-11T08:28:58.857+0200 [ERROR] backend/local: apply returned nil state
�[31m╷�[0m�[0m
�[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mfailed to read schema for myprovider_res5.test in registry.terraform.io/hashicorp/myprovider: failed to instantiate provider "registry.terraform.io/hashicorp/myprovider" to obtain schema: unavailable provider "registry.terraform.io/hashicorp/myprovider"�[0m
�[31m│�[0m �[0m
�[31m│�[0m �[0m�[0m
�[31m╵�[0m�[0m
2022-10-11T08:28:58.857+0200 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
2022-10-11T08:28:58.857+0200 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock

@jbardin jbardin self-assigned this Oct 11, 2022
@jbardin jbardin removed waiting-response An issue/pull request is waiting for a response from the community waiting for reproduction unable to reproduce issue without further information labels Oct 11, 2022
@jbardin
Copy link
Member

jbardin commented Oct 11, 2022

Thanks @weeniearms, that gives us all the info we need!

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 12, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug v1.3 Issues (primarily bugs) reported against v1.3 releases
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants