Resources not found after update 0.35.0

[mrMoe]

Hi there,

after the update of the last few days I tried to reuse my old plan but suddenly some resources are not found anymore.

Initializing the backend...

Initializing provider plugins...
- Finding latest version of paultyng/unifi...
- Using previously-installed paultyng/unifi v0.38.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
unifi_network.private_network: Refreshing state... [id=6260993a6c8f85000f9a968c]
unifi_network.wan: Refreshing state... [id=6260901c6c8f85000f9a95b5]
unifi_firewall_rule.accept_connection_to_iot_service: Refreshing state... [id=63029d7a21f7e8000d36eac1]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # unifi_firewall_rule.accept_connection_to_iot_service will be updated in-place
  ~ resource "unifi_firewall_rule" "accept_connection_to_iot_service" {
        id                     = "63029d7a21f7e8000d36eac1"
        name                   = "accept_connection_to_iot_service"
      - src_address            = "192.168.28.19" -> null
        # (17 unchanged attributes hidden)
    }

  # unifi_network.private_network will be updated in-place
  ~ resource "unifi_network" "private_network" {
        id                           = "6260993a6c8f85000f9a968c"
      - ipv6_pd_start                = "::2" -> null
      - ipv6_pd_stop                 = "::7d1" -> null
        name                         = "private.home"
        # (28 unchanged attributes hidden)
    }

  # unifi_network.wan will be updated in-place
  ~ resource "unifi_network" "wan" {
        id                           = "6260901c6c8f85000f9a95b5"
        name                         = "Default (WAN1)"
      - wan_type_v6                  = "disabled" -> null
        # (26 unchanged attributes hidden)
    }

Plan: 0 to add, 3 to change, 0 to destroy.
unifi_network.private_network: Modifying... [id=6260993a6c8f85000f9a968c]
unifi_network.wan: Modifying... [id=6260901c6c8f85000f9a95b5]
unifi_firewall_rule.accept_connection_to_iot_service: Modifying... [id=63029d7a21f7e8000d36eac1]
╷
│ Error: not found
│ 
│   with unifi_firewall_rule.accept_connection_to_iot_service,
│   on firewall.tf line 110, in resource "unifi_firewall_rule" "accept_connection_to_iot_service":
│  110: resource "unifi_firewall_rule" "accept_connection_to_iot_service" {
│ 
╵
╷
│ Error: not found
│ 
│   with unifi_network.wan,
│   on main.tf line 19, in resource "unifi_network" "wan":
│   19: resource "unifi_network" "wan" {
│ 
╵
╷
│ Error: not found
│ 
│   with unifi_network.private_network,
│   on main.tf line 39, in resource "unifi_network" "private_network":
│   39: resource "unifi_network" "private_network" {

I'm not sure, if I'm doing something wrong or if it is the updates but I already removed the resources from the state and reimportetd them again.
Same result.

Hope you can assist to solve that.

Best regards
Moritz Herrmann

[paultyng]

Hmm, this is odd. I updated my home network to the latest on the provider and besides an odd upgrade diff that worked itself out with the introduction of the IPv6 attributes, its remaining stable on subsequent plans and showing no diff. I did not get any not found errors either. I wonder if its because you are using more of the IPv6 stuff as I'm just on IPv4 on my network right now. I'm going to see if I can simulate this in the test controller.

[mrMoe]

I think I know what the problem is. If I do not set the wan_type_v6 to any value then terraform wants it to set it to null but this seems to be an invalid value. It has to be disabled.
Btw. I'm IPv4 only as well.

For the other two problems I completely deleted the resources und re-added them and that seems to have fixed them.

[pvicol]

I just ran into this issue. I am using version 0.41.0

The plan sees the drift:

terraform plan -target module.iot_network
module.iot_network.unifi_network.this: Refreshing state... [id=641939a7b17a3e10ccfd10c6]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.iot_network.unifi_network.this will be updated in-place
  ~ resource "unifi_network" "this" {
      ~ dhcp_v6_lease                = 86400 -> 0
        id                           = "641939a7b17a3e10ccfd10c6"
      ~ ipv6_ra_preferred_lifetime   = 14400 -> 0
      ~ ipv6_ra_valid_lifetime       = 86400 -> 0
        name                         = "IoT"
        # (25 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

When I try to apply, I get the not found error:

terraform apply -target module.iot_network
module.iot_network.unifi_network.this: Refreshing state... [id=641939a7b17a3e10ccfd10c6]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.iot_network.unifi_network.this will be updated in-place
  ~ resource "unifi_network" "this" {
      ~ dhcp_v6_lease                = 86400 -> 0
        id                           = "641939a7b17a3e10ccfd10c6"
      ~ ipv6_ra_preferred_lifetime   = 14400 -> 0
      ~ ipv6_ra_valid_lifetime       = 86400 -> 0
        name                         = "IoT"
        # (25 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
╷
│ Warning: Resource targeting is in effect
│ 
│ You are creating a plan with the -target option, which means that the result of this plan may not represent all of the changes requested by the current configuration.
│ 
│ The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use it as part
│ of an error message.
╵

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.iot_network.unifi_network.this: Modifying... [id=641939a7b17a3e10ccfd10c6]
╷
│ Warning: Applied changes may be incomplete
│ 
│ The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output values may not be fully updated. Run the following
│ command to verify that no other changes are pending:
│     terraform plan
│ 
│ Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests
│ to use it as part of an error message.
╵
╷
│ Error: not found
│ 
│   with module.iot_network.unifi_network.this,
│   on ../modules/network/main.tf line 9, in resource "unifi_network" "this":
│    9: resource "unifi_network" "this" {

TF Version:

terraform version
Terraform v1.4.2
on darwin_arm64
+ provider registry.terraform.io/paultyng/unifi v0.41.0

[joshuaspence]

Can you post debug logs?

[pvicol]

Logs posted here - https://gist.github.com/pvicol/26bddc684440573f9a475acf68f0380f

[joshuaspence]

So what's happening here is that the provider updates the network by hitting /proxy/network/api/s/default/rest/networkconf/641939a7b17a3e10ccfd10c6 and Unifi Network responds with the following response:

{"meta":{"rc":"ok"},"data":[]}

This is unexpected as it should respond with the properties of the network. It only ever responds with data: [] when an object is deleted. Will have to debug further

[pvicol]

Looking through the debug logs, the data payload sent by the provider does not include the fields noted in the changed plan.

Terraform says these will be changed

Terraform used the selected providers to generate the following execution plan. Resource
actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.management_network.unifi_network.this will be updated in-place
  ~ resource "unifi_network" "this" {
      ~ dhcp_v6_lease                = 86400 -> 0
        id                           = "61844081522ffc043a923bba"
      - ipv6_pd_interface            = "wan" -> null
      - ipv6_pd_start                = "::2" -> null
      - ipv6_pd_stop                 = "::7d1" -> null
      ~ ipv6_ra_preferred_lifetime   = 14400 -> 0
      - ipv6_ra_priority             = "high" -> null
        name                         = "Management"
        # (26 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

However, in the debug log, the payload is missing those fields

2023-04-04T20:51:18.852-0700 [DEBUG] provider.terraform-provider-unifi_v0.41.0: Sending HTTP Request: Host=ROUTER_ENDPOINT tf_http_req_method=PUT tf_http_req_version=HTTP/1.1 @module=provider Content-Length=1827 Content-Type="application/json; charset=utf-8" Cookie=TOKEN=COOKIE_TOKEN tf_http_trans_id=152386cc-ac36-d90a-50ca-05477fcd1ad8 tf_resource_type=unifi_network tf_rpc=ApplyResourceChange Accept-Encoding=gzip tf_http_req_body={"_id":"61844081522ffc043a923bba","site_id":"default","auto_scale_enabled":false,"dhcpd_boot_enabled":false,"dhcpd_boot_server":"","dhcpd_dns_1":"192.168.40.50","dhcpd_dns_2":"192.168.1.1","dhcpd_dns_3":"","dhcpd_dns_4":"","dhcpd_dns_enabled":true,"dhcpd_enabled":true,"dhcpd_gateway":"","dhcpd_gateway_enabled":false,"dhcpd_ip_1":"","dhcpd_ip_2":"","dhcpd_ip_3":"","dhcpd_leasetime":86400,"dhcpd_mac_1":"","dhcpd_mac_2":"","dhcpd_mac_3":"","dhcpd_ntp_1":"","dhcpd_ntp_2":"","dhcpd_ntp_enabled":false,"dhcpd_start":"192.168.1.4","dhcpd_stop":"192.168.1.250","dhcpd_time_offset_enabled":false,"dhcpd_unifi_controller":"","dhcpdv6_dns_auto":false,"dhcpdv6_enabled":false,"dhcpd_wins_1":"","dhcpd_wins_2":"","dhcpd_wins_enabled":false,"dhcp_relay_enabled":false,"dhcpguard_enabled":false,"dpi_enabled":false,"dpigroup_id":"","domain_name":"","enabled":true,"exposed_to_site_vpn":false,"gateway_device":"","igmp_fastleave":false,"igmp_querier":"","igmp_snooping":false,"igmp_supression":false,"ipsec_dynamic_routing":false,"ipsec_pfs":false,"ip_subnet":"192.168.1.1/24","ipv6_interface_type":"none","ipv6_pd_prefixid":"","ipv6_ra_enabled":false,"internet_access_enabled":true,"intra_network_access_enabled":true,"is_nat":false,"l2tp_allow_weak_ciphers":false,"lte_lan_enabled":false,"mac_override":"","mac_override_enabled":false,"mdns_enabled":false,"name":"Management","networkgroup":"LAN","pptpc_require_mppe":false,"purpose":"corporate","radiusprofile_id":"","remote_site_id":"","report_wan_event":false,"require_mschapv2":false,"upnp_lan_enabled":false,"usergroup_id":"","vlan_enabled":false,"vpn_client_default_route":false,"vpn_client_pull_dns":false,"wan_dns1":"","wan_dns2":"","wan_dns3":"","wan_dns4":"","wan_gateway_v6":"","wan_ipv6":"","wan_provider_capabilities":{},"wan_smartq_enabled":false,"wan_vlan_enabled":false} tf_provider_addr=provider @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.25.0/helper/logging/logging_http_transport.go:159 User-Agent=terraform-provider-unifi/0.1 X-Csrf-Token=CSRF_TOKEN tf_http_op_type=request tf_http_req_uri=/proxy/network/api/s/default/rest/networkconf/61844081522ffc043a923bba tf_req_id=def754e7-da9b-0df2-f8c3-ab67ba808330 timestamp=2023-04-04T20:51:18.851-0700

From the PUT request method above, the payload is missing the fields being changed.

Using Python requests, I discovered that if fields being changed are not added to the payload, the response is empty. After adding the appropriate fields, the response returned is the object/resource changed.

Notice after adding ipv6_ra_preferred_lifetime to the payload, it returned the proper response.

(Pdb) network_data = {"_id":"641939a7b17a3e10ccfd10c6","site_id":"default","auto_scale_enabled":False,"dhcpd_boot_enabled":False,"dhcpd_boot_server":"","dhcpd_dns_1":"192.168.40.50","dhcpd_dns_2":"192.168.60.1","dhcpd_dns_3":"192.168.1.1","dhcpd_dns_4":"","dhcpd_dns_enabled":True,"dhcpd_enabled":True,"dhcpd_gateway":"","dhcpd_gateway_enabled":False,"dhcpd_ip_1":"","dhcpd_ip_2":"","dhcpd_ip_3":"","dhcpd_leasetime":86400,"dhcpd_mac_1":"","dhcpd_mac_2":"","dhcpd_mac_3":"","dhcpd_ntp_1":"","dhcpd_ntp_2":"","dhcpd_ntp_enabled":False,"dhcpd_start":"192.168.60.4","dhcpd_stop":"192.168.60.250","dhcpd_time_offset_enabled":False,"dhcpd_unifi_controller":"","dhcpdv6_dns_auto":False,"dhcpdv6_enabled":False,"dhcpd_wins_1":"","dhcpd_wins_2":"","dhcpd_wins_enabled":False,"dhcp_relay_enabled":False,"dhcpguard_enabled":False,"dpi_enabled":False,"dpigroup_id":"","domain_name":"","enabled":True,"exposed_to_site_vpn":False,"gateway_device":"","igmp_fastleave":False,"igmp_querier":"","igmp_snooping":False,"igmp_supression":False,"ipsec_dynamic_routing":False,"ipsec_pfs":False,"ip_subnet":"192.168.60.1/24","ipv6_interface_type":"none","ipv6_pd_prefixid":"","ipv6_ra_enabled":False,"internet_access_enabled":True,"intra_network_access_enabled":True,"is_nat":False,"l2tp_allow_weak_ciphers":False,"lte_lan_enabled":False,"mac_override":"","mac_override_enabled":False,"mdns_enabled":True,"name":"IoT","networkgroup":"LAN","pptpc_require_mppe":False,"purpose":"corporate","radiusprofile_id":"","remote_site_id":"","report_wan_event":False,"require_mschapv2":False,"upnp_lan_enabled":False,"usergroup_id":"","vlan":60,"vlan_enabled":True,"vpn_client_default_route":False,"vpn_client_pull_dns":False,"wan_dns1":"","wan_dns2":"","wan_dns3":"","wan_dns4":"","wan_gateway_v6":"","wan_ipv6":"","wan_provider_capabilities":{},"wan_smartq_enabled":False,"wan_vlan_enabled":False, 'dhcp_v6_lease': 0}
(Pdb) resp = sesh.put(network_url, headers = put_headers, json = network_data)
(Pdb) resp.status_code
200
(Pdb) resp.text
'{"meta":{"rc":"ok"},"data":[]}'
(Pdb) network_data = {"_id":"641939a7b17a3e10ccfd10c6","site_id":"default","auto_scale_enabled":False,"dhcpd_boot_enabled":False,"dhcpd_boot_server":"","dhcpd_dns_1":"192.168.40.50","dhcpd_dns_2":"192.168.60.1","dhcpd_dns_3":"192.168.1.1","dhcpd_dns_4":"","dhcpd_dns_enabled":True,"dhcpd_enabled":True,"dhcpd_gateway":"","dhcpd_gateway_enabled":False,"dhcpd_ip_1":"","dhcpd_ip_2":"","dhcpd_ip_3":"","dhcpd_leasetime":86400,"dhcpd_mac_1":"","dhcpd_mac_2":"","dhcpd_mac_3":"","dhcpd_ntp_1":"","dhcpd_ntp_2":"","dhcpd_ntp_enabled":False,"dhcpd_start":"192.168.60.4","dhcpd_stop":"192.168.60.250","dhcpd_time_offset_enabled":False,"dhcpd_unifi_controller":"","dhcpdv6_dns_auto":False,"dhcpdv6_enabled":False,"dhcpd_wins_1":"","dhcpd_wins_2":"","dhcpd_wins_enabled":False,"dhcp_relay_enabled":False,"dhcpguard_enabled":False,"dpi_enabled":False,"dpigroup_id":"","domain_name":"","enabled":True,"exposed_to_site_vpn":False,"gateway_device":"","igmp_fastleave":False,"igmp_querier":"","igmp_snooping":False,"igmp_supression":False,"ipsec_dynamic_routing":False,"ipsec_pfs":False,"ip_subnet":"192.168.60.1/24","ipv6_interface_type":"none","ipv6_pd_prefixid":"","ipv6_ra_enabled":False,"internet_access_enabled":True,"intra_network_access_enabled":True,"is_nat":False,"l2tp_allow_weak_ciphers":False,"lte_lan_enabled":False,"mac_override":"","mac_override_enabled":False,"mdns_enabled":True,"name":"IoT","networkgroup":"LAN","pptpc_require_mppe":False,"purpose":"corporate","radiusprofile_id":"","remote_site_id":"","report_wan_event":False,"require_mschapv2":False,"upnp_lan_enabled":False,"usergroup_id":"","vlan":60,"vlan_enabled":True,"vpn_client_default_route":False,"vpn_client_pull_dns":False,"wan_dns1":"","wan_dns2":"","wan_dns3":"","wan_dns4":"","wan_gateway_v6":"","wan_ipv6":"","wan_provider_capabilities":{},"wan_smartq_enabled":False,"wan_vlan_enabled":False, 'dhcp_v6_lease': 0, 'ipv6_ra_preferred_lifetime': 0}
(Pdb) resp = sesh.put(network_url, headers = put_headers, json = network_data)
(Pdb) resp.status_code
200
(Pdb) resp.text
'{"meta":{"rc":"ok"},"data":[{"setting_preference":"manual","dhcpdv6_dns_auto":false,"dhcpd_ntp_1":"","vpn_client_default_route":false,"dhcpd_ntp_2":"","dhcpd_gateway_enabled":false,"dhcpd_dns_1":"192.168.40.50","vpn_client_pull_dns":false,"dhcpd_start":"192.168.60.4","dhcpd_unifi_controller":"","ipv6_ra_enabled":false,"dpigroup_id":"","domain_name":"","mac_override":"","ip_subnet":"192.168.60.1/24","dhcpd_dns_4":"","require_mschapv2":false,"ipv6_interface_type":"none","dhcpd_dns_2":"192.168.60.1","dhcpd_dns_3":"192.168.1.1","dhcpd_ip_1":"","dhcpd_ip_2":"","pptpc_require_mppe":false,"dhcpd_ip_3":"","dhcpd_wins_2":"","dhcpd_boot_server":"","wan_dns4":"","is_nat":false,"wan_dns3":"","dhcpd_wins_1":"","wan_dns2":"","dhcpd_dns_enabled":true,"dhcpdv6_enabled":false,"wan_dns1":"","internet_access_enabled":true,"usergroup_id":"","nat_outbound_ip_addresses":[],"dhcp_relay_enabled":false,"radiusprofile_id":"","wan_provider_capabilities":{},"dhcpd_wins_enabled":false,"ipsec_pfs":false,"ipv6_pd_prefixid":"","intra_network_access_enabled":true,"name":"IoT","site_id":"6184407c522ffc043a923ba7","dhcpdv6_leasetime":86400,"_id":"641939a7b17a3e10ccfd10c6","lte_lan_enabled":false,"l2tp_allow_weak_ciphers":false,"dhcpd_leasetime":86400,"purpose":"corporate","igmp_snooping":false,"dhcpd_time_offset_enabled":false,"dhcpguard_enabled":false,"dhcpd_mac_3":"","igmp_supression":false,"dhcpd_mac_2":"","dhcpd_mac_1":"","ipv6_ra_preferred_lifetime":0,"dhcpd_stop":"192.168.60.250","enabled":true,"dhcpd_enabled":true,"vlan":60,"dhcpd_wpad_url":"","igmp_querier":"","networkgroup":"LAN","exposed_to_site_vpn":false,"ipv6_ra_valid_lifetime":86400,"vlan_enabled":true,"remote_site_id":"","mac_override_enabled":false,"igmp_fastleave":false,"gateway_type":"default","dhcpd_gateway":"","ipsec_dynamic_routing":false,"wan_ipv6":"","dhcpd_boot_enabled":false,"wan_smartq_enabled":false,"upnp_lan_enabled":false,"dhcpd_ntp_enabled":false,"wan_vlan_enabled":false,"dpi_enabled":false,"report_wan_event":false,"mdns_enabled":true,"dhcpd_tftp_server":"","auto_scale_enabled":false,"wan_gateway_v6":""}]}'

[john-terrell]

New user here. I just set up a new terraform config using this provider (v.0.41.0) Any interaction with the API returns "Not found" as above.

[pvicol]

@paultyng - any insights into this issue?

[MaienM]

There is a related issue in the go-unifi library. I fixed an instance of this issue for the Device resource a while back, and in that case the issue was that the request omitted a field, just like has been observed by @pvicol above for this case. The field you mention (ipv6_ra_preferred_lifetime) has the omitEmpty flag, which is why it is absent from the request.

If anyone wants to do the work to get this fixed, here's the things I had to do last time to do so (with links to the relevant PRs):

• PR Create a test case in this repository that reproduces this issue.
• PR Alter the code generator in the go-unifi library so that this field does not get the omitEmpty flag.