Support Terraform Imports

[dkistner]

What this PR does / why we need it:
Add support for Terraform imports to the Terraformer.
There are certain cases, primarily migration scenarios, which require to import existing resources into the Terraform state. See here for an example.

The imports can be defined in the *.infra.tf-config configmap via .data.imports.
This need to be look like this:

  imports: |
   resource_type.name1 abc
   resource_type.name2 xyz

The resources will be imported in the order as they are specified. Only if all imports succeed the state will be persisted. After that the *.infra.tf-config configmap will be patched to remove .data.imports as the imports are now part of the state.

Special notes for your reviewer:
Here is an example to test the import behaviour based on the mentioned issues above. You need the azurerm provider with version >= 2.12.0. You can use this image to test: dominickistner/terraformer:test-import-1 (test image contains a bash for debugging)

1. Apply this configmap with the Terraformer configuration (don't forget to replace the subscription id in the imports field):

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: some-name.infra.tf-config
  namespace: default
data:
  main.tf: |
    provider "azurerm" {
      subscription_id = var.subscription_id
      tenant_id       = var.tenant_id
      client_id       = var.client_id
      client_secret   = var.client_secret
      version         = "2.12.0"
      features {}
    }

    resource "azurerm_resource_group" "rg" {
      name     = "test-nat"
      location = "westeurope"
    }

    resource "azurerm_public_ip" "natip1" {
      name                = "nat-ip1"
      location            = "westeurope"
      resource_group_name = azurerm_resource_group.rg.name
      allocation_method   = "Static"
      sku                 = "Standard"
    }

    resource "azurerm_public_ip" "natip2" {
      name                = "nat-ip2"
      location            = "westeurope"
      resource_group_name = azurerm_resource_group.rg.name
      allocation_method   = "Static"
      sku                 = "Standard"
    }

    resource "azurerm_nat_gateway" "nat" {
      name                    = "nat-gateway"
      location                = "westeurope"
      resource_group_name     = azurerm_resource_group.rg.name
      sku_name                = "Standard"
      public_ip_address_ids   = [azurerm_public_ip.natip1.id, azurerm_public_ip.natip2.id] # Remove me later. Step 4.
    }

    # Enable those resources later. Step 4.
    # resource "azurerm_nat_gateway_public_ip_association" "nat-ip-association-1" {
    #   nat_gateway_id       = azurerm_nat_gateway.nat.id
    #   public_ip_address_id = azurerm_public_ip.natip1.id
    # }

    # resource "azurerm_nat_gateway_public_ip_association" "nat-ip-association-2" {
    #   nat_gateway_id       = azurerm_nat_gateway.nat.id
    #   public_ip_address_id = azurerm_public_ip.natip2.id
    # }

  imports: |
   azurerm_nat_gateway_public_ip_association.nat-ip-association-1 /subscriptions/<subscription-id>/resourceGroups/test-nat/providers/Microsoft.Network/natGateways/nat-gateway|/subscriptions/<subscription-id>/resourceGroups/test-nat/providers/Microsoft.Network/publicIPAddresses/nat-ip1
   azurerm_nat_gateway_public_ip_association.nat-ip-association-2 /subscriptions/<subscription-id>/resourceGroups/test-nat/providers/Microsoft.Network/natGateways/nat-gateway|/subscriptions/<subscription-id>/resourceGroups/test-nat/providers/Microsoft.Network/publicIPAddresses/nat-ip2

  variables.tf: |
    variable "subscription_id" {
      type = string
    }
    variable "tenant_id" {
      type = string
    }
    variable "client_id" {
      type = string
    }
    variable "client_secret" {
      type = string
    }

---
apiVersion: v1
kind: Secret
metadata:
  name: some-name.infra.tf-vars
  namespace: default
type: Opaque
data:
  terraform.tfvars: |
    b64(
      subscription_id = "my-subscription-id"
      tenant_id = "my-tenant-id"
      client_id = "my-spn-id"
      client_secret = "my-spn-secret"
    )

2. Add an empty state configmap:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: some-name.infra.tf-state
  namespace: default
data:
  terraform.tfstate: ""

3. Run a terraform apply:

kubectl apply -f example/30-terraformer-apply-pod.yaml

4. Modify the configuration configmap with the following changes and apply the modified configmap.
   You need to remove the public_ip_address_ids field from the azurerm_nat_gateway resource and enable the two azurerm_nat_gateway_public_ip_association resources.

5. Try to apply again. It should fail due to import error like that:

Error: A resource with the ID "/subscriptions/<subscription-id>/resourceGroups/test-nat/providers/Microsoft.Network/natGateways/nat-gateway|/subscriptions/<subscription-id>/resourceGroups/test-nat/providers/Microsoft.Network/publicIPAddresses/nat-ip1" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_nat_gateway_public_ip_association" for more information.

6. Now run the import job:

kubectl apply -f example/50-terraformer-import-pod.yaml

7. The azurerm_nat_gateway_public_ip_associationresources should be now part of the state. Run import once more. There should be no diff.

8. Apply once more. It should now succeed.

9. Finally destroy the created resources again.

kubectl apply -f example/40-terraformer-destroy-pod.yaml

Release note:

Terraformer support now also Terraform imports.

[ialidzhikov]

/assign @ialidzhikov

[vlerenc]

Any plans to go forward with this one @ialidzhikov or @dkistner ?

[ialidzhikov]

Any plans to go forward with this one @ialidzhikov or @dkistner ?

My proposal is gardener/gardener-extension-provider-azure#54 (comment).

[CLAassistant]

All committers have signed the CLA.

[rfranzke]

/rotten
/close