Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The aws route public_internet_gateway of the route table getting deleted then recreated #1070

Open
1 task done
SadmiB opened this issue Apr 22, 2024 · 3 comments
Open
1 task done

Comments

@SadmiB
Copy link

SadmiB commented Apr 22, 2024

Description

We upgraded our VPC module from version 3.14 to 5.5.2, when we did the terraform plan and apply there were default resources that were created including the default security group, the vpc network acl, the default route table, and that's related to the breaking changes introduced since version 4.0.0. the following are the resources that asked to be created:

 # module.vpc.aws_default_network_acl.this[0] will be created
  + resource "aws_default_network_acl" "this" {
      + arn                    = (known after apply)
      + default_network_acl_id = "acl-fa3eee90"
      + id                     = (known after apply)
      + owner_id               = (known after apply)
      + tags                   = {
          + "Name"    = "main-production-default"
          + "project" = "project"
          + "stack"   = "vpc-infrastructure"
          + "stage"   = "production"
          + "type"    = "infra"
        }
      + tags_all               = {
          + "Name"    = "main-production-default"
          + "project" = "project"
          + "stack"   = "vpc-infrastructure"
          + "stage"   = "production"
          + "type"    = "infra"
        }
      + vpc_id                 = (known after apply)

      + egress {
          + action          = "allow"
          + from_port       = 0
          + ipv6_cidr_block = "::/0"
          + protocol        = "-1"
          + rule_no         = 101
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }
      + egress {
          + action          = "allow"
          + cidr_block      = "0.0.0.0/0"
          + from_port       = 0
          + protocol        = "-1"
          + rule_no         = 100
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }

      + ingress {
          + action          = "allow"
          + from_port       = 0
          + ipv6_cidr_block = "::/0"
          + protocol        = "-1"
          + rule_no         = 101
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }
      + ingress {
          + action          = "allow"
          + cidr_block      = "0.0.0.0/0"
          + from_port       = 0
          + protocol        = "-1"
          + rule_no         = 100
          + to_port         = 0
            # (1 unchanged attribute hidden)
        }
    }

  # module.vpc.aws_default_route_table.default[0] will be created
  + resource "aws_default_route_table" "default" {
      + arn                    = (known after apply)
      + default_route_table_id = "rtb-3b338a51"
      + id                     = (known after apply)
      + owner_id               = (known after apply)
      + route                  = (known after apply)
      + tags                   = {
          + "Name"    = "main-production-default"
          + "project" = "project"
          + "stack"   = "vpc-infrastructure"
          + "stage"   = "production"
          + "type"    = "infra"
        }
      + tags_all               = {
          + "Name"    = "main-production-default"
          + "project" = "peterpark"
          + "stack"   = "vpc-infrastructure"
          + "stage"   = "production"
          + "type"    = "infra"
        }
      + vpc_id                 = (known after apply)

      + timeouts {
          + create = "5m"
          + update = "5m"
        }
    }

  # module.vpc.aws_default_security_group.this[0] will be created
  + resource "aws_default_security_group" "this" {
      + arn                    = (known after apply)
      + description            = (known after apply)
      + egress                 = (known after apply)
      + id                     = (known after apply)
      + ingress                = (known after apply)
      + name                   = (known after apply)
      + name_prefix            = (known after apply)
      + owner_id               = (known after apply)
      + revoke_rules_on_delete = false
      + tags                   = {
          + "Name"    = "main-production-default"
          + "project" = "project"
          + "stack"   = "vpc-infrastructure"
          + "stage"   = "production"
          + "type"    = "infra"
        }
      + tags_all               = {
          + "Name"    = "main-production-default"
          + "project" = "project"
          + "stack"   = "vpc-infrastructure"
          + "stage"   = "production"
          + "type"    = "infra"
        }
      + vpc_id                 = "vpc-7a808711"
    }

After we applied this we had connectivity issues and when planning the module again we got this:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:

  # module.vpc.aws_route.public_internet_gateway[0] has been deleted
  - resource "aws_route" "public_internet_gateway" {
      - id                          = "r-rtb-3b338a511080289494" -> null
        # (18 unchanged attributes hidden)
    }

and terraform asking to create the public_internet_gateway knowing that we didn't delete it from the aws console!

  • ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

  1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
  2. Re-initialize the project root to pull down modules: terraform init
  3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

  • Module version [Required]: 5.5.2

  • Terraform version: v1.8.0

  • Provider version(s):

provider registry.terraform.io/hashicorp/aws: v4.1.0

Reproduction Code [Required]

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.5.2"

  name = "main-${var.stage}"
  cidr = var.vpc_cidr

  azs             = var.azs
  private_subnets = var.private_subnet_cidrs
  public_subnets  = var.public_subnet_cidrs

  map_public_ip_on_launch = true

  enable_nat_gateway = true
  enable_vpn_gateway = false

  enable_dns_hostnames = true
  enable_dns_support   = true

  single_nat_gateway = var.stage == "staging"

  tags = local.tags

  # k8s tags
  private_subnet_tags = merge(var.additional_private_subnet_tags, {
    "kubernetes.io/role/internal-elb"                 = "1"
    "kubernetes.io/cluster/cluster-${var.stage}"      = "shared"
    "kubernetes.io/cluster/cluster-mesh-${var.stage}" = "shared"
  })

  public_subnet_tags = merge(var.additional_public_subnet_tags, {
    "kubernetes.io/role/elb"                          = "1"
    "kubernetes.io/cluster/cluster-${var.stage}"      = "shared"
    "kubernetes.io/cluster/cluster-mesh-${var.stage}" = "shared"
  })
}

module "external_outputs" {
  source = "./external_outputs"

  tags  = local.tags
  stage = var.stage

  vpc_id          = module.vpc.vpc_id
  private_subnets = module.vpc.private_subnets
  public_subnets  = module.vpc.public_subnets
}

Steps to reproduce the behavior:

Expected behavior

The aws route public_internet_gateway is not getting deleted.

Actual behavior

The public_internet_gateway got deleted and recreated in second apply.

Terminal Output Screenshot(s)

Added in the description above.

Additional context

Copy link

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

@github-actions github-actions bot added the stale label May 23, 2024
@richardgavel-ordinaryexperts

Is this an issue with the module, or the way the aws_default_route_table Terraform resource works?

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/default_route_table

When Terraform first adopts a default route table, it immediately removes all defined routes. It then proceeds to create any routes specified in the configuration.

@SadmiB
Copy link
Author

SadmiB commented May 23, 2024

In this case we lost route public_internet_gateway entirely in first apply, until it got recreated after reapplying again without any changes!

@github-actions github-actions bot removed the stale label May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants