When docker service need to be replaced but not destroy exists one

[elct9620]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and docker Provider) Version

Terraform v1.5.3
on darwin_amd64
+ provider registry.terraform.io/cloudflare/cloudflare v4.11.0
+ provider registry.terraform.io/hashicorp/random v3.5.1
+ provider registry.terraform.io/kreuzwerker/docker v3.0.2

Affected Resource(s)

• docker_service

Terraform Configuration Files

resource "docker_network" "gateway" {
  name   = "ingress_gateway"
  driver = "overlay"

  labels {
    label = "com.docker.stack.namespace"
    value = "ingress"
  }
}

variable "cloudflare_account_id" {
  type        = string
  description = "The CloudFlare Account ID"
  sensitive   = true
  nullable    = false

  validation {
    condition     = length(var.cloudflare_account_id) > 0
    error_message = "The CloudFlare Account ID must be provided"
  }
}

module "ingress" {
  source = "github.com/StarPortal/terraform-swarm-stack//modules/cloudflare-gateway"

  account_id  = var.cloudflare_account_id
  name        = "cloudflare"
  namespace   = "ingress"
  networks    = [docker_network.gateway.id]
  description = "example ingress"

  agent_version = "2023.7.1"
  constraints = [
    "node.role==manager"
  ]

  ingress = [
    {
      service = "http_status:404"
    }
  ]
}

Debug Output

From GitLab CI not prepared

Panic Output

Terraform will perform the following actions:
  # module.ingress.docker_service.this must be replaced
+/- resource "docker_service" "this" {
      ~ id   = "gpxvv3djmaqk3dhvnwpkhlpfm" -> (known after apply)
        name = "ingress_cloudflare_agent"
      - endpoint_spec {
          - mode = "vip" -> null
        }
      ~ mode {
            # (1 unchanged attribute hidden)
        }
      ~ task_spec {
          ~ force_update = 0 -> (known after apply)
          ~ runtime      = "container" -> (known after apply)
          ~ container_spec {
              - command           = [] -> null
              - groups            = [] -> null
              - read_only         = false -> null
              ~ stop_grace_period = "0s" -> (known after apply)
              - sysctl            = {} -> null
                # (4 unchanged attributes hidden)
              - dns_config {}
              - healthcheck {
                  - interval     = "0s" -> null
                  - retries      = 0 -> null
                  - start_period = "0s" -> null
                  - test         = [] -> null
                  - timeout      = "0s" -> null
                }
              + labels { # forces replacement
                  + label = "com.docker.stack.namespace"
                  + value = "ingress"
                }
            }
          ~ placement {
              - max_replicas = 0 -> null
              - prefs        = [] -> null
                # (1 unchanged attribute hidden)
            }
          ~ restart_policy {
              - max_attempts = 0 -> null
                # (3 unchanged attributes hidden)
            }
            # (2 unchanged blocks hidden)
        }
        # (2 unchanged blocks hidden)
    }
Plan: 1 to add, 0 to change, 1 to destroy.

Acquiring state lock. This may take a few moments...
module.ingress.docker_service.this: Creating...
╷
│ Error: Error response from daemon: rpc error: code = AlreadyExists desc = name conflicts with an existing object: service ingress_cloudflare_agent already exists
│ 
│   with module.ingress.docker_service.this,
│   on .terraform/modules/ingress/modules/cloudflare-gateway/main.tf line 57, in resource "docker_service" "this":
│   57: resource "docker_service" "this" {
│ 
╵

Expected Behaviour

The docker_service should be destroyed first and replaced with a new one.

Actual Behaviour

The docker_service is created directly that got name conflict.

Steps to Reproduce

1. Define service without labels
2. terraform apply
3. Add labels to the service
4. terraform apply

The network is changed triggering this error, too.

Important Factoids

Heavily depend on the customize module which generated a lot of default config.

Some error is triggered by changing the module to add a new optional config, e.g. label

References

N/A

[matthewbaggett]

I would be delighted to know if anyone has a workaround for this. In our CI/CD pipeline, when this occurs, just about the only thing I can do to handle it in an automated way is to destroy everything in the stack and recreate it which feels like an incredibly blunt instrument.

[cwoelk]

No workaround, but I believe that the reasoning of #152 was only valid for containers, but is not for services as they update labels in-place just fine - at least on the resource level in my case. Maybe using the

  lifecycle {
    create_before_destroy = false
  }

on the resource level works for you @matthewbaggett though this should anywayse be the default. 🤷