Template should be created from VM, not directly

[tomaszkiewicz]

Describe the bug
The provider creates a template by sending the request to API to create vm with "template" option set to true.
That approach doesn't work when we deal with qcow2 based images and ZFS datastore for VMs because it skips some Proxmox conversion logic that enables linked clone from template.

To Reproduce

1. Create any template using the provider, e.g.:

resource "proxmox_virtual_environment_download_file" "flatcar3760_qcow2_img" {
  content_type = "iso"
  datastore_id = local.cloud_images_datastore_id
  node_name    = local.main_node
  file_name    = "flatcar-3760-test.qcow2.img"
  url          = "https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_qemu_image.img"
}

resource "proxmox_virtual_environment_vm" "flatcar3760_vm_template" {
  name        = "flatcar-3760"

  template = true

  started  = false

  node_name = local.main_node
  vm_id     = 801

  disk {
    datastore_id = local.main_datastore_id // IMPORTANT: ZFS based storage!
    file_id      = proxmox_virtual_environment_download_file.flatcar3760_qcow2_img.id
    interface    = "virtio0"
  }

  network_device {
    bridge = "vmbr0"
  }

  serial_device {}
}

2. Create a VM from the template using Linked Clone mode (e.g. manually from web console)
3. Get error: "Linked clone feature is not supported for drive 'virtio0' (500)" (virtio0 has nothing to do with the issue, I checked all possible drivers like scsi and ide)
4. Change template = true to template = false in above code and apply TF, it'll replace the template with VM
5. Go do web console and convert the new VM to a template or issue qm template 801 from CLI
6. From that template create a new VM, again in Linked Clone mode
7. This time it succeeds

Expected behavior
Provider should create a VM and then issue "convert to template" operation on Proxmox side as some additional logic (I'm not sure what exactly) happens during that process.

Additional context

• Single or clustered Proxmox: N/A
• Provider version (ideally it should be the latest version): 0.45.0
• Terraform version: 1.3.5, but N/A
• OS (where you run Terraform from): Fedora on WSL2

[bpg]

@tomaszkiewicz could you please share the configuration of the virtio0 hard disk the provider created after step 2? I.e.

[tomaszkiewicz]

After creating the template (step 1):

Step 2/3 fails:

Then after step 4 (recreation as VM) we have a VM with the following:

After manual conversion to template (step 5):

Then new VM created from template:

Maybe that link will help understanding what's going on when converting to template:

https://forum.proxmox.com/threads/qm-template-what-does-it-_do_-what-makes-a-template-different-from-a-k-vm.77455/#post-344027

It seems that when converting, this ZFS plugin method is called:

https://git.proxmox.com/?p=pve-storage.git;a=blob;f=PVE/Storage/ZFSPlugin.pm;h=63b95517adad6a544bd15ba1d06328e1fcf9119a;hb=refs/heads/master

 249 sub create_base {
 250     my ($class, $storeid, $scfg, $volname) = @_;
 251 
 252     my $snap = '__base__';
 253 
 254     my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) =
 255         $class->parse_volname($volname);
 256 
 257     die "create_base not possible with base image\n" if $isBase;
 258 
 259     my $newname = $name;
 260     $newname =~ s/^vm-/base-/;
 261 
 262     my $newvolname = $basename ? "$basename/$newname" : "$newname";
 263 
 264     $class->zfs_delete_lu($scfg, $name);
 265     $class->zfs_request($scfg, undef, 'rename', "$scfg->{pool}/$name", "$scfg->{pool}/$newname");
 266 
 267     my $guid = $class->zfs_create_lu($scfg, $newname);
 268     $class->zfs_add_lun_mapping_entry($scfg, $newname, $guid);
 269 
 270     my $running  = undef; #fixme : is create_base always offline ?
 271 
 272     $class->volume_snapshot($scfg, $storeid, $newname, $snap, $running);
 273 
 274     return $newvolname;
 275 }

[dmbrownlee]

FWIW, I have the same problem using NFS storage rather than local-zfs. I was thinking it has to do with the path name for the disk image in the template getting created with the "vm-..." prefix rather than the "base-..." prefix you get when you manually convert a VM to a template. I tried using the experimental "path_in_datastore"
(https://registry.terraform.io/providers/bpg/proxmox/latest/docs/resources/virtual_environment_vm#path_in_datastore) to override the default file name, but that did not work for me.

[dmbrownlee]

This time with proper quotes...

To test the hypothesis about the file name prefix, I used the proxmox node shell and renamed the disk image file and updated the config for the template in /etc/pve/virtual-guest/nodes/<pve3>/qemu-server/.conf. Simply changing the disk image filename from vm-<id>-disk-0.qcow2 to base-<id>-disk-0.qcow2 allowed the previously broken template to be cloned.

[murdats]

Some of the comments here gave me enough info to get this workaround approach to fully automate template and linked clone provisioning working, also for flatcar like the original comment:

resource "proxmox_virtual_environment_file" "ignition_file" {
  node_name    = var.node
  content_type = "snippets"
  datastore_id = "local"

  source_raw {
    data = var.config
    file_name = "${var.vm_id}-ignition-file.ign"
  }
}

resource "proxmox_virtual_environment_vm" "flatcar_template" {
  node_name = var.node
  vm_id     = var.vm_id
  name      = var.name

  on_boot  = false
  started  = false
  template = false

  kvm_arguments = "-fw_cfg name=opt/org.flatcar-linux/config,file=/var/lib/vz/snippets/${proxmox_virtual_environment_file.ignition_file.file_name}"

  disk {
    datastore_id = "local-zfs"
    file_format  = "raw" # seems to cause issues if not explicitly set
    file_id   = var.cloud_image_file_id
    interface = "virtio0"
  }

  network_device {
    bridge = "vmbr0"
  }

  serial_device {}

  connection {
    type     = "ssh"
    user     = var.username
    password = var.password
    host     = var.host
  }

  # Workaround to resolve disk issues caused by creating template vs creating VM then converting to template
  provisioner "remote-exec" {
    inline = ["qm template ${self.vm_id}"]
  }

  lifecycle {
    ignore_changes = [
      template # Ignore template flag being changed by provisioner on re-run
    ]
  }
}