Issue building Windows 11 images on Microsoft Azure (while not having no issue with other Windows versions)

[tdefise]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Overview of the Issue

Unable to create Windows 11 using Packer on Microsoft Azure, while being able to create Windows 10, Windows Server 2019 and Windows Server 2022.

Reproduction Steps

1. Use the packer pluging "github.com/hashicorp/azure" with a version "~> 1"
2. Implement our requirements within the temporary Packer Azure Virtual Machine (Windows updates, Security Hardening, ...)
3. Sysprep the image

Packer version

1.9.4

Simplified Packer Template

build {
  sources = ["source.azure-arm.autogenerated_1"]

  # Deploying latest Microsoft updates
  provisioner "file" {
    max_retries = 3
    source      = "./packer_scripts/windowsupdates.ps1"
    destination = "C:\\temp\\windowsupdates.ps1"
  }

  # Deploy the Windows Update (Admin/System Rights are needed)
  provisioner "powershell" {
    max_retries       = 3
    elevated_user     = "SYSTEM"
    elevated_password = ""
    inline = ["powershell.exe -ExecutionPolicy Bypass -File 'C:\\temp\\windowsupdates.ps1' -Verb RunAs",
      "Write-Output \"Uninstalling the PowerShell module 'PSWindowsUpdate'\"",
      "Uninstall-Module -Name PSWindowsUpdate -Force",
    "Remove-Item 'C:\\temp\\windowsupdates.ps1' -Force"]
  }

  provisioner "windows-restart" {
    check_registry        = true
    restart_timeout       = "5m"
    max_retries           = "${var.max_restart_retries}"
    restart_check_command = "powershell -command \"& {Write-Output 'restarted.'}\""
  }

  # Remove of unwanted softwares/features/roles
  provisioner "powershell" {
    max_retries       = 3
    elevated_user     = "SYSTEM"
    elevated_password = ""
    script            = "./packer_scripts/removesoftwares.ps1"
  }

  provisioner "windows-restart" {
    check_registry        = true
    restart_timeout       = "5m"
    max_retries           = "${var.max_restart_retries}"
    restart_check_command = "powershell -command \"& {Write-Output 'restarted.'}\""
  }

  provisioner "powershell" {
    max_retries       = 3
    elevated_user     = "SYSTEM"
    elevated_password = ""
    inline = ["powershell.exe -ExecutionPolicy Bypass -File 'C:\\temp\\winlogbeats.ps1' -Verb RunAs",
    "Remove-Item 'C:\\temp\\winlogbeats.ps1' -Force", ]
  }

  provisioner "windows-restart" {
    check_registry        = true
    restart_timeout       = "5m"
    max_retries           = "${var.max_restart_retries}"
    restart_check_command = "powershell -command \"& {Write-Output 'restarted.'}\""
  }

  # Set required audit settings
  provisioner "file" {
    max_retries = 3
    source      = "./packer_scripts/auditsettings.ps1"
    destination = "C:\\temp\\auditsettings.ps1"
  }

  provisioner "powershell" {
    max_retries       = 3
    elevated_user     = "SYSTEM"
    elevated_password = ""
    inline = ["powershell.exe -ExecutionPolicy Bypass -File 'C:\\temp\\auditsettings.ps1' -Verb RunAs",
      "Remove-Item 'C:\\temp\\auditsettings.ps1' -Force"
    ]
  }

  # Configure cryptography requirements
  provisioner "powershell" {
    scripts = [
    "./packer_scripts/cryptography.ps1"]
  }

  provisioner "powershell" {
    inline = ["& C:\\Windows\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit > sysprep.log 2>&1", "Get-Content sysprep.log", "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select-Object ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 5 } else { break } }"]
  }

Operating system and Environment details

• image_publisher = "MicrosoftWindowsDesktop"
• image_offer = "Windows-10"
• image_sku = "win10-22h2-entn"
• vm_size = "Standard_DS4_v2"
• winrm_insecure = true
• winrm_timeout = "5m"
• winrm_use_ssl = true

Log Fragments and crash.log files

None

[nywilken]

Hi @tdefise thanks for opening up this issue. In order for us to best assist could you provide details on what is not working and if possible a log of the build. You can generate a log by setting the environment variable PACKER_LOG=1 before executing the build.

[tdefise]

Hi @nywilken

Thanks for the instruction.
I will execute that and come back to you.

[tdefise]

Hi @nywilken,

I added this variable and launched packer using the "-debug" flag.
The builds timeout.

I will retry with "PACKER_LOG=1" only

[tdefise]

Here is the output I get when doing the sysprep

2023/10/26 10:48:31 packer.exe plugin: Found command: Get-Content sysprep.log
2023/10/26 10:48:31 packer.exe plugin: Found command: while($true) { $imageState = Get-ItemProperty HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\State | Select-Object ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 5 } else { break } }
==> azure-arm.autogenerated_1: Provisioning with powershell script: C:\Users\VSSADM1\AppData\Local\Temp\powershell-provisioner1941092003
2023/10/26 10:48:31 packer.exe plugin: Opening C:\Users\VSSADM1\AppData\Local\Temp\powershell-provisioner1941092003 for reading
2023/10/26 10:48:31 packer.exe plugin: Uploading env vars to c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1
2023/10/26 10:48:31 packer.exe plugin: [INFO] 80 bytes written for 'uploadData'
2023/10/26 10:48:31 [INFO] 80 bytes written for 'uploadData'
2023/10/26 10:48:31 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:31 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:31 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-bc2db86d-2be5-4dfe-4964-47473fbf2f93.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50268->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:33 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:33 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:33 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:33 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:33 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-c2373dda-93b5-4d1e-79e6-adca30515b7a.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50270->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:35 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:35 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:35 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:35 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:35 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-a7c56b77-4f63-4575-7af3-c38588b26287.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50271->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:37 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:37 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:37 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:37 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:37 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-1cc20456-c453-4e35-4aff-306e60c7117a.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50272->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:39 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:39 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:39 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:39 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:39 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-c6cf6033-c0e4-4065-7843-990ecb5cf796.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50273->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:41 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:41 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:41 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:41 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:41 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-28ddf4d0-553b-41e3-4fe3-32b9ab0f0e94.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50274->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:43 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:43 [INFO] 0 bytes written for 'uploadData'

[nywilken]

Hi @tdefise thanks for updating the issue with the debug logs. Looking at the logs it appears that Packer is unable to connect to the Azure instance after the sysprep step. This is indicated by the logs entries copied below.

Error uploading file to $env:TEMP\winrmcp-c2373dda-93b5-4d1e-79e6-adca30515b7a.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50270->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:35 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:35 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:35 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:35 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:35 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-a7c56b77-4f63-4575-7af3-c38588b26287.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50271->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:37 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:37 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:37 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:37 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:37 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-1cc20456-c453-4e35-4aff-306e60c7117a.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50272->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:39 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:39 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:39 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:39 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:39 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-c6cf6033-c0e4-4065-7843-990ecb5cf796.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50273->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:41 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:41 [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:41 packer-plugin-azure_v2.0.1_x5.0_windows_amd64.exe plugin: 2023/10/26 10:48:41 Uploading file to 'c:/Windows/Temp/packer-ps-env-vars-653a43ff-3f08-e144-ce5c-3f1b48359b84.ps1'
2023/10/26 10:48:41 packer.exe plugin: Retryable error: Error uploading ps script containing env vars: Error uploading file to $env:TEMP\winrmcp-28ddf4d0-553b-41e3-4fe3-32b9ab0f0e94.tmp: Couldn't create shell: unknown error Post "https://13.74.218.215:5986/wsman": read tcp 10.1.0.34:50274->13.74.218.215:5986: wsarecv: An existing connection was forcibly closed by the remote host.
2023/10/26 10:48:43 packer.exe plugin: [INFO] 0 bytes written for 'uploadData'
2023/10/26 10:48:43 [INFO] 0 bytes written for 'uploadData'

In the Azure portal, what is the state of the instance after running the sysprep command?

I suspect that if the same template works for previous Windows versions that there may be a difference in behavior for Windows 11 and sysprep. We had a similar issue open on Packer hashicorp/packer#12663 for Windows 2022, where the sysprep was terminating the WinRM service. To determine if it is the sysprep command I recommend removing the sysprep command to see if Packer exits correctly. If it does then I would check if sysprep is disabling WinRM or if there is something preventing it from rebooting properly. Running a quick search for Azure, Windows 11, stuck after sysprep I found this post on the Windows forums.