Worker hangs up and drakrun waits forever for injection to finish

[psrok1]

Describe the bug

Drakrun worker can't recover from Inject.CreateProc cmd.exe /c ipconfig /release >nul when it takes forever due to tenporary DHCP connection issues

I see that there is timeout=120 set in provisioning part:

drakvuf-sandbox/drakrun/drakrun/main.py

Lines 608 to 615 in 5017f9c

	if self.net_enable:
	self.log.info("Setting up network...")
	injector.create_process(
	"cmd /C ipconfig /release >nul", wait=True, timeout=120
	)
	injector.create_process(
	"cmd /C ipconfig /renew >nul", wait=True, timeout=120
	)

but timeout argument handling is not implemented:

drakvuf-sandbox/drakrun/drakrun/injector.py

Lines 68 to 73 in 5017f9c

	def create_process(
	self, cmdline: str, wait: bool = False, timeout: int = 60
	) -> subprocess.CompletedProcess:
	injector_cmd = self._get_cmdline_createproc(cmdline, wait=wait)
	""" Create a process inside the VM with given command line """
	return subprocess.run(injector_cmd, check=True)

Output of the status checking commands

Dec 07 12:29:01 ... drakrun[8661]: DRAKVUF injector v0.8-git+-1 Copyright (C) 2014-2021 Tamas K Lengyel
Dec 07 12:29:01 ... drakrun[8661]: tcpdump: listening on vif30446.0-emu, link-type EN10MB (Ethernet), capture size 262144 bytes
Dec 07 12:29:01 ... drakrun[8661]: [2022-12-07 12:29:01,218][INFO] Using command: regsvr32 /s C:\Users\...\Desktop\malwar.dll
Dec 07 12:29:01 ... drakrun[8661]: INFO:karton.drakrun-oss:Using command: regsvr32 /s C:\Users\...\Desktop\malwar.dll
Dec 07 12:29:01 ... drakrun[8661]: [2022-12-07 12:29:01,221][INFO] Setting up network...
Dec 07 12:29:01 ... drakrun[8661]: INFO:karton.drakrun-oss:Setting up network...
Dec 07 12:29:01 ... drakrun[8661]: DRAKVUF injector v0.8-git+-1 Copyright (C) 2014-2021 Tamas K Lengyel
Dec 07 12:29:01 ... drakrun[8661]: {"Plugin": "inject", "TimeStamp": "1670412541.519323", "Method": "CreateProc", "Status": "Success", "ProcessName": "", "Arguments": "/C ipconfig /release >nul", "InjectedPid": 2800, "InjectedTid": 2804}
Dec 07 12:29:01 ... drakrun[8661]: DRAKVUF injector v0.8-git+-1 Copyright (C) 2014-2021 Tamas K Lengyel

[BonusPlay]

If DHCP fails, then VM will have no internet. What would be the preferable outcome? Run analysis without internet risking that dropper won't download main payload? Also, please provide drakvuf-sandbox commit hash/version. @psrok1

[psrok1]

It should mark current analysis task as failed when timeout is reached or retry ipconfig few times with hope that DHCP will be alive after short downtime.

Right now workers are just dying over time, one by one, waiting forever for ipconfig to finish.

Drakvuf is installed from .deb package:

$ apt show drakrun
Package: drakrun
Version: 0.18.2
Status: install ok installed
Priority: optional
Section: admin
Maintainer: Unmaintained snapshot
Installed-Size: 87.0 MB
Pre-Depends: dpkg (>= 1.16.1), python3.7, libpython3.7, python3-distutils
Depends: tcpdump, genisoimage, qemu-utils, bridge-utils, dnsmasq
Download-Size: unknown
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: DRAKVUF Sandbox Worker (standalone)

[psrok1]

Well it seems like it's injector fault:

# trying to run ipconfig /release
Dec 07 12:29:01 ... drakrun[8661]: DRAKVUF injector v0.8-git+-1 Copyright (C) 2014-2021 Tamas K Lengyel
# ipconfig /release succeed
Dec 07 12:29:01 ... drakrun[8661]: {"Plugin": "inject", "TimeStamp": "1670412541.519323", "Method": "CreateProc", "Status": "Success", "ProcessName": "", "Arguments": "/C ipconfig /release >nul", "InjectedPid": 2800, "InjectedTid": 2804}
# trying to run ipconfig /renew
Dec 07 12:29:01 ... drakrun[8661]: DRAKVUF injector v0.8-git+-1 Copyright (C) 2014-2021 Tamas K Lengyel

After these logs we usually see some DHCP logs from dnsmasq-dhcp but there is nothing. I don't see any cmd.exe processes in taskmgr, but explorer.exe is still alive and has the same PID.

I will come back with more info

[psrok1]

Uh it looks like bug in injector?

Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.164965 libdrakvuf initialized
Mar 18 02:19:51 kuku2 drakrun[5248]: Target PID 1896 to start 'cmd /C ipconfig /renew >nul'
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166210 Found PEB @ 0x7fffffd4000. LDR @ 0x77da2640. INLOADORDER @ 0x2826f0.
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166278 Found module Explorer.EXE at 0xff590000
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166307 Found module ntdll.dll at 0x77c70000
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166316 Found module kernel32.dll at 0x77a50000
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166475 #011ResumeThread @ 0x77a613a0
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166482 Found PEB @ 0x7fffffd4000. LDR @ 0x77da2640. INLOADORDER @ 0x2826f0.
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166493 Found module Explorer.EXE at 0xff590000
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166501 Found module ntdll.dll at 0x77c70000
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166508 Found module kernel32.dll at 0x77a50000
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166551 #011CreateProcessW @ 0x77a71bb0
Mar 18 02:19:51 kuku2 drakrun[5248]: Starting injection loop
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.166584 Started DRAKVUF polling loop
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.180139 CR3 cb on vCPU 1: 0x9edd4000
Mar 18 02:19:51 kuku2 drakrun[5248]: CR3 changed to 0x9edd4000. PID: 384 PPID: 360 TID: 444
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.181001 CR3 cb on vCPU 1: 0x39e64000
Mar 18 02:19:51 kuku2 drakrun[5248]: CR3 changed to 0x39e64000. PID: 1896 PPID: 1852 TID: 2024
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.181106 Breakpoint VA 0xfffff800026e3e70 -> PA 0x26e3e70
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.181128 Physmap populated? 0
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.181188 Copied trapped page to new location
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.181194 Activating remapped gfns in the altp2m views!
Mar 18 02:19:51 kuku2 drakrun[5248]: 1679102391.181257 #011#011Trap added @ PA 0x26e3e70 RPA 0xff006e70 Page 9955 for entry.
Mar 18 02:19:51 kuku2 drakrun[5248]: Got return address 0xfffff800026e3e70 from trapframe and it's now trapped!

All workers end their life trying to make a trap on 0xfffff800026e3e70 which appears in KTRAP_FRAME->Rip

https://github.com/tklengyel/drakvuf/blob/main/src/libinjector/win/win_injector.c#L297

I see this address is assumed to be in user-mode address space (what is expected indeed) and it's part of trapping process in user-mode context to start injection. Drakvuf waits for int3 trap to be executed on 0xfffff800026e3e70 but it never happens.

[psrok1]

I reopened an issue to track the problem here