Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proper way to snapshot export and import #761

Open
fabionitto opened this issue Jan 27, 2023 · 2 comments
Open

Proper way to snapshot export and import #761

fabionitto opened this issue Jan 27, 2023 · 2 comments
Assignees
@chivay

Comments

@fabionitto
Copy link

fabionitto commented Jan 27, 2023 •
edited

Host system: Debian 10
Guest: window 7 x64 pt-br
Running over proxmox 7.2-3

I'm trying to use snapshotting feature to "facilitate" deployment of drakvuf-sandbox. The idea is that, after manually installing guest (win7x64), i could make a minimal snapshot to minio, and then, in a new fresh installation, instead of doing draksetup install I could do draksetup snapshot import --name <NAME> --bucket <BUCKET>

I don't know if snapshotting should be used for this case, specially after I found out the snapshot import appears to not be working correctly.

After trying to import, I get this error:

root@debian:/home/automation# draksetup snapshot import --name Win7x64_Inicial --bucket draksnap
[2023-01-27 15:01:35,883][DEBUG] Starting new HTTP connection (1): 10.61.193.153:9000
[2023-01-27 15:01:35,887][DEBUG] http://10.61.193.153:9000 "GET /draksnap?location= HTTP/1.1" 200 128
[2023-01-27 15:01:35,889][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/ HTTP/1.1" 200 0
[2023-01-27 15:01:35,889][INFO] Downloading installation info
[2023-01-27 15:01:35,892][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/Win7x64_Inicial/install.json HTTP/1.1" 200 0
[2023-01-27 15:01:35,915][DEBUG] http://10.61.193.153:9000 "GET /draksnap/Win7x64_Inicial/install.json HTTP/1.1" 200 324
[2023-01-27 15:01:35,917][INFO] Downloading VM config
[2023-01-27 15:01:35,918][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/Win7x64_Inicial/cfg.template HTTP/1.1" 200 0
[2023-01-27 15:01:35,925][DEBUG] http://10.61.193.153:9000 "GET /draksnap/Win7x64_Inicial/cfg.template HTTP/1.1" 200 755
[2023-01-27 15:01:35,947][INFO] Downloading VM disk image
[2023-01-27 15:01:35,951][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/Win7x64_Inicial/disk.img HTTP/1.1" 200 0
[2023-01-27 15:01:35,977][DEBUG] http://10.61.193.153:9000 "GET /draksnap/Win7x64_Inicial/disk.img HTTP/1.1" 200 7670530048
[2023-01-27 15:03:28,879][INFO] Importing VM disk
[2023-01-27 15:04:55,027][INFO] Generated VM configuration for vm-0
qemu-img: /var/lib/drakrun/volumes/vm-0.img: Error: Trying to create an image with the same filename as the backing file
Traceback (most recent call last):
  File "/usr/bin/draksetup", line 5, in <module>
    ds.main()
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/draksetup.py", line 1197, in snapshot_import
    backend.rollback_vm_storage(0)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/storage.py", line 266, in rollback_vm_storage
    check=True,
  File "/usr/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['qemu-img', 'create', '-f', 'qcow2', '-F', 'qcow2', '-o', 'backing_file=/var/lib/drakrun/volumes/vm-0.img', '/var/lib/drakrun/volumes/vm-0.img']' returned non-zero exit status 1.

I edited /opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/draksetup.py and removed the lines 1196 e 1197
backend = get_storage_backend(install_info)
backend.rollback_vm_storage(0)
since it appears to be unnecessary, and leads the error above.

After doing this, snapshot import seems to work correctly. I can VNC to vm-0 just fine.
But when running draksetup postinstall I get InjectorTimeout Error from 'C:\Windows\System32\ntdll.dll':

root@debian:/home/automation# draksetup snapshot import --name Win7x64_Inicial --bucket draksnap
Detected local snapshot. It will be REMOVED. Continue? [y/N]: y
[2023-01-27 16:11:15,458][DEBUG] Starting new HTTP connection (1): 10.61.193.153:9000
[2023-01-27 16:11:15,462][DEBUG] http://10.61.193.153:9000 "GET /draksnap?location= HTTP/1.1" 200 128
[2023-01-27 16:11:15,464][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/ HTTP/1.1" 200 0
[2023-01-27 16:11:15,465][INFO] Downloading installation info
[2023-01-27 16:11:15,467][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/Win7x64_Inicial/install.json HTTP/1.1" 200 0
[2023-01-27 16:11:15,477][DEBUG] http://10.61.193.153:9000 "GET /draksnap/Win7x64_Inicial/install.json HTTP/1.1" 200 324
[2023-01-27 16:11:15,480][INFO] Downloading VM config
[2023-01-27 16:11:15,481][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/Win7x64_Inicial/cfg.template HTTP/1.1" 200 0
[2023-01-27 16:11:15,505][DEBUG] http://10.61.193.153:9000 "GET /draksnap/Win7x64_Inicial/cfg.template HTTP/1.1" 200 755
[2023-01-27 16:11:15,528][INFO] Downloading VM disk image
[2023-01-27 16:11:15,532][DEBUG] http://10.61.193.153:9000 "HEAD /draksnap/Win7x64_Inicial/disk.img HTTP/1.1" 200 0
[2023-01-27 16:11:15,589][DEBUG] http://10.61.193.153:9000 "GET /draksnap/Win7x64_Inicial/disk.img HTTP/1.1" 200 7670530048
[2023-01-27 16:13:06,096][INFO] Importing VM disk
[2023-01-27 16:14:44,681][INFO] Generated VM configuration for vm-0
[2023-01-27 16:14:44,698][INFO] Created bridge drak0
[2023-01-27 16:14:44,756][INFO] Bridge drak0 is up
Parsing config from /etc/drakrun/configs/vm-0.cfg
[2023-01-27 16:14:45,730][INFO] Minimal snapshots require postinstall to work correctly
[2023-01-27 16:14:45,731][INFO] Please VNC to the port 5900 to ensure the OS booted correctly
[2023-01-27 16:14:45,731][INFO] After that, execute this command to finish the setup
[2023-01-27 16:14:45,731][INFO] # draksetup postinstall
root@debian:/home/automation# draksetup postinstall
[2023-01-27 16:17:03,156][INFO] Cleaning up leftovers(if any)
[2023-01-27 16:17:03,157][INFO] Ejecting installation CDs
[2023-01-27 16:17:03,813][INFO] Determined PDB GUID: 3844dbb920174967be7aa4a2c20430fa2
[2023-01-27 16:17:03,813][INFO] Determined kernel filename: ntkrnlmp.pdb
[2023-01-27 16:17:03,813][INFO] Fetching PDB file...
[2023-01-27 16:17:03,817][DEBUG] Starting new HTTPS connection (1): msdl.microsoft.com:443
[2023-01-27 16:17:05,179][DEBUG] https://msdl.microsoft.com:443 "GET /download/symbols/ntkrnlmp.pdb/3844dbb920174967be7aa4a2c20430fa2/ntkrnlmp.pdb HTTP/1.1" 302 0
[2023-01-27 16:17:05,181][DEBUG] Starting new HTTPS connection (1): vsblobprodscussu5shard10.blob.core.windows.net:443
[2023-01-27 16:17:10,063][DEBUG] https://vsblobprodscussu5shard10.blob.core.windows.net:443 "GET /b-4712e0edc5a240eabf23330d7df68e77/3361580E1DAA2301EF4C62D105FB67166BD89EA03FCDE3C800EACFAF71EE01C200.blob?sv=2019-07-07&sr=b&si=1&sig=wvHgOJWmf%2FPm7L3wRXGaSyX6Ahhs5IxzRXTVPDe1v3Y%3D&spr=https&se=2023-01-28T20%3A12%3A04Z&rscl=x-e2eid-1fe296f9-5d384a10-9ad9ca7b-06e49c05-session-a3b2efc5-afda4a33-b7180fb6-e8c3c234 HTTP/1.1" 200 8768512
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8.77M/8.77M [02:45<00:00, 52.8kiB/s]
[2023-01-27 16:19:56,116][INFO] Generating profile out of PDB file...
[2023-01-27 16:20:18,154][INFO] Saving profile...
[2023-01-27 16:20:18,157][INFO] Deleted /var/lib/drakrun/profiles/ntkrnlmp.pdb
[2023-01-27 16:20:19,340][INFO] Saving runtime profile...
[2023-01-27 16:20:19,342][INFO] Saving VM snapshot...
[2023-01-27 16:20:19,342][INFO] Saving VM vm-0
Saving to /var/lib/drakrun/volumes/snapshot.sav new xl format (info 0x3/0x0/2064)
xc: info: Saving domain 9, type x86 HVM
xc: Frames: 1114240/1114240  100%
xc: End of stream: 0/0    0%
[2023-01-27 16:21:13,575][INFO] Snapshot was saved succesfully.
[2023-01-27 16:21:13,576][INFO] Snapshotting persistent memory...
[2023-01-27 16:21:13,581][DEBUG] Starting new HTTPS connection (1): drakvuf.cert.pl:443
[2023-01-27 16:21:15,901][DEBUG] https://drakvuf.cert.pl:443 "POST /usage/draksetup HTTP/1.1" 200 2
[2023-01-27 16:21:16,116][INFO] Generated VM configuration for vm-1
[2023-01-27 16:21:16,192][INFO] Bridge drak1 already exists.
[2023-01-27 16:21:16,269][INFO] Bridge drak1 is up
Formatting '/var/lib/drakrun/volumes/vm-1.img', fmt=qcow2 size=107374182400 backing_file=/var/lib/drakrun/volumes/vm-0.img backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16
[2023-01-27 16:21:16,710][INFO] Restoring VM vm-1
Loading new save file /var/lib/drakrun/volumes/snapshot.sav (new xl fmt info 0x3/0x0/2064)
 Savefile contains xl domain config in JSON format
Parsing config from /etc/drakrun/configs/vm-1.cfg
xc: info: Found x86 HVM domain from Xen 4.16
xc: info: Restoring domain
xc: info: Restore successful
xc: info: XenStore: mfn 0xfeffc, dom 0, evt 1
xc: info: Console: mfn 0xfefff, dom 0, evt 2
[2023-01-27 16:21:21,668][INFO] Fetching rekall profile for Windows/System32/ntdll.dll
[2023-01-27 16:22:21,834][INFO] Deleted /var/lib/drakrun/profiles/amd64_ntdll_profile
Traceback (most recent call last):
  File "/usr/lib/python3.7/subprocess.py", line 474, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.7/subprocess.py", line 939, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.7/subprocess.py", line 1682, in _communicate
    self._check_timeout(endtime, orig_timeout)
  File "/usr/lib/python3.7/subprocess.py", line 982, in _check_timeout
    raise TimeoutExpired(self.args, orig_timeout)
subprocess.TimeoutExpired: Command '['injector', '-o', 'json', '-d', 'vm-1', '-r', '/var/lib/drakrun/profiles/kernel.json', '-i', '1888', '-k', '0x187000', '-m', 'readfile', '-e', 'C:\\Windows\\System32\\ntdll.dll', '-B', '/var/lib/drakrun/profiles/amd64_ntdll_profile']' timed out after 60 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/draksetup.py", line 586, in create_rekall_profile
    cmd = injector.read_file(guest_dll_path, local_dll_path)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/injector.py", line 66, in read_file
    return subprocess.run(injector_cmd, timeout=timeout, capture_output=True)
  File "/usr/lib/python3.7/subprocess.py", line 479, in run
    stderr=stderr)
subprocess.TimeoutExpired: Command '['injector', '-o', 'json', '-d', 'vm-1', '-r', '/var/lib/drakrun/profiles/kernel.json', '-i', '1888', '-k', '0x187000', '-m', 'readfile', '-e', 'C:\\Windows\\System32\\ntdll.dll', '-B', '/var/lib/drakrun/profiles/amd64_ntdll_profile']' timed out after 60 seconds

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/draksetup", line 5, in <module>
    ds.main()
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/draksetup.py", line 817, in postinstall
    create_missing_profiles()
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/draksetup.py", line 864, in create_missing_profiles
    create_rekall_profile(injector, profile, True)
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/draksetup.py", line 634, in create_rekall_profile
    f"Injector timed out for {file.path}", raise_on_error, e
  File "/opt/venvs/drakrun/lib/python3.7/site-packages/drakrun/draksetup.py", line 570, in on_create_rekall_profile_failure
    raise Exception(f"[REQUIRED DLL] {msg}") from exception
Exception: [REQUIRED DLL] Injector timed out for Windows/System32/ntdll.dll

Is it possible that I am doing something wrong? What are the proper steps to make a snapshot import?

...
@BonusPlay
Copy link
Collaborator

This is probably caused by VM not starting, as explained in #762.

@fabionitto
Copy link
Author

This is probably caused by VM not starting, as explained in #762.

Yes. I checked this. After importing the snapshot it is necessary to wait a few minutes so that Guest SO boots up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chivay chivay

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@fabionitto @BonusPlay @chivay