VM Guest instant crash

[xSalice]

Everytime I try to run a plugin, it will instantly crash the VM. I've tried to run with syscall only. Ubuntu 20.04 host and guest.

Logs:

(XEN) Dom0 has maximum 32 VCPUs
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) Freed 612kB init memory
(XEN) d2v1 failed to set entry for 1f9800 -> e39800 altp2m 1, rc -12
(XEN) domain_crash called from p2m.c:2119
(XEN) Domain 2 (vcpu#1) crashed on cpu#40:
(XEN) ----[ Xen-4.15.0  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    40
(XEN) RIP:    0010:[<ffffffff86bcd378>]
(XEN) RFLAGS: 0000000000010082   CONTEXT: hvm guest (d2v1)
(XEN) rax: 0000000000000036   rbx: 0000000000000040   rcx: 0000000000000006
(XEN) rdx: 0000000000000000   rsi: 0000000000000036   rdi: ffffffff87c6a880
(XEN) rbp: ffffb10dc00c4f10   rsp: ffffb10dc00c4f00   r8:  ffff8aa2f9800b6a
(XEN) r9:  ffff8aa2f9800b68   r10: 0000000000000000   r11: ffffffff87c6a888
(XEN) r12: ffff8aa2f0955940   r13: 0000000000000000   r14: ffff8aa2f9e5f000
(XEN) r15: ffff8aa2faf2a7a0   cr0: 0000000080050033   cr4: 00000000007606e0
(XEN) cr3: 00000001f209e001   cr2: 00007f093d39f7d0
(XEN) fsb: 0000000000000000   gsb: ffff8aa2faf00000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0018   cs: 0010

[tklengyel]

Can you post your xen boot options, the VM config and the output of xl info?

[xSalice]

Can you post your xen boot options, the VM config and the output of xl info?

I'm using the config package from drakvuf bundle inside of drakvuf-sandbox.

xl info:

host                   : msl-arch105
release                : 5.4.0-88-generic
version                : #99-Ubuntu SMP Thu Sep 23 17:29:00 UTC 2021
machine                : x86_64
nr_cpus                : 32
max_cpu_id             : 63
nr_nodes               : 2
cores_per_socket       : 16
threads_per_core       : 1
cpu_mhz                : 2095.143
hw_caps                : bfebfbff:77fef3ff:2c100800:00000121:0000000f:d39ffffb:00000008:00000100
virt_caps              : pv hvm hvm_directio pv_directio hap shadow iommu_hap_pt_share vmtrace
total_memory           : 65185
free_memory            : 23718
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 15
xen_extra              : .0
xen_version            : 4.15.0
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          :
xen_commandline        : placeholder dom0_mem=32592M,max:32592M dom0_max_vcpus=16 dom0_vcpus_pin=1 force-ept=1 ept=ad=0 hap_1gb=0 hap_2mb=0 altp2m=1 hpet=legacy-replacement smt=0 sched=credit no-real-mode edd=off
cc_compiler            : gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
cc_compile_by          : root
cc_compile_domain      :
cc_compile_date        : Tue Aug 10 13:41:07 UTC 2021
build_id               : 91d4df26f8863cde122b234a8f44faf15e52b94a
xend_config_format     : 4

VM config:

arch = 'x86_64'
name = "ubuntu20"
maxmem = 8000
memory = 8000
vcpus = 1
maxvcpus = 1
type = "hvm"
boot = "cd"
hap = 1
acpi = 1
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "destroy"
vnc=1
vnclisten="[::]:1,websocket=6400"
#vncpasswd="BjanWMdkyBFov9jqjyKB"
usb = 1
usbdevice = "tablet"
altp2m = 2
shadow_memory = 16
audio=1
soundhw='hda'
cpuid="host,htt=0"
vga="stdvga"
vif = [ 'type=ioemu,model=e1000,bridge=drak0' ]
disk = [ "tap:qcow2:/home/ubuntu/vms/ubuntu20.qcow2,xvda,w", "file:/home/ubuntu/iso/ubuntu-20.04.2-desktop-amd64.iso,hdc:cdrom,r" ]

[tklengyel]

I don't see anything wrong with your config. Probably worth trying another machine.

[xSalice]

Yeah that's unfortunate. I tried a windows 10 as well for a guest VM and that crashes. Really weird... All the libvmi example works to. Just crashes on drakvuf only. I'll provide more info if I see anything...

[tklengyel]

Yea, the question is why this happens:

(XEN) d2v1 failed to set entry for 1f9800 -> e39800 altp2m 1, rc -12
(XEN) domain_crash called from p2m.c:2119

At that point Xen just propagates some entries from one EPT to another, but if fails to set the entry in view 1. Probably worth digging into Xen and adding some printk's to see why it would fail to do that (and why only happens on this machine you have).

[tklengyel]

Oh, one thing to try, increase the shadow_memory parameter of the VM, 16 may be too low for an 8GB VM. Try 128.

[xSalice]

Oh wow this works perfectly! It's running great now :D. Thanks for the help @tklengyel. If you guys ever have time would be nice to have a guide on some of the explanation on xen settings related to alt2pm ^_^. Couldn't find much documentations on it.

[tklengyel]

Shadow memory for hvm guests is used to store the actual EPTs. With more memory it requires more space since it requires more entries to map the memory. Not sure why 16 is set as the default in the sandbox setup, my recommended minimum is 32 on the website.

[xSalice]

Got it. Thanks for all the help :). closing issue now