vm analysis – episode 4 - polytechnique...
TRANSCRIPT
VM Analysis – Episode 4
Wait analysis of virtualized environments using host kernel tracing
Hani Nemati
May 5, 2017
Polytechnique Montréal
Laboratoire DORSAL
POLYTECHNIQUE MONTREAL – Hani Nemati
Agenda
Introduction● Research update and research motivation
New Investigations● Wait analysis of virtualized environments using host kernel tracing
● Sate of the art
● Proposed Algorithm
● Demo
● KVM-Tool for eBPF
Conclusion and in-progress
POLYTECHNIQUE MONTREAL – Hani Nemati
Available Trace-Points in different layers
HardwareHardware
Host Kernel Host KernelKVM.KOKVM.KO
QemuQemu
GuestKernel
VM 1
QemuQemu
GuestKernel
VM 2
QemuQemu
GuestKernel
VM n
Guest Kernel TraceGuest Kernel Trace
Qemu TraceQemu Trace
KVM TraceKVM Trace
Host Kernel TraceHost Kernel Trace
Hardware PMCHardware PMC
Previously on “VM Analysis”
POLYTECHNIQUE MONTREAL – Hani Nemati
Available Trace-Points in different layers
HardwareHardware
Host Kernel Host KernelKVM.KOKVM.KO
QemuQemu
GuestKernel
VM 1
QemuQemu
GuestKernel
VM 2
QemuQemu
GuestKernel
VM n
KVM TraceKVM Trace
Host Kernel TraceHost Kernel Trace
Previously on “VM Analysis”
POLYTECHNIQUE MONTREAL – Hani Nemati
Resource View for VM without tracing the VM
Previously on “VM Analysis”
POLYTECHNIQUE MONTREAL – Hani Nemati
Previously on “VM Analysis”
VirtFlow: Execution Flow Analysis of Virtual Machine
POLYTECHNIQUE MONTREAL – Hani Nemati
Two Nested VMs and One VM are preempting each other
Previously on “VM Analysis”
POLYTECHNIQUE MONTREAL – Hani Nemati
MotivationWhy the VM is waiting?
POLYTECHNIQUE MONTREAL – Hani Nemati
MotivationLet's use the Critical Flow view of Trace Compass?
POLYTECHNIQUE MONTREAL – Hani Nemati
InvestigationsMethodology Vec from kvm_inj_virq
CR3 from vcpu_enter_guest
Vec from kvm_inj_virqCR3 from vcpu_enter_guestCR3 from vcpu_enter_guest
If (Vec == (Block I/O irq)) {Block State = Block I/O State
} else if (Vec == (network irq)) {Block State = Network State
}
POLYTECHNIQUE MONTREAL – Hani Nemati
InvestigationsMethodology
If (Vec == 239) {Block State = Timer
} else if (Vec == 251) {Block State = Task
}
Vec from kvm_inj_virqCR3 from vcpu_enter_guest
CR3 from vcpu_enter_guest
Vec from kvm_inj_virqCR3 from vcpu_enter_guest
POLYTECHNIQUE MONTREAL – Hani Nemati
Investigations
Demo
POLYTECHNIQUE MONTREAL – Hani Nemati
InvestigationsWhat do you need to test this project?
● Access to Host only
● Run LTTng on Host with my new added tracepoint (vcpu_enter_guest)
● Clone TraceCompass from my github (virtFlow)● https://github.com/Nemati
● Open Resource View of TraceCompass
POLYTECHNIQUE MONTREAL – Hani Nemati
Investigations
One More Thing ...
KVM-ToolsFor
eBPF
POLYTECHNIQUE MONTREAL – Hani Nemati
POLYTECHNIQUE MONTREAL – Hani Nemati
Conclusion and in-progress
Inferences
● Wait Analyzing of process inside VM● A process is waiting for
● A Block request to finish● A network packet to receive ● Another process● A timer to fire
What you will see in Episode 5
● Wait Analyzing of process inside Nested VM