virtualkd on linux

Author

“POLITEHNICA” University of

Bucharest

Automatic Control and Computers

Faculty

Computer Science

Department

Scientific Advisors

VirtualKd on Linux

Tudor [email protected]

As. Dr. Ing. Răzvan DeaconescuIng. Cosmin Rohat

Bachelor Presentation Session - July 2012

Contents

• Introduction to Kernel Debugging• About the VirtualKd Project• Implementation• Conclusion & Further work

19.04.23 Bachelor Presentation Session - July 2012 2

• Project idea : To speed-up kernel module debugging on

Linux, using native VMware communication mechanisms.


Kernel Debugging: Overview

• Traditional approach: “printk debugging”• Easy to use• Low/No overhead• Some errors are difficult to find

• Serial port debugging• Better error catching• Requires kernel recompilation• Slow


Motivation for the VirtualKd Project

• Virtual machines emulate the serial port

• Problem: Serial port is slow (~20 KB/s)

• Why not let the VM hypervisor handle the data transfer?

• VirtualKd

• GuestRPC

• Available for the Windows kernel


Installing VirtualKd


First task: Installing VirtualKd and creating a debugging session on Windows

Project Overview


• Keywords: VMware; virtual console; debugger

Project Steps

1. Console driver that captures debugging output on the guest

2. Sending the debugging output to a process on the host

3. Attaching a debugger


Registering a Console

• Capturing kernel printk() messages Registering a virtual console driver

• Problem : can not use printk() inside write method(recursive printk’s)

• =>• Write data in memory• Display it after console

is unregistered


VMware Communication

• Used Open-Vm-Tools

• VMCI Sockets for VM-Host communication

• API similar to BSD sockets• Works on a single physical machine


VMCI-based Kernel Sender


Two components:

Client•On the guest •Inside a kernel module•Registers a console•Sends debug output through a socket

Server•On the host•User space program•Receives the debug output

Integrating VMCI into the Console Driver

• Issues Encountered• Lack of documentation for Open-Vm-Tools

• Kernel panic when sending the data through a socket in write()

=> Need to disable interrupts

• Write() will be called each time a printk() call is made=> The protocol between the client and server should be

as simple as possible


Adding debugger support

• Debugging solution used: kgdb

• Kernel patched with kgdb support

• Uses gdb on the Linux kernel

• Kernel debugging packets : put_packet(), get_packet()


Kgdb support


Idea:Using kprobes API, register hooks to put_packet/get_packet

Performance analysis


Solution Average Transfer Rate (KB/s)

Netconsole 600

VirtualKd on Linux 350-400

Serial Port 30

Similar functionality with Netconsole

No networking required!

Results

• Installed and configured VirtualKd for Windows

• Managed to capture printk() messages

• Finished host-guest VMCI communication

• Relies on VMCI kernel module=> Early boot messages can not be captured


Conclusion

• Not as fast as Netconsole

.... but – Still better than using serial port– Can be used in networkless environments– Can potentially be improved, using other VMware

communication APIs


Future Work

• Possibly test other VMware transport mechanisms– GuestRPC– VMware Shared Memory API

• Capturing “early printk” output

• Adding support for a kgdb I/O driver


Questions


virtualkd on linux

Documents

bachelor presentation

debugging session

packetstudent paper

windowsstudent paper

windows kernel

kernel moduleregisters

kernel debuggingabout

debugging output