paravirtualized usb support for xen · modern webcams support •the uvc (usb video class) driver...

Copyright 2008 FUJITSU LIMITED

Paravirtualized USB Support

for Xen

Noboru Iwamatsu

[email protected]

FUJITSU LABORATORIES LTD.

mailto:[email protected]

Background: Client-side virtualization and USB

Copyright 2008 FUJITSU LIMITED1

In client-side virtualization, special Service-VMs work in the background of User-VM, and provide various functions. To give some examples,

Authentication VM: Authenticate with USB-connected smartcard or biometrics.

Communication VM: VoIP/Messenger application using USB-connected webcam or audio.

In this situation, both User-VM and Service-VMs require using the USB devices at the same time by the same USB host controller.

Hypervisor

User VM Service VM Service VM

LaptopUSB Host Controller

User OS Service OS Service OS

AuthenticationVoIP/

Messenger

Office/Music

apps

USB devices

Usage example of

client-side virtualization

History and Motivation: Xen and USB support


Xen 2.0.x(in PV-only age)

Paravirtualized USB support was temporarily into the tree.

But eventually removed and not released.

Xen 3.0 Qemu-dm supported UHCI emulation for HVM domain.

Xen 3.2PCI pass-through with IOMMU supported attaching USB

Host Controller to HVM domain.

Xen 3.4(current

unstable)

Not well supported

USB1.1 only

Works well. But the entire controller is assigned to

single domain and can’t be used from other domains.

Existing options are not suitable for the client-side

virtualization. So, we have started developing

paravirtualized USB driver, and proposed in XCI.

Now, “PV USB support” is on the roadmap.

Today, we would like to report our implementation details

and development status.

Overview: What’s paravirtualized USB driver?


A split drivers for handling USB devices from guest domains.

frontend: Virtual USB 2.0 Host Controller driver acting as a proxy for backend.

backend: USB function driver handling the USB devices.

urb (USB request block):

In linux kernel, all USB drivers and subsystems

communicate with urb. Urb is a USB request block

structure (described in include/linux/usb.h).

It is much like “struct skbuff” in the networking code.

Frontend transfers urbs to backend through Xen, and backend transfers

the received urbs to the USB devices.

Xen Hypervisor

Dom 0

Dom0 OS

Guest VM 1 Guest VM 2

Guest OSGuest OS

PV USB

frontend driver

PV USB

frontend driver

PV USB

backend driver

USB native driver

USB Host Controller

USB devices

urb transferring

Implementation details(1/5):

USB basics: Driver architecture and urb lifecycle in kernel


USB drivers in linux kernel is implemented as 3-layer module stacks.

USB Function Driver - Driver for each USB device (e.g. storage, printer, hid)

A urb is created by this driver, and is submitted to USB core.

If the urb is successfully transferred to the USB device, the completion handler of the urb is called.

USB Core - USB subsystem in linux kernel

The urb is enqueued to specific host controller driver for the specified device by this layer.

USB Host Controller Driver – Driver for USB host controller hardware

The enqueued urb is transferred to the hardware, and is backed to USB core after the driver get response.

USB Function Driver

USB Core

USB Host Controller Driver

usb_submit_urb()

usb_hcd_giveback_urb().urb_enqueue()

calling completion handler

usb_alloc_urb() usb_free_urb()

Hardware

(USB Host Controller)

User-space or other kernel subsystems

urb urb

Kernel-space


Paravirtualized USB driver architecture and urb lifecycle

Frontend driver - Implemented as a USB host controller driver

The enqueued urb is mapped to RING request, and is sent to backend.

When the RING response is received, the urb is backed to USB core after substituting that status-value.

Backend driver - Implemented as a USB function driver

The urb is re-created from the RING request, and is submitted to the specified device.

When the urb is completed, the status of the urb is mapped to RING response and returned to frontend.

All existing code need not be modified, and maybe all USB drivers in kernel can work.


PV USB backend

PV USB frontend

Xen frontend interface

RING xenbus

Virtual USB 2.0

Host Controller Driver

USB Function Driver

USB Core

Host Controller Driver

USB Core

USB Function Driver

Xen backend interface

RINGxenbus

urb urb

Cloned

urb

Dom0 kernel-space DomU kernel-space

Cloned

urb

Hardware

(USB Host Controller)

.urb_enqueue() usb_hcd_giveback_urb()

usb_submit_urb () completion handler

allocfree


How to map urb to RING request/response?

The fields of the urb structure is mapped to the RING as follows.

The data buffers of urb are sharing between frontend and backend by using the grant table operations.

If the number of shared pages is 10, RING_SIZE is set to 32. Most of function drivers may be all right by this setting :-)


struct urb

{

unsigned int pipe;

unsigned int transfer_flags;

void *transfer_buffer;

int transfer_buffer_length;

unsigned char *setup_packet;

int interval;

int start_frame;

int number_of_packets;

struct usb_iso_packet_descriptor iso_frame_desc[0];

int status;

int actual_length;

int error_count;

…

};

struct usbif_response

{

...

};

struct usbif_request

{

…

segs[];

};

Mapping to RING request

Mapping to RING response

Sharing pages with grant table


Virtual USB host controller internals


Virtual USB Host Controller

submit_waiting_queue

submit_in_progress_queue

giveback_waiting_queue

urb

.urb_enqueue() usb_hcd_giveback_urb()

RING request

RING response

urb

urb urb urb urb

urb

urb

after send, move to tail

move matched urb to tail

add to tail

delete from queue

urb urb

Virtual host controller has 3 queues for scheduling urbs.

submit_waiting_queue

The enqueued urb is added to the tail of this queue, and waits its turn to be sent to backend.

submit_in_progress_queue

While the RING request is sent and waits for the response, the urb is added to this queue.

giveback_waiting_queue

While the RING response is received and waits for backing to the USB core, the urb is added to this

queue.

RING responses are called

from interrupt context and

submit_wating_queue and

givback_wating_queue are

flushed by timer functions.


How to plug the devices to guest domains?


Guest VM 1

(domain ID:1)

Virtual

Host Controller

(vusb-0)Virtual

roothub

Guest VM 2

(domain ID:2)

Virtual

Host Controller

(vusb-0)Virtual

roothub

device

Dom 0

Host Controller

(usb1)

roothub

hub

device

device device device device

device

1-4

1-2.1 1-2.3

Hotplug-rule is set from sysfs interface in the backend driver.

The hotplug-rule format

• <usbbusname>:<domid>:<vusb number>:<virtual port number>

Example settings% echo 1-2.3:1:0:3 > /sys/bus/usb/drivers/usbback/new_vport

% echo 1-4:2:0:1 > /sys/bus/usb/drivers/usbback/new_vport

% echo 1-2.1:2:0:2 > /sys/bus/usb/drivers/usbback/new_vport

When new device is connected, the backend driver claims the device if its busnameis found on the settings. The frontend driver is notified of hotplug by xenbus.

Development Status

Paravirtualized USB driver has just started working.

Tested devices

The performance is about 6MB/s(Read/Write throughput of flash drive).

• 1/5 of native performance, but more than 4 times faster than USB1.1.

• There is still room for improvement.

Implementation Status

• Hotplug and disconnect functions are not completed yet.


Type Name Manufacturer Driver Status

Keyboard FKB-108-EU FILCO usbhid

Mouse Cordless Notebook Mouse Logitech usbhid

Flash drive RUF2-R2GS Buffalo usb-storage

Flash drive RUF-C1G/U2 Buffalo usb-storage

HDD HDCN-U500 IO DATA usb-storage [1]

Webcam WebCam 3 USB Creative Labs ov511 [2]

Works Works with issues Not worked yet

1. SCSI command [READ_CAPCITY] fails.

2. Horizontal stripes are into video streaming image.

TODO & Future work

TODO

Many cleanups and bugfixes.

Performance upgrade.

Modern webcams support

• The UVC (USB Video Class) driver requires high bandwidth.

Suspend / Resume support.

Xend support.

Future work

Netchannel2 support.

Porting the frontend driver to stub domain.


We will post the code by the end of this year!

paravirtualized usb support for xen · modern webcams support •the uvc (usb video class) driver...

Documents