linux kernel 3.0 release - cnblogs.com...similar as linux system call • event channel • grant...
TRANSCRIPT
![Page 1: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/1.jpg)
Linux kernel 3.0 release
![Page 2: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/2.jpg)
IO Data Flow Hook On [email protected][email protected]
![Page 3: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/3.jpg)
What?
Why?
How?
![Page 4: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/4.jpg)
Agenda
IO hook general idea IO hook on virtualization platform IO hook achievement on Xen
![Page 5: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/5.jpg)
IO Hook Philosophy
![Page 6: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/6.jpg)
Utility
transparent encryption (e.g. Linux dm-crypt) virtual block device driver (e.g. Linux softraid) file hidden virtual filesystem (e.g. FUSE) firewall (e.g. Netfilter) virtual network device driver (e.g. bond & vlan)
![Page 7: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/7.jpg)
IT World Is Changing …
Physical Hardware
Host OSApp AppApp
Physical HardwareHypervisorGuest OS
App AppApp
从对于变化的研究来研究技术的变化
![Page 8: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/8.jpg)
Change Our Cognition
glibc glibc
VFS
vim/cat … hook app
NFS
Ext3
Hook module
User-Space
KernelOS
Physical Server
Hook App
OS
Physical Server
Hook App
OS
Hook App
OS
Hook App
OS
Physical Server
OS OS
Hook App
现实意义?
![Page 9: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/9.jpg)
Advantage
reduce management cost uniform configuration interface frequent patch/hot fix migration virtual appliance shipping
management task heterogeneous -> uniform
![Page 10: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/10.jpg)
Xen IO Overview
![Page 11: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/11.jpg)
Filesystem Hook Overview
Xen Hypervisor
Domain-0 Windows HVMDomain-X
Physical NIC
Physical CPU
Physical Memory
mini-filter driver
front-end handler
Application
shared memory
back-endhandler (kernel)
hook app(in user-space)
Qemu device manager
device drivers
Emulated device 1
Emulated device 2
BlockDevice
![Page 12: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/12.jpg)
Xen Filesystem IO Hook (1)
notepad.exe hook app
IO Manager
WindowsPlatform
KernelUser-Space
NTFS driver
device driver
Filter driver
1. callback2. intercept
![Page 13: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/13.jpg)
Xen Filesystem IO Hook (2)
One agent on each windows Guest OS mini-filter driver filter/delete/quarantine do not need binary update
Deployment challenge convince user “no harm” VM template? Good idea?
![Page 14: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/14.jpg)
Network Hook Overview
Xen Hypervisor
Domain-0 Domain-UDomain-X
Physical NIC
Physical CPU
Physical Memory
bridge
hook point
netback driver netfront driver
TCP/IP stack
Application
shared memory
handler(in kernel)
hook app(in user-space)
BlockDevice
![Page 15: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/15.jpg)
Xen Network IO Hook (1)Device Driver
(input)Device Driver
(output)
Routing Routing
Local Process
NF_IP_PRE_ROUTING
NF_IP_POST_ROUTING
NF_IP_FORWARD
incoming packets outgoing packets
NF_IP_LOCAL_IN NF_IP_LOCAL_OUT
Device Driver(input)
Device Driver(output)
Forwarding Forwarding
IP Layer
NF_BR_PRE_ROUTING
NF_BR_POST_ROUTING
NF_BR_FORWARD
incoming packets outgoing packets
NF_BR_LOCAL_IN
NF_BR_LOCAL_OUT
![Page 16: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/16.jpg)
Xen Network IO Hook (2)
Where to hook? Layer-2 (bridge) [√] Layer-3 (ip) [√] Layer-4 (tcp) [X]
Xen uses bridge-network by default /etc/xen/xend-config.sxp
(network-script network-bridge)
![Page 17: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/17.jpg)
Xen Network IO Hook (3)
Layer-2 hook vs. Layer-3 hook mac address permanent while ip address maybe
dynamic (DHCP) ARP packet to Dom0 cannot be hooked in IP Layer
• proxy ARP & ARP spoof
easy to cooperate with Open vSwitch
![Page 18: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/18.jpg)
Data Handling (1)
Where to handle these hooked data? Dom0 one dedicated PV domain, “DomX” [√]
Data transfer between domains TCP/IP socket transmit? memory sharing? [√]
• event notification?• synchronization?
![Page 19: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/19.jpg)
Data Handling (2)
Difference in filesystem hook & network hook Filesystem hook
• Domain U <-> share memory <-> Domain X
Network hook• Domain 0 <-> share memory <-> Domain X
![Page 20: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/20.jpg)
Data Handling (3)
Memory sharing between 2 domains grant table provided by Xen
• allocate page & grant reference id on initiator side• map grant reference id on other side• who should be initiator?
alternative channel organization• place metadata & data in the channel• place metadata in the channel while put data out-band
![Page 21: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/21.jpg)
Data Handling (4)
Event notification between 2 domains event channel provided by Xen
• similar as POSIX signal• local port <-> remote port• bind local port with one virtual irq handler• initialization
1. where to keep remote domid & port? xenstore
• when to trigger virtual irq handler?
domain switch to -> ret_from_intr -> test_all_events -> event_do_upcall -> virtual irq handler (Xen-3.4.0)
![Page 22: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/22.jpg)
Data Handling (5)
Memory access sync between 2 domains shared memory organized as ring-buffer xen/include/public/io/ring.h (xen-4.0.1)
• one reader & one writer• memory barrier
filesystem hook• one reader & multiple writer
![Page 23: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/23.jpg)
Xen Programming Interface
Xen hypercall similar as Linux system call
• event channel• grant table• domain control• …
Linux wrapper interfaces trap Guest OS kernel to Xen hypervisor
• normal kernel routines may trap to Xen hypervisorschedule -> update_rq_clock -> native_read_tsc -> “rdtsc” -> invalid op exception -> trap into Xen (linux-2.6.24-29-xen)
![Page 24: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/24.jpg)
Potential Issue
PV driver in HVM PCI through VMDq …
![Page 25: Linux kernel 3.0 release - cnblogs.com...similar as Linux system call • event channel • grant table • domain control • … Linux wrapper interfaces trap Guest OS kernel to](https://reader035.vdocuments.us/reader035/viewer/2022080718/5f7856bb69770d5f4a733067/html5/thumbnails/25.jpg)
Q & A