an introduction to the linux kernel and device drives (ntu 2016.03)
TRANSCRIPT
Note: The Copyrights of the referenced materials and photos go to their original authors. As a result, this slide is for personal reference only.
For the contents created in this document, the Copyright belongs to William W.-Y. Liang. © 2005-2016 All Rights Reserved.
NTU CSIE Open Source System Software
2016.03.22
An Introduction to the Linux Kernel with Device Drivers
William W.-Y. Liang (梁文耀), Ph. D.
http://www.ntut.edu.tw/~wyliang
for 台大資工系開源系統軟體課程
hosted by Prof. Shih-Hao Hung
© 2016 William W.-Y. Liang, All Rights Reserved.
General-purpose Operating Systems
Characteristics
Development of applications can be logically separated from hardware
Complete software stacks and middleware can be created to support easy
development of complex and versatile applications.
Example:
Linux (with many Distributions), Android, Chrome OS, Firefox OS,
Brillo, uClinux, MacOS, iOS, Windows, etc.
Application development
System independent application development: General applications,
GUI/Window programming, Web programming
System dependent software development: System programming, Device
drivers, Protocol software
2 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
The Linux Operating System
Open source and free
Maintained by numerous volunteer experts and hackers
Robust, efficient, and reliable
Modular, configurable, and scalable
Tons of applications available
Suitable for application types from low-end embedded systems, mid-range consumer
products, personal computer, and up to the high performance supercomputers
3 NTU OSSSP 2016: Linux Kernel
https://en.wikipedia.org/wiki/Linux_kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Some References
Linux Kernel Wiki
https://en.wikipedia.org/wiki/Linux_kernel
Linux Kernel Source
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Reference books for Beginners
Linux Device Driver, 3rd Edition, by Jonathan Corbet, Alessandro
Rubini and Greg Kroah-Hartman, O'Reilly (Also available online
https://lwn.net/Kernel/LDD3/)
Linux Kernel in a NutShell, by Greg Kroah-Hartman, O'Reilly (Also
available online http://www.kroah.com/lkn/)
Understanding the Linux Kernel, 3rd Edition, by Daniel P. Bovet, Marco
Cesati, O'Reilly
4 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Chitchat about Linux and the Kernel
5 NTU OSSSP 2016: Linux Kernel
* Taiwanese slang :^)
© 2016 William W.-Y. Liang, All Rights Reserved.
A Systems View -- from Hardware to Software
Hardware
Software development environment
Tool chain and Library
Boot loader
OS Kernel / RTOS
Middleware*
Applications
6 NTU OSSSP 2016: Linux Kernel
* Example: In Android, the
middleware is called the
Application Framework.
Source: Qing Li and Caroline Yao, “real-time concepts for embedded systems”
Embedded System Development Environment Example
© 2016 William W.-Y. Liang, All Rights Reserved.
The Episode: System Booting
Source: Qing Li and Caroline Yao, “real-time concepts for embedded systems”
7 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Boot Sequence Example
8 NTU OSSSP 2016: Linux Kernel
http://www.at91.com/linux4sam/bin/view/Linux4SAM/GettingStarted
© 2016 William W.-Y. Liang, All Rights Reserved.
Booting the Kernel – Before the C code
9 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Booting the Kernel – Entering the C World
10 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Prepare for Tracing the Kernel
Tools, always the most important thing
git, repo, vim, ctags, grep, find, etc.
Basics for understanding the Kernel
Data structures
11 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Introduction to the Linux Kernel
12 NTU OSSSP 2016: Linux Kernel
Let’s try to understand the Linux
kernel by introducing the basics
for the device drivers and the
kernel features, and discussing
on some related issues.
© 2016 William W.-Y. Liang, All Rights Reserved.
Software View from User Program to Hardware
For a general purpose OS
such as Linux
User level (user space)
Coding
Compilation
Linking
Execution
Kernel (kernel space)
Program loading
System calls (API)
Device drivers
Hardware manipulation
Bare metal hardware
13 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Device Drivers in Linux
Device drivers are usually treated as black boxes for
the application developers.
They resemble as a software layer
lying between the applications and
the actual devices.
14 NTU OSSSP 2016: Linux Kernel
http: //www.ni.com/tutorial/3789/en/
© 2016 William W.-Y. Liang, All Rights Reserved.
Device Drivers in Linux Kernel
15 NTU OSSSP 2016: Linux Kernel
ALESSANDRO RUBINI and JONATHAN CORBET, Linux Device Drivers, Second Edition
© 2016 William W.-Y. Liang, All Rights Reserved.
Major Classes of Devices for Linux
Character devices
A character (char) device is one that can be accessed as a stream
of bytes (just like a normal file).
Block devices
A block device is accessed in a unit of fixed-size block.
The device can be accessed randomly.
Network interfaces
Others: File Systems, Protocol Stack, USB, PCI, etc.
Refers to <kernel-source>/Documentation/devices.txt”
16 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Device Files
File Operations vs. Device Accesses
Device Initialization vs. File Open
Device Accesses vs. File Read/Write
Device Control vs. File Control
Device Termination vs. File Close
UNIX/Linux systems implement device files to enable
users to employ familiar commands and functions such
as open, read, write, and close when working with
some peripherals.
17 NTU OSSSP 2016: Linux Kernel
int open( char *pathname, int flags, … );
int read( int fd, void *buf, size_t count );
int write( int fd, void *buf, size_t count );
int ioctl(struct inode *, struct file *, unsigned int, unsigned long );
int close( int fd );
© 2016 William W.-Y. Liang, All Rights Reserved.
How Device File and Driver Modules Work?
18 NTU OSSSP 2016: Linux Kernel
☼ Discussion: taking “Hello, World” as an example
© 2016 William W.-Y. Liang, All Rights Reserved.
Example: Applications using a Device
19 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Device Number
Example crw-rw-rw- 1 root root 1, 3 Mar 2 2015 null
crw-rw-rw- 1 root root 1, 5 Mar 2 2015 zero
crw------- 1 root root 4, 1 Mar 2 2015 tty1
crw-rw-rw- 1 root tty 4, 64 Mar 2 2015 ttys0
crw-rw---- 1 root uucp 4, 65 Mar 2 2015 ttyS1
crw------- 1 root root 10, 1 Mar 2 2015 psaux
crw-rw-rw- 1 root root 123, 0 Mar 2 2015 androint
A System Administrator may create device files with the “mknod” command. For example,
mknod /dev/androint c 123 0
‘/dev/androint’ is the file’s pathname, ‘c’ indicates that it’s a character-mode device, 123 is its (unique) ‘major number’, and 0 is its ‘minor number’.
20 NTU OSSSP 2016: Linux Kernel
☼ Discussion: relationship between device files and drivers
© 2016 William W.-Y. Liang, All Rights Reserved.
Loadable Kernel Modules
A great mechanism for OS ‘extensibility’
No need to recompile and then reboot
A kernel module differs from a normal C program
A kernel module cannot call any of the familiar
functions from the standard C runtime libraries
A module will be run as part of the kernel
Any ‘bug’ can cause a system malfunction or a
complete crash!
21 NTU OSSSP 2016: Linux Kernel
☼ Discussion: GPL issues with the Linux kernel modules
© 2016 William W.-Y. Liang, All Rights Reserved.
Example
22 NTU OSSSP 2016: Linux Kernel
☼ Discussion: how multiple program entries work?
© 2016 William W.-Y. Liang, All Rights Reserved.
How to get a module to work in Kernel
23 NTU OSSSP 2016: Linux Kernel
ALESSANDRO RUBINI and JONATHAN CORBET, Linux Device Drivers, Second Edition
Driver Functions
© 2016 William W.-Y. Liang, All Rights Reserved.
Example: Device Registration
24 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Driver Methods -- File Operations
The kernel uses the file_operations structure to access the driver’s methods.
int (*open) (struct inode *, struct file *);
ssize_t (*read) (struct file *, char *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char *, size_t, loff_t *);
int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long);
int (*release) (struct inode *, struct file *);
…
25 NTU OSSSP 2016: Linux Kernel
☼ Discussion: drivers vs. the object-orient concept
© 2016 William W.-Y. Liang, All Rights Reserved.
Kernel and User Space
26 NTU OSSSP 2016: Linux Kernel
☼ Discussion: address space for kernel and processes
© 2016 William W.-Y. Liang, All Rights Reserved.
Virtual to Physical Space
27 NTU OSSSP 2016: Linux Kernel
https://en.wikipedia.org/wiki/Operating_system
☼ Discussion: how the memory space is utilized?
ALESSANDRO RUBINI and JONATHAN CORBET, Linux Device Drivers, Second Edition
© 2016 William W.-Y. Liang, All Rights Reserved.
Data Transfer between User and Kernel Space
28 NTU OSSSP 2016: Linux Kernel
Read: kernel->user
Write: user->kernel
☼ Discussion: issues for cross space data accesses
ALESSANDRO RUBINI and JONATHAN CORBET, Linux Device Drivers, Second Edition ,
© 2016 William W.-Y. Liang, All Rights Reserved.
Example: User & Kernel Space Data Transfer
29 NTU OSSSP 2016: Linux Kernel
© 2016 William W.-Y. Liang, All Rights Reserved.
Kernel Memory Allocation
Small size allocation
kmalloc and kfree dptr->data = kmalloc(qset * sizeof(char *), GFP_KERNEL);
memset(dptr->data, 0, qset * sizeof(char *));
Limitations
Large size allocation
vmalloc and vfree
Restrictions
Page allocation
Slab and memory pool
30 NTU OSSSP 2016: Linux Kernel
☼ Discussion: performance issues in memory management
© 2016 William W.-Y. Liang, All Rights Reserved.
Multi-tasking: the Central Part of the OS
Process states in OSes
Running: own CPU
Ready: To be scheduled,
in ready queue (run-queue)
Waiting:
Pending on I/O or Event
Delay itself for some duration
Request a resource but is not yet available
Linux Task States
Tasks: user-process, user-thread, kernel thread
States: TASK_RUNNING, TASK_INTERRUPTIBLE, etc.
Scheduling Policy
SCHED_OTHER, SCHED_FIFO, SCHED_RR, etc
31 NTU OSSSP 2016: Linux Kernel
☼ Discussion: scheduling and preemption issues
© 2016 William W.-Y. Liang, All Rights Reserved.
Concurrency Problems
Problems: Data inconsistency, Deadlock
Race Condition: Occurs when multiple tasks want to
access the shared resources, such as shared data or
hardware devices.
32 NTU OSSSP 2016: Linux Kernel
☼ Discussion: what’s the exact cause?
© 2016 William W.-Y. Liang, All Rights Reserved.
Thread Safe
Potential Causes of Thread-Unsafe Problems
Global variables
Static variables
Indirect accesses
Solutions
Reentrancy
Atomic operations
Thread-local storage
Mutual exclusion
33 NTU OSSSP 2016: Linux Kernel
Concurrency in Kernel and Device Drivers
Driver code vs. Tasks
Single Core vs. Multicore
IRQ impact
Soft-IRQ
☼ Discussion: performance issues for synchronizations
© 2016 William W.-Y. Liang, All Rights Reserved.
Synchronization Operations
Semaphore down(), up(), etc.
Mutex mutex_lock(), mutex_unlock(), etc.
Spin locks spin_lock(), spin_unlock(), etc.
Atomic operations
atomic_set(), atomic_add(), etc
Bit operations
set_bit(), clear_bit(), etc.
34 NTU OSSSP 2016: Linux Kernel
☼ Discussion: semaphore vs. spinlock, usage & performance
© 2016 William W.-Y. Liang, All Rights Reserved.
Interrupt Handling
35 NTU OSSSP 2016: Linux Kernel
☼ Discussion: cost and performance issues
of IRQ handling and solutions
© 2016 William W.-Y. Liang, All Rights Reserved.
Task Blocking and Waiting Queue
Processes are usually blocked due to I/O waiting.
No data available for read, or no space
available for write or device busy
Waiting queues
Blocked task is put into a waiting queue
before it is woke up.
36 NTU OSSSP 2016: Linux Kernel Linux Device Drivers, Second Edition
☼ Discussion: how it works?
Linux Device Drivers, Second Edition
© 2016 William W.-Y. Liang, All Rights Reserved.
The Endless Journey on Discovering Linux
To be Continued…
This brief introductory lecture is just a beginning.
Still many interesting things out there
Take it as interesting adventure, by learning from the
wonderful open source world!
37 NTU OSSSP 2016: Linux Kernel
https://en.wikipedia.org/wiki/Linux_kernel#/media/File:Linux_kernel_map.png
Let’s keep the discussion next week
on the next topic – Android Framework.
Note: The Copyrights of the referenced materials and photos go to their original authors. As a result, this slide is for personal reference only.
For the contents created in this document, the Copyright belongs to William W.-Y. Liang. © 2005-2016 All Rights Reserved.
Q & A
http://www.ntut.edu.tw/~wyliang
http://www.facebook.com/william.wyliang