spring 2014 silicon valley university confidential 1 introduction to embedded systems dr. jerry...

Spring 2014SILICON VALLEY UNIVERSITY

CONFIDENTIAL 1

Introduction to Embedded Systems

Dr. Jerry Shiao, Silicon Valley University

SILICON VALLEY UNIVERSITY CONFIDENTIAL 2Spring 2014

Section 5 Kernel Internals Kernel Modules

Functionality Object file that extends the running Kernel without rebuilding and

rebooting Kernel. Loaded and unloaded upon demand. Drivers to support new hardware or filesystems. Support system calls.

Module Organization Invoked by user application needing its service. Does not execute

on its own. Allows application communicate to kernel:

proc file system (i.e. /proc). System calls and ioctl calls.

- Process Control (i.e. load/execute/create/terminate process, allocate/free memory.

- File management (i.e. create/delete/open/close/read/write file).- Device Management (i.e. request/release/read/write device).- Information Maintenance (i.e. get/set system data, get/set device attributes).

module_init() and module_exit(). Module installation (Insmod, modprobe), removal (rmmod), list

(lsmod).


Section 5 Kernel Internals Kernel Modules

Without Loadable Kernel Modules Need large, monolithic kernel for all possible functionality.

Kernel code memory not fragmented. LKM load/unload fragments Kernel code memory.

Rebuild and reboot required for new functionality. Waste memory on unused modules. NOTE: With embedded systems, hardware will not change. During

embedded system development, modules are loadable and for production, modules are made static and placed in the kernel.

License Issues LKM are derived work of the Kernel. “Tainted” Kernel when non-GPL LKM loaded (code not available to public).

MODULE_LICENSE macro for GPL. EXPORT_SYMBOL_GPL macro resolves ONLY for GPL modules.

Configuration .config file.

CONFIG_INSMOD=y CONFIG_RMMOD=y CONFIG_LSMOD=y

/lib/modules ( extension “.ko” ).


Section 5 Kernel Internals

Kernel ModulesLoading Modules Into the Kernel.

kmod subsystem:

execs modprobe

Kernel Module:

request_module() System call

/etc/modprobe.conf-- Aliase to module.…alias eth0 e100

/etc/modules/<version>/modules.dep-- Dependency before loading.…/lib/modules/…/e100.ko: /lib/modules/…/mii.ko/lib/modules/…/ext2.ko: /lib/modules/…/mbcache.ko

Kernel Initialization (/etc/rc.d scripts):depmod –a-- Create dependency file to be used by modprobe.

Insmod:



Kernel Modules System Call – Explicit request to the kernel made through a software interrupt.

Service provided in the kernel, cross the user-space / kernel boundary.

Linux C library – Provides the wrapper routines to issue a system call.

main() {

…

xyz()

}

User Application

xyz() {

…

“SYSCALL”

}

Linux C library

Wrapper routine

User Mode

system_call()

…

sys_call_table[] -> sys_xyz()

…

“SYSEXIT”

sys_xyz() {

…

}

System Call Handler System Call Service Routine

Load arguments:

1) Register eax = system call index.

2) Software Interrupt (Interrupt 0x80).

Kernel Mode



Kernel Modules#include <linux/module.h>#include <linux/kernel.h>…MODULE_LICENSE(“GPL”);MODULE_DESCRIPTION(“Mytime Module”);MODULE_AUTHOR(“Linux:Embedded Class (SVU).…Int init_mytime_module( void ){ printk(KERN_ALERT "Hello, world\n");… return ret;} /* init_mytime_module */

Int cleanup_mytime_module( void ){… return ret;} /* cleanup_my_time_module */

module_init( init_mytime_module );module_exit( cleanup_mytime_module);

Required for all modules. Defines for MODULE_XXX macros. Contained in ELF header in .modinfo section.

Identify whether the module is GPL.

NOTE: EXPORT_SYMBOL_GPL() only available to modules with GPL compatible license.

Describes what module does.

Author of the module.

Kernel module initialization entry point. Modules only in kernel space. Called by insmod.

Kernel module exit entry point. Called by rmmod.


Section 5 Kernel Internals Linux Device Drivers

Provides access to hardware devices, hiding details through set of standardized APIs independent to specific driver.

Drivers provide APIs through system calls or through ioctl for the user to access the actual hardware device.

Independent modules separate from Kernel. Dynamic Loadable Module linkable and removable at runtime. /dev directory contains all hardware devices accessable through file

system abstraction (required before device driver can access). Device driver catagories.

Character Drivers. Does not need buffering, byte I/O. Accessed as stream of bytes. Do not need fixed block size.

/dev/console, /dev/ttyS0 Block Drivers.

Buffers to respond to requests. Send and receive through blocks. The block size is determined by driver module. Usually a filesystem (i.e. RAMDisk, USB stick).

Network Drivers. Interface is hardware device for transmit/receive packets, but could be software. Network subsystem, not in /dev.

Accessed through name (i.e. eth0).



Linux Device Drivers


Section 5 Kernel Internals Linux Device Drivers Major number used by kernel to identify device driver. Minor number used by device driver to distinguish devices of the

same type. Hard drive multiple partitions, each partition minor number. Multiple serial ports.

mknod command creates the device file in /dev. Udev manages /dev directory.

Automatically selects the major numbers. Prevents clutter of /dev with unused devices.

Only present devices in /dev. Linux Assigned Name and Number Authority (LANANA) to get major

number. /sys initially created to perform driver level debugging out of /proc.

/sys became useful to other subsystems. Contain exported information about devices and drivers in user space.

/sys/devices are all devices in system. /sys/class shows devices grouped according to classes. /sys/block contains the block devices.



Linux Char Device Driver Components#include <linux/module.h>#include <linux/kernel.h>…MODULE_LICENSE(“GPL”);MODULE_DESCRIPTION(“Device Driver”);MODULE_AUTHOR(“Linux:Embedded Class (SVU).…Int init_device_driver( void ){ major = register_chrdev(major, “foo”, &fops);…} /* init_device_driver */

Int cleanup_device_driver( void ){ unregister_chrdev(major, “foo”);…} /* cleanup_device_driver */

module_init( init_device_driver );module_exit( cleanup_device_driver);

Registers device driver for character device. “fops” is structure with callback functions of file operations for this device.

struct file_operations fops = { .owner = THIS_MODULE, .open = <device_open_cb>, .release = <device_release _cb>, .read = <device_read_cb>, .write = <device_write_cb>, .ioctl = <device _ioctl_cb>}

Unregisters device driver by releasing the major number for character device, “foo”.

Character device created using:#mknod /dev/foo c <major> 0


Section 5 Kernel Internals Linux Block Device Driver Components…MODULE_LICENSE(“GPL”);…Int init_device_driver( void ){ major = register_blkdev(major, “foo”); foo.gd = alloc_disk( 16 ); foo.gd->major = major; foo.gd->minors = 16; set_capacity(foo.gd, <disk_size_in_sectors>); strcpy(foo.gd->disk_name, “foo”); foo.gd->fops = &foo_ops;…} /* init_device_driver */

Int cleanup_device_driver( void ){ unregister_blkdev(major, “foo”);…} /* cleanup_device_driver */

module_init( init_device_driver );

module_exit( cleanup_device_driver);

Registers device driver for block device. “foo_ops” is structure with callback functions of file operations for this device.

struct file_operations foo_ops = { .owner = THIS_MODULE, .open = <device_open_cb>, .release = <device_release _cb>, .read = <device_read_cb>, .write = <device_write_cb>, .ioctl = <device _ioctl_cb>}

Unregisters device driver by releasing the major number for block device, “foo”.

Block device created using:#mknod /dev/foo b <major> 0

Gendisk structure created, with 15 partition descriptors.Initialize gendisk structure capacity.


Section 5 Kernel Internals Assignment 5: Kernel Modules

/proc file system is used to store many system configuration parameters.

Use the /proc file system to return the number of clock ticks since the system initialized.

Kernel module to create /proc file structure (struct proc_dir_entry) with file system read/write APIs.

Kernel variable jiffies represents the number of clock ticks since system initialized.

Kernel variable HZ represents the clock tick rate, the frequency of clock ticks per second (i.e. HZ = 250 means hardware will have 250 ticks/second or 4 milliseconds per tick.

User application use system calls (i.e. read/write) to interface with kernel module.

Insmod installs loadable kernel module into running kernel.


Section 6 Kernel Scheduling

Kernel processes heart of the Linux system. Process and files are fundamental concept in

Unix Operating System. Process is a program in execution. Consists of:

Executing program code. Set of resources (i.e. open files). Internal kernel data (i.e. struct task_struct). Address space. One or more threads of execution. Data section (i.e. global variables).


Section 6 Kernel Scheduling Linux multitasking operating system (tasks

running concurrently). Process is an independently running program that has its own set of

resources (i.e. output file). Process managed with process descriptors (struct task_struct) in

circular doubly-linked list. Use light weight processes or threads to support multithreaded

operation. Light weight processes share same address space with each other

and parent process. Communication simple. Heavy weight processes contains its own address space.

Communication through IPCs.

Linux process state: TASK_RUNNING: Executing in CPU or waiting. TASK_INTERRUPIBLE: Suspended waiting on some condition. TASK_UNINTERRUPTIBLE: Suspended until a given event. TASK_STOPPED: Stopped from receiving signal. TASK_ZOMBIES: Child process terminated, but not waited on by its

parent. Does not release its process ID.



ps (Process Status) command:

[sau@localhost ~]$ ps –f

UID PID PPID STIME TTY TIME CMD

…

hermie 24 1 00:35:28 pts/1 0:01 bash

hermie 137 24 00:36:39 pts/1 0:00 ps –f

…

[sau@localhost ~]$ ps –ax

PID TTY STAT TIME COMMAND

…

181 pts/1 S 0:01 -bash

231 ? T 0:07 emacs

274 pts/1 R 0:00 ps

…

PID – Process ID (used in command “kill”).

TTY – Controlling terminal or running from a shell.

STAT – State. S = Shell is suspended.

T = Running, but suspended.

R = Running.

TIME – CPU time used so far.

COMMAND – Command running.

UID – Owning user.

PID – Process ID (used in command “kill”).

PPID – Parent process ID.

STIME – When the task started.

TTY – Controlling terminal or running from a shell.

STAT – State. S = Shell is suspended.

T = Running, but suspended.

R = Running.

TIME – CPU time used so far.

CMD – Command running.


Section 6 Kernel Scheduling Process Scheduling

Scheduler responsible for best utilizing the processor time by deciding which processor to run next from a set of runnable processes.

Scheduling gives impression that multiple processes are executing concurrently.

Multitasking operating systems has two options: Preemptive multitasking.

Process involuntary suspended. Scheduler decides when process is context switched through time

slice. Scheduler manages the time slice, prevents process monopolizing

CPU. Interrupts will cause context switch.

Cooperative multitasking. Process must voluntarily suspend (i.e. driver’s responsibility). Scheduler does not decide when process is switched. Process can monopolize CPU, hang the system. Group Scheduling by userid or groups of processes to not starve

processes. Real time systems are multitasked with key activities taking a greater share

of process time.


Section 6 Kernel Scheduling Scheduling Policy

Objectives: Set of rules determining when and how a process is to run.

Fast process response time, good throughput background jobs, avoid process starvation, reconcilitate low and high priority processes.

Time sharing principle based on timer interrupts. Each process has time slice or quantum. When quantum expires,

context switch occurs (process appear to run concurrently). Epoch is time when all runnable processes has exceeded their

quantum. Timer interrupts used by Linux to measure process quantum.

Quantum Linux 2.4: 210ms, Linux 2.6: 100ms. Linux dynamically changes process priority.

Priority adjusted higher for releasing CPU before quantum (I/O-bound process).

Priority adjusted lower for using entire quantum (CPU-bound process).

Process dynamically changes its own priority: nice() and setpriority(). Device driver executing I/O and expects to take a long time.



Scheduling Policy Scheduling Algorithmn

o Epoch is division of CPU time or a period of time

o Every process has a maximum time quantum computed at beginning of each epoch.

o When process waiting for I/O its quantum suspended and can be used again in the epoch until its fully used.

o Epoch ends when all runnable processes have used all of their quantum.

Process_3

Quantum_3

epoch

Process_2

Quantum_2Process_1

Quantum_1

Process_3

Quantum_3Process_1

Quantum_1

Process_1

Quantum_1



Process Priority Static priorities: Real Time processes. Range 1 to 99.

Always higher than dynamic priority of other processes. Dynamic priorities: Other processes. I/O and CPU bound

based on time used. Sum of the base time quantum and the number of “clock ticks” left in the current epoch.

I/O-bound priority bonus ( -5 ). CPU-bound priority bonus ( +5 ).

Process Quantum Process assigned at least base time quantum. Process does not use its entire quantum, bonus calculated

from unused quantum carried over into the next epoch. Increased quantum used to calculate priority in next epoch.

When a new child process is created, the parent’s quantum is split in two halves, one for the parent and one for the child.



Process Priority Normal process static priority range 100 – 139 New process inherits static priority of its parent. Dynamic priorities: Process either I/O and CPU bound based

on time used or the average sleep time. Used by scheduler to select the next process to run. Max(100, min (static priority – bonus + 5, 139)) Bonus determined by the average sleep time.

Sleep 0-99 msec, bonus = 0. Sleep 100-199 msec, bonus = 1. … Sleep 900-1000 msec, bonus = 10

.Highest process priority: max( 100, min( 100-10+5, 139 ) ) = 100. Lowest process priority: max( 100, min( 100-0+5, 139 ) ) = 139. CPU-bound priority bonus ( static priority – 0 + 5 ) is +5. I/O-bound priority bonus ( static priority – 10 + 5 ) is -5.

Real-Time process static priority range 1 – 99. Always higher than dynamic priority of normal processes.



Process Classification: Interactive Process. Interactive processes are I/O-bound, spending a lot of

time waiting for I/O operations. Interactive processes waiting for human input.

Response time must be quick (latency 50-150ms). Command shells, text editors, graphical

applications, mouse and keyboard processes. Processes heavily utilizing I/O (i.e. hard drive, CD).

I/O bound process “blocked”, pending arrival of data, will receive interrupt when data has arrived and continue.

Higher priority access to CPU. Scheduled multiple times within epoch. CPU released

before quantum completes.



Scheduling Policy Process Classification: Batch Processes.

Batch processes are CPU-bound, heavily ultilize CPU time for computations.

Batch processes typically executing in background, i.e. compilers, database search engines, and scientific computiations.

No user interactions. Lower priority access to CPU. Preempted by I/O-bound process. Scheduled less than I/O-bound process within

epoch. CPU usually ultilized up to quantum.



Scheduling Policy Process Classification: Real-time Processes.

Real-time processes highest priority, always higher than I/O or Batch processes.

“Hard” real-time require bounded response time (absolute deadline must be meet to guarantee deterministic behavior).

Linux is “soft” real-time. Allows time tolerance when event occurs in effort to minimize latency. Internal latency cannot guarantee deterministic behavior.

Streaming video/audio, robot controllers, programs collecting data from physical sensors.


Section 6 Kernel Scheduling Linux 2.6 Scheduler Algorithm

Runqueues Keep track of all runnable tasks for a CPU. Two priority arrays, active and expired array.

Tasks at same priority are RR scheduled Dynamic priority changes based on CPU usage.

Special threads: idle thread and migration thread. runqueue structure

*curr: Pointer to the currently running task. *idle: Pointer to the CPU’s idle task. cpu_load: CPU load info used by Migration thread to balance CPU

loads by scheduling threads to other CPUs. *active: Pointer to active priority array containing tasks with unexpired

quantums. *expired: Pointer to expired priority array containing tasks with used up

quantums. arrays[]: Actual active and expired priority arrays. *migration_thread: Thread that handles task migrations. Every 200 ms,

checks to see whether the CPU loads are unbalanced.



Priority Queues Priority arrays are the data structure that provide O(1)

scheduling. 140 priority levels. Lower the value, higher the priority. Two priority-ordered arrays (active and expired) per CPU

managed by runqueue structure. Transition between active and expired handled by moving pointers

in runqueue structure. When each task's quantum reaches zero, its quantum is

recalculated before it is moved to the expired array. Multiple tasks at same priority scheduled round robin.

Active/Expired Array elements linked list of task_structs. queue structure.

bitmap[]: Bitmap represents the priorities for which active tasks exist in the priority array.

*list_head_queue[]: Array of linked lists. One list in the array for each priority level (140 priority levels).



Linux 2.6 Scheduler Algorithm runqueue per CPU.

runqueue

Migration Thread

. . .Active Task 1 Task 2 Task N Priority: 1

Task 3 Task MTask 4 Priority: 140

Expired Task 5 Task 6 Task X Priority: 1

Task 7 Task ZTask 68

. . .

Priority: 140

. . .

1 0 0 . . . 0 0 1

140 . . . 3 2 1Priority

Bitmap



Linux 2.6.23 CFS (Completely Fair Scheduler) Fair queuing algorithm. Aim to maximize overall CPU utilization, especially

interactive performance. Uses Red-Black tree, instead of runqueues.

Red-Black tree implements “time-line” of future task execution.

Spent processor time as key, efficiently finding process with least amount of time (leftmost node).

Nanosecond granularity does not need dynamic +- bonus priority adjustments.

Choosing process is O(1). Inserting process is O(log N), because of RB tree.

Uses same concept of rewarding sleeping or waiting processes.



schedule() Kernel Function Implements the Linux scheduler, finds a process in the

runqueue and assigns CPU. Direct Invocation

Current process is blocking (sleeping), waiting for resource. Current process state changed to

TASK_INTERRUPTIBLE or TASK_UNINTERRUPTIBLE and placed in wait queue.

schedule() called. Resource becomes available, process removed from wait

queue. Device driver checks for long iterations, and schedules if too

long. Lazy Invocation

Current process used up its quantum. Process added to the runqueue with higher priority than

currently executing process (i.e. wake_up_process()). Process calls sched_yield() to release CPU.



schedule() Kernel Function Handles Linux internal “ housekeeping” tasks (i.e.

accounting, maintaining system uptime). Scans the runqueue for highest priority process.

If all processes used up quantum, start new epoch. Assign new quantum for all processes.

Found process with higher priority than current process. Scheduler performs context switch.

Real-time process: Current process state is SCHED_RR and quantum used up, provide new quantum and place at end of runqueue.

No runnable process in runqueue, for multi-processor system, attempt load balancing.

Interrupt handler bottom-half processing.



Scheduling Classes SCHED_FIFO: Real-time FIFO scheduling preempts other

processes. Does not have quantum. Runs until: Process completes. Blocked by an I/O call. Higher priority SCHED_FIFO process becomes runnable. sched_yield() system calls. Group scheduling reserves minimum time for other processes

(prevent FIFO process locking CPU). SCHED_RR: Real-time RR scheduling similar with

SCHED_FIFO, but with quantums and can be preempted by SCHED_FIFO process.

SCHED_NORMAL: Conventional time-shared process. Non-Realtime.

SCHED_BATCH: Similar to SCHED_NORMAL, but process cannot be considered interactive process (quantum stays the same).

Always background process (i.e. tape backup process).

spring 2014 silicon valley university confidential 1 introduction to embedded systems dr. jerry...

Documents

monolithic kernel

running kernel

tainted kernel

rebooting kernel

loadable kernel modules

module system

kernel mode slide

execs modprobe kernel