linux driver writing - itcourseware...companies, names and data used in examples herein are...

Object Innovations Course 314

Student Guide Revision 1.0

Linux Driver Writing

Rev. 1.0 Copyright ©2001 Object Innovations, Inc. ii All Rights Reserved

Linux Driver Writing Student Guide Information in this document is subject to change without notice. Companies, names and data used in examples herein are fictitious unless otherwise noted. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without the express written permission of Object Innovations. Product and company names mentioned herein are the trademarks or registered trademarks of their respective owners. Copyright ©2001 Object Innovations, Inc. All rights reserved. Object Innovations, Inc. 420 Boston Turnpike Shrewsbury, MA 01545 (508) 845-1195 www.ObjectInnovations.com Printed in the United States of America.

Rev. 1.0 Copyright ©2001 Object Innovations, Inc. iii All Rights Reserved

Table of Contents

Chapter 1 Introduction to Linux Driver Development Chapter 2 Device Drivers Chapter 3 Linux Kernel Facilities Chapter 4 Modules Chapter 5 Character Devices Chapter 6 Hardware Aspects Chapter 7 Block Drivers Chapter 8 Network Chapter 9 Network Devices Chapter 10 SCSI Subsystem Chapter 11 Device Driver Debugging Appendix A Learning Resources Appendix B Data Structures Appendix C Labs Appendix D Solutions


Rev. 1.0 Copyright © 2001 Object Innovations, Inc. 1-1 All Rights Reserved

Chapter 1

Introduction to Linux Driver Writing



Introduction to Linux Driver Writing

Objectives

After completing this unit you will be able to:

• Setup the environment that will be used for Kernel development.

• Understand the version numbering scheme used for the Linux Kernel, along with the associated compatibility issues.

• Identify the main components of the Linux Kernel.

• Identify the steps involved in the process of developing a driver for the Linux operating system.

• Have an understanding of the basic performance, stability and security issues involved in device driver development.



Environmental Setup

• The following describes what software components we will need in this course.

• PCs running Linux:

− Any computer on which you have access to a shell and a compiler could theoretically serve the purpose of Kernel development.

− More speed essentially means a shorter compile time as well as faster execution and debugging.

− Having access to recent technologies (Pentium for example) allows you to optimize the Kernel for this specific subset of the Intel architecture. Kernels optimized for another Intel processor will not work on other (older) version of the Intel processor.

• GNU cc compiler:

− This compiler is called on the command line with the gcc command.

− The quality of the compiler depends on the architecture used. For PCs, gcc competes with commercial grade compilers. The results may vary depending on which architecture you are developing for.

− The Linux Kernel is gcc-dependent. Thus, no other compiler could be used in order to build a working version of the Linux Kernel.



Environmental Setup (Cont’d)

• Bash command interpreter

− The Bash shell is the most commonly used and distributed shell on the Linux Operating System. It is greatly similar to the Bourne shell, on which its design was based.

− We will use exclusively the Bash shell in the examples and exercises provided in this course. Feel free to use any shell that you like for Kernel development. However, keep in mind that most scripts included in the Kernel rely on Bash.

• These software components are usually provided by the distribution with which Linux was installed on your computer.

− This course will focus on the RedHat 7.0 distribution, which provides all the tools needed for Kernel development.

− Other distributions may be used as well, as long as they contain the components listed earlier.

− We will also focus in the course on the Intel x86 architecture. However, most aspects of this course may also be applied to other architectures.



Kernel Versions

• Linux kernel versions are divided in two series, which are downloadable separately:

− Experimental (odd series, e.g. 2.3.x). These are fast moving versions that are used to test new features, updates, device drivers, etc. By their own nature the experimental kernels may behave in unpredictable ways, so one may experience data losses, random machine lockups, corrupted file systems, etc., depending on the parts of the Kernel which are being used.

− Production (even series, e.g. 1.2.xx, 2.0.xx, 2.2.xx or the new 2.4.x). Production kernels are known for being very stable, they have almost no known bugs, and problems like file system corruption and system crashes are very unlikely to occur. These are always the ones that are installed with Linux distributions.

• This course will focus on the 2.4.x production series.

− The 2.4.0 version was available during the development of this course, but everything should also be applicable to subsequent versions of the Linux Kernel.



Components of the Kernel

• The Kernel is delimited under specific subdirectories:

− arch/ - Contains architecture-specific code, like assembly files (.S).

− drivers/ - Contains the code for each device drivers supported directly by the Kernel.

− fs/ - The actual implementations of the various supported file systems (ext2, fat, hpfs, etc) are placed in this directory.

− include/ - Header files associated with most .c files in the other directories.

− init/ - Contains the main.c file for the initialization of the Kernel.

− kernel/ - Contains core Kernel functions, which are used in other parts of the Kernel (for example: printk()and panic())

− lib/ - Various library routines. Some are deprecated and not used anymore.

− mm/ - Memory management routines

− net/ - Implementation of the various network protocols available on Linux.

− scripts/ - Contains some Kernel configuration scripts (menuconfig, xconfig) and debugging programs.



Components of the Kernel (Cont'd)

• The following is a list of the major components of the Linux Kernel along with their approximate weight in the source tree:

Size Directory Files

90Mb /usr/src/linux 7645

4.5 Mb Documentation 380

16.5 Mb arch 1685

54 Mb drivers 2256

5.6 Mb fs 489

14.2 Mb include 2262

28 Kb init 2

120 Kb ipc 6

332 Kb kernel 25

80 Kb lib 8

356 Kb mm 19

5.8 Mb net 453

400 Kb scripts 42



Aims of Driver Development

• Writing device drivers corresponds to write parts of the Linux Kernel. Programmers should therefore pay a particular attention to the following aspects:

− Stability: Device drivers are executed in Kernel mode, and could therefore make the whole Kernel unstable.

− Performance: The Linux Kernel, as most UNIX operating systems, is non-preemptive. This means that a specific driver could slow down the rest of the system if it does not allow other tasks to run.

• Device driver programmers are also usually constrained by the following aspects:

− Name space: Developing a device driver often involves integrating variables and functions in the Kernel. Programmers should be careful not to create any conflict with other parts of the Kernel.

− Memory allocation: Kernel memory is not swappable, thus much more limited than memory allocated from normal programs. Programmers should try to keep the memory consumption of their device drivers at a bare minimum.

• Device drivers should also aim at providing a clean software interface to physical devices.

− It should not, however, add too many constraints to the way users of the driver will use the physical device.



How Device Drivers Work

• The main mechanisms provided by device drivers are the following:

− A set of routines that communicate with hardware devices and provide a clear interface to the Linux Kernel. These routines are automatically called when the device driver is opened. In most cases, the interface that these routines provide is made available by the VFS (Virtual Filesystem Switch).

− Management of data flow between user programs and a peripheral device. This is usually the primary functionality expected from a device driver.

− Device drivers provide control mechanisms to the user of the device. As we will see, this functionality is usually provided by ioctl’s.

− It allows a program or a physical peripheral device to appear as a file in /dev to the rest of the system's software.

− Optionally, a self-contained component that can be added to, or removed from, the operating system dynamically. As we will see, modules provide this functionality in the Linux Kernel.



Summary

• The environmental setup needed for Kernel development is very minimal.

• Most Linux distributions provide the required setup in their default installation. It is the case for Red Hat 7.0, on which this course is based.

• Two versions of the Linux Kernel are usually available at any given time: the production and experimental (or development) versions.

• Device drivers aim at providing a clean and easy to use interface between the user and the actual physical device. This is achieved by implementing device drivers in the Kernel.



Chapter 7

Block Drivers



Block Drivers

Objectives


• Identify the basic components of a typical block device driver.

• Implement the function in charge of the device registration in the Kernel.

• Identify the file operations implemented by most block device drivers.

• Modify the content of the blk.h in order to take in charge new block device drivers.

• Understand how requests to block devices are managed in the Kernel.

• Understand the code involved in the mounting process.



Block Device Registration

• The registration of block devices in the Kernel is provided by the register_blkdev() function. The argument needed by this function are as follows:

− A number that represents the major number associated with the device. As for character devices, the file Documentation/devices.txt should be consulted before associating a new major number with a device.

− The name of device, as a string of characters.

− The valid file operations that may be executed on the device, which are contained in a file_operations structure.

• Similarly, the device must be unregistered when it is removed from the system. This is accomplished by the unregister_blkdev() function, which expects the following argument:

− The major number associated with the device

− The name of the device



Block Device Registration

• Block device registration could be viewed as the following diagram:

Device driver Kernel

Initialization Boot register_blkdev()

fops block_read()

block_write()

open () , release()

request()



Block Device File Operations

• The file_operations structure associated with a block device does not usually contain the same fields as the one associated with other types of devices.

− The read, write and fsync fields of the file_operations structure are not implemented by the drivers.

− Each one of those fields typically invokes block_read(), block_write() and block_fsync() respectively, which are generic functions made available by fs/block_dev.c.

− The generic read and write functions are used to increase performance, since they also involve some buffering mechanisms before accessing the actual devices.

• However, each driver must implement a “request” function that is used to interact with the actual physical device.

− The blk_init_queue() function is used to associate the request function associated with a device with its major number.

− In the case of the floppy disk driver, it is used as follows:

blk_init_queue(BLK_DEFAULT_QUEUE(MAJOR_NR), DEVICE_REQUEST);

− In this case, DEVICE_REQUEST is defined as do_fd_request, which is the request function provided by the floppy disk driver.



blk.h

• All block devices must contain the following lines: #define MAJOR_NR XX #include <linux/blk.h>

− The first line defines the major number used for the device driver. MAJOR_NR must be defined since it is often used in the blk.h header file.

• The blk.h file contains several macros and definitions that are common to all block device drivers.

− This file contains several macro definitions that are used in most block device drivers, as well as prototypes for the initialization function corresponding to every driver.

• Each device also has a corresponding block of code in blk.h that is used for several definitions.

− Those blocks of code are defined at compile-time, depending on the block device driver that is being compiled in the Kernel.



blk.h (Cont’d)

• The block of definitions corresponding to the floppy driver looks as follows:

#elif (MAJOR_NR == FLOPPY_MAJOR) static void floppy_off(unsigned int nr); #define DEVICE_NAME "floppy" #define DEVICE_INTR do_floppy #define DEVICE_REQUEST do_fd_request #define DEVICE_NR(device) ( (MINOR(device) & 3) | ((MINOR(device) & 0x80 ) >> 5 )) #define DEVICE_OFF(device) floppy_off(DEVICE_NR(device))

• The following symbols are defined in blk.h:

− MAJOR_NR is the major number of the block device.

− DEVICE_NAME is the name of the device.

− DEVICE_INTR is the symbol used to refer to the bottom-half handler associated with the block device.

− DEVICE_REQUEST usually defines the request function used to access the hardware itself.

− DEVICE_NR defines an operation that is used to fetch the minor number of the device.

− DEVICE_OFF defines a function that is called when the device needs to be turned off. In the case of the floppy driver, it will be used to turn the motor off.



Block Device Requests

• As we have seen in the previous pages, the request function implemented in a block device driver contains the operations used to interact with the physical device itself.

− The request function is the one that is actually called when the device needs to be read or written.

• Read or write requests to the device are placed in a queue until they are actually processed.

• The request function takes the current block operation and performs the following steps:

− It first checks if the request is valid, i.e., if the queue is active and actually contains some requests.

− The data transfer (read or write) is executed. The CURRENT symbol is usually required to fetch some information about the request.



Block Device Requests (Cont’d)

• The CURRENT symbol resolves to a request structure. This data structure includes the following fields:

− Information about the device that is being accessed. This takes the form of a kdev_t structure, which stores the major and minor numbers associated with the device.

− The type of command (cmd), which could be READ or WRITE.

− The first sector to which the request refers.

− The size of the request operation, in numbers sectors of the block device.

− The location in the buffer cache to which data should be written.

• There are also other, less important fields contained in the request data structure. They may be viewed in include/linux/blkdev.h.



Mounting Block Devices

• One of the major difference between block and character devices is that the former type of device may be mounted (or attached) to the root filesystem.

• The mounting process is first activated by the mount() system call.

− The sys_mount() Kernel function is basically a wrapper for do_mount() function. It is used to pass the options, filesystem type and device name information to do_mount().

• The do_mount() function is the one doing the real work.

− Most operations in this function are triggered in order to deal with the filesystem used on the block device.

• The get_sb_dev() function is the one which actually interacts with the block device.

− It analyzes what device is opened for mounting, and ultimately calls the open() method associated with the block device.

− The opening stage of the device is launched by blkdev_get(), which invokes the following code:

bdev->bd_op->open(fake_inode, &fake_file);



Questions

• Why don’t block device drivers use read and write functions to provide access to their corresponding devices?

• What Kernel component handles all block device accesses before actually reading or writing to the disk?



Summary

• Registration is a critical step necessary when using block devices on a Linux system.

• A large part of a block device drivers consists in implementing some methods that show up in the file_operations data structure.

• The blk.h header file contains the initialization information for all block devices supported on the system.

• Block device requests are placed in a queue before a device could actually execute them.

• From the viewpoint of a block device driver, mounting a block device basically means executing the open() method implemented in the driver.



Chapter 10

SCSI Subsystem



SCSI Subsystem

Objectives


• Understand the basic architecture under which the SCSI subsystem is built.

• Identify the layers through which SCSI commands are communicated to the hardware.

• Describe the mechanisms and functions associated with each of the layers used in the SCSI subsystem.

• Have a basic understanding of how a SCSI controller driver is implemented in Linux.



SCSI Architecture Overview

• The SCSI subsystem is integrated in the Kernel as a 3-level architecture.

− The upper level is the one being closest to User space.

− The lower level is closest to the hardware.

− The mid level is the unifying layer between the upper and low level.

• Any operation using the SCSI subsystem will involve one driver at each of these three levels.

− For example, a User space application will access a device implemented by the upper level, which will call the mid level for requests to the physical device.

− The mid level will in turn communicate with the lower level, which implements the SCSI controller driver and allows access to the physical device.

• SCSI involves one particular aspect: the only hardware-related code is related to the SCSI controller.

− The SCSI standard is strictly defined so that the upper layers handles the details about access to disks, CDROM or tapes.

− Therefore, the only code that might change if the hardware is modified is the controller device driver.

− Generally, no device drivers are necessary for devices such as disks and CDROMs.



SCSI Architecture Overview (Cont’d)

• The SCSI architecture may be viewed as the following diagram:

Kernel Space

User Space

SD Block

Devices

SR Block

Devices

SG Character Devices

ST Character Devices

SCSI Mid Level

Upper Level

Low Level SCSI Adapter

Pseudo Driver for non-SCSI Services



Names and Conventions

• A naming convention must be used in cases where the user and the Kernel need to refer to a specific device.

• SCSI Device names are located in the /dev directory. Just as every other device, they are identified with their device type (block or character), major and minor device numbers.

− Each major number can support 256 minor numbers.

− Major numbers associated with a specific device are listed in Documentation/devices.txt. You should always refer to this file before associating a device with a given major number.

• Eight major numbers are reserved for SCSI disks (sd) devices: 8,65,66,67,68,69,70 and 71.

− A single major number, for example 8, allows devices to span from /dev/sda (which is allocated major 8 and minor 0) to /dev/sdp15 (which is allocated major 8 and minor 255).

− The next major number, 65, will span from /dev/sdq to /dev/sdaf15.

− Note that a maximum of 15 partitions are possible on a single SCSI disk, so the ending digit of any SCSI device will not be larger than 15.

− Devices that do not end with a digit (e.g., /dev/sda) refer to a whole SCSI disk.

− The maximum number of SCSI disks is therefore 128 (compared to 20 for the IDE subsystem).



Names and Conventions (Cont’d)

• SCSI CDROM (sr) devices have been allocated the block major number 11.

− 256 CDROM devices are therefore supported on a single system.

− Most recent Linux distributions refer to sr devices as “scd” in the /dev directory.

• SCSI Tape devices are allocated the char major number 9.

− 32 tapes are supported on a single system, and each one of them may be access in four different modes. This takes up 128 minor numbers.

− Tape devices may also be accessed in rewind or non-rewind mode. Minor numbers from 0 to 127 are used for rewind mode, whereas minor number 128 to 256 are used for non-rewind mode.

− For example, /dev/st0 refers to tape number 0, mode 0 in rewind mode. /dev/nst0 refers to tape number 0, mode 0 in non-rewind mode. /dev/st0l, /dev/st0m and /dev/st0a refer to mode 1, 2 and 3 respectively.

• SCSI generic (sg) devices are allocated the char major number 21 and support 256 sg devices, which are identified from /dev/sg0 to /dev/sg255.



Upper Level SCSI Layer

• The upper level drivers are usually referred to with two-character device names:

− sd stands for SCSI disks, like hard drives. These are always block devices.

− sr devices correspond to CDROM and DVD drives, which are also block devices.

− st stands for SCSI tapes, which are character devices.

− sg is for pass through character devices, which are commonly used in scanner drivers.

• The upper level implements the User-Kernel interfaces.

• The files in the Kernel source tree that implement the upper level are the following:

− drivers/scsi/sd.c and drivers/scsi/sr.c provide the SCSI disk and CDROM device drivers, respectively.

− drivers/scsi/st.c and drivers/scsi/sg.c provide the SCSI character and generic device drivers, respectively.

• The modules corresponding to each device type are sd_mod.o, sr_mod.o, st.o and sg.o. They are needed for SCSI disks, SCSI CDROMs, SCSI tapes and generic devices, respectively.



Block Devices

• Block devices are represented as the “sd” and “sr” component of the upper layer.

• The most common operation on a block device is to mount a filesystem.

− For a sd (SCSI disk) device, this is typically done with the following command:

mount –t ext2 /dev/sda6 /home

− For a sr (CDROM for example) device, this would be achieved in the following way:

mount –t iso9660 /dev/sr0 /mnt/cdrom

− The dd (disk dump) device can also be used to read or write from block devices. In this case, the block size parameter (bs) needs to be set to the block size of the device, which is usually 512 bytes.

− The fdisk command also takes charge of sd devices.

− The hdparm utility, which is usually used to change settings related to IDE hard disks, can also be used to change some options for SCSI disks.



CDROM And Char Devices

• The “sr” component of the upper layer is part of the CD-ROM subsystem.

− Just as for SCSI disks, these kinds of devices may be mounted with the mount command.

− Audio CD’s may also be read. This does not involve mounting the disk. This operation rather makes use of IOCTLs in order to gain access to the CD-ROM disk.

• Character devices include the st component of the upper layer for reading and writing to tape devices.

− General-purpose commands such as tar and dd may be used for these devices, but the mt command has been specially designed for the purpose of reading and writing tape devices.



Generic drivers

• The Linux sg driver is a upper level SCSI subsystem device driver that is used to handle devices that are not covered by the other upper level drivers: sd (disks), sr (CDROMs) and st (tapes).

• General-purpose commands such as dd and mount may not be used for sg devices.

• This device is usually used by applications such as SANE (for scanners) and cdrecord (for CD writers).

• This driver supports system calls that are expected to work on any character device, like open(), close(), write(), read() and ioctl().

• For each of these system calls, the sg_io_hdr_t data structure is used to convey information about the length of data to be read/written by the associated SCSI command. The following fields are included:

− The dxfer_direction determine if the current operation is a read or write.

− The cmd_len is the command length that cmdp points to.

− timeout is the time elapsed before a command aborts.

− status contains the status byte, defined in the SCSI standard.

− info contains some information about the current operation.



Mid Level SCSI Layer

• This layer is mostly implemented in scsi.c.

• The SCSI mid level is common to all operations, independently of the upper level component that is used and the SCSI controller installed on the system.

• It provides internal interfaces and common services to the upper and lower level drivers.

• The mid level uses the Scsi_Host_Template object, which provides an interface to low-level drivers.

− This object offers the interface that is used by the mid level layer to communicate to the low level layer.

− This is the only way that the SCSI controller (and therefore the SCSI devices) may be accessed.

− As long as the interface of the Scsi_Host_Template object does not change, new SCSI controller drivers may be added in the Kernel without modifying any other code. This is one of the main benefits that made the Kernel “object-oriented”.

• The mid layer also uses the scsi_cmnd data structure.

− This data structure is passed as an argument to most of the functions implemented by the low-level device driver.

− It is used by the high-level code to specify a SCSI command for execution by the low-level code.

− This data structure contains a private and public part. Low-level drivers use only the public part.



Low Level SCSI Layer

• A SCSI controller driver basically implements a series of functions as defined in the SHT data structure. This provides an interface that the mid-level layer will use in order to access the SCSI device.

• The functions implemented by the device driver are defined in the Scsi_Host_Template object. The following are the fields that are most commonly implemented:

− The proc_info() function is used to export information about the device to the /proc filesystem.

− name and procname contain the name of the device driver.

− The detect() function is called by the SCSI layer when the driver is initialized. This function is called at boot time or when a SCSI device module is loaded.

− The info() function returns a description of the actual controller itself.

− The queuecommand() function issues a SCSI command and does not wait for it to finish.

− The command() function issues a SCSI command and then waits for it to complete. This usually uses queuecommand().

− The abort() function is used to handle error situations. For example, if a time out occurred in the mid level layer, it may want to abort the command.



Summary

• The Linux SCSI subsystem relies on three specific layers:

− The low level layer interfaces directly with the hardware.

− The mid level layer receives commands from upper levels and sends requests to the low level drivers.

− The high level layer provides the user with devices that can be used accessed with common system calls.

• SCSI devices are separated in four different types:

− SCSI Disks (sd)

− SCSI CDROMs (sr)

− SCSI character or tape devices (st)

− SCSI generic devices (sg)

• The implementation of SCSI controllers is object oriented. A set of well-defined functions needs to be implemented in order to have a fully functioning driver.



Chapter 11

Device Drivers Debugging



Device Drivers Debugging

Objectives


• Use the printk function to debug device drivers.

• Query device drivers in order to fetch some information about their internal performance.

• Use entries in the /proc filesystem in order to debug a specific device driver.

• Install and use some of the optional Kernel debuggers and tracing tools currently available for Linux.



Printing With printk

• The printk() function, which is similar to the User space printf() function, allows us to easily monitor the Kernel operations.

• The printk() function could be used in the same way that printf() is invoked in User space. However, printk() provides some other features that are worth discussing here:

− Messages may be sent with varying degree of priorities. This allows the Kernel log daemon to determine if a specific message is important enough to be added to the logs.

− The following is the list of all valid message priorities:

#define KERN_EMERG "<0>" #define KERN_ALERT "<1>" #define KERN_CRIT "<2>" #define KERN_ERR "<3>" #define KERN_WARNING "<4>" #define KERN_NOTICE "<5>" #define KERN_INFO "<6>" #define KERN_DEBUG "<7>"

• The printk() function may be invoked without any priority. In this case, the priority of the message will default to DEFAULT_MESSAGE_LOGLEVEL, which is usually set to 4 (KERN_WARNING):

printk("Normal message from the Kernel\n");



Printing With printk (Cont’d)

• In the case were we need to send a critical message to the Kernel logs, we may use printk() as follows:

printk(KERN_CRIT "Kernel has crashed!\n");

• This line will be expanded by the preprocessor to the following line of code:

printk(<4> "Kernel has crashed!\n");

− This is also a valid way to invoke printk().

• In the implementation of printk(), you could see that the message will be printed on the console if the priority passed with the printk() function is less than DEFAULT_CONSOLE_LOGLEVEL.

• This value can also be monitored by looking at the fourth field in the /proc/sys/kernel/printk file.




• The printk() function can be very useful for debugging new code.

• For example, most drivers may be compiled with the debug option enabled, which will generate a considerable amount of output in most cases.

• In the case of the Realtek 8139 driver, the following code handles the debug statements: #ifdef RTL8139_DEBUG #define DPRINTK(fmt, args...) printk(KERN_DEBUG "%s: " fmt, __FUNCTION__ , ## args) #else #define DPRINTK(fmt, args...) #endif

− As you see, if the driver was compiled with RTL8139_DEBUG as being defined, then the DPRINTK statements will return some information of the driver to the Kernel logs.

− No messages will be returned if RTL8139_DEBUG was not defined, since any invocation of the preprocessor expands DPRINTK to nothing.

• The driver may thus use the DPRINTK macro in the following fashion: DPRINTK ("EXIT, returning %d\n", retval);

• Note that this is a compile time feature. Conditional statements may be used for runtime debugging.




• Debugging with printk() statements could however considerably slow down the system.

− The syslogd daemon synchronizes the content of the Kernel log with the log file located on the disk.

− You can therefore assume that a disk operation will take place for every line printed by the Kernel (so that the log could be retrieved after a crash).

− Adjusting the logging level with klogd in a way that only the most important lines will be printed is one way to limit the number of lines printed to the disk.

− The /proc/kmesg file may also be directly read from a application such as “cat” or “tail”. Using this method after killing klogd is another way to avoid too many disk operations due to the log file synchronization.

• Depending on the type of code that is being debugged, the amount of messages generated by printk() statements could also grow quite large.

− In this case, the debugging mechanisms need to be completely redesigned.

− We often rely on queries in order to solve this problem. This will allow to user to query data about a Kernel element, instead of interpreting the data generated by the Kernel.



Debugging by Querying

• Queries are usually considered a cleaner way to debug Kernel code in cases where the amount of data generated by printk() statements would grow too large.

• In this case the Kernel code must provide entry points where User space processes may read some information about a specific Kernel subsystem.

• We will focus here on two different methods that involve querying devices drivers:

− We will see that the /proc filesystem could be used in order to insert entries which represent the internal operations of a driver. This method is often the easiest one for users who need to fetch some information about a particular driver.

− The ioctl system call can also be used in order to gather some internal information about a driver and a device. Using ioctl’s is however more complex than querying entries in the /proc filesystem.



/proc Entries

• Files located in the /proc filesystem are not associated with any block device.

− All the files that are found in this directory are generated on the fly by the corresponding Kernel component when users read them.

− Files in /proc are usually text files. This allows user or application to easily monitor the content of those files, without having to interpret the content of the files in a complex way.

− For example, every process running on the system has an associated directory in /proc. This directory contains files providing information about a particular process.

• Similarly to the major Kernel subsystems (memory manager, VFS, scheduler), device drivers may also dynamically add entries in the /proc filesystem.

− This allows device driver developers to add entries that serve the task of monitoring the state of the device.

− Some aspects of the device driver (variables for examples) may also be modified since the file entries in /proc could also be modified if they were set correspondingly.



Adding /proc Entries

• Adding an entry in /proc is simple. This is basically achieved in the following steps, which may be executed at boot time or when a module is loaded):

− We must first initialize the entry in the /proc filesystem:

struct proc_dir_entry *res = create_proc_entry("filename",0,NULL); res->read_proc=proc_read; return 0;

− The next step is to implement the function which will generate the characters to output when the /proc entry will be read:

int proc_read(char *buff, char **start, off_t fpos, int length, int *eof, void *data) { int len =0; len += sprintf(buff, "%d %d\n", int1, int2); return len; }

• Note that the entire proc_read() function will be executed whenever a process reads the corresponding file.

• The file may need to be removed when the driver is unloaded, which could be done with the following code:

remove_proc_entry("filename",NULL);



IOCTL’s

• As we have seen in Chapter 5, ioctl is a system call which acts on a file descriptor representing a device special file.

• By passing it a number representing a command and (optionally) an argument, we can send commands to a specific device without actually invoking the read and write operations on this device.

• However, the User space aspects involved with ioctl’s are a bit trickier.

− Instead of reading a file (as for /proc entries), a User space process needs to invoke the ioctl system call and deal with the output of the command.

− Thus, for debugging this means that the developer has to maintain another program to debug the device driver.

• Ioctl’s are useful in cases where reading data from a file in /proc would be time consuming, or when data needs to be interpreted in a specific way before being presented to the user.

• Debugging-related ioctl’s also got the benefit of not being obstructive once they are integrated in a driver. They could very well remain in the production driver, even if they are no longer needed.



IOCTL’s

• For example, the ncr885e.c network device driver defines an ioctl to provide some information about the card’s registers. The code is defined as follows:

static int debug_ioctl( struct net_device *dev, struct ifreq *req, int cmd ) { unsigned long ioaddr = dev->base_addr; struct ncr885e_private *data; struct ncr885e_regs *regs; unsigned long flags; struct ncr885e_regs dump temp; switch ( cmd ) { case NCR885E_GET_REGS: regs = (struct ncr885e_regs *) &req->ifr_data; if ( verify_area( VERIFY_WRITE, &req->ifr_data,

sizeof( struct ncr885e_regs ))) return -EFAULT;

temp.dump.tx_status=inl(ioaddr+TX_CHANNEL_STATUS ); temp.dump.rx_status=inl(ioaddr+RX_CHANNEL_STATUS ); temp.dump.mac_config=inl( ioaddr + MAC_CONFIG ); temp.dump.tx_cmd_ptr = inl(ioaddr+TX_CMD_PTR_LO ); temp.dump.rx_cmd_ptr = inl(ioaddr+RX_CMD_PTR_LO); copy_to_user( regs, (char *) &temp.dump, sizeof( struct ncr885e_regs )); break; default:

return -EOPNOTSUPP; } return 0; }



The System Request Key

• The system request key is a feature of the Linux Kernel that allows the user on the physical console to get some information about the state of the Kernel.

− The SysReq option should be set in the Kernel configuration prior to the compilation in order to include this feature.

− In some distributions, the file /proc/sys/kernel/sysrq should contain a “1” to enable the system request key.

• Once the System Request Key has been properly configured and compiled in the Kernel, it could be launched by typing ALT-SysRQ and a key corresponding to the needed information.

− Note that on some computers, the SysRq key is also labeled “Print Screen”.



The System Request Key (Cont’d)

• The valid keys are the following:

− p will dump the content of the major registers of the CPU.

− t provides information about the tasks running on the system. The returned data includes the name, stack address, PID, father, child and siblings of each task.

− m returns some information about the current memory usage.

− k kills all programs running on the current console.

− e sends the SIGTERM signal to all processes except init, whereas i sends this SIGKILL signal to every process except init.

− l will send the SIGKILL signal to every single process in the system, including init.

− b immediately reboots the computer.

− s may be used to synchronize the filesystems with their physical disks, which is useful if a crash occured.

− u remounts the filesystems.

− Any number from 0 to 9 will set a new log level. A value of 0 will let almost nothing be presented on the console.



The Linux Kernel Debugger (kdb)

• The Kernel Debugger (kdb) is a more advanced debugger if we compare it with the System Request Key.

− It basically gives you access to the same features as the System Request Key.

− However, KDB allows you to have more control over the execution of the Kernel.

• KDB is not integrated in the Kernel by default. It must be installed by patching the Kernel with the KDB patch.

− Once the patch is applied to the Kernel, the Kernel configuration utilities will contain an option related to KDB.

− Setting this option will allow you to compile KDB in the Kernel.

• Once the Kernel has been compiled and installed with this new debugger, KDB may be launched by pressing the “pause” key.

− At this point the leds on the keyboard will begin flashing in order to let the user know that the Kernel was switched in debugging mode.

− The KDB debugger switches the Kernel in a kind of command line mode where the user needs to type in some commands which we will describe in the next section.



The Linux Kernel Debugger (Cont’d)

• The following commands are used to access the system memory:

− In the next commands, the memory address may be passed as a numerical value, a symbol or a register name.

− md will display the content of a memory address, and mdr will do the same thing but for a specified number of bytes.

− Just like the last two commands, mds read a memory address but tries to associate the output with a known symbol. With it fails to do so, the ASCII representation of the memory content will be shown.

− mm allows the user of KDB to modify a memory area by transferring the new value in the address specified as the first parameter.

• KDB can also modify the execution of the Kernel code with the following commands:

− bp allows the user to set a breakpoint in the Kernel. The only parameter to this command will be the breakpoint itself, which may be an a numerical value, a symbol or a register name. The “bp schedule” command, for example, will stop the Kernel at each execution of the scheduler.

− The bl command lists all breakpoints currently active in KDB.

− bd deletes the breakpoint specified as the parameter. A passed value of 0, for example, will remove the first breakpoint. Similarly, bc will remove all breakpoints.



The Linux Kernel Debugger (Cont’d)

• Registers may also be manipulated by the use of the rd amd rm commands:

− Similarly to the functionality provided by the System Request Key, rd will display all registers of the CPU.

− The rm command modifies the value of the registers of the CPU. The first parameter is the name of the register, and the second parameter is the new value.

• KDB also supports the following miscellaneous commands:

− The sections command gives information on all known memory section. The memory address where modules, data, text section will be presented to the user.

− The lsmod and rmmod command may be invoked in order to respectively list and remove modules from the Kernel.

− The go command will resume the execution of the Kernel. This command should be entered in order to quit the KDB command line.

− reboot immediately reboots the system.

− ps lists all processes currently stored in the task table.

− id disassemble the CPU instructions located at the memory address passed in argument.



Questions

• Can the System Request Key and KDB coexist on the same Kernel?

• In what case would you want to close all programs running on the current console?

• Why is KDB heavily dependent on the Kernel version it is installed on?

• Why is it considered dangerous to execute the “b” system request key? What command should you execute before in order to avoid problems?

• What KBD command would you use in order to stop the Kernel at each invocation of the printk() function?



Summary

• The printk() function is one of the most precious function when it comes to debugging Kernel code. However, great care must be taken in order to avoid saturating the Kernel logs.

• The /proc filesystem may be used to provide an interface to device drivers, so that their internal behavior could easily be monitored and modified by the user.

• The ioctl system call allows the user to send predefined commands to a device driver, which could allow the user to monitor the way a driver is working in the Kernel.

• The System Request Key is a helpful tool to gain some knowledge of the Kernel internals, but it does not provide the basic functionality expected from a real debugger.

• Even though it is sometimes limited, KDB provides far more features. It allows the user to monitor, modify and control the internal operations of the Kernel.

linux driver writing - itcourseware...companies, names and data used in examples herein are...

Documents