Threads Primer

A Guide to Multithreaded Programming

Bil Lewis Daniel J. Berg

SunSoft PressA Prentice Hall Title

Contents

Acknowledgments ........................................................................................................... xxi

Preface .............................................................................................................................xxiii

1. Introduction ................................................................................................................1

2. Concepts ......................................................................................................................5Background: Traditional Operating Systems..........................................................5What Is a Thread?....................................................................................................... 8Kernel Interaction..................................................................................................... 11

System Calls........................................................................................................... 12Signals......................................................................................................................12Synchronization.......................................................................... 12Scheduling..............................................................................................................12

The Value of Using Threads.....................................................................................13Parallelism..............................................................................................................13Throughput........................................................................................................... 14Responsiveness..................................................................................................... 15Communications................................................................................................... 15System Resources................................................................................................. 16Distributed Objects...............................................................................................16Same Binary for Uniprocessors and Multiprocessors.....................................17Program Structure................................................................................................. 17Single Source for Multiple Platforms................................................................18

What Kind of Programs to Thread?........................................................................ 18Inherently MT Programs.....................................................................................19Not Obviously MT Programs............................................................................ 19Automatic Threading...........................................................................................20Programs Not to Thread...................................................................................... 20

What About Shared Memory?................................................................................ 21

V

Threads Standards.................................................................................................... 21Performance.............................................................................................................. 22

Solaris 2...................................................................................................................22NFS.........................................................................................................................22SPECfp 92...............................................................................................................23PAR93.....................................................................................................................23Dakota Scientific Software, Inc............................................................................24MARC Analysis Inc...............................................................................................24Delphax and Uniq Inc...........................................................................................24Adobe.....................................................................................................................24Facet Decision Systems, Inc................................................................................. 24The ImagiNation Network................................................................................. 25Visual Numerics.................................................................................................... 25Vantage Analysis Systems, A Viewlogic Company........................................ 25Landmark Graphics..............................................................................................25University of Houston..........................................................................................26Geophysical Development Corporation...........................................................26InfoGraphix Technologies Incorporated...........................................................26Scitex.......................................................................................................................26Western Geographical..........................................................................................26

3. Foundations.............................................................................................................. 27Implementation vs. Specification........................................................................... 27Thread Libraries........................................................................................................ 27The Process Structure................................................................................................29Lightweight Processes..............................................................................................30

Threads and LWPs................................................................................................ 31Solaris Multithreaded Model................................................................................. 34System Calls...............................................................................................................35Signals.........................................................................................................................37

Don't Use Signal Handlers!............................................................................... 39Per-Thread Timers................................................................................................ 39

4. Scheduling.................................................................................................................41Different Models of Kernel Scheduling.................................................................41

Threads Are Not Always User-level Entities.................................................. 44Thread Scheduling.................................................................................................... 44

vi Threads Primer

Thread Scheduling Details.................................................................................. 45Context Switching.................................................................................................49Preemption............................................................................................................. 52How Many LWPs?.................................................................................................52Bound Threads and Real-time LWPs................................................................53Binding LWPs to Processors.............................................................................. 54When Should You Care About Scheduling?................................................... 55

Thread Creation and Destruction.......................................................................... 56Returning Status and Memory.......................................................................... 57That's Not a Bug, That's a Feature!....................................................................58Exiting the Process...............................................................................................58Cancellation........................................................................................................... 59

5. Synchronization ...................................................................................................... 61Synchronization Issues.............................................................................................61

Memory Model..................................................................................................... 61Critical Sections..................................................................................................... 63Lock Your Global Variables!................................................................................ 63

Synchronization Variables.......................................................................................63Mutexes..................................................................................................................64Reader/Writer Locks...........................................................................................65Condition Variables.............................................................................................67Semaphores........................................................................................................... 69Barriers....................................................................................................................70Event Semaphores................................................................................................. 71OS/2 Critical Sections...........................................................................................71Spin Locks............................................................................................................. 71Adaptive Locks..................................................................................................... 72Cross-process Synchronization Variables......................................................... 72Synchronization Variable Initialization and Destruction...............................74

Synchronization Problems.......................................................................................74Deadlocks................................................................................................................74Race Conditions..................................................................................................... 76Recovering from Deadlocks................................................................................ 76Priority Inversion.................................................................................................77FIFO Mutexes......................................................................................................... 78

Contents vii

Thread-specific Data.................................................................................................. 80Thread Local Storage............................................................................................83

Global Variables, Constants, and Cheating...........................................................84Comparing the Four Libraries............................................................................... 84

6. Operating System Issues........................................................................................87The Solaris Threads Library....................................................................................87Solaris Multithreaded Kernel................................................................................. 88

Concurrency vs. Parallelism............................................................................... 88Solaris Symmetric Multiprocessing...................................................................91Kernel Scheduling................................................................................................92

Are Libraries Safe?.................................................................................................... 93Async Safety.......................................................................................................... 95

New Semantics for System Calls........................................................................... 96Forking New Processes........................................................................................96Executing a New Program................................................................................. 97The New System Call sigwait (2)..................................................................... 97

The Solaris Operating Environment....................................................................... 97SVR4 and SPEC 1170 Compliant....................................................................... 98Real-Time, Fully Preemptable Kernel...............................................................98SMP.........................................................................................................................98Dynamically Loadable Kernel Modules...........................................................99Auto-Configuration..............................................................................................99Dynamically Loadable, Shared Memory Modules........................................99Memory-Mapped Files........................................................................................99Optimized Swap Space........................................................................................99Transport-Layer Interfaces................................................................................100Cache File System and Auto File System.......................................................100Interactive Scheduling Class............................................................................. 100In-kernel rlogin/telnet.............................. 100Wabi and MAE.................................................................................................... 100Network Information Service Plus................................................................. 101CDE....................................................................................................................... 101Project DOE.........................................................................................................101Multiple Hardware Architectures................................................................... 101

Solaris Threads Debugging Interface................................................................... 102

L

Comparisons of Different Implementations........................................................102

7. POSIX Threads (pthreads) ...................................................................................105Pthreads Introduction.............................................................................................105Attribute Objects..................................................................................................... 106

Threads and Attribute Objects.......................................................................... 107Synchronization Variables and Attribute Objects......................................... 110

Cancellation..............................................................................................................112Thread Scheduling................................................................................................... 115Moving Between the Two APIs.............................................................................117

8. Programming Tools............................................................................................... 119LockLint................................................................................................................119LoopTool................................................................................................................120SPARC works Debugger.....................................................................................120Thread Analyzer................................................................................................. 120

Examples of Using the Tools.................................................................................120Example One : Threading the Mandelbrot Program................................. 121Example Two : Checking a Program with LockLint................................. 125Example Thr ee: Parallelizing Loops with LoopTool................................. 128Example Fou r: Using the SPARC works Debugger................................... 131

For More Information.............................................................................................135

9. Programming With Threads.................................................................................137Global Variables (errno).................................137Static Local Variables............................................................................................... 138Synchronizing Threads...........................................................................................140

Single-Lock Strategy...........................................................................................140Reentrance........................................................................................................... 140Avoiding Deadlock.............................................................................................141Scheduling Problems.........................................................................................143Locking Guidelines.............................................................................................143

Following Some Basic Guidelines........................................................................ 144Creating Threads................................................................................................. 144Thread Concurrency...........................................................................................145Bound Threads................................................................................................... 145

Contents ix

10. Examples .................................................................................................................147Using thr_create() and thr_join()........................147Producer / Consumer............................................................................................ 154Matrix Multiplication.............................................................................................. 159Threads and Windows............................................................................................ 168Socket Server.............................................................................................................173Using Many Threads.............................................................................................. 177Real-time Thread.................................................................................................... 180POSIX Cancellation................................................................................................ 182Deadlock...................................................................................................................186Software Race Condition........................................................................................188Signal Handler........................................................................................................ 190Interprocess Synchronization................................................................................192Arrays ....................................................................................................................... 194

A. Answers ................................................................................................................... 197

B. Solaris Signal Implementation ..........................................................................203

C. Threads on the Net.................................................................................................205Threads on the World Wide Web......................................................................... 205The SPILT Package for Solaris/POSIX Compatibility......................................205Threads Newsgroup................................................................................................ 205Code Examples........................................................................................................ 206The Authors on the Net..........................................................................................206

D. Timings..................................................................................................................... 207

E. Common Bugs......................................................................................................... 209Failure to Check Return Values for Errors...........................................................209Using errno Without Checking the Return Value............................................ 209Not Joining on Nondetached Threads.................................................................209Failure to Verify that Library Calls Are MT Safe.............................................. 209Insufficient Swap Space..........................................................................................210Creating Excessive LWPs........................................................................................210Forgetting to Include the -D_REENTRANT Flag................................................ 210Using Spin Locks.................................................................................................... 210Depending upon Scheduling Order..................................................................... 210

x Threads Primer

Using errno for Threads Library Functions......................................................211Not Recognizing Shared Data...............................................................................211

F. Compile and Debug Specifics ............................................................................. 213Compiling a Multithreaded Application............................................................213

Using libthread....................................213Debugging Multithreaded Programs..................................................................214

Using adb......................................... 215Using dbx......................................... 215

G. Application Program Interface ...........................................................................217Condition Functions...............................................................................................219Mutex Functions..................................................................................................... 222Reader/Writer Lock Functions............................................................................ 224Semaphore Functions.............................................................................................228Thread Functions..................................................................................................... 230

H. Pthreads API (POSIX 1003.1c) ..............................................................................239Condition Functions...............................................................................................241Condition Attribute Functions.............................................................................. 246Mutex Functions..................................................................................................... 249Mutex Attribute Functions.....................................................................................254Pthread Functions................................................................................................... 261Pthread Attribute Functions.................................................................................. 275Semaphore Functions.............................................................................................286

Glossary .............................................................................................................................291

Index .................................................................................................................................. 305

Contents xi