This page intentionally left blank

WAVELET THEORY

This page intentionally left blank

WAVELET THEORY An Elementary Approach With Applications

David K. Ruch Metropolitan State College of Denver

Patrick J. Van Fleet University of St. Thomas

WILEY A JOHN WILEY & SONS, INC., PUBLICATION

The image in Figure 7.6(a) appears courtesy of David Kubes. The photographs on the front cover, Figure 4.4, and Figure 4.11 appear courtesy of Radka Tezaur.

Copyright © 2009 by John Wiley & Sons, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., I l l River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Ruch, David K., 1959-Wavelet theory: an elementary approach with applications / David K. Ruch, Patrick J. Van Fleet

p. cm. Includes bibliographical references and index. ISBN 978-0-470-38840-2 (cloth) 1. Wavelets (Mathematics) 2. Transformations (Mathematics) 3. Digital Images—

Mathematics. I. Van Fleet, Patrick J., 1962- II. Title. QA403.3.V375 2009 515\2433—dc22 2009017249

Printed in the United States of America.

10 9 8 7 6 5 4 3 2 1

To Pete and Laurel for a lifetime of encouragement

(DKR)

To Verena, Sam, Matt, and Rachel for your unfailing support

(PVF)

This page intentionally left blank

CONTENTS

Preface xi

Acknowledgments xix

1 The Complex Plane and the Space L2(R) 1

1.1 Complex Numbers and Basic Operations 1 Problems 5

1.2 The Space L2(R) 7 Problems 16

1.3 Inner Products 18 Problems 25

1.4 Bases and Projections 26 Problems 28

2 Fourier Series and Fourier Transformations 31

2.1 Euler's Formula and the Complex Exponential Function 32 Problems 36

2.2 Fourier Series 37

vii

Vi i i CONTENTS

Problems 49 2.3 The Fourier Transform 53

Problems 66 2.4 Convolution and 5-Splines 72

Problems 82

3 Haar Spaces 85

3.1 The Haar Space Vo 86 Problems 93

3.2 The General Haar Space Vj 93 Problems 107

3.3 The Haar Wavelet Space W0 108 Problems 119

3.4 The General Haar Wavelet Space Wj 120 Problems 133

3.5 Decomposition and Reconstruction 134 Problems 140

3.6 Summary 141

4 The Discrete Haar Wavelet Transform and Applications 145

4.1 The One-Dimensional Transform 146 Problems 159

4.2 The Two-Dimensional Transform 163 Problems 171

4.3 Edge Detection and Naive Image Compression 172

5 Multiresolution Analysis 179

5.1 Multiresolution Analysis 180 Problems 196

5.2 The View from the Transform Domain 200 Problems 212

5.3 Examples of Multiresolution Analyses 216 Problems 224

5.4 Summary 225

6 Daubechies Scaling Functions and Wavelets 233

6.1 Constructing the Daubechies Scaling Functions 234

CONTENTS IX

Problems 246 6.2 The Cascade Algorithm 251

Problems 265 6.3 Orthogonal Translates, Coding, and Projections 268

Problems 276

7 The Discrete Daubechies Transformation and Applications 277

7.1 The Discrete Daubechies Wavelet Transform 278 Problems 290

7.2 Projections and Signal and Image Compression 293 Problems 310

7.3 Naive Image Segmentation 314 Problems 322

8 Biorthogonal Scaling Functions and Wavelets 325

8.1 A Biorthogonal Example and Duality 326 Problems 333

8.2 Biorthogonality Conditions for Symbols and Wavelet Spaces 334 Problems 350

8.3 Biorthogonal Spline Filter Pairs and the CDF97 Filter Pair 353 Problems 368

8.4 Decomposition and Reconstruction 370 Problems 375

8.5 The Discrete Biorthogonal Wavelet Transform 375 Problems 388

8.6 Riesz Basis Theory 390 Problems 397

9 Wavelet Packets 399

9.1 Constructing Wavelet Packet Functions 400 Problems 413

9.2 Wavelet Packet Spaces 414 Problems 424

9.3 The Discrete Packet Transform and Best Basis Algorithm 424 Problems 439

9.4 The FBI Fingerprint Compression Standard 440

Appendix A: Huffman Coding 455

X CONTENTS

Problems 462

References 465

Topic Index 469

Author Index 479

Preface

This book presents some of the most current ideas in mathematics. Most of the theory was developed in the past twenty years, and even more recently, wavelets have found an important niche in a variety of applications. The filter pair we present in Chapter 8 is used by JPEG2000 [59] and the Federal Bureau of Investigation [8] to perform image and fingerprint compression, respectively. Wavelets are also used in many other areas of image processing as well as in applications such as signal denoising, detection of the onset of epileptic seizures [2], modeling of distant galaxies [3], and seismic data analysis [34, 35].

The development and advancement of the theory of wavelets came through the efforts of mathematicians with a variety of backgrounds and specialties, and of engi-neers and scientists with an eye for better solutions and models in their applications. For this reason, our goal was to write a book that provides an introduction to the essential ideas of wavelet theory at a level accessible to undergraduates and at the same time to provide a detailed look at how wavelets are used in "real-world" appli-cations. Too often, books are heavy on theory and pay little attention to the details of application. For example, the discrete wavelet transform is but one piece of an im-age compression algorithm, and to understand this application, some attention must be given to quantization and coding methods. Alternatively, books might provide a detailed description of an application that leaves the reader curious about the theoret-ical foundations of some of the mathematical concepts used in the model. With this

XI

Xi i PREFACE

book, we have attempted to balance these two competing yet related tenets, and it is ultimately up to the reader to determine if we have succeeded in this endeavor.

To the Student

If you are reading this book, then you are probably either taking a course on wavelets or are working on your own to understand wavelets. Very often students are natu-rally curious about a topic and wish to understand quickly their use in applications. Wavelets provide this opportunity— the discrete Haar wavelet transformation is easy to understand and use in applications such as image compression. Unfortunately, the discrete Haar wavelet transformation is not the best transformation to use in many applications. But it does provide us with a concrete example to which we can refer as we learn about more sophisticated wavelets and their uses in applications. For this reason, you should study carefully the ideas in Chapters 3 and 4. They provide a framework for all that follows. It is also imperative that you develop a good working knowledge of the Fourier series and transformations introduced in Chapter 2. These ideas are very important in many areas of mathematics and are the basic tools we use to construct the wavelet filters used in many applications.

If you are a mathematics major, you will learn to write proofs. This is quite a change from lower-level mathematics courses where computation was the main objective. Proof-writing is sometimes a formidable task and the best way to learn is to practice. An indirect benefit of a course based on this book is the opportunity to hone your proof-writing skills. The proofs of most of the ideas in this book are straightforward and constructive. You will learn about proof by induction and contraposition. We have provided numerous problems that ask you to complete the details of a portion of a proof or mimic the ideas of one case in a proof to complete another. We strongly encourage you to tackle as many of these problems as possible. This course should provide a good transition from the proofs you see in a sophomore linear algebra course to the more technical proofs you might see in a real analysis course.

Of course, the book also contains many computational problems as well as prob-lems that require the use of a computer algebra system (CAS). It is important that you learn how to use a CAS — both to solve problems and to investigate new concepts. It is amazing what you can learn by taking examples from the book and using a CAS to understand them or even change them somewhat to see the effects. We strongly encourage you to install the software packages described below and to visit the course Web site and work through the many labs and projects that we have provided.

To the Instructor

In this book we focus on bridging the gap often left between discrete wavelet trans-formations and the traditional multiresolution analysis-based development of wavelet theory. We provide the instructor with an opportunity to balance and integrate these ideas, but one should be wary of getting bogged down in the finer details of either

PREFACE XI I i

topic. For example, the material on Fourier series and transforms is a place where instructors should use caution. These topics can be explored for entire semesters, and deservedly so, but in this course they need to be treated as tools rather than the thrust of the course.

The heart of wavelet theory is covered in Chapters 3,5, and 6 in a comprehensive approach. Extensive details and examples are given or outlined via problems, so students should be able to gain a full understanding of the theory without hand-waving at difficult material. Having said that, some proofs are omitted to keep a nice flow to the book. This is not an introductory analysis book, nor is the level of rigor up to that of a graduate text. For example, the technical proofs of the completeness and separation properties of multiresolution analyses are left to future courses. The order of infinite series and integration are occasionally swapped with comment but not rigorous justification. We choose not to develop fully the theory of Riesz bases and how they lead to true dual multiresolutions of L2(R), for this would leave too little time for the very real applications of biorthogonal filters. We hope students will whet their appetites for future courses from the taste of theory they are given here!

We feel that the discrete wavelet transform material is essential to the spirit of the book, and based on our experience, students will find the applications quite gratifying. It may be tempting to expand on our introduction to these ideas after the Haar spaces are built, but we hope sufficient time is left for the development of multiresolution analyses and the Daubechies wavelets, which can take considerable time.

We also hope that instructors will take the time for thorough treatment of the connections between standard wavelet theory and discrete wavelet transforms. Our experience, both personally and with teaching other faculty at workshops, is that these connections are very rewarding but are not obvious to most beginners in the field. Some interesing problems crop up as we move between L 2 (R) and finite-dimensional approximations.

Text Topics

In Chapter 1, we provide a quick introduction to the complex plane and L 2(R), with no prior experience assumed, emphasizing only the properties that will be needed for wavelet development. We believe that the fundamentals of wavelets can be studied in depth without getting into the intricacies of measure theory or the Lesbesgue integral, so we discuss briefly the measure of a set and convergence in norm versus pointwise convergence, but we do not dwell heavily on these ideas.

In Chapter 2 we present Fourier series and the Fourier transform in a limited and focused fashion. These ideas and their properties are developed only as tools to the extent that we need them for wavelet analysis. Our goal here is to prepare quickly for the study of wavelets in the transform domain. For example, the transform rules on translation and dilation are given, since these are critical for manipulating scaling function symbols in the transform domain. 5-splines are introduced in this chapter as an important family of functions that will be used throughout the book, especially in Chapter 8.

XIV PREFACE

In Chapter 3 we begin our study of wavelets in earnest with a comprehensive examination of Haar spaces. All the major ideas of multiresolution analysis are here, cast in the accessible Haar setting. The properties and standard notations of approximation spaces Vj and detail spaces Wj are developed in detail with numerous examples.

Students may be ready for some applications after the long Haar space analysis, and we present some classics in Chapter 4. The ideas behind filters and the discrete Haar wavelet transform are introduced first. The basics of processing signals and images are developed in Sections 4.1 and 4.2, with sufficient detail so that students can carry out the calculations and fully understand what software is doing while processing large images. The attractive and very accessible topics of image compression and edge detection are introduced as applications in Section 4.3.

In Chapter 5 we generalize the Haar space concepts to a general multiresolution analysis, beginning with the main properties in the time domain. Section 5.2 begins the development of critical multiresolution properties in the transform domain. In Section 5.3 we present some concrete examples of functions satisfying multiresolution properties. In addition to Haar, the Shannon wavelet and I?-splines are discussed, each of which has some desirable properties but is missing others. This also provides some motivation for the formidable challenge of developing Daubechies wavelets. We return to 5-splines in Chapter 8.

Chapter 6 centers on the Daubechies construction of continuous, compactly sup-ported scaling functions. After a detailed development of the ideas, a clear algorithm is given for the construction. The next two sections are devoted to the cascade algo-rithm, which we delay presenting until after the Daubechies construction, with the motivation of plotting these amazing scaling functions with only a dilation equation to guide us. The cascade algorithm is introduced in the time domain, where examples make it intuitively clear, and is then discussed in the transform domain. Finally, we study the practical issue of coding the algorithm with discrete vectors.

After the rather heavy theory of Chapters 5 and 6, an investigation of the discrete Daubechies wavelet transform and applications in Chapter 7 provides a nice change of pace. An important concept in this chapter is that of handling the difficulties encountered when the decomposition and reconstruction formula are truncated, which are investigated in Section 7.2. Our efforts are rewarded with applications to image compression, noise reduction and image segmentation in Section 7.3.

In Chapter 8 we introduce scaling functions and wavelets in the biorthogonal set-ting. This is a generalization of an orthogonal multiresolution analysis with a single scaling function to a dual multiresolution analysis with a pair of biorthogonal scaling functions. We begin by introducing several new ideas via an example from 5-splines, with an eye toward creating symmetric filters to be used in later applications. The main structural framework for dual multiresolution analyses and biorthogonal wavelets is developed in Section 8.2. We then move to constructing a family of biorthogonal filters based on J3-splines using the methods due to Ingrid Daubechies in Section 8.3. The Cohen-Daubechies-Feauveau CDF97 filter pair is used in the JPEG2000 and FBI fingerprint compression standards, so it is natural to include them in the book. The method of building biorthogonal spline filters can be adjusted fairly easily to

PREFACE XV

create the CDF97 filter pair, and this construction is part of Section 8.3. The pyramid algorithm can be generalized for the biorthogonal setting and is presented in Sec-tion 8.4. The discrete biorthogonal wavelet transform is discussed in Section 8.5. An advantage of biorthogonal filter pairs is that they can be made symmetric, and this desirable property affords a method, also presented in Section 8.5, of dealing with edge conditions in signals or digital images. A fundamental theoretical under-pinning of dual multiresolution analyses is the concept of a Riesz basis, which is a generalization of orthogonal bases. The very formidable specifics of Riesz bases have been suppressed throughout most of this chapter in an effort to provide a balance between theory and applications. As a final and optional topic in this chapter, a brief examination of Riesz bases is provided in Section 8.6.

Wavelet packets, the topic of Chapter 9, provide an alternative wavelet decomposi-tion method but are more computationally complex since the decomposition includes splitting the detail vectors as well as the approximations. We introduce wavelet packet functions in Section 9.1 and wavelet packet spaces in Section 9.2. The dis-crete wavelet packet transform is presented in Section 9.3 along with the best basis algorithm. The wavelet packet decomposition allows for redundant representations of the input vector or matrix, and the best basis algorithm chooses the "best" rep-resentation. This is a desirable feature of the transformation as this algorithm can be made application-dependent. The FBI fingerprint compression standard uses the CDF97 biorthogonal filter pair in conjunction with a wavelet packet transformation, and we outline this standard in Section 9.4.

Prerequisites

The minimal requirements for students taking this course are two semesters of calculus and a course in sophomore linear algebra. We use the ideas of bases, linear indepen-dence, and projection throughout the book so students need to be comfortable with these ideas before proceeding. The linear algebra prerequisite also provides the nec-essary background on matrix manipulations that appear primarily in sections dealing with discrete transformations. Students with additional background in Fourier series or proof-oriented courses will be able to move through the material at a much faster pace than will students with the minimum requirements. Most proofs in the book are of a direct and constructive nature, and some utilize the concept of mathematical in-duction. The level of sophistication assumed increases steadily, consistent with how students should be growing in the course. We feel that reading and writing proofs should be a theme throughout the undergraduate curriculum, and we suggest that the level of rigor in the book is accessible by advanced juniors or senior mathematics students. The constant connection to concrete applications that appears throughout the book should give students a good understanding of why the theory is important and how it is implemented. Some algorithms are given and experience with CAS software is very helpful in the course, but significant programming experience is not required.

XVI PREFACE

Possible Courses for this Book

The book can serve as a stand-alone introduction to wavelet theory and applications for students with no previous exposure to wavelets. If a brisk pace is kept in line with the prerequisites discussed above, the course could include the first six chapters plus the discrete Daubechies transform and a sample of its applications. While considerable time can be spent on applied projects, we strongly recommend that any course syllabus include Chapter 6, on Daubechies wavelets. The construction of these wavelets is a remarkable mathematical achievement accomplished during our lifetime (if not those of our students) and should be covered if at all possible.

Some instructors may prefer to first cover Chapters 3 and 4 on Haar spaces before introducing the Fourier material of Chapter 2. This approach will work well since aside from a small discussion of the Fourier series associated with the Haar filter, no ideas from Fourier analysis are used in Chapters 3 and 4.

A very different course can be taught if students have already completed a course using Van Fleet's book Discrete Wavelet Transformations: An Elementary Approach with Applications [60]. Our book can be viewed as a companion text, with consistent notation, themes, and software packages. Students with this experience can move quickly through the applications, focusing on the traditional theory and its connections to discrete transformations. Students completing the discrete course should have a good sense of where the material is headed, as well as motivation to see the theoretical development of the various discrete transform filters. In this case, some sections of the text can be omitted and the entire book could be covered in one semester.

A third option exists for students who have a strong background in Fourier anal-ysis. In this case, the instructor could concentrate heavily on the theoretical ideas in Chapters 5, 6, 8, and 9 and develop a real appreciation for how Fourier methods can be used to drive the theory of multiresolution analysis and filter design.

Problem Sets, Software Package, and Web Site

Problem solving is an essential part of learning mathematics, and we have tried to provide ample opportunities for the student to do so. After each section there are problem sets with a variety of exercises. Many allow students to fill in gaps in proofs from the text narrative, as well as to provide proofs similar to those given in the text. Others are fairly routine paper-pencil exercises to ensure that students understand examples, theorem statements, or algorithms. Many require computer work, as discussed in the next paragraph. We have provided 430 problems in the book to facilitate student comprehension material covered. Problems marked with a * should be assigned and address ideas that are used later in the text.

Many concepts in the book are better understood with the aid of computer vi-sualization and computation. For these reasons, we have built the software pack-age ContinuousWavelets to enhance student learning. This package is modeled after the DiscreteWavelets package that accompanies Van Fleet's book [60]. These packages are available for use with the computer algebra systems (CAS)

PREFACE XVÜ

Mathematica®, Matlab®, and Maple . This new package is used in the text to investigate a number of topics and to explore applications. Both packages contain modules for producing all the filters introduced in the course as well as discrete trans-formations and their inverses for use in applications. Visualization tools are also provided to help the reader better understand the results of transformations. Modules are provided for applications such as data compression, signal/image denoising, and image segmentation. The Cont inuousWavelet s package includes routines for con-structing scaling functions (via the cascade algorithm) and wavelet functions. Finally, there are routines to easily implement the ideas from Chapter 3 — students can easily construct piecewise constant functions and produce nice graphs of projections into the various Vj and Wj spaces.

The course Web site is

http ://www.stthomas.edu/wavelets

On this site, visitors will find the software packages described above, several computer labs and projects of varying difficulty, instructor notes on teaching from the text, and some solutions to problems.

D A V I D K. R U C H

PATRICK J. VAN FLEET

Denver, Colorado USA St. Paul, Minnesota USA March 2009

This page intentionally left blank

Acknowledgments

We are grateful to several people who helped us with the manuscript. Caroline Haddad from SUNY Geneseo, Laurel Rogers, and University of St. Thomas mathematicians Doug Dokken, Eric Rawdon, Melissa Shepart-Loe, and Magdalena Stolarska read versions of the first four chapters. They caught several errors and made many sug-gestions for improving the presentation.

We gratefully acknowledge the National Science Foundation for their support through a grant (DUE-0717662) for the development of the book and the computer software. We wish to thank our editor Susanne Steitz-Filler, for her help on the project. Radka Tezaur and David Kubes provided digital images that were essential in the presentation. We also wish to salute our colleagues Peter Massopust, Wasin So, and Jianzhang Wang, with whom we began our journey in this field in the 1990s.

Dave Ruch would like to thank his partner, Tia, for her support and continual reminders of the importance of applications, and their son, Alex, who worked through some of the Haar material for a school project.

Patrick Van Fleet would like to express his deep gratitude to his wife, Verena, and their three children, Sam, Matt, and Rachel. They allowed him time to work on the book and provided support in a multitude of ways. The project would not have been possible without their support and sacrifice.

D.K.R. and P. V.F.

XIX

This page intentionally left blank

CHAPTER 1

THE COMPLEX PLANE AND THE SPACE L2(R)

We make extensive use of complex numbers throughout the book. Thus for the purposes of making the book self-contained, this chapter begins with a review of the complex plane and basic operations with complex numbers. To build wavelet functions, we need to define the proper space of functions in which to perform our constructions. The space L2(R) lends itself well to this task, and we introduce this space in Section 1.2.

We discuss the inner product in L2(R) in Section 1.3, as well as vector spaces and subspaces. In Section 1.4 we talk about bases for L2(R). The construction of wavelet functions requires the decomposition of L2(R) into nested subspaces. We frequently need to approximate a function f(t) G L2(R) in these subspaces. The tool we use to form the approximation is the projection operator. We discuss (orthogonal) projections in Section 1.4.

1.1 COMPLEX NUMBERS AND BASIC OPERATIONS

Any discussion of the complex plane starts with the definition of the imaginary unit:

i = \TI\ Wavelet Theory: An Elementary Approach with Applications. By D. K. Ruch and P. J. Van Fleet 1 Copyright © 2009 John Wiley & Sons, Inc.

2 THE COMPLEX PLANE AND THE SPACE L 2 (R)

We immediately see that

i2 = ( v ^ ) 2 = - 1 , i3 = i2 · i = -i, i4 = (-1) · (-1) = 1

In Problem 1.1 you will compute in for any integer n. A complex number is any number of the form z — a + bi where a, b G M. The

number a is called the real part oiz and b is called the imaginary part oiz. The set of complex numbers will be denoted by C. It is easy to see that R c C since real numbers are those complex numbers with the imaginary part equal zero.

We can use the complex plane to envision complex numbers. The complex plane is a two-dimensional plane where the horizontal axis is used for the real part of complex numbers and the vertical axis is used for the imaginary part of complex numbers. To plot the number z — a + bi, we simply plot the ordered pair (a, b). In Figure 1.1 we plot some complex numbers.

• 2 -1 +2i

1

Real -2 -1

• -1 -2-i

_2«

Imaginary

1 2

► - 2 i

Figure 1.1 Some complex numbers in the complex plane.

Complex Addition and Multiplication

Addition and subtraction of complex numbers is a straightforward process. Addition of two complex numbers u = a + bi and v = c + di is defined as y = u + v = (a + c) + (b + d)z. Subtraction is similar: z = t¿ — ^ = (a — c) + (6 — d)z.

To multiply the complex numbers u — a-\-bi and v = c + di, we proceed just as we would if a + 6z and c + di were binomials:

i¿ · i; = (a + 6z)(c + di) = ac + adi -f bei + frdz2 = (ac — bd) + (ad + bc)i

Example 1.1 (Complex Arithmetic) Let u = 2 + i, v = — 1 — i, y = 2z, awJ 2 = 3 + 2i. Compute u + v, z — v, u · y, and v · 2.

1.1 COMPLEX NUMBERS AND BASIC OPERATIONS 3

Solution

u + v = (2-l) + (l-l)i = l Z-V = (Z- (_i)) + (2 - (_1))¿ = 4 + 3z

u · y = (2 + z) · 2i = 4z + 2i2 = - 2 + 4z v · 2 = ( -1 - i) - (3 + 2i) = (3 ( - l ) - (-1)2) + (3 ( - l ) + 2(-l))z - - 1 - hi

Complex Conjugation

One of the most important operations used to work with complex numbers is conju-gation.

Definition 1.1 (Conjugate of a Complex Numbers) Let z — a + bi e C. The conjugate ofz, denoted by ~z, is defined by

~z = a — bi

Conjugation is used to divide two complex numbers and also has a natural relation to the length of a complex number.

To plot z = a + bi, we plot the ordered pair (a, b) in the complex plane. For the conjugate ~z — a — bi, we plot the ordered pair (a, —b). So geometrically speaking, the conjugate ~z of z is simply the reflection ofz over the real axis. In Figure 1.2 we have plotted several complex numbers and their conjugates.

• u = - l +2 i

teal - 1

F = - l - 2 i •

2

l·

~1<

~2

Imaginary

►v = i

w = w = 1

Î

►r = - i

• z = 2

2

5 = 2 •

Figure 1.2 Complex numbers and their conjugates in the complex plane.

4 THE COMPLEX PLANE AND THE SPACE L2 (R)

A couple of properties of the conjugation operator are immediate and we state them in the proposition below. The proof is left as Problem 1.3.

Proposition 1.1 (Properties of the Conjugation Operator) Let z complex number. Then

(a) ~z = z

(b) Z G ! if and only if~z = z

a -f bi be a

Proof: Problem 1.3. ■

Note that if we graph the points z = cos Θ + i sin θ as Θ ranges from 0 to 2π, we trace a circle with center (0,0) with radius 1 in a counterclockwise manner. Note that if we produce the graph of z = cos Θ — i sin Θ as Θ ranges from 0 to 2π, we get the same picture, but the points are drawn in a clockwise manner. Figure 1.3 illustrates this geometric interpretation of the conjugation operator.

z = cos v + i sm t

z = cos Θ - i sin Θ

Figure 1.3 A circle is traced in two ways. Both start at Θ — 0. As Θ ranges from 0 to 2π, the points z trace the circle in a counterclockwise manner while the points ~z trace the circle in a clockwise manner.

Modulus of a Complex Number

We can use the distance formula to determine how far the point z = a+bi is away from 0 = 0 + 0i in the complex plane. The distance is y/(a — 0)2 + (b — 0)2 = Va2 + b2. This computation gives rise to the following definition.

Definition 1.2 (Modulus of a Complex Number) The modulus of the complex num-ber z = a + bi is denoted by \z\ and is defined as

\z\ = y/a2 + b2

PROBLEMS 5

Other names for the value \z\ are length, absolute value, and norm ofz. There is a natural relationship between \z\ and z. If we compute the product z · z

where z = a + bi, we obtain

z - z = (a + bi)(a - bi) = a2 - 62i2 = a2 + b2

The right side of the equation above is simply \z\2 so we have the following useful identity:

\\z\2 = z-z\ (1.1)

In Problem 1.5 you are asked to compute the norms of some complex numbers.

Division of Complex Numbers

We next consider division of complex numbers. That is, given z — a + bi and y — c + di Φ 0, how do we express the quotient z/y as a complex number? We proceed by multiplying both the numerator and denominator of the quotient by y\

z a + bi a + bi c — di (ac + bd) H- {be — ad)i ac + bd be — ad . y c + di c + di c — di c2 + d? c2 + d? c2 + d2

PROBLEMS

1.1 Let n be any integer. Find a closed formula for in.

1.2 Plot the numbers 3 — i, bi, —1, and cos # + i sinö for Θ = 0, π /4 ,π /2 , 5π/6,π in the complex plane.

1.3 Prove Proposition 1.1.

1.4 Compute the following values.

(a) (3 - i) + (2 + i)

(b) (i + i ) - ( 3 T i )

(c) -i3 · ( -2 + 3i)

(d) (2 + 5i) · (4 - i)

(e) (2 + 5i) · (4 - i)

(f) ( 2 - i ) - i

(g) (l + ¿)-=-( l -¿)

1.5 For each complex number z, compute \z\.

(a) z = 2 + 3i

6 THE COMPLEX PLANE AND THE SPACE L 2 (R)

(b) z = 5

(c) z = -U

(d) z = tanfl + i where Θ G ( - § , f )

(e) z satisfies z · z = 6

1.6 Let 2 = a + 6z and y = c + di. For parts (a) - (d) show that:

(a) ψ~ζ = y-z

(b) |*| = \z\

(c) \yz\ = \y\ · |*|

(d) y + z = y + z

(e) Find the real and imaginary parts of z _ 1 = - . z

*1.7 Suppose z = a + bi with |z| = 1. Show that ~z = z - 1 .

*1.8 We can generalize Problem 1.6(d). Suppose that Zk = α^ + M , for fc = 1 , . . . ,n. Show that

n n n n Y2zk = ^2z^=^ak-iy^bk fc=l fc=l fc=l fe=l

*1.9 Suppose that J^ a& and ^ bk are convergent series where ak,bk G M. For fcez fcez

Zk = dk + ibk, k e Z, show that OO C O OO OO

fc=l fc=l fc=l fc=l

*1.10 The identity in this problem is key to the development of the material in Section 6.1. Suppose that z, w G C with \z\ = 1. Show that

\(z -w)(z- l/w)\ = \UJ\-1 \z - w\2

The following steps will help you organize your work:

(a) Using the fact that \z\ — 1, expand \z — w\ = [z — w)(z — w) to obtain

\z — w\ = 1 + \w\2 — vSz — wz

1.2 THE SPACE L 2 ( R ) 7

(b) Factor —wz l from the right side of the identity in part (a) and use Problem 1.7 to show that

, ,2 i / 2 ! + M 2 , w

\z — w\ = —wz ( z ——z + — w w

(c) Show that the quadratic on the right-hand side of part (b) can be factored as (z — w){z — 1/w).

(d) Take norms of both sides of the identity obtained in part (c) and simplify the result to complete the proof.

1.2 THE SPACE L2(R)

In order to create a mathematical model with which to build wavelet transforms, it is important that we work in a vector space that lends itself to applications in digital imaging and signal processing. Unlike RN, where elements of the space are iV-tuples v = (i>i,..., VN)T, elements of our space will be functions. We can view a digital image as a function of two variables where the function value is the gray-level intensity, and we can view audio signals as functions of time where the function values are the frequencies of the signal. Since audio signals and digital images can have abrupt changes, we will not require functions in our space to necessarily be continuous. Since audio signals are constructed of sines and cosines and these functions are defined over all real numbers, we want to allow our space to hold functions that are supported (the notion of support is formally provided in Definition 1.5) on R. Since rows or columns of digital images usually are of finite dimension and audio signals taper off, we want to make sure that the functions f{t) in our space decay sufficiently fast as t —> ±00. The rate of decay must be fast enough to ensure that the energy of the signal is finite. (We will soon make precise what we mean by the energy of a function.) Finally, it is desirable from a mathematical standpoint to use a space where the inner product of a function with itself is related to the size (norm) of the function. For this reason, we will work in the space L2(R). We define it now.

L2(R) Defined

Definition 1.3 (The Space L2(R)) We define the space L2(R) to be the set

LZ(R) = {f:R-^C\ / \f(t)\zdt<oo /l/WI (1.2)

8 THE COMPLEX PLANE AND THE SPACE L 2 (R)

Note: A reader with some background in analysis will understand that a rigorous definition of L2(R) requires knowledge of the Lebesgue integral and sets of measure zero. If the reader is willing to accept some basic properties obeyed by Lebesgue integrals, then Definition 1.3 will suffice.

We define the norm of a function in L2(R) as follows:

Definition 1.4 (The L2(R) Norm) Let f(t) G L2(R). Then the norm off(t) is

(1.3)

The norm of the function is also referred to as the energy of the function. There are several properties that the norm should satisfy. Since it is a measure of energy or size, it should be nonnegative. Moreover, it is natural to expect that the only function for which | | /(t) | | = 0 is f(t) = 0. Some clarification of this property is in order before we proceed.

If f(t) = 0 foralli G R, then certainly \f(t)\2 = 0, so that | | /(t) | | = 0. But what about the function that is 0 everywhere except, say, for a finite number of values? It is certainly possible that a signal might have such abrupt changes at a finite set of points. We learned in calculus that such a finite set of points has no bearing on the integral. That is, for a < c < b, /(c) might not even be defined, but

rb pL pb / f(t)dt= lim / / ( t ) d í + lim / f(t)dt

could very well exist. This is certainly the case when f(t) = 0 except at a finite number of values.

This idea is generalized using the notion of measurable sets. Intervals (a, b) are measured by their length b — a, and in general, sets are measured by writing them as a limit of the union of nonintersecting intervals. The measure of a single point a is 0, since for an arbitrarily small positive measure e > 0, we can find an interval that contains a and has measure less than e (the interval (a — e/4, a + e/4) with measure e/2 works). We can generalize this argument to claim that a finite set of points has measure 0 as well. The general definition of sets of measure 0 is typically covered in an analysis text (see Rudin [48], for example).

The previous discussion leads us to the notion of equivalentfunctions. Two func-tions f(t) and g {t) are said to be equivalent if fit) = g(t) except on a set of measure 0.

We state the following proposition without proof.

Proposition 1.2 (Functions for Which ||f(t)|| = 0) Suppose that f(i) G L2(R). Then \\f(t) || = 0 if and only if' f{t) = 0 except on a set of measure 0. ■