discrete sampling in continuous time, a dissertationch771fw4323/... · 2013. 6. 17. · john pauly,...
TRANSCRIPT
![Page 1: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/1.jpg)
DISCRETE SAMPLING IN CONTINUOUS TIME,
BAND-LIMITED SIGNALS
A DISSERTATION
SUBMITTED TO THE DEPARTMENT OF ELECTRICAL
ENGINEERING
AND THE COMMITTEE ON GRADUATE STUDIES
OF STANFORD UNIVERSITY
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
Fernando Gomez-Pancorbo
March 2012
![Page 2: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/2.jpg)
http://creativecommons.org/licenses/by-nc/3.0/us/
This dissertation is online at: http://purl.stanford.edu/ch771fw4323
Includes supplemental files:
1. Matlab code that reproduces the numerical examples mentioned in the dissertation
(matlab_files.zip)
© 2012 by Fernando Gomez Pancorbo. All Rights Reserved.
Re-distributed by Stanford University under license with the author.
This work is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.
ii
![Page 3: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/3.jpg)
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Brad Osgood, Primary Adviser
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
John Pauly, Co-Adviser
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Julius Smith, III
Approved for the Stanford University Committee on Graduate Studies.
Patricia J. Gumport, Vice Provost Graduate Education
This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.
iii
![Page 4: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/4.jpg)
iv
![Page 5: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/5.jpg)
To my parents Pedro and Marıa Dolores,
and the loving memory of my friend May Zhou
v
![Page 6: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/6.jpg)
vi
![Page 7: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/7.jpg)
Abstract
During more than 60 years, the paradigm implied by the celebrated Shannon sampling
theorem has been the basis of most ADC and DAC architectures, with the exception
of the so called oversampling converters. The improvements in semiconductor tech-
nology allowed for little change in these design architectures while faster, lower power
converters and new generations of manufacturing processes were developed. Over the
last 5 to 10 years, the point of diminishing returns with respect to scaling seems to
have been reached. As a result, there is a renewed interest in researching alternative
sampling paradigms to Shannon’s.
Discrete Sampling is one of the numerous sampling alternatives proposed recently.
In this research we show how Discrete Sampling can be extended to work with the
class of continuous time band-limited signals. In addition to several mathematical
results on the matter, the actual implementation of the paradigm is also considered.
It will be shown that the proposed scheme results in a particular type of Hybrid Filter
Bank. Several examples of such banks are presented together with their performance
when sampling and reconstructing real life audio signals.
vii
![Page 8: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/8.jpg)
viii
![Page 9: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/9.jpg)
Acknowledgments
I would not have completed successfully this dissertation without the help, guidance
and mentorship of numerous people whom I want to sincerely thank for their support.
First and foremost I am deeply thankful to my adviser, Professor Brad Osgood, for
giving me the opportunity to work with him. Incidentally, his famous Fourier course
EE261 is the first class that I took as a graduate student in Electrical Engineering.
He set his teaching standard so high that no other professor has come close to it
in any of the classes that I have taken to complete the coursework requirements of
the Master of Science and PhD degrees. In fact, the only reason I took the Linear
Dynamical Systems course EE263 is that it so happened that he taught it during the
Spring Quarter of the 2006-2007 academic year. It goes without saying that I have
nothing but great admiration for the fantastic professors that I have had in my other
classes. In addition, he has been a great research adviser, mentor, and an even better
human being who has always been concerned about my well being and my graduate
student experience. It has been a great honor to work with him during the last three
years.
I want to thank Professor Boris Murmann for advising me during my first two years
as PhD student. Under his guidance, I learned a great deal about what makes data
converters work in real life. The lessons I learned with him inspired a great deal of my
work in extending Discrete Sampling to work with infinitely long, square summable
discrete sequences. During that time, I was co-advised by Professor Tsachy Weiss-
man whom I also want to thank for his critical support and for the many stimulating
discussions on modeling noise. My deep gratitude also goes to Professor John Pauly,
ix
![Page 10: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/10.jpg)
who graciously accepted to serve as my research co-advisor and a reader for this dis-
sertation, Professor Julius Smith, for the wonderful learning experience in his course
Music 421 on Audio Applications of the FFT and for agreeing to be a reader of this
thesis, Professor Jonathan Abel, for accepting to be part of my PhD Orals committee
in a very short notice, and Professor John Eaton, for kindly chairing my PhD Orals.
I am highly indebted to the various people and organizations that have provided me
with financial assistance during my graduate student years, allowing me to gradu-
ate without significant financial burden. Professor Butrus Khuri-Yakub’s rotating
research assistantship funded my first year which I spent doing research at the labo-
ratories of Professors Robert Byer, Philip Wong and Boris Murmann. My next three
years were funded by Professor Boris Murmann with grants from the Robert Bosch
RTC, Research Technology Center. Christoph Lang, manager at RTC, was instru-
mental in setting up this collaboration which not only allowed me to learn a great
deal about automotive electronics but also resulted in United States Patent 7,424,407.
Finally, the funding for my final years was provided through a consultant position
at the Information Surfaces Laboratory at Hewlett Packard Laboratories, that was
offered to me by managers Carl Taussig and Rich Elder. The last three years working
with their team have been exhilarating; they have put together an exceptional group
of researchers who are also exceptional human beings.
Life with the administrative side of the Electrical Engineering Department was made
easy thanks to several of its staff members whose assistance I want to acknowledge:
Denise Murphy, Kelly Yilmaz, Ann Guerra, Natasha Newson, Diane Shankle, Vicky
Carrillo, Amy Duncan and Rafael Ulate. Over the years, Rafael and I became great
friends; I am still in touch with him even after he moved to Santa Clara University.
I have made many friends during my years as graduate student, I want to thank
them for their continuing support and for making my graduate student life richer
on the human side. I want to name a few special ones, although that doesn’t mean
that I have forgotten about the rest: Oren Feinstein, Mahdieh Bagher Shemirani,
Babak Javid, Shen Ren, Pedram Lajevardi, John Cunningham, Grace Gao, Henrique
Miranda, Daniel Fernandes, Isaac Martinez, Deji Akinwande, Mehdi Jahanbakht,
Maryam Fathi, Mohammad Hekmat, Gunhan Ertosun, Kunal Ghosh, Guangyu Shi,
x
![Page 11: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/11.jpg)
Luis Adarve, Tom Lee, Kamil Dada, Xiaowei Ding, James Hohmann, Paul Gurney,
Kivanc Ozonat and Borja Manuel Peleato-Inarrea.
Of course I couldn’t leave out of the acknowledgments section the colleagues that
have made my experience at the different labs enjoyable and fun: for Professor’s
Murmann Group I want to thank Drew Hall, Yoonyoung Chung, Parastoo Nikaeen,
Ross Walker, Alex Guo, Ray Nguyen, Noam Dolev, Clay Daigle, Jim Salvia, Jason
Hu, Echere Iroaga, Alireza Dastgheib, Manar El-Chammas, Yangjin Oh, Christo-
pher Anderson, Justin Kim, Wei Xiong, Donghyun Kim; for Bosch Research and
Technology Center I want to thank Vladimir Petkov, Xinyu Xing, Johan Vanderhae-
gen, Chinwuba Ezekwe and Thomas Rocznik; for HP Labs I want to thank Marcia
Almanza-Workman, Warren Jackson, Hao Luo, Lihua Zhao, Steven Trovinger, Mark
Smith, Craig Perlov, Ping Mei, John Maltabes, Alejandro de la Fuente Vornbrock,
Ohseung Kwon, Han-Jun Kim, Albert Jeans, Mehrban Jam, Edward Holland, Robert
Garcia, Robert Cobene, James Brug, Alesha Cater and Lois Masciola; and last, but
certainly not least, for Professors Brad Osgood and John Gill groups I want to thank
William Wu, Joseph Koo, Aditya Siripuram and Ulrich Barnhoefer.
Amongst the most significant events that happened during my graduate career, going
back to God, after spending many years as an atheist, outshines the rest. In fact,
it’s more accurate to say that He found me and welcomed me back with open arms.
My Bible study brothers have helped my faith grow and have been very supportive
all these years. I am particularly thankful to Kary Eldred for inviting me to attend
the group meetings. My Bible study group members are Alan Hsai, John Lovewell,
Bob Larson, Brian Keating, Derek Blazensky, Erik Rannala, Gabriel Yu, Gibs Sung-
Ku Song, Harry Bims, John Shenk, John Mumford, Ken Eldred, Ken Korea, Kevon
Saber, Lee Kenna, Paul Kim, Paul Chau, Phil Larson, Phil Taylor, Philip Larson,
Preston Butcher, Robert Larson and Scott Feamster.
Of course, I would not have been able to come to Stanford University without the
great education that I received during my K-12 schooling and college years. I am very
thankful to the continuous support and encouragement from my teachers at Primary
School “Virgen de La Oliva” in Carcastillo, Spain, and at High School “Marques de
Villena” in Marcilla, Spain as well as from Professors Rafael Cabeza, Sonia Porta
xi
![Page 12: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/12.jpg)
and Antonio Lumbreras at UPNA, Universidad Publica de Navarra in Pamplona,
Spain. Professor Antonio Lumbreras greatly assisted me in continuing my under-
graduate education at ESIEE-Paris, Ecole Superieure d’Ingenieurs en Electronique et
Electrotechnique de Paris, where I spent two amazing years. I want to thank Profes-
sors Jean Michel Le Notre and Martine Villegas for their support during my time at
ESIEE-Paris.
My parents Pedro and Marıa Dolores worked very hard so that my siblings, Marıa
Guadalupe, Pedro, Laura, and I could have the opportunity to go to college. My
older sister Marıa Guadalupe was the first member of the family to go to college. I
was the first member of the family to get a scientific degree. What my parents could
not envision is that one day I would be getting a doctoral degree from one of the
greatest research universities in the world. My PhD degree is a testament to their
dedication and sacrifice.
During my Stanford years, I also had the bad fortune of losing two good friends in
tragic circumstances. I want to honor the memory of Archie Li, and of my fellow
graduate student and classmate May Zhou. May’s death marked a before and after
moment in my life; her loss made me value the many things that I had taken for
granted until then. It made me understand what truly matters in life and to value
the precious gift of being alive. I just wish that I could have learned these lessons in
a different way.
xii
![Page 13: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/13.jpg)
Preface
Sampling is one of the most fascinating areas in the field of signal processing. The
idea that one can put information in the form of a finite length string of zeros and ones
that can be used at a later time to reconstruct a continuous time signal is mind bog-
gling. When Shannon presented in 1949 his famous sampling theorem that explained
how that feat can be accomplished, provided that the continuous time signal is band-
limited, he wasn’t probably aware of the revolution that he was about to unleash.
Today we take high quality digital cameras, audio CDs, DVDs and so many other
technologies that rely on analog to digital conversion, such as wireless technology, for
granted; in fact, for the people of my age or younger it’s hard to imagine a world
in which none of these existed. The vast majority of Analog to Digital Converters,
ADCs, designed over the years are based on Shanon’s paradigm that one can recover
a continuous time band-limited signal, with frequency components up to BHz, from
samples of the signal taken at regularly spaced time points with the distance between
points equal to 12B
. During the decades following Shannon’s paper most research
about solving the sampling problem differently was motivated by mere intellectual
curiosity since for most ADCs the Shannon paradigm, combined with the benefits of
increasingly small transistors allowed by semiconductor technology advances, made it
possible to build faster and more accurate converters. Today, technology scaling has
reached the point where further miniaturization only provides diminishing returns if
we insist in using the Shannon paradigm. As a result there is a new push for research
in sampling theory this time motivated by applications in the real world rather than
by academic reasons. In this thesis I present the results of my doctoral research,
xiii
![Page 14: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/14.jpg)
aimed at extending one of the several alternative sampling approaches developed re-
cently, Discrete Sampling, to the class of continuous time band-limited signals. The
question explored was whether Discrete Sampling could produce a conversion scheme
that would allow the recovery of such signals when sampled at rates below Shannon’s
rate, also known as Nyquist’s rate. As we shall see, the answer in some cases is posi-
tive, provided that we make extra assumptions about the continuous time signal, in
addition to its band-limitedness. In addition, we showed that any continuous time,
band-limited signal can be fully recovered with a scheme that while needing as many
samples as in the case of the Shannon’s paradigm, it consists of several converters
working in parallel, each at a sampling rate below Shannon’s.
Chapter 1 of the thesis gives details about how the Shannon paradigm is implemented
with electrical circuits and explores in detail the reasons that drive the current interest
in sampling theory. This chapter also gives an introduction to Discrete Sampling, a
theory that allows the reconstruction of finite length discrete signals through discrete
interpolation from a set of samples whose size is strictly smaller than the length of the
original signal. Chapter 2 gives an introduction to the mathematical theory of frames,
which will be later used in Chapter 3 to extend Discrete Sampling to infinitely long
discrete signals that belong to `2(R). Chapter 4 explores the circuit implementation
of the new sampling scheme implied by the mathematical results developed in Chap-
ter 3. These two Chapters, 3 and 4, contain my research contributions. Chapters 1
and 2 summarize existing knowledge which is a requisite to develop the new results
contained the the later chapters. Finally in Chapter 5, I reflect on my research work
during the last three years as well as propose research areas that could be explored
in the future that would facilitate the adoption of the new sampling scheme by the
community of ADC designers.
xiv
![Page 15: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/15.jpg)
Contents
v
Abstract vii
Acknowledgments ix
Preface xiii
1 Challenges in Practical Sampling 1
1.1 Real Life Analog to Digital Conversion . . . . . . . . . . . . . . . . . 1
1.1.1 Amplitude Quantization . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 SNDR and ENOB . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.3 Practical Anti-Aliasing Filters . . . . . . . . . . . . . . . . . . 4
1.1.4 Practical Sampling . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.5 Quantization, ADC Architectures, Energy Trade-offs . . . . . 8
1.2 Real Life Digital to Analog Conversion . . . . . . . . . . . . . . . . . 9
1.3 Introduction to Discrete Sampling . . . . . . . . . . . . . . . . . . . . 11
1.4 Other Research Approaches . . . . . . . . . . . . . . . . . . . . . . . 13
2 Introduction to Frame Theory 17
2.1 Bases in Infinite-Dimensional Hilbert Spaces . . . . . . . . . . . . . . 17
2.1.1 Orthonormal Bases . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.2 Riesz Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Frames in Infinite-dimensional Hilbert Spaces . . . . . . . . . . . . . 21
xv
![Page 16: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/16.jpg)
2.2.1 Definition and Properties of Frames . . . . . . . . . . . . . . . 21
2.2.2 Frames and Riesz Bases . . . . . . . . . . . . . . . . . . . . . 24
2.2.3 Frames and Orthogonal Projections . . . . . . . . . . . . . . . 24
2.2.4 An Illustrating Example, the PWTs Space . . . . . . . . . . . 25
3 Discrete Sampling for Signals in PWTs 27
3.1 Infinitely Long Discrete Signals in `2(Z) . . . . . . . . . . . . . . . . 28
3.1.1 Equivalence of Continuous Time Inner Products With Discrete
Time Inner Products . . . . . . . . . . . . . . . . . . . . . . . 30
3.2 Interpolation of Signals in PWTs . . . . . . . . . . . . . . . . . . . . . 31
3.2.1 Interpolation in Continuous Time, Band-Limited, Shift Invari-
ant Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.3 Frames of Shift Invariant Subspaces of PWTs . . . . . . . . . . . . . 42
3.3.1 Orthogonal Interpolating Atoms . . . . . . . . . . . . . . . . . 46
3.3.2 Computing the Dual Riesz Basis For Compactly Supported c[k] 46
3.3.3 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3.4 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4 Discrete Sampling and HBF’s 53
4.1 A Bit on History of HBF’s . . . . . . . . . . . . . . . . . . . . . . . . 54
4.2 Union of Orthogonal SI Subspaces With ON Generators . . . . . . . 55
4.3 Very Brief Overview of Wavelet Theory . . . . . . . . . . . . . . . . . 56
4.3.1 Multiresolution Representations . . . . . . . . . . . . . . . . . 57
4.3.2 A Multiresolution Approximation is a Collection of Shift In-
variant Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.3.3 Conjugate Mirror Filters . . . . . . . . . . . . . . . . . . . . . 59
4.3.4 Wavelet Decomposition . . . . . . . . . . . . . . . . . . . . . . 60
4.3.5 Fast Orthogonal Wavelet Transform . . . . . . . . . . . . . . . 61
4.4 Discrete Sampling for PWTs Meets Wavelet Theory . . . . . . . . . . 64
4.4.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.5 Noise Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.5.1 A More Detailed Look at the Discrete Interpolation Process . 71
xvi
![Page 17: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/17.jpg)
4.5.2 Linear Filtering of a Zero Mean Gaussian iid Process . . . . . 73
4.5.3 Noise Analysis in an HBF Designed with Discrete Sampling . 74
4.6 Practical Imprementation of Continuous Time Filters . . . . . . . . . 78
4.6.1 Computing H(s) for Impulse Response u(−t) . . . . . . . . . 80
4.6.2 Design Example . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5 Conclusions and Future Research 91
5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Bibliography 95
xvii
![Page 18: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/18.jpg)
xviii
![Page 19: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/19.jpg)
List of Tables
3.1 u[n] Associated with φ12, Wavelet Decomposition Symmlet 8 J=12,
L=10; see figure 4.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.2 c[k] Associated with u[n] Shown in Table 3.1 . . . . . . . . . . . . . . 49
3.3 Coefficients u[nM ] computed from c[k] shown in Table 3.2 . . . . . . 49
3.4 u[n] Associated With u[n] Shown in Table 3.1 . . . . . . . . . . . . . 50
3.5 d[k] Associated with c[k] Shown in Table 3.2 . . . . . . . . . . . . . . 50
4.1 Numerical Results for 32s Segment of Pageant . . . . . . . . . . . . . 69
4.2 ||uqk||2 , Example of Subsection 4.4.1 . . . . . . . . . . . . . . . . . . 78
4.3 H1(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.4 H2(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
xix
![Page 20: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/20.jpg)
xx
![Page 21: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/21.jpg)
List of Figures
1.1 a) Sampling Grid b)Block Diagrams of Practical ADCs and DACs [1] 2
1.2 a) Practical Anti-Aliasing Filter b)Design Rules [1] . . . . . . . . . . 4
1.3 a) Ideal Track & Hold Stage b)Waveforms [1] . . . . . . . . . . . . . 5
1.4 Maximum Input Signal Frequency vs SNRaperture for Different Jitter
Scenarios [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 a) Zero Order Hold b) Effect of Zero Order Hold in the Spectrum of a
Signal [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.6 State of the Art in ADC design, 2011 [5] . . . . . . . . . . . . . . . . 14
3.1 Sampling Scheme With Interpolation for Continuous Time, Band-Limited
Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2 Magnitude of the Spectrum of cos(2π 132n) . . . . . . . . . . . . . . . 33
3.3 Magnitude of the Spectrum of ↑↓ cos(2π 132n) . . . . . . . . . . . . . . 33
3.4 Magnitude of the Spectrum of cos(2π 132n).w[n] . . . . . . . . . . . . . 34
3.5 Magnitude of the Spectrum of ↑↓ {cos(2π 132n).w[n]} . . . . . . . . . . 34
3.6 Interpolating Function u[n] for cos(2π 132n).w[n] . . . . . . . . . . . . 35
3.7 cos(2π 132n).w[n] (red) and its Reconstruction (blue) via Interpolation
With u[n] Shown in Figure 3.6 . . . . . . . . . . . . . . . . . . . . . 36
3.8 Example of a Signal that Doesn’t Satisfy Equation (3.4) with M=8 . 37
3.9 Spectrum of ↑↓ {x[n].w[n]} Associated With the Signal Shown in Fig-
ure 3.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.10 Illustration of Shift Invariant Subspace Sampling When {u[n−iM ]}k∈Zis a Riesz Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
xxi
![Page 22: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/22.jpg)
4.1 Hybrid Filter Bank Architecture, Adapted from [34] . . . . . . . . . . 54
4.2 DWT Decomposition (top), Inverse (bottom) [28] . . . . . . . . . . . 61
4.3 WPT Decomposition (top), Inverse (bottom) [28] . . . . . . . . . . . 62
4.4 HFB Associated with Decomposition Symmlet 8 J=12, L=10 . . . . . 64
4.5 Discrete Equivalent of Figure 4.4 . . . . . . . . . . . . . . . . . . . . 65
4.6 Symmlet 8 Generators With J=12, L=10 . . . . . . . . . . . . . . . . 66
4.7 Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With φ12 67
4.8 Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ12 67
4.9 Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ11 68
4.10 Smoothed Magnitude Spectrum of Signals x[n][blue], xφ12 [n] [red] and
xφ12ψ12 [n] [green] for 32s Segment of Theme Pageant . . . . . . . . . . 70
4.11 Polyphase Decomposition of uφ12 [n], from Wavelet Decomposition Symm-
let 8 With J=12, L=10 . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.12 Interpolation From The Polyphase Decomposition Point of View . . . 74
4.13 Noise Propagation in a Discrete Sampling HBF . . . . . . . . . . . . 76
4.14 Idealized Filter Frequency Responses [43] . . . . . . . . . . . . . . . . 80
4.15 Analog Filter Design Parameters [43] . . . . . . . . . . . . . . . . . . 81
4.16 Magnitude of L{uφ12(−t)}(jω), Example Subsection 4.4.1 . . . . . . . 82
4.17 uψ[n] and uψ[n] Associated with ψ[n], Beylkin Wavelet Decomposition
2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.18 uφ[n] and uφ[n] Associated with φ[n], Beylkin Wavelet Decomposition
2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.19 Tuncated uψ[−n] and uφ[−n] for Beylkin Wavelet Decomposition 2
Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.20 DuTψ(j2πfn) and DuTφ(j2πfn) for Beylkin Wavelet Decomposition 2
Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.21 Block Diagram of HBF for Beylkin Wavelet Decomposition 2 Subspaces 89
4.22 Discrete Sinc Based Interpolating Filter Used to Resample x(t) at Rate
0.25Hz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
xxii
![Page 23: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/23.jpg)
Chapter 1
Challenges in Practical Sampling
A research work about sampling must consider practical implementation if it hopes
to be adopted by the community of mixed signal circuit designers, the very people
who design and implement real life data converters. In this chapter I will give an
overview of how sampling is implemented today as well as the numerous challenges
that must be dealt with when designing a real life Analog to Digital Converter, ADC.
We will see that some of these challenges can be alleviated with Discrete Sampling
at the expense of a more complicated analog front end. I will also talk briefly about
other approaches currently being investigated by other researchers to address these
difficulties.
1.1 Real Life Analog to Digital Conversion
The observations of this section are based on [1]. In addition to taking this class
during my PhD studies, I also had the opportunity to serve as the lead TA when the
class was taught as EE315 in Spring 2008.
1.1.1 Amplitude Quantization
When the problem of sampling is presented in a signal processing class, it is usually
the case that the exposition begins by assuming that there is some continuous time
1
![Page 24: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/24.jpg)
2 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
signal x(t) which we want to fully recover from its samples taken at regular time
instances nTs, where n is an integer that goes from −∞ to +∞. Thus the problem is
how we get x(t) back, assuming that one has measured x[n] = x(nTs). The celebrated
Shannon-Nyquist sampling theorem [2] says that if the Fourier Transform of x(t) is
contained in [ −12Ts, 1
2Ts] then x(t) can be recovered exactly by applying the following
reconstruction formula, which is known in the literature as the sinc interpolation
formula,
x(t) =+∞∑
n=−∞
x[n]sinc
(t− nTsTs
)(1.1)
Figure 1.1: a) Sampling Grid b)Block Diagrams of Practical ADCs and DACs [1]
In practical analog to digital conversion, the Shannon sampling theorem only ad-
dresses part of the problem. Since computers must work with finite precision, all
samples x[n] must be discretized in amplitude before they can be passed to the next
processing stage. As shown in figure 1.1a), the sampling process must fit a signal,
which is continuous in time and amplitude, into a grid that results from discretiz-
ing time and amplitude. The discretization in amplitude is usually uniform with
the quantization step equal to ∆. If we have an nb bits converter, the error in am-
plitude incurred by considering a discrete point in the grid, instead of x(nTs), will
be bound by ±∆2
as long as the amplitude of x(t) stays within the so called full
![Page 25: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/25.jpg)
1.1. REAL LIFE ANALOG TO DIGITAL CONVERSION 3
scale range, FSR = 2nb∆. For this analysis to hold, it doesn’t matter whether
the amplitude of x(t) stays within [0, FSR], [−FSR2, FSR
2] or other arrangements as
long as |max(x(t)) −min(x(t))| = FSR. With this setup, the effect of quantization
in amplitude can be modeled via a zero mean uniform random i.i.d. process, eq[n]
∼ U(−∆2, ∆
2),[3],[1], that is added to x(nTs). Thus, instead of measuring x[n] = x(nTs)
we will have
x[n] = x(nTs) + eq[n] (1.2)
This model works well in practice for signals whose amplitude varies “sufficiently”
from one sample to the next, such as sinusoids of frequency less than 12Ts
, whose
amplitude covers the full scale range. The average energy error due to quantization
can be computed with this model as the variance of the process, e2q[n] = ∆2
12.
1.1.2 SNDR and ENOB
We introduce the metric known in the ADC community as SNDR, which stands for
Signal to Noise and Distortion Ratio, and its companion ENOB, which stands for
Equivalent Number Of Bits. Suppose we have a test signal, typically a full scale
range sinusoid of frequency below fs2
, with fs the Nyquist frequency of the ADC, that
we inject to the ADC in order to measure its non idealities. Then,
SNDR = 10 log10
(Psignal
Pquantization error + Pnoise + Pdistortion
)(1.3)
Where Psignal is the total power of the signal used in the test of the ADC, Pquantization error
is the quantization error described in the previous section, Pnoise is the total power of
electrical noise present at the output, and Pdistortion, is the total power of nonlinear
distortion at the output. The metric ENOB defined as SNDR[dB]−1.766.03
gives us the
resolution of a perfect ADC whose SNDR is affected only by quantization noise that
has the same SNDR as the ADC under consideration. For example, say that we have
a 14 bit ADC whose SNDR is 74.12 dB. The resulting ENOB is 12 bit which means
that this 14 bit ADC is behaving in terms of accuracy as a perfect 12 bit ADC that
![Page 26: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/26.jpg)
4 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
has no noise or nonlinear distortion, only quantization error.
1.1.3 Practical Anti-Aliasing Filters
Figure 1.2: a) Practical Anti-Aliasing Filter b)Design Rules [1]
As shown in figure 1.1b), it is a common practice to place a so called anti-aliasing
filter prior to sampling x(t), in order to prevent information contained in frequencies
|f | ≥ 12Ts
from appearing in equation (1.1) during reconstruction. An ideal anti-
aliasing filter has a brick wall frequency response. The real situation is more like the
one shown in figure 1.2 a) where a practical low pass filter, with a smooth transition
between the pass band and the stop band, is used as antialiasing filter. The higher
the order of the low pass filter the steeper the transition but the more complicated is
the design as well. Also the bigger the sampling frequency is than twice the maximum
frequency of the pass band, the more attenuated are the parasitic tones that could
get into the pass band. Figure 1.2b) shows the design trade off between filter order
and the ratio of the sampling frequency to the maximum pass band frequency. In
high speed ADCs, a high sampling frequency might not be an option, leaving no
alternative other than a high order for the low pass filter in such cases.
![Page 27: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/27.jpg)
1.1. REAL LIFE ANALOG TO DIGITAL CONVERSION 5
Figure 1.3: a) Ideal Track & Hold Stage b)Waveforms [1]
1.1.4 Practical Sampling
The block identified as “sampling” in figure 1.1b), is usually implemented via a so
called Track and Hold, T&H, stage. Figure 1.3 shows an ideal T&H stage: during the
track phase the switch is closed so that Vout tracks Vin. When the switch is open the
voltage of the capacitor is held constant. During the hold phase quantization takes
place. Ideally, Ts = Ttrack + Thold with Ttrack = Thold = 0.5Ts. The ideal model is
never encountered in practice. All track and hold circuits suffer from a list of non
idealities:
• Finite acquisition time: this refers to the fact that, in practice, the switch shown
in figure 1.3a) is implemented via a transistor. When the transistor is on, it
has a resistance R greater than zero. Thus, during the track phase, the stage
becomes an RC circuit. In the worst case, the capacitor is initially discharged
and a voltage equal to the FSR is applied in Vin, thus Vout = VFSR(1 − e−tτ )
where τ = RC. Since VFSR = 2nb∆, the time it takes for Vout to be within 0.5∆
of Vin depends on the ADC’s nb. A quick calculation shows that settling Vout
with a 10 bit resolution takes 7.6 time constants τ ; settling Vout with a 14 bit
resolution takes 10.4 time constants τ , and so on. To alleviate this problem, we
would need to make τ = RC small. There is little that can be done to make R
smaller, since it depends on the technology. Making C smaller might or might
not be an option since a smaller C also means a higher electrical noise, as we
![Page 28: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/28.jpg)
6 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
will see in the next item.
• KTC
noise: the effect of R being greater than zero also means that it adds a noisy
thermal voltage, generated by the resistor being at a temperature greater than
the absolute zero, to the capacitor during the track phase. It can be shown that
if the number of time constants that we wait to settle is high, such as what is
required for achieving a resolution of 6 bits and higher, the effect on the discrete
time signal that results from measuring the capacitor voltage during the hold
phase is that of adding a discrete zero mean white Gaussian noise of varianceKTC
, where T is the temperature in Kelvin and K the Boltzmann constant. As
pointed out in the previous item, the value of the capacitor C results from a
trade off resolution, speed. A fast T&H stage will require a small capacitor
which in turn will mean higher KTC
noise. In such case the maximum ENOB
achievable in the absence of distortion, dictated by when KTC
becomes as big as
the quantization error, will be limited by thermal noise. On the other hand a
slow ADC allows for a bigger C, which in turns results in a smaller KTC
noise,
allowing for a higher resolution.
• Aperture uncertainty: this is due to the jitter present in the clock that feeds
the switch, which means that the actual moment at which the switch is opened
varies with each sample. A simple way to characterize its effect is to consider
the sampling of a sinusoid A cos(2πfint). If we represent the jitter by the zero
mean random variable τ , and it is small with respect to the sampling period,
we can say that the ∆Vin ≈ dVindtτ . Thus, the energy of the error introduced by
the clock jitter can be approximated by
E[∆V 2
in
]≈ E
[(dVindt
)2
τ 2
]= E
[(dA cos(2πfint)
dt
)2]E[τ 2] ≈ 1
2(2πAfin)2σ2
τ
(1.4)
since the jitter and the signal are statistically independent . We can then define
![Page 29: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/29.jpg)
1.1. REAL LIFE ANALOG TO DIGITAL CONVERSION 7
SNRaperture = 10 log10
(A2
212(2πAfin)2σ2
τ
)= 10 log10
(1
(2πfin)2σ2τ
)(1.5)
Figure 1.4 shows the input frequency fin associated with a level of SNRaperture.
A plot is provided for three different jitter scenarios. If we asume that there is
no source of error other than clock jitter then SNDR = SNRaperture. Under
that assumption the maximum sampling frequency, for a given level of SNDR
or ENOB, is twice fin. The plot highlights the fin associated with an SNDR
of 60 dB, which corresponds to 9.7 ENOB. The maximum sampling frequencies
in this scenario are 31.8 Mhz, 318 Mhz and 3.18 Ghz with 10ps, 1ps and 0.1ps
jitter rms, στ , respectively. Jitter is the non ideality that limits most severely
the ability of designing fast ADCs. Thus, if we are limited by the clock jitter
we will be interested in sampling approaches that use slow ADCs.
Figure 1.4: Maximum Input Signal Frequency vs SNRaperture for Different JitterScenarios [1]
• Signal dependent sampling instant: since the switch is usually implemented
![Page 30: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/30.jpg)
8 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
via a transistor, we have to deal with the fact that real transistors become
conductive only when the gate voltage, to which the clock is connected, minus
Vin is greater than a technology dependent threshold voltage. Therefore, the
sampling moment will depend on Vin. This problem can be alleviated if the
clock fall time is much bigger than the maximum dVindt
.
• Track mode nonlinearity: when the transistor becomes conductive it behaves as
a resistor whose resistance depends on Vin. Thus, there is a nonlinear response
added to Vout during track mode. It can be shown that if Vin is a sinusoid
of frequency fin, the nonlinearity caused by this effect is proportional to finfs
,
where fs is the sampling frequency. This non-ideality is usually dealt with using
circuit design techniques.
• Charge injection nonlinearity: when the switch is turned off, the charge stored
in the channel of the transistor doesn’t disappear instantaneously. Part of the
charge ends up in the output capacitor, which adds another nonlinear effect to
Vout. This effect can be mitigated by circuit design techniques as well.
The list of non-idealities presented above is non exhaustive but should give the reader
an idea of the challenges involved in the design of actual ADCs.
1.1.5 Quantization, ADC Architectures, Energy Trade-offs
Once we have designed a Track & Hold stage that meets our specifications, the prob-
lem remains of performing the quantization of the signal in amplitude. The direct
method, implemented by the so called flash ADCs, consists of comparing Vout with
a set of predefined voltage levels through an equal number of voltage comparators.
It is practical only for converters up to 6 bits. For a higher resolution, quantization
is performed in stages: a coarse quantization is performed in the first stage while
later stages perform finer and finer quantizations until the desired resolution level is
reached. The different ADC architectures, pipeline, SAR and cyclic, differ on how
this procedure is implemented.
![Page 31: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/31.jpg)
1.2. REAL LIFE DIGITAL TO ANALOG CONVERSION 9
All architectures mentioned up to this point fall in the category of so called Nyquist
ADCs since they implement the Shannon-Nyquist sampling paradigm. There exists
a different architecture called sigma-delta which works with a different paradigm.
Sigma-delta ADCs are also known as oversampling ADCs because they sample the
signal at rates several times higher, 64 to 128 are typical, than the Nyquist frequency,
although they still output samples at the Nyquist rate. In exchange, they are capable
of very high resolutions, like 16 bits and beyond, with Nyquist sample rates of several
tenths of Mhz possible. A detailed description of each of these architectures is beyond
the scope of this discussion; the interested reader is referred to [1].
Before moving to the next topic the subject of energy dissipation needs to be ad-
dressed since it’s the final item that motivates the research of alternative sampling
paradigms. Generally speaking, the power consumption of an ADC is proportional to
its Nyquist sampling frequency, which means that doubling the sampling frequency
requires doubling the energy dissipated. For resolution, the situation is even worse.
A bit of extra resolution involves doubling, quadrupling or, in some cases such a flash
ADC limited by thermal noise, multiply by eight the power dissipation.
1.2 Real Life Digital to Analog Conversion
Figure 1.5: a) Zero Order Hold b) Effect of Zero Order Hold in the Spectrum of aSignal [1]
![Page 32: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/32.jpg)
10 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
Much of the effort of researchers in the field of mixed circuit design focuses on solving
the problems present in practical ADCs because the errors introduced by the sampling
circuit will degrade the signal quality irreparably. On the other hand, in a DAC, the
circuit designer has control over the signal that is generated. That is not to say that
the design of DACs is without challenges but generally speaking ADCs are harder to
design than DACs. I will only discuss here how an actual DAC differs from the ideal
DAC that implements equation (1.1). Figure 1.1b) shows the block diagram of a real
life DAC. As we can see, instead of a train of Dirac deltas, low pass filtered, we have
a three stage process that involves some mechanism to convert a digital sequence to
an electric analog signal, a signal hold, and a reconstruction filter. The typical DAC
converts binary samples to an electrical signal by generating a current, or voltage,
whose value is the analog representation of the sample. If we had the ability to
generate a train of Deltas with the samples, the spectrum of that signal would be,
1
Ts
∞∑n=−∞
X
(f − n
Ts
)
where Ts is the sampling period. Instead, we generate the signal shown in figure 1.5
a), a so called zero order hold approximation in which each sample is held for a time
Tp. The spectrum of the zero order hold approximation is,
TpTs
sinc(fTp)e−jπfTp︸ ︷︷ ︸
Amplitude Envelope
∞∑n=−∞
X
(f − n
Ts
)(1.6)
The spectrum that would be obtained by the train of Deltas is here multiplied by a
sinc function in the frequency domain. Figure 1.5b) shows an example of the effect of
this envelope on the overall spectrum of a signal for Tp = Ts. To finalize the digital to
analog conversion process we need, just as in the ideal case involving a train of deltas,
to filter the zero order hold approximation with a low pass filter with pass band equal
to 12Ts
. Unlike the ideal case, this filter not only cannot have a brick wall transfer
function but in addition it needs to correct for the sinc distortion due to the zero
order hold approximation. In terms of practical implementation, the design problem
![Page 33: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/33.jpg)
1.3. INTRODUCTION TO DISCRETE SAMPLING 11
of the reconstruction filter is similar to the design problem of a practical anti-aliasing
filter with the extra complication of requiring a transfer function in the pass band
that compensates for the sinc distortion introduced by the zero order hold.
1.3 Introduction to Discrete Sampling
From what has been presented so far in this chapter, it is clear that a new sampling
paradigm that would involve samplers working at frequencies below the Nyquist rate
in the analog domain would have at its disposal a rich ammunition of circuitry that has
been already developed to work at the Nyquist rate. Discrete sampling fits perfectly
the existing technology for analog to digital conversion. Say that we have a signal
whose bandwidth is 12Ts
; with discrete sampling we could sample it at a rate 1MTs
,
where M is a positive integer, and then digitally interpolate the missing samples.
This would allow the design of an ADC working at a sampling frequency lower than
Nyquist’s but whose output is the signal sampled at the Nyquist rate. Immediate
benefits are the mitigation of the sampling error due to jitter and probably energy
savings due to the ADC working at a lower rate. The final energy balance however will
depend on the complication added in the analog front end necessary to implement
Discrete Sampling. Of course there are many questions that can be raised about
how to accomplish that goal. This thesis is precisely about providing answers to
this proposition. For the purpose of this introductory chapter, I will just give a
summary of the most basic results of Discrete Sampling in a finite dimensional setting.
Understanding how the theory works in this setting will allow the reader to grasp the
mathematical ideas on which Discrete Sampling is based. The following chapters
of the thesis deal with the extension of Discrete Sampling to infinitely long discrete
signals. The reader interested in a detailed exposition of Discrete Sampling in a finite
dimensional setting is referred to the work by William Wu [4].
In the typical Discrete Sampling setting, one is given a finite dimensional space X
of dimension N , typically RN or CN , and a subspace Y ⊂ X, of dimension k, whose
vectors one wants to reconstruct. Discrete Sampling’s goal is to recover a vector v ∈ Ya la Shannon , ie, by measuring v in a number of places of its vector representation,
![Page 34: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/34.jpg)
12 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
then interpolating v in those places where v is not measured. Indeed, Shannon’s
sampling theorem tells us that a continuous time, band limited signal of bandwidth 12Ts
can be reconstructed in its entirety by sampling it at regularly spaced time instances.
From the samples taken at times nTs, we can generate the signal at every time instance
through continuous time interpolation with a set of translated sinc functions. Discrete
Sampling takes its name from its interest in reconstructing discrete signals through
discrete interpolation.
Definition 1.1. Given a vector space X of dimension N , a subspace Y ⊂ X of
dimension k < N , and an index set I consisting of k distinct positive integers taken
from the set {1, 2, ..., N}, an Interpolation System is the pair (I,U) where
1. I is the sampling set
2. U is a k-dimensional basis {ui}i∈I, represented in matrix form by a N×k matrix
U, such that ∀ f ∈ Yf =
∑i∈I
f [i]ui. (1.7)
The definition requires that the coordinate of vector f for element ui in basis U be
precisely f [i]. Thus, if we have an interpolation system, we can just measure f [i]i∈I
then recover the full vector f via equation (1.7). Equation (1.7) is an interpolating
equation that imposes on matrix U a very particular structure: the k-dimensional
submatrix made of the rows of U indexed by I is Ik, the k × k identity matrix. We
have the following theorem regarding existence and characterization of interpolation
systems in a finite dimensional space.
Theorem 1.1. Let X, N , Y , k, U and U be as in definition 1.1. Let R be a N × kmatrix whose columns are a basis for Y and ET
I a matrix of 0′s and 1′s that selects
out rows with indexes in I , then there exists U , a matrix whose columns form an
interpolating basis for subspace Y for some sampling set I.
However, matrix U doesn’t necessarily exist for every sampling set. Furthermore,
(I,U) is an interpolation system for Y if and only if
1. ETIR is invertible
![Page 35: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/35.jpg)
1.4. OTHER RESEARCH APPROACHES 13
2.
U = R(ETIR)−1 (1.8)
Equation (1.8) plays a fundamental role in the theory of Discrete Sampling. In fact,
the remaining of the theory of Discrete Sampling for finite dimensional spaces deals
with how equation (1.8) can be applied for particular choices of R. In [4] the cases
where R is a submatrix of the Discrete Fourier Transform matrix and where R is
a submatrix of the Haar wavelet matrix are studied throughly. Another topic of
interest for Discrete Sampling in the finite dimensional setting is finding orthogonal
interpolation systems, i.e., interpolation systems for which the corresponding matrix
U has orthogonal columns. Orthogonality is a desired property for it can be shown
that orthogonal U ’s can be computed in a manner that does not require matrix
inversion as equation (1.8) does for a generic interpolating matrix. Orthogonality also
helps with situations where we have noisy measurements, since the impact of the noise
in the reconstruction can be minimized with orthogonal interpolation matrices. Unlike
non orthogonal interpolation basis, it is not guaranteed that a particular subspace
has an orthogonal interpolation basis. Algorithms to find which subspaces have an
orthogonal interpolation basis for the Discrete Fourier spaces and Haar wavelet spaces
is another contribution of [4].
1.4 Other Research Approaches
The topic of sampling with methods different than the Shannon-Nyquist paradigm has
been the focus of a great deal of research work. During much part of the decades after
Shannon the driver behind research on sampling was mostly intellectual curiosity since
the Shannon paradigm worked very well in practice and existing technology allowed
the Nyquist ADCs to meet the engineering specs. Today however, there is the extra
motivation that existing ADC architectures seem to have reached the point where
miniaturization only provides diminishing returns. As recently as five to ten years
ago, if an engineer wanted to design a lower power, faster and more accurate ADC,
he or she just needed to wait for the technology of the next generation. Scaling made
![Page 36: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/36.jpg)
14 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
Figure 1.6: State of the Art in ADC design, 2011 [5]
existing architectures meet the new, more stringent specs. Today the approach of
waiting for the next technology improvement no longer works. Figure 1.6 shows the
performance in terms of bandwidth, defined as fs2
, and resolution for ADCs presented
at the two most prestigious circuits conferences, the International Solid-State Circuits
Conference, ISSCC, and the VLSI Symposium, during the last 15 years. Each point
corresponds to a design presented at the corresponding conference, as indicated in
the legend. Shown also are the theoretical limits imposed by the clock jitter. We see
that the limit has been already reached for a jitter of 1ps rms and that there is not
much room even if jitter can be reduced to 0.1ps rms. Reducing the clock jitter is far
from easy.
Discrete Sampling is just one of the numerous research initiatives in the subject of
sampling. Others include,
• The work of P. P. Vaidyanathan and his coauthors, [6], [7], [8], [9], which relies
heavily in the concepts of multi rate signal processing [10].
• The work of Yonina Eldar and her coauthors, [11], [12], [13], [14] which is
![Page 37: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/37.jpg)
1.4. OTHER RESEARCH APPROACHES 15
based on different ways of modeling the analog signal which results in sampling
architectures that are able to sample data at rates below Nyquist for signals
that meet the analog signal model. Their different proposals fall under the
umbrella of the so called Xampling framework. The most limiting factor of these
approaches is that they require mixing the analog signal with binary patterns
whose clock is higher than the Nyquist rate, thus jitter becomes a problem.
• The approach known as Compressed Sensing, initiated by Candes, Tao and
Donoho [15], [16] but with a very rich set of results by other researchers that
followed such as [17]. In Compressed Sensing the signal is assumed to fit the
so called “sparse model”. This means that there exist some basis in which the
signal representation consists of coefficients that are mostly zero, with a few k
non zero ones. To find those coefficients, Compressed Sensing proposes doing
a set of linear measurements whose number is close to k. Reconstruction is
performed by solving a `1 minimization problem that recovers the k coefficients
from the measured data. Although there is abundant literature in both theory
and algorithms for recovery there are few practical implementations of Com-
pressed Sensing ADCs. The limits in the current chip implementations relate
to jitter, just as in the case of Xampling. The Compressed Sensing ADCs mea-
sure samples at rates below Nyquist but they need to mix the analog signal
with a pseudo random binary sequence whose clock needs to be precise in time.
Such clock works at the Nyquist rate or higher.
• The work by Unser and Aldroubi, [18], [19] which relays in precise models of the
analog signals. The analog signal is sampled after being generated as output
of a custom analog filter, not necessarily band-limited. Part of the recovery is
done in the digital domain via correction with a digital filter. Since the sampled
signals are not necessarily band-limited either, a comparison with the Nyquist
rate is not appropriate. The difficulty of their work is in coming up with signal
models that are good for real signals and whose analog filters models are easy
to invert in the digital domain.
• Theoretical work on signal models that consist of a union of subspaces, such as
![Page 38: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/38.jpg)
16 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING
[20] by Lu and Do, and [21] by Blumensath and Davies.
• Theoretical work by Walter on sampling wavelet subspaces [22].
• The work of Serdijn and his coauthors on practical implementation of wavelet
samplers for biomedical applications [23],[24].
![Page 39: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/39.jpg)
Chapter 2
Infinite-Dimensional Vector
Spaces, Bases and Frames
In this chapter I give a summary of frame theory, which I will use to extend the
theory of Discrete Sampling to an infinite dimensional space. I assume that the
reader is familiar with the following concepts, which are taught in an undergraduate
level course in analysis: the notion of basis of an infinite dimensional space, operators,
norms, inner products, convergence, Cauchy completeness and Hilbert spaces. A good
reference that describes these topics in detail is [25]. Although frames can be defined
in finite dimensional spaces as well, they become particularly useful when they are
used to analyze infinite dimensional spaces. My exposition is mainly inspired by the
book “Frames and Bases, an Introductory Course” from author Ole Christensen [26],
to which I refer the reader interested in an expanded treatment of the subject.
2.1 Bases in Infinite-Dimensional Hilbert Spaces
2.1.1 Orthonormal Bases
Definition 2.1. Consider a sequence {ek}∞k=1 of vectors in the Hilbert space H.
1. The sequence {ek}∞k=1 is a basis for H if for each f ∈ H, ∃ unique scalar
coefficients {ck(f)}∞k=1 such that f =∑k=∞
k=1 ck(f)ek.
17
![Page 40: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/40.jpg)
18 CHAPTER 2. INTRODUCTION TO FRAME THEORY
2. A basis {ek}∞k=1 is an unconditional basis if∑k=∞
k=1 ck(f)ek converges uncon-
ditionally for every f ∈ H.
3. A basis {ek}∞k=1 is an orthonormal basis if ∀ek, ej ∈ {ek}∞k=1 〈ek, ej〉 = δk,j
with δk,j = 1 if k = j and δk,j = 0 if k 6= j .
We note that the definition is very similar to the definition of basis in a finite dimen-
sional space except that here convergence plays a prominent role. Obviously, we are
primarily interested in unconditional bases. The following three theorems also show
why orthonormal basis are preferred to other unconditional bases.
Theorem 2.1. Consider an orthonormal sequence {ek}∞k=1 of vectors of a Hilbert
space H, the following 6 statements are equivalent:
1. {ek}∞k=1 is an orthonormal basis.
2. f =∑k=∞
k=1 〈f, ek〉ek ∀f ∈ H.
3. 〈f, g〉 =∑k=∞
k=1 〈f, ek〉〈ek, g〉 ∀f, g ∈ H.
4.∑k=∞
k=1 |〈f, ek〉|2 = ||f ||2 ∀f ∈ H. This is called Parseval’s identity.
5. span{ek}∞k=1 = H.
6. if 〈f, ek〉 = 0 ∀k ∈ N, then f = 0.
Theorem 2.2. Consider {ek}∞k=1 an orthonormal basis of Hilbert space H; then, each
f ∈ H has an unconditionally convergent expansion f =∑k=∞
k=1 〈f, ek〉ek.
Theorem 2.3. Every separable Hilbert space H has an orthonormal basis. Separable,
in the context of Hilbert spaces, means that there exists a countable sequence of vectors
{fk}∞k=1 such that span{fk}∞k=1 = H.
To prove Theorem 2.3 we just apply the Gram-Schmidt orthogonalization process to
finite sequences of vectors taken from {fk}∞k=1. Orthonormal bases have the inter-
esting, and useful from an engineering point of view, property that the coordinates
of any vector in f ∈ H can be computed as the inner products of f with elements
![Page 41: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/41.jpg)
2.1. BASES IN INFINITE-DIMENSIONAL HILBERT SPACES 19
of the basis. Parseval’s identity is also very useful since it shows how to compute
the norm of a vector, which in many engineering applications equals the energy of
some signal, from the same coordinates. Inner products can be implemented by linear
filters so the result of this formalism is that the decomposition of infinite duration
signals, or of very long finite duration signals modeled as infinite length signals, can
be performed through filtering followed by sampling. The following theorems give
three other interesting results for orthonormal bases.
Theorem 2.4. Every separable infinite-dimensional Hilbert space H is isometrically
isomorphic to `2(N).
Theorem 2.5. For k ∈ N, let ek be the sequence in `2(N) whose k-th element equals
1 while the rest elements in the sequence are equal to zero.Then {ek}∞k=1 is an or-
thonormal basis of `2(N). It is known as the canonical orthonormal basis of
`2(N).
Theorem 2.6. Let {ek}∞k=1 be an orthonormal basis of a Hilbert space H. All other
orthonormal bases of H are sequences of vectors in the form {Uek}∞k=1 with U : H →H a unitary operator.
Theorem 2.4 will play a very important role in the extension of Discrete Sampling to
infinite dimensional spaces while the last theorem takes us directly to the next topic,
Riesz bases.
2.1.2 Riesz Bases
If orthonormal bases are so great, why bother studying other types of bases? The
main reason is that for a basis to be orthonormal, the requirements are very stringent.
We would like to have bases with characteristics similar to Theorem 2.1.2 and 2.1.4
but easier to build. We begin with a characterization of Riesz bases similar, but
weaker, to Theorem 2.6 for orthonormal bases.
Definition 2.2. Let H be a Hilbert space with an orthonormal basis {ek}∞k=1. A Riesz
basis is a sequence {Uek}∞k=1 where U : H → H is a bounded bijective operator.
![Page 42: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/42.jpg)
20 CHAPTER 2. INTRODUCTION TO FRAME THEORY
We have the following theorem which, among other things, states that a Riesz basis
is actually a basis,
Theorem 2.7. Let {fk}∞k=1 = {Uek}∞k=1 be a Riesz basis in Hilbert space H, then
{fk}∞k=1 is a basis for H and there exists a unique sequence {gk}∞k=1 in H such that
f =∑∞
k=1〈f, gk〉fk unconditionally ∀f ∈ H. Furthermore, the sequence {gk}∞k=1,
gk = (U−1)∗ek, is also a Riesz basis, called the dual basis of {fk}∞k=1.
Since the sequence {gk}∞k=1 in Theorem 2.7 is also a Riesz basis, it makes sense to ask
which is the dual of {gk}∞k=1. It follows immediately that the answer is {fk}∞k=1. For
this reason, the couple {gk}∞k=1,{fk}∞k=1 is called a pair of dual Riesz bases. We have
the following theorem that relates the two.
Theorem 2.8. Let’s {fk}∞k=1 and {gk}∞k=1 be a pair of dual Riesz bases for Hilbert
space H, then
1. {fk}∞k=1 and {gk}∞k=1 are biorthogonal. This means that 〈fk, gj〉 = δk,j.
2. given a Riesz basis {fk}∞k=1 of H, there is a unique set of vectors biorthogonal
to {fk}∞k=1, namely, its dual basis {gk}∞k=1 [27].
3. ∀f ∈ H we have that f =∑∞
k=1〈f, gk〉fk =∑∞
k=1〈f, fk〉gk.
Thus, if we work with a pair of dual Riesz bases we have many of the good things
of orthonormal bases. In general, however, we don’t have an equivalent to Parseval’s
identity. Instead we have the following theorem, which is used in many references
[28] as the definition of a Riesz basis.
Theorem 2.9. Let {fk}∞k=1 = {Uek}∞k=1 be a Riesz basis for Hilbert space H then
there exist constants A,B > 0 such that ∀f ∈ H,
A||f ||2 ≤∞∑k=1
|〈f, fk〉|2 ≤ B||f ||2 (2.1)
The largest possible value for constant A is called the optimal lower bound and equals1
||U−1||2 . The smallest possible value for B is called the optimal upper bound and is
![Page 43: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/43.jpg)
2.2. FRAMES IN INFINITE-DIMENSIONAL HILBERT SPACES 21
equal to ||U ||2. For the dual Riesz basis {gk}∞k=1 = {(U−1)∗ek}∞k=1, if we consider the
optimal bounds A and B, we have,
1
B||f ||2 ≤
∞∑k=1
|〈f, gk〉|2 ≤1
A||f ||2 (2.2)
Theorem 2.9 leads us to the study of frames.
2.2 Frames in Infinite-dimensional Hilbert Spaces
The main feature of a basis in a Hilbert spaceH is that it allows us to uniquely express
every element of f ∈ H as an infinite linear combination of the elements in the basis.
There are situations however, in which we would gladly give up the uniqueness of
the representation if in exchange we get back flexibility about properties of the basis
elements. Frames are generalizations of the concept of basis that provide for that
flexibility. In fact, we will see that just as orthonormal bases can be seen as particular
cases of Riesz bases for which both bounds in equation 2.1 are equal to one, Riesz
bases are particular cases of frames for which we impose that the elements of the frame
be linearly independent. Frames were introduced in 1952 by Duffin and Schaeffer as
a tool to study non uniform sampling. They remained a somehow obscure tool until
the mid 1980s when they started to be used by Daubechies, Mallat and those that
ignited the wavelet revolution [28]. We note that frames are not required to work
with an infinite dimensional space if we have orthonormal or Riesz bases that meet
our needs.
2.2.1 Definition and Properties of Frames
Definition 2.3. Let H 6= {0} be a Hilbert space. A sequence of elements {fk}∞k=1 ∈ His a frame of H if there exists two constants A,B > 0 such that ∀f ∈ H,
A||f ||2 ≤∞∑k=1
|〈f, fk〉|2 ≤ B||f ||2 (2.3)
![Page 44: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/44.jpg)
22 CHAPTER 2. INTRODUCTION TO FRAME THEORY
Equation (2.3) is commonly known as the frame condition. It can be shown that the
frame condition is satisfied for H if it is satisfied for a subspace V ⊂ H which is dense
in H. Equation (2.3) looks very similar to equation (2.1), the main difference is that
the elements of a frame don’t have to be linearly independent. If we consider the
frame operator S, defined below, the infimum over all upper bounds B is the optimal
upper frame bound and equals ||S||. Similarly, the supremum among all lower bounds
A is the optimal lower frame bound and equals ||S−1||−1. If A = B in equation (2.3),
the frame is said to be tight. In a tight frame, equation (2.3) becomes a Parseval-like
identity. From the definition of frame it follows that if {fk}∞k=1 ∈ H is a frame for Hthen span{fk}∞k=1 = H. To see why, compute the orthogonal complement of {fk}∞k=1;
the frame condition imposes that if f is perpendicular to all vectors in the frame,
then f = 0 which is equivalent to say that span{fk}∞k=1 = H. We might be interested
in sequences that are frames of its closed linear span but not necessarily frames of H.
Definition 2.4. Consider a sequence {fk}∞k=1 in a Hilbert space H. We say that this
sequence is a frame sequence if it is a frame for span{fk}∞k=1.
Definition 2.5. Consider {fk}∞k=1, a frame of Hilbert space H; we define the pre-
frame operator or synthesis operator the operator T : `2(N) → H such that
T{ck}k=∞k=1 =
∑k=∞k=1 ckfk. It can be shown that the frame condition makes T bounded.
It can also be shown that the adjoint of T is given by T ∗ : H → `2(N) such that
T ∗f = {〈f, fk〉}∞k=1. T ∗ is called the analysis operator. Composing T and T ∗, we
get the frame operator S : H → H as Sf = TT ∗f =∑∞
k=−1〈f, fk〉fk.
The following theorem lists important properties of S.
Theorem 2.10. Let {fk}∞k=1 be a frame of a Hilbert space H, with frame bounds
A,B, and S its associated frame operator. Then:
1. S converges unconditionally for all f ∈ H.
2. S is bounded, invertible, self-adjoint, and positive.
3. {S−1fk}∞k=1, called the canonical dual frame of {fk}∞k=1, is a frame with
frame operator S−1 and frame bounds 1B, 1A
.
![Page 45: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/45.jpg)
2.2. FRAMES IN INFINITE-DIMENSIONAL HILBERT SPACES 23
4. If A,B are the optimal frame bounds for frame {fk}∞k=1, then 1B, 1A
are the
optimal frame bounds for {S−1fk}∞k=1.
The reason {S−1fk}∞k=1 is called the canonical dual frame becomes clear in the follow-
ing theorem, which is the most important frame result because it shows how every
element in H admits a representation as an infinite linear combination of the frame
elements.
Theorem 2.11. Let {fk}∞k=1 be a frame of a Hilbert space H with frame operator S.
Then ∀f ∈ H
f =k=∞∑k=1
〈f, S−1fk〉fk (2.4)
and
f =k=∞∑k=1
〈f, fk〉S−1fk (2.5)
Thus {fk}∞k=1 and {S−1fk}∞k=1 give us a way to represent elements of H akin to
the situation with Riesz bases where {fk}∞k=1 plays the role of a Riesz basis and
{S−1fk}∞k=1 plays the role of its dual. For this reason we call {fk}∞k=1 and {S−1fk}∞k=1
a dual frame pair. Unlike the situation with Riesz bases though, here a frame can
have more than one dual frame. However there is an incentive in using the canonical
dual frame. It can be shown that the frame coefficients 〈f, S−1fk〉 have minimal `2
norm among all sequences representing f . The main difficulty of frame theory is that
it requires the computation of the inverse frame operator or at least the effect it has on
all the frame elements fk . A tight frame facilitates things because the canonical dual
frame of a tight frame is {A−1fk}∞k=1. We can even normalize the frame vectors such
that A = 1, in which case the representations of Theorem 2.11 adopts the same form
as the representation with an orthonormal basis. Before we move on, we must notice
that given a sequence of vectors {fk}∞k=1 in an infinite-dimensional Hilbert spaceH the
frame condition is truly necessary to represent every f ∈ H as a linear combination
of vectors {fk}∞k=1. For certain infinite dimensional Hilbert spaces H, it is possible to
have span{fk}∞k=1 = H and find vectors in H that cannot be represented as a linear
combination of vectors {fk}∞k=1 because such vectors don’t satisfy the frame condition.
![Page 46: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/46.jpg)
24 CHAPTER 2. INTRODUCTION TO FRAME THEORY
In contrast, finite-dimensional spaces do have the property that if span{fk}Nk=1 = Hfor some natural number N, then every vector in H can be represented as a linear
combination of vectors in {fk}Nk=1.
2.2.2 Frames and Riesz Bases
From the exposition so far, it is clear that every Riesz basis is a frame. Furthermore,
the Riesz basis bounds are equal to the frame bounds and the dual Riesz basis equals
the canonical dual frame. The frames that are not Riesz bases are called overcomplete
or redundant as shown in the following theorem.
Theorem 2.12. Let {fk}∞k=1 be a frame for Hilbert space H then these two statements
are equivalent:
1. {fk}∞k=1 is a Riesz basis for H
2. If∑∞
k=1 ckfk = 0 for some {ck}∞k=1, then ck = 0 ∀k ∈ N.
Another way of stating this is that if {fk}∞k=1, a frame for H, is not a basis, there
exist {ck}∞k=1 such that∑∞
k=1 ckfk = 0 and not all ck equal zero. Thus, such frame is
composed of vectors that are linearly dependent. The linear dependence of the frame
vectors has several effects in the representation. As we already said above, frames
which are not bases have more than one dual frame. It can also be shown that if
{fk}∞k=1 is a redundant frame for H, then every f ∈ H has several representations
with vectors {fk}∞k=1. With the right normalization of the frame vectors, the frame
bounds tell whether a frame is redundant. If we scale the vectors such that ||fk|| = 1
∀k, then A ≤ 1 ≤ B if the frame is a basis, with A = B = 1 if and only if the frame
is an orthonormal basis. Under such normalization A is greater than one if the frame
is redundant; thus A can be interpreted as the redundancy factor in the normalized
frame.
2.2.3 Frames and Orthogonal Projections
Orthogonal projections play a fundamental role in approximation theory. The orthog-
onality principle states that when we approximate a vector v ∈ H, with an element
![Page 47: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/47.jpg)
2.2. FRAMES IN INFINITE-DIMENSIONAL HILBERT SPACES 25
v in subspace V ⊂ H such that v /∈ V , the approximation error ||v− v|| is minimized
by picking v ∈ V such that 〈v, v − v〉 = 0. Such v is known as the orthogonal pro-
jection of v onto V because it is orthogonal to the approximation error v − v. When
this problem arises in a finite dimensional setting, the matrix whose columns span
V is full rank, tall and skinny. The orthogonal projection in that case is given by
the Moore-Penrose pseudoinverse of that matrix applied to vector v. In the infinite
dimensional setting, if the subspace V is spanned by a frame we can easily compute
the orthogonal projection, provided that its canonical dual frame is known.
Theorem 2.13. Let {fk}∞k=1 be a sequence of vectors in a Hilbert space H, and P
the orthogonal projection of H onto a closed subspace V ⊂ H, then
1. If {fk}∞k=1 is a frame for H with frame bounds A,B, then {Pfk}∞k=1 is a frame
for V with frame bounds A,B.
2. If {fk}∞k=1 is a frame for V with frame operator S, then the orthogonal projection
of H onto V is given by
Pf =∞∑k=1
〈f, S−1fk〉fk. (2.6)
∀f ∈ H
Theorem 2.13 is valid for all frames, in particular it is true for Riesz bases. Equation
(2.6) is at the heart of the extension of Discrete Sampling to `2(Z).
2.2.4 An Illustrating Example, the PWTsSpace
Consider the Hilbert space composed of real or complex valued square integrable
continuous time signals H = L2(R). With the inner product of f(t), g(t) defined as
〈f(t), g(t)〉 =
∫ +∞
−∞f(t)g(t) dt
we say that f(t) ∈ L2(R) if
![Page 48: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/48.jpg)
26 CHAPTER 2. INTRODUCTION TO FRAME THEORY
〈x(t), x(t)〉 =
∫ +∞
−∞|f(t)|2 dt <∞
We define PWTs ⊂ L2(R) as the space of square integrable, continuous time, band-
limited signals. More precisely, f(t) ∈ PWTs if the support of its Fourier transform
is included in [ −12Ts, 1
2Ts]. PWTs is known as the Paley-Wiener space associated with
bandwidth 12Ts
. PWTs is also a Hilbert space with the same inner product. The
previous results, together with the Shannon sampling theorem, boil down to saying
that { 1√Ts
sinc(t−nTsTs
)}∞−∞ is frame of PWTs with A = B = 1. Not only this frame and
its dual are equal but, as a consequence of having A = 1 and since the vectors of the
frame are normalized to have their norm equal to one, this frame is non redundant; ie,
it’s a Riesz basis of PWTs . It’s also an orthonormal basis of PWTs . Theorem 2.13 tells
us that if we have a signal f(t) ∈ L2(R) which is not band limited, if we compute the
inner products of f(t) with the frame vectors, then use these as coordinates of vectors
{ 1√Ts
sinc(t−nTsTs
)}∞−∞, we get the orthogonal projection of f(t) onto PWTs . This is
equivalent to the standard practice of applying an antialiasing filter of bandwidth 12Ts
prior to sampling a non band limited signal f(t) at rate 1Ts
. The signal reconstructed
through sinc interpolation with these samples is the orthogonal projection of f(t)
onto PWTs .
![Page 49: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/49.jpg)
Chapter 3
Discrete Sampling for Continous
Time, Band-Limited Signals
Discrete Sampling has a great deal of results in the finite dimensional setting, and
all signals we work with in real life are finite dimensional, therefore the question of
why bother with infinite dimensional signals is justified. The answer is that there
are situations in which pretending that our signal is infinitely long allows us to build
simpler models for the signals under consideration. Take for instance an application
that deals with CD quality audio. The application will get 44100 samples per each
second -88200 in case of stereo signals- of sampled signal. Using the finite dimensional
setting for such signals is impractical since we would need to work with very big
matrices, on the order of several million rows, to process signals that are just a few
minutes long. Although one approach could be to break up the signal into smaller
pieces, so that each of which would then be processed independently, there are still
problems. Say, for instance, that you take pieces of 1024 samples; that’s over 40
manipulations of matrices with 1024 rows per second of signal. Even if we manage to
solve the matrix operation problem efficiently with numerical methods adapted to the
matrices in question, there is the additional problem of putting the pieces together
in a meaningful way. It is very unlikely that each of the 1024 long pieces will belong
to the same subspace of R1024; thus, the application would need to work with several
interpolation matrices, have a way to know which one works with a particular block
27
![Page 50: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/50.jpg)
28 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
of signal, then put all processed pieces together to get the original long signal. In
digital signal processing a similar situation arises when one tries to use the FFT as
a tool to do filtering of long signals. In the case of the DFT it is possible to process
a very long signal by cleverly overlapping the result of applying the DFT to small
blocks of the long signal; it’s the so called overlap-add method [29]. It is not clear that
a similar approach can be used to extend the finite dimensional methods of Discrete
Sampling to very long signals. This chapter is about using the ideas presented in
Chapter 2 to extend Discrete Sampling to work with infinitely long signals.
3.1 Infinitely Long Discrete Signals in `2(Z)
“the 2WT evenly spaced samples of a signal can be thought of as coordi-
nates of a point in a space of 2WT dimensions. Each particular selection
of these numbers corresponds to a particular point in this space. Thus
there is exactly one point corresponding to each signal in the band W and
with duration T . The number of 2WT dimensions will be, in general, very
high. A 5-Mc television signal lasting for an hour would be represented
by a point in a space with 2 × 5 × 106 × 602 = 3.6 × 1010 dimensions ...
The advantage of this geometrical representation of the signals is that we
can use the vocabulary and the results of geometry in the communication
problem... Passing a signal through an ideal filter corresponds to project-
ing the corresponding point onto a certain region in the space. In fact, in
the frequency-coordinate system those components lying in the pass band
of the filter are retained and those outside are eliminated, so that the
projection is on one of the coordinate lines, planes, or hyperplanes. Any
filter performs a linear operation on the vectors of the space, producing a
new vector linearly related to the old one.”
The above quote is from Shannon’s 1949 paper [2] “Communication in the Presence
of Noise”. The geometric point of view is already used to some extent in the theory
of Discrete Sampling for finite length signals. To deal with the signals like the one
![Page 51: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/51.jpg)
3.1. INFINITELY LONG DISCRETE SIGNALS IN `2(Z) 29
described by Shannon, we can take the idea further by considering that our signals,
while discrete, are infinitely long. From a practical point of view it also makes sense
to focus on signals of finite energy. Mathematically, the Hilbert space H = `2(Z) of
square summable sequences gives us the appropriate ambient space for our signals.
Note that since there is a one to one correspondence between Z and N, we could
consider instead signals in `2(N). With the inner product of two infinite signals
x[n], y[n] defined as,
〈x[n], y[n]〉 =∞∑
n=−∞
x[n]y[n]
we say x[n] ∈ `2(Z) if
〈x[n], x[n]〉 =∞∑
n=−∞
|x[n]|2 <∞
The following theorem, referred to in [27] as the Riesz-Fisher Theorem, allows us to
make a one to one correspondence between `2(Z) and PWTs
Theorem 3.1. Let H be a Hilbert space, {fn}∞n=−∞ an orthonormal basis for H and
{cn}∞n=−∞ a sequence of scalars. Then there exists an f ∈ H such that∑∞
n=−∞ cnfn
converges to f if and only if∑∞
n=−∞ |cn|2 < ∞. Under either condition cn = 〈f, fn〉and there cannot exist g ∈ H, distinct from f , for which cn = 〈g, fn〉.
To see how we can apply the theorem, we remember that the set { 1√Ts
sinc(t−nTsTs
)}∞k=−∞
is an orthonormal basis of PWTs and that the Shannon sampling theorem says that
we can reconstruct any x(t) ∈ PWTs via the sinc interpolation formula
x(t) =∞∑
n=−∞
√Tsx[n]√Ts
sinc
(t− nTsTs
)(3.1)
therefore
• Suppose we have x(t) ∈ PWTs , then by equation (3.1) 〈x(t), 1√Ts
sinc(t−nTsTs
)〉 =
√Tsx[n] with x[n] = x(nTs). By Theorem 3.1, {
√Tsx[n]}∞n=−∞ ∈ `2(Z), thus
{x[n]}∞n=−∞ ∈ `2(Z).
![Page 52: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/52.jpg)
30 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
• Conversely, suppose we have {c[n]}∞n=−∞ ∈ `2(Z), thus {√Tsc[n]}∞n=−∞ ∈ `2(Z),
then by Theorem 3.1 there exists a unique x(t) ∈ PWTs such that√Tsc[n] =
〈x(t), 1√T
sinc(t−nTT
)〉 and
x(t) =∞∑
n=−∞
√Tsc[n]√Ts
sinc
(t− nTsTs
)
Since the representation of x(t) in basis { 1√T
sinc(t−nTT
)}∞k=−∞ is unique c[n] =
x[n] = x(nTs).
As a corollary of the above reasoning we find a very important, if surprising, fact
about PWTs, which is that despite being composed of continuous time signals, PWTs
has a discrete nature [30].
Extending Discrete Sampling to work with signals x[n] ∈ `2(Z) will therefore make
the theory of Discrete Sampling work with the class of continuous time band-limited
signals as well. The objective then is to come up with an equation analog to equation
(1.8) so that we can interpolate infinitely long signals out of samples indexed by
some sampling set “smaller” than Z, provided that we know that x[n] belongs to a
subspace of `2(Z). I use the word smaller in between quotes because such sampling
set will likely be, although not necessarily, a countable set whose cardinal is the
same as the cardinal of the integer numbers. This difficulty didn’t exist in the finite
dimensional case. We will see in the next sections that we can greatly simplify the task
by focusing on regular subsampling sets (or schemes) and shift invariant subspaces
of `2(Z). Furthermore, these simplifications will allow us to work with any signal
x[n] ∈ `2(Z).
3.1.1 Equivalence of Continuous Time Inner Products With
Discrete Time Inner Products
The isomorphism between PWTs and `2(Z) implies that inner products of band-
limited continuous time signals in PWTs can be computed from inner products of
discrete time signals. If we take any two band-limited signals x(t) and y(t) we have,
![Page 53: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/53.jpg)
3.2. INTERPOLATION OF SIGNALS IN PWTS 31
〈x(t), y(t)〉 =
〈+−∞∑n1=−∞
x[n1]sinc
(t− n1Ts
Ts
),
+−∞∑n2=−∞
y[n2]sinc
(t− n2Ts
Ts
)〉 =
∫ +∞
−∞{
+−∞∑n1=−∞
x[n1]sinc
(t− n1Ts
Ts
).
+−∞∑n2=−∞
y[n2]sinc
(t− n2Ts
Ts
)} dt =
+∞∑n1=−∞
Tsx[n1]y[n1] = Ts〈x[n], y[n]〉. (3.2)
The penultimate line follows from the orthogonality of the translated sinc functions,
which makes all terms of the double sum of continuous time integrals vanish except
when n1 = n2. When n1 = n2, the continuous time integrals equal Ts, and what
remains is an inner product of discrete sequences.
3.2 Interpolation of Continuous Time, Band-Limited
Signals
Extending Discrete Sampling to `2(Z) can be difficult if we insist in keeping all the
features present in the finite dimensional case, like non regular sampling. By making
the choice of working only with regular sampling schemes we can simplify the task.
This choice is practical in an engineering context.
Consider then the following setup,
• x(t) is a continuous time signal, that belongs to the space PWTs . This is
equivalent to saying that the Fourier transform of x(t), X(f), equals to zero for
|f | > 12Ts
. As an additional technical condition, we require that X(f) 6= 0 for
|f | ≤ 12Ts
. We let B = 1/2Ts and equivalently say that x(t) is a continuous time
band-limited signal whose frequency support is included in [−B,+B].
• x[n] is a discrete sequence such that x[n] = x(nTs); by Theorem 3.1 we have
x[n] ∈ `2(Z). Thus, x[n] is the discrete sequence that results from sampling
![Page 54: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/54.jpg)
32 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
x(t) at the Shannon-Nyquist rate.
We have a theorem that tells us when it is possible to recover {x[n]}n∈Z from samples
{x[nM ]}n∈Z
Theorem 3.2. Interpolation of Undersampled Continuous Time Band-Limited Sig-
nals
The following three statements are equivalent
i) There exists a discrete sequence u[n] ∈ `2(Z) and a positive integer M > 1 such
that x[n] can be expressed as
x[n] =+∞∑i=−∞
x[iM ]u[n− iM ] (3.3)
u[n] has the property known in the literature as Nyquist(M) [9], [10], ie, u[0] = 1
and u[n] = 0 for n = ±M,±2M,±3M, . . .
ii) X(z), the z-transform of x[n] satisfies that
|M−1∑k=0
X(ze−j2πMk)| > 0, ∀z = ej2πf (3.4)
iii) There exists a signal u(t) ∈ PWTs such that the function x(t) can be written as
x(t) =+∞∑i=−∞
x[iM ]u(t− iMTs) (3.5)
A key aspect of the theorem is that we are interested in signals that satisfy equation
(3.4). This property guarantees that the DTFT of the discrete signal that results from
downsampling x(t) by a factor M , then upsampling again by a factor M , is non zero
in the range of frequency [−B,+B]. Theorem 3.2 says that such band-limited signals
x(t) can be sampled at rate 1MTs
without loss of information. The reconstruction of
x(t) is performed by first interpolating the samples x[n], out of samples x[nM ], then
applying continuous time sinc interpolation with a low pass filter of bandwidth B; in
![Page 55: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/55.jpg)
3.2. INTERPOLATION OF SIGNALS IN PWTS 33
fact we will apply its practical version, a zero-order hold followed by the appropriate
low pass / reconstruction filter of bandwidth B. The setup is shown in figure 3.1.
Figure 3.1: Sampling Scheme With Interpolation for Continuous Time, Band-LimitedSignals
Figure 3.2: Magnitude of the Spectrum of cos(2π 132n)
Figure 3.3: Magnitude of the Spectrum of ↑↓ cos(2π 132n)
Proof.
i)⇒ ii) Because x(t) ∈ PWTs , we can write it as,
![Page 56: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/56.jpg)
34 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
Figure 3.4: Magnitude of the Spectrum of cos(2π 132n).w[n]
Figure 3.5: Magnitude of the Spectrum of ↑↓ {cos(2π 132n).w[n]}
x(t) =+∞∑
n=−∞
x[n]sinc
(t− nTsTs
)(3.6)
Putting equations (3.3) and (3.6) together we have,
x(t) =+∞∑
n=−∞
+∞∑i=−∞
x[iM ]u[n− iM ]sinc
(t− nTsTs
)=
+∞∑i=−∞
x[iM ]+∞∑
n=−∞
u[n− iM ]sinc
(t− nTsTs
)(3.7)
if we call
u(t) =+∞∑
n=−∞
u[n]sinc
(t− nTsTs
)(3.8)
by Theorem 3.1 u(t) ∈ PWTs , and we can rewrite equation (3.7) as
![Page 57: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/57.jpg)
3.2. INTERPOLATION OF SIGNALS IN PWTS 35
Figure 3.6: Interpolating Function u[n] for cos(2π 132n).w[n]
x(t) =+∞∑i=−∞
x[iM ]u(t− iMTs) (3.9)
[Note that equation (3.9) is identical to equation (3.5)]. If we further develop
equation (3.9) we get
x(t) =+∞∑i=−∞
x[iM ]u(t− iMTs) = u(t) ∗+∞∑i=−∞
x[iM ]δ(t− iMTs) (3.10)
Since x(t), u(t) ∈ PWTs , their continuous time Fourier transforms are 0 for
|f | > 12Ts
, and we can equivalently work with equation (3.10) in the z domain.
Thus, the right hand side of equation (3.10) is the convolution of u[n] with
a downsampled version, by factor M, then upsampled, again by factor M, of
x[n]. For the purpose of explanation, we can call the latter signal ↑↓ x[n].
In the z domain the convolution becomes a product. Defining X(z) as the z
transform of x[n], downsampling x[n] gives a signal whose z transform is a sum
of delayed, and aliased, versions of X(z). Upsampling this latter signal results
in stretching it in the z domain. Using the z transform operator Z{} we can
show mathematically [29] that Z{↑↓ x[n]} = 1M
∑M−1k=0 X(ze−j
2πMk). Thus, we
have
![Page 58: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/58.jpg)
36 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
Figure 3.7: cos(2π 132n).w[n] (red) and its Reconstruction (blue) via Interpolation
With u[n] Shown in Figure 3.6
X(z) = U(z)Z{↑↓ x[n]} = U(z)1
M
M−1∑k=0
X(ze−j2πMk) (3.11)
this gives us
U(z) =X(z)
1M
∑M−1k=0 X(ze−j
2πMk)
(3.12)
which is well defined provided that |∑M−1
k=0 X(ze−j2πMk)| > 0 ∀z = ej2πf . This
condition is precisely equation (3.4). The fact that we required X(f) 6= 0
for |f | ≤ 12Ts
allows us to not consider the case where both numerator and
denominator are zero.
ii)⇒ iii) Suppose that equation (3.4) is true, we can build U(z) as indicated in (3.12).
By construction of U(z), u(t) shown in equation (3.8) is the signal in PWTs
whose samples taken at rate 1Ts
, are u[n]. Going back to the continuous time
domain, u(t) allows us to reconstruct x(t) with equation (3.5).
iii)⇒ i) Suppose (3.5) is true. By assumption, u(t) ∈ PWTs, so it admits an expansion
as shown in equation (3.8). Plugging (3.8) into (3.5) we get (3.7). Since (3.7)
![Page 59: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/59.jpg)
3.2. INTERPOLATION OF SIGNALS IN PWTS 37
Figure 3.8: Example of a Signal that Doesn’t Satisfy Equation (3.4) with M=8
must be equal to equation (3.6),
+∞∑n=−∞
x[n]sinc
(t− nTsTs
)=
+∞∑n=−∞
{+∞∑i=−∞
x[iM ]u[n− iM ]}sinc
(t− nTsTs
)(3.13)
the expansion of x(t) in basis {sinc(t−kTsTs
)}∞k=−∞ is unique so we have,
x[n] =+∞∑i=−∞
x[iM ]u[n− iM ] (3.14)
which is identical to equation (3.3). This completes the proof of the theorem.
Note that equation (3.12) resembles equation (1.8) in the finite dimensional Discrete
![Page 60: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/60.jpg)
38 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
Figure 3.9: Spectrum of ↑↓ {x[n].w[n]} Associated With the Signal Shown in Figure3.8
Sampling case. Intuitively it makes sense that both equations share a similar spirit.
In the finite dimensional setting, the effect of ETI is to keep only certain rows of R
which is analogous to downsampling in the infinite dimensional case. In both cases
the “downsampled” signal is then applied as an inverse to the signal under consider-
ation.
As to how stringent is the requirement imposed by equation (3.4) on real life signals,
the answer is not much. One would expect that the signal x[n] = cos(2π 132n), whose
DTFT magnitude is shown in figure 3.2, would not meet this requirement since its
spectrum is mostly zero. Moreover, figure 3.3 shows the magnitude of the spectrum
of ↑↓ x[n], the signal that results from downsampling, then upsampling, x[n], both by
a factor of M=4. The z transform of ↑↓ x[n] goes into the denominator of equation
(3.12), how can we expect such a signal to give us a valid interpolating sequence?
The answer is that in real life, one never considers signals such as this x[n]. In
practice there is always a window w[n], by default the rectangular window, multiplying
x[n]. We can improve our ability to reconstruct signals such as this cosine by using
other windows. Multiplying x[n] by a 1024 long Hann window [31] results in signals
x[n]w[n] and ↑↓ {x[n].w[n]} whose magnitude spectra are shown in figures 3.4 and 3.5
respectively. Since the Fourier transform of the Hann window is a sinc-like function,
but with a main lobe twice as wide than the sinc function’s, and the multiplication
in the time domain results in convolution in the frequency domain, we clearly see
![Page 61: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/61.jpg)
3.2. INTERPOLATION OF SIGNALS IN PWTS 39
such functions, in lieu of Delta functions, in these figures. Figure 3.6 shows the
interpolating function associated with x[n]w[n] via equation (3.12) while figure 3.7
compares cos(2π 132n).w[n] with its reconstruction via interpolation with the function
shown in figure 3.6 . We can define
PSNR = 10 log10
(N
max(|x[n]|)2
||x[n]− xrec[n]||2
)(3.15)
where xrec[n] is the reconstructed signal and N the length of x[n] and xrec[n]. In
this example max(|x[n]|)2 = 1 and N = 1024. The reconstruction gives a PSNR
of 74.58dB. This means that the mean squared error is 3.4832e-008 or 1.8663e-004
rms. We notice that cos(2π 132n).w[n] is indistinguishable from its reconstruction in
figure 3.7. In Chapter 4, we will consider examples, derived from wavelet theory, that
behave even better numerically. This same method will give interpolating functions
that achieve an even higher PSNR, showing that perfect reconstruction is not only
achievable in theory but also in practice. These good news don’t mean that (3.4) is
satisfied for all real life signals and all M . Figure 3.8 shows an example of a signal,
obtained from a wavelet decomposition, that doesn’t satisfy equation (3.4) for M
= 8 as the spectrum associated with ↑↓ {x[n].w[n]}, where w[n] is the rectangular
window, shows in figure 3.9. The effect of windowing however means that we will be
able to work with many real life signals that satisfy equation (3.4) even though their
theoretical model might indicate otherwise.
Another simplification in the infinite dimensional setting is to study the case of shift
invariant subspaces. We’ll see in Chapter 4 that we can decompose `2(Z) into a finite
direct sum of discrete shift invariant subspaces, each of which can be analyzed with
the framework presented in subsection 3.2.1 and section 3.3.
3.2.1 Interpolation in Continuous Time, Band-Limited, Shift
Invariant Subspaces
Suppose that x(t) belongs to a shift invariant subspace S ⊂ PWTs, meaning that x(t)
can be expressed in the following way, with s(t) ∈ S,
![Page 62: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/62.jpg)
40 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
x(t) =+∞∑
k=−∞
c[k]s(t− kMTs) (3.16)
where M is a positive integer. Note that c[k] are not necessarily samples of x(t). We
are interested in representing x(t) as
x(t) =+∞∑
k=−∞
x(kMTs)u(t− kMTs) (3.17)
where u(t) ∈ S is an interpolating function. There are a few generic results about
this problem in [28], [32]. If s(t) ∈ PWTs, we can get a very simple result when we
apply Theorem 3.2 to s(t).
Theorem 3.3. Suppose x(t) ∈ PWTs admits a representation as in equation (3.16).
Suppose also that S(z), the z transform of the discrete signal that results from sam-
pling s(t) at rate 1Ts
, satisfies equation (3.4). Then, x(t) admits a representation as
in equation (3.17) where u(t) depends on s(t), and is independent of c[k].
Proof. Since s(t) satisfies theorem 3.2, we can define U(z), as well as u(t) ∈ PWTs ,
from S(z),
U(z) =S(z)
1M
∑M−1k=0 S(ze−j
2πMk)
u(t) =+∞∑
n=−∞
u[n]sinc
(t− nTsTs
)s(t) can be then expressed as,
s(t) =+∞∑j=−∞
s[jM ]u(t− jMTs) (3.18)
plugging this expression for s(t) in equation (3.16) we get,
![Page 63: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/63.jpg)
3.2. INTERPOLATION OF SIGNALS IN PWTS 41
x(t) =+∞∑
k=−∞
c[k]s(t− kMTs) =+∞∑
k=−∞
c[k]+∞∑j=−∞
s[jM ]u(t− kMTs − jMTs) =
+∞∑j=−∞
+∞∑k=−∞
c[k]s[jM ]u(t− kMTs − jMTs) =+∞∑j=−∞
+∞∑i=−∞
c[i− j]s[jM ]u(t− iMTs) =
+∞∑i=−∞
+∞∑j=−∞
c[i− j]s[jM ]u(t− iMTs) =+∞∑i=−∞
b[i]u(t− iMTs) (3.19)
On the right hand side, second line, of equation (3.19), we have made the change of
variable i = k + j; b[i] is defined as
b[i] =+∞∑j=−∞
s[jM ]c[i− j] = s[iM ] ∗ c[i] (3.20)
we can conclude that b[i] = x[iM ] by taking inner products with 1Ts
sinc(t−mMTs
Ts
)on
both sides of equation (3.19),
〈 1
Tssinc
(t−mMTs
Ts
), x(t)〉 = 〈 1
Tssinc
(t−mMTs
sT
),
+∞∑i=−∞
b[i]u(t− iMTs)〉 =
〈 1
Tssinc
(t−mMTs
Ts
),
+∞∑i=−∞
b[i]+∞∑
n=−∞
u[n]sinc
(t− (iM + n)Ts
Ts
)〉 =
+∞∑i=−∞
b[i]+∞∑
n=−∞
u[n]〈 1
Tssinc
(t−mMTs
Ts
), sinc
(t− (iM + n)Ts
Ts
)〉
=+∞∑i=−∞
b[i]u[(m− i)M ] = b[m] (3.21)
![Page 64: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/64.jpg)
42 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
In the last line of equation (3.21), we use two properties. First, the orthogonality
of the delayed sinc functions implies that 〈sinc(t−mMTs
Ts
), sinc
(t−(iM+n)Ts
Ts
)〉 = 0
except when iM + n = mM , ie, n = (m − i)M , in which case the inner product is
equal to Ts. Second, since u[n] is Nyquist(M), u[(m − i)M ] = 0 for all (m − i)M
except for (m − i)M = 0 ⇒ m = i, in which case we have u[0] = 1 . Now, since1Ts〈sinc
(t−mMTs
Ts
), x(t)〉 = x[mM ] , we have b[m] = x[mM ].
If we suspect that our signal fits the model (3.16), we just need to sample the output
of the filter whose impulse response is s(t) at rate 1/Ts to get s[n]. If s[n] satisfies
the conditions of theorem 3.2, we build u[n]. The signal x(t), can then be sampled
at rate 1MTs
without loss of information. The sampling of s(t) at rate 1/Ts need not
be physical; it can just be that we have a good theoretical model for our signal from
which we obtain s[n]. From now on, we will call s[n] the atom that generates the
shift invariant subspace while u[n], if it can be defined, will be the interpolation atom
associated with the shift invariant subspace.
3.3 Frame Theory in the Context of Discrete Sam-
pling in Continuous Time Band-Limited Spaces
We now shift gears to study how frame theory can be used to analyze Discrete Sam-
pling in continuous time, band-limited, shift invariant subspaces.
Theorem 3.4. Suppose that we have the setting explained in subsection 3.2.1, that is,
we have signal x(t) ∈ S = span{s(t− kMTs)}k∈Z ⊂ PWTs. In addition assume that
〈s(t−iMTs), s(t−jMTs)〉 = δij, that is, we assume that the vectors {s(t−kMTs)}k∈Zform and orthonormal basis for S. We also assume that s(t) satisfies the conditions
of Theorem 3.3, thus we know that there exists an interpolating function u(t) ∈ S
such that x(t) =∑+∞
i=−∞ x[iM ]u(t− iMT ).
Then {u(t− kMT )}k∈Z is a Riesz basis for S. Furthermore, define c[i− j] = 〈u[n−iM ], u[n− jM ]〉. If c[i− j] is compactly supported, then we can compute explicitly the
optimal frame bounds as well as the dual Riesz basis associated with {u(t−kMTs)}k∈Z.
![Page 65: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/65.jpg)
3.3. FRAMES OF SHIFT INVARIANT SUBSPACES OF PWTS 43
Proof. Consider u[n] = u(nTs) and s[n] = s(nTs). From theorem 3.3, we know that
the following U(z), S(z) and Uf (z) are well defined,
U(z) = S(z)
Uf (z)︷ ︸︸ ︷1
1M
∑M−1k=0 S(ze−j
2πMk)
(3.22)
Equation (3.22) can equivalently be expressed in the discrete time domain as u[n] =
s[n] ∗ uf [n], where uf [n] is the sequence whose z transform is Uf (z). By the assump-
tions of this theorem, the operator ∗uf [n] : `2(Z) → `2(Z), such that ∗uf [n]{y[n]} =
y[n] ∗ uf [n] ∀y[n] ∈ `2(Z), is a linear, bounded and bijective operator. Further-
more, one of the properties of the z transform is that if Z{x[n]} = X(z) then
Z{x[n − kM ]} = z−kMX(z); thus, since u[n] = s[n] ∗ uf [n], it follows that Z{u[n −kM ]} = z−kMU(z) = z−kMS(z)Uf (z) = Z{s[n− kM ]}Uf (z). Therefore u[n− kM ] =
s[n− kM ] ∗ uf [n].
By definition 2.2, {u[n − kM ]}k∈Z = {s[n − kM ] ∗ uf [n]}k∈Z is a Riesz basis of
span{s[n− kM ]}k∈Z and therefore {u(t− kMTs)}k∈Z is a Riesz basis of S.
Now working with u[n], define the function c[i − j] = 〈u[n − iM ], u[n − jM ]〉 =
〈u[n], u[n− (i−j)M ]〉 . Since this function depends only on i−j we can call k = i−jand say that c[i, j] = c[i−j] = c[k]. In fact, it follows very quickly that c[k] = ruu[kM ],
ie, c[k] is the downsampled version, by a factor M, of the autocorrelation function of
u[n], ruu[k]. As an immediate consequence, c[k] is symmetric while c[0] = ||u[n]||2.
There are a few steps ahead before we can compute the dual basis of {u(t−kMTs)}k∈Z.
Let’s start by computing the norm of x(t) =∑+∞
i=−∞ x[iM ]u(t− iMTs); we have
||x(t)||2 = 〈x(t), x(t)〉 =+∞∑i=−∞
x[iM ]〈x(t), u(t− iMTs)〉 =
+∞∑i=−∞
x[iM ]+∞∑j=−∞
Tsx[jM ]c[i− j] = Ts
+∞∑i=−∞
+∞∑j=−∞
x[iM ]x[jM ]c[i− j] > 0 ∀x(t) 6= 0
(3.23)
![Page 66: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/66.jpg)
44 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
Given that,
〈x(t), u(t− jMTs)〉 = 〈+∞∑i=−∞
x[iM ]u(t− jMTs), u(t− iMTs)〉 =
+∞∑i=−∞
x[iM ]〈u(t− jMTs), u(t− iMTs)〉 =+∞∑i=−∞
x[iM ]Tsc[i− j]
An interesting consequence of equation (3.23) is that ||x(t)|| is entirely determined
by samples x[mM ]. Since x[n] ∈ `2(Z) ⇒ x[nM ] ∈ `2(Z). In addition, since ||x(t)||is well defined, given that x(t) is square integrable, equation (3.23) means that the
sequence c[i− j] is positive definite. If c[i− j] is compactly supported, meaning that
c[k] = 0 for |k| > N , equation (3.23) has two important consequences. First, we can
define a (2N + 1) × (2N + 1) Toeplitz matrix C, with Ci,j = c[i − j], which is also
positive definite, that will play an important role when we compute the dual basis.
The second important implication is that the DTFT of c[k], C(ejω), gives us the upper
and lower bounds for the eigenvalues of the banded matrix C with Ci,j = c[i− j]i,j∈Z.
If we call λC,min > 0 the minimum over ω of |C(ejω)| and λC,max > 0 the maximum
over ω of |C(ejω)| we get that the positive definiteness of c[k] gives [33],
λC,minTs||x[nM ]||2 ≤ ||x(t)||2 ≤ λC,maxTs||x[nM ]||2 (3.24)
We can then derive the frame condition in the following way,
|〈x(t), u(t− lMTs)〉|2 = T 2s
+∞∑i=−∞
+∞∑j=−∞
x[iM ]x[jM ]c[j − l]c[i− l] > 0⇒
![Page 67: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/67.jpg)
3.3. FRAMES OF SHIFT INVARIANT SUBSPACES OF PWTS 45
+∞∑l=−∞
|〈x(t), u(t− lMTs)〉|2 = T 2s
+∞∑i=−∞
+∞∑j=−∞
x[iM ]x[jM ]+∞∑l=−∞
c[j − l]c[i− l] > 0
(3.25)
Let’s define d[i− j] as
d[i− j] = d[i, j] =+∞∑l=−∞
c[l − i]c[l − j] (3.26)
Since d[i − j] depends only on k = i − j, we can say d[i − j] = d[k]. It follows that
d[k] is the autocorrelation sequence of c[k]. The frame condition, equation (3.25),
implies that d[k] is a positive definite sequence. If c[k] is compactly supported, then
d[i − j] is also compactly supported. Consider the DTFT of d[k], D(ejω) and the
banded matrix D with Di,j = d[i − j]i,j∈Z. If we call λD,min > 0 the minimum over
ω of |D(ejω)| and λD,max > 0 the maximum over ω of |D(ejω)| we get that the frame
condition is equivalent to,
λD,minT2s ||x[nM ]||2 ≤
+∞∑k=−∞
|〈x(t), u(t− kMT )〉|2 ≤ λD,maxT2s ||x[nM ]||2
combined with the inequality shown in equation (3.24) gives,
TsλD,minλC,max
||x(t)||2 ≤+∞∑
k=−∞
|〈x(t), u(t− kMT )〉|2 ≤ TsλD,maxλC,min
||x(t)||2 (3.27)
Thus, the optimal frame bounds are A = TsλD,minλC,max
, B = TsλD,maxλC,min
.
![Page 68: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/68.jpg)
46 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
3.3.1 Orthogonal Interpolating Atoms
With this framework, the case of orthogonal interpolating atoms boils down to c[i−j] = c[0]δ[i − j] with c[0] = ||u[n]||2. It follows very quickly that in this case, A =
B = Tsc[0], since λD,min = λD,max = c[0]2 and λC,min = λC,max = c[0]. In turn, this
means that ||x(t)||2 is Tsc[0] = Ts||u[n]|2 multiplied by ||x[nM ]||2, the downsampled
version of x[n]. In addition, the dual basis of {u[n−kM ]}k∈Z is {u[n−kM ]}k∈Z with
u[n− iM ] = u[n− iM ]/||u[n]||2 = u[n− iM ]/c[0] .
3.3.2 Computing the Dual Riesz Basis For Compactly Sup-
ported c[k]
Frame theory provides for the existence of the dual frame but it is less generous with
algorithms to compute the dual frame in the general setting. Fortunately, because of
the particularities of interpolating frames in `2(Z), if we make the extra assumptions
that c[k] is compactly supported, and that {u[n − iM ]}k∈Z is a Riesz basis, there is
a very straightforward algorithm that gives us the dual frame associated with {u[n−iM ]}k∈Z. These two assumptions are by no means restrictive since the most widely
studied and understood shift invariant subspaces come from Wavelet Theory, which
provides several families of shift invariant subspaces whose generators are compactly
supported and satisfy the requisites of Theorem 3.4. To facilitate the manipulation,
let’s assume that c[k] is 2N + 1 long. Suppose that the dual frame is in the form
{u[n−iM ]}k∈Z, which is a reasonable assumption to make that will be later confirmed
by the result. We know that u[n] ∈ span{u[n− iM ]}, thus we can write
u[n] =+∞∑i=−∞
u[iM ]u[n− iM ]
But we also have that a Riesz basis and its dual must satisfy the biorthogonality
condition, thus
〈u[n], u[n−mM ]〉 = 〈+∞∑i=−∞
u[iM ]u[n− iM ], u[n−mM ]〉 = δ(m)
![Page 69: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/69.jpg)
3.3. FRAMES OF SHIFT INVARIANT SUBSPACES OF PWTS 47
−∞∑i=−∞
u[iM ]〈u[n− iM ], u[n−mM ]〉 =m+N∑i=m−N
u[iM ]c[i−m] = δ(m)
These equations give us the perfect way to estimate the 2N+1 coefficients u[iM ]Nk=−N .
If we rescue the (2N + 1) × (2N + 1) Toeplitz matrix C, such that Ci,j = c[i − j],then we can create the following system of equations,
Cu = b
Where b is a (2N + 1)x1 vector for which bN+1,1 = 1 but bi,1 = 0 for any other i. b
is basically a discrete Delta function. Finally, u is a (2N + 1)x1 vector for which we
want ui,1 = u[(i− (N + 1))M ] . Since C is positive definite, it is also full rank, thus
u = (C)−1b
And with that we get the generator of the dual frame, u[n], which in general is not
an interpolating atom unless {u[n− iM ]}k∈Z are orthogonal,
u[n] =N∑
i=−N
u[iM ]u[n− iM ] (3.28)
From u[n] we get the dual frame as {u[n − iM ]}k∈Z. By construction, the vectors
{u[n − iM ]}k∈Z satisfy the biorthogonality condition 〈u[n], u[n − mM ]〉 = 〈u[n +
mM ], u[n]〉 = δ(m). Thus, since Theorem 2.8 says that the dual of a Riesz basis is
the unique set of vectors whose vectors are biorthogonal to the vectors of the original
Riesz basis, {u[n − iM ]}k∈Z is the dual Riesz basis of {u[n − iM ]}k∈Z. Finally, we
can build u(t) as well,
u(t) =+∞∑
n=−∞
u[n]sinc
(t− nTsTs
)
![Page 70: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/70.jpg)
48 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
Mallat’s book [28] reports a general version of this result in its Theorem 7.20 by
starting with a general setup in which φ(t) is defined as an interpolating continuous
time function, not necessarily band-limited, with the interpolation interval normalized
to 1. That result, attributed to Aldroubi and Unser, states that if Φ(ω) is the Fourier
transform of φ(t) and Φ(ω) is the Fourier transform of its dual φ(t), the continuous
time interpolating function and its dual are related by the expression,
Φ(ω) =Φ(ω)∑+∞
k=−∞ |Φ(ω + 2kπ)|2
which is more complicated than the method showed here for band-limited shift in-
variant spaces.
3.3.3 Numerical Example
To illustrate the concepts described in this chapter, let’s consider the shift invariant
subspace spanned by φ12, taken from the wavelet decomposition Symmlet 8 J=12,
L=10, with M = 4. Chapter 4, subsection 4.4.1, will describe in more detail what
this means; for the purpose of this example it is enough to think of φ12 as a subspace
generator that satisfies the requisites of Theorem 3.4 with M = 4, including that
the set {φ12[n − 4k]}k∈Z is an orthonormal basis for the subspace. Its associated
interpolating atom is shown in table 3.1. In this example, u[n] is truncated to have
length equal to 33.
u[−16] = −7.869 · 10−18 u[−15] = 0.005092 u[−14] = 0.008966 u[−13] = 0.006037u[−12] = 3.572 · 10−17 u[−11] = −0.01098 u[−10] = −0.01841 u[−9] = −0.005762u[−8] = −3.954 · 10−17 u[−7] = −0.01938 u[−6] = −0.04299 u[−5] = −0.06309u[−4] = 2.223 · 10−16 u[−3] = 0.2185 u[−2] = 0.5197 u[−1] = 0.8244
u[0] = 1.0 u[1] = 0.9127 u[2] = 0.6399 u[3] = 0.2976u[4] = −3.539 · 10−16 u[5] = −0.1278 u[6] = −0.1232 u[7] = −0.06625u[8] = 7.408 · 10−17 u[9] = 0.02373 u[10] = 0.01503 u[11] = 0.006445u[12] = 4.235 · 10−17 u[13] = −0.0007221 u[14] = 0.00238 u[15] = 0.001992u[16] = 6.069 · 10−17
Table 3.1: u[n] Associated with φ12, Wavelet Decomposition Symmlet 8 J=12, L=10;see figure 4.6
We can then compute the sequence c[k] associated with u[n]. It turns out that not
![Page 71: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/71.jpg)
3.3. FRAMES OF SHIFT INVARIANT SUBSPACES OF PWTS 49
only c[k] is compactly supported, as expected, in addition, we can get a very good
numerical result by assuming that all coefficients c[k] such that |k| ≥ 5 are zero.
Thus, we can work with the c[k] shown in table 3.2. Inspecting the table, we also
verify that c[k] is symmetric and that c[0] = ||u[n]||2.
c[−4] = 0.01744 c[−3] = 0.01895 c[−2] = −0.1952 c[−1] = 0.4793c[0] = 3.372 c[1] = 0.4793 c[2] = −0.1952 c[3] = 0.01895c[4] = 0.01744
Table 3.2: c[k] Associated with u[n] Shown in Table 3.1
We can easily compute matrix C, 3.372 0.4793 −0.1952 0.01895 0.01744 0 0 0 00.4793 3.372 0.4793 −0.1952 0.01895 0.01744 0 0 0
−0.1952 0.4793 3.372 0.4793 −0.1952 0.01895 0.01744 0 00.01895 −0.1952 0.4793 3.372 0.4793 −0.1952 0.01895 0.01744 00.01744 0.01895 −0.1952 0.4793 3.372 0.4793 −0.1952 0.01895 0.01744
0 0.01744 0.01895 −0.1952 0.4793 3.372 0.4793 −0.1952 0.018950 0 0.01744 0.01895 −0.1952 0.4793 3.372 0.4793 −0.19520 0 0 0.01744 0.01895 −0.1952 0.4793 3.372 0.47930 0 0 0 0.01744 0.01895 −0.1952 0.4793 3.372
as well as it inverse C−1,
0.3048 −0.04747 0.02567 −0.008352 0.001454 −0.000602 8.583 · 10−5 −1.239 · 10−5 2.592 · 10−6
−0.04747 0.3122 −0.05147 0.02697 −0.008579 0.001548 −0.0006153 8.754 · 10−5 −1.239 · 10−5
0.02567 −0.05147 0.3144 −0.05217 0.02709 −0.00863 0.001555 −0.0006153 8.583 · 10−5
−0.008352 0.02697 −0.05217 0.3146 −0.05221 0.02711 −0.00863 0.001548 −0.0006020.001454 −0.008579 0.02709 −0.05221 0.3146 −0.05221 0.02709 −0.008579 0.001454
−0.000602 0.001548 −0.00863 0.02711 −0.05221 0.3146 −0.05217 0.02697 −0.008352
8.583 · 10−5 −0.0006153 0.001555 −0.00863 0.02709 −0.05217 0.3144 −0.05147 0.02567
−1.239 · 10−5 8.754 · 10−5 −0.0006153 0.001548 −0.008579 0.02697 −0.05147 0.3122 −0.04747
2.592 · 10−6 −1.239 · 10−5 8.583 · 10−5 −0.000602 0.001454 −0.008352 0.02567 −0.04747 0.3048
The middle column of C−1 are the coefficients u[nM ] which are shown in table 3.3.
u[−16] = 0.001454 u[−12] = −0.008579 u[−8] = 0.02709 u[−4] = −0.05221u[0] = 0.3146 u[4] = −0.05221 u[8] = 0.02709 u[12] = −0.008579
u[16] = 0.001454
Table 3.3: Coefficients u[nM ] computed from c[k] shown in Table 3.2
With these u[nM ] we can construct u[n] via interpolation with atom u[n] as shown in
equation (3.28); the result is in table 3.4. The plots of both u[n] and u[n] are shown
in figure 4.7.
Finally, we can compute the d[k] sequence associated with c[k], which is shown in
table 3.5; d[k] = 0 for |k| ≥ 3.
Using Theorem 3.4, via the DTFTs of sequences c[k] and d[k], we can compute
λC,max = 4.0129, λC,min = 2.0199, λD,max = 12.2556 and λD,min = 11.5600. Therefore,
the optimal frame bounds for the continuous time frame {u(t− k4Ts)}k∈Z associated
![Page 72: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/72.jpg)
50 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
u[−16] = 0.001454 u[−15] = 0.001103 u[−14] = −0.0009104 u[−13] = −0.006149u[−12] = −0.008579 u[−11] = −0.004804 u[−10] = 0.004396 u[−9] = 0.02085u[−8] = 0.02709 u[−7] = 0.009068 u[−6] = −0.02104 u[−5] = −0.05379u[−4] = −0.05221 u[−3] = 0.01809 u[−2] = 0.1283 u[−1] = 0.2451u[0] = 0.3146 u[1] = 0.2826 u[2] = 0.18 u[3] = 0.05255u[4] = −0.05221 u[5] = −0.08306 u[6] = −0.05845 u[7] = −0.01379u[8] = 0.02709 u[9] = 0.037 u[10] = 0.02385 u[11] = 0.006281
u[12] = −0.008579 u[13] = −0.01244 u[14] = −0.008106 u[15] = −0.002859u[16] = 0.001454
Table 3.4: u[n] Associated With u[n] Shown in Table 3.1
d[−2] = 0.0003042 d[−1] = 0.1739 d[0] = 11.91 d[1] = 0.1739 d[2] = 0.0003042
Table 3.5: d[k] Associated with c[k] Shown in Table 3.2
with discrete frame {u[n− 4k]}k∈Z are,
A = TsλD,minλC,max
= 2.8807Ts
B = TsλD,maxλC,min
= 6.0674Ts
3.3.4 Remarks
Figure 3.10: Illustration of Shift Invariant Subspace Sampling When {u[n− iM ]}k∈Zis a Riesz Basis
Frame theory brings a new perspective to Discrete Sampling in an infinite dimensional
setting. If the signal x[n] that we are interested in recovering lies in S, a shift-invariant
![Page 73: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/73.jpg)
3.3. FRAMES OF SHIFT INVARIANT SUBSPACES OF PWTS 51
subspace of `2(Z) generated by shifts of s[n] ∈ S, for which {s[n− kM ]}k∈Z form an
orthonormal basis of S, and s[n] satisfies equation (3.4), we can explicitly find an
interpolating basis for the subspace which is also shift invariant, {u[n − iM ]}k∈Z;
this interpolation basis is a Riesz basis for S. In such a basis, the coordinates of
x[n] are given by the samples x[nM ]. Thus, we just need to measure the signal at
times x[nM ], then build x[n] via interpolation. The samples x[nM ] happen to be as
well the inner products of x[n] with the dual Riesz basis {u[n − iM ]}k∈Z. The dual
Riesz basis plays an even more important role if x[n] doesn’t lie in S. In that case
taking the samples x[nM ] and interpolating with basis {u[n − iM ]}k∈Z will give us
a signal in S which might or might not be close to x[n]. Theorem 2.13 gives us a
better alternative. By considering the inner products 〈x[n], u[n− iM ]〉k∈Z, instead of
x[nM ], as the coordinates in basis {u[n− iM ]}k∈Z, we get the orthogonal projection
of x[n] onto S, x[n]. By the orthogonality principle, the signal reconstructed, x[n], is
the signal in S which is closest to x[n]. These concepts are illustrated in figure 3.10.
![Page 74: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/74.jpg)
52 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS
![Page 75: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/75.jpg)
Chapter 4
A New Method to Design Hybrid
Filter Banks
The framework presented in Chapter 3 has a very direct application to the problem
of designing Hybrid Filter Banks, also known as HBF’s. Figure 4.1 shows the block
diagram of an HFB. The basic idea is to use M analog filters, instead of a single an-
tialiasing filter, to process an analog signal of bandwidth 12Ts
prior to taking samples.
Sampling is performed through M Analog to Digital Converters, ADC, working at a
rate of 1TsM
, ie, M times below the Nyquist rate. Each ADC samples one of the filter
outputs. The signal is then recovered via digital filters that process the respective
ADC outputs, upsampled by factor M. The output of the HBF is the sum of the out-
puts of the digital filters; the objective in HBF design is that the output is a delayed
version of the input analog signal sampled at rate 1Ts
. Even though perfect recovery
in a HBF setting requires to collect as many data points as in Shannon sampling,
the fact that the ADC’s work at a lower rate has important practical consequences,
as explained in Chapter 1. In the same chapter we mentioned that except for the
so called sigma delta ADCs, which achieve high resolution at the expense of reduced
sampling rates and a complicated analog design, the architecture, from a paradigm
point of view, of most ADCs has remained unchanged over the last 60 years. HBFs
present an alternative paradigm although nonidealities have prevented the concept of
HBF ADCs from becoming mainstream.
53
![Page 76: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/76.jpg)
54 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.1: Hybrid Filter Bank Architecture, Adapted from [34]
This chapter is about designing HBFs with the tools developed in Chapter 3. In
addition to showing that Discrete Sampling for continuous time, band-limited signals
fits perfectly the problem of designing HBFs, although not every HBF might be
designed with Discrete Sampling, we will study the restrictions imposed on a practical
implementation of an HBF designed with this technique as well as how the noise
present at the output of each analog filter propagates to the HBF output. Examples
will be derived along the way to illustrate each of these items.
4.1 A Bit on History of HBF’s
That one could use a multichannel analog front end to sample signals with converters
working below the Nyquist rate was already suggested by Papoulis in 1977 [35][36].
Papoulis proposed that the reconstruction be analog as well. The work of Velazquez et
al.[37] is considered the first to suggest a hybrid approach. More recent work on both
design and implementation of HBFs can be found here [34][38] and here [39]. Going
![Page 77: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/77.jpg)
4.2. UNION OF ORTHOGONAL SI SUBSPACES WITH ON GENERATORS 55
back to figure 4.1, these works see the problem of designing an HBF as one in which
one has to compute the coefficients of H1(s) to HM(s), a family of M continuous time
filters, and the coefficients of F1(z) to FM(z), a family of M discrete time filters. The
filters H1(s) to HM(s) are known as the analysis filters. The filters F1(z) to FM(z) are
known as the synthesis filters. The objective of the design problem is to pick the filter
coefficients so that for any signal x(t) with bandwidth 12Ts
, y[n] is a delayed version
of x[n] = x(nTs), property known as perfect reconstruction in the literature. The
problem is usually formulated in terms of a numerical optimization. Theoretical
analysis on the filter bank is performed using classic Fourier theory applied to the
output of the synthesis filters. In particular, aliasing terms, due to the samplers
working at 1MTs
instead of the Nyquist rate, are analytically identified at the output
of the HBF y[n]; the designer attempts to drive to zero these components. Once
the aliasing terms are minimized, the remaining constraint is that the bank achieves
perfect reconstruction, or quasi perfect recontruction if some error margin is allowed.
With Discrete Sampling extended to `2(Z), we introduce a novel design method that
might help HBFs become more widely used. It is important to note that the method
we are about to present doesn’t work for every HBF, but given its generality, it’s a
step in the direction of making HBF’s more practical.
4.2 Union of Mutually Orthogonal Shift Invariant
Subspaces With Orthonormal Generators
Let’s use the ideas of Discrete Sampling to introduce a new point of view to the
problem of HBF design. Suppose that it is possible to decompose `2(Z) as `2(Z) =⊕Q−1q=0 span{sq[n − lMq]}l∈Z, with Q a positive integer and {sq[n − lMq]}l∈Z an or-
thonormal basis for the subspace spanned by these vectors; equivalently, suppose that
for any x[n] ∈ l2(Z) we have
x[n] =
Q−1∑q=0
+∞∑l=−∞
〈x[n], sq[n− lMq]〉sq[n− lMq] (4.1)
![Page 78: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/78.jpg)
56 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
with 〈sq1[n − l1Mq1], sq2[n − l2Mq2]〉 = 0 ∀q1, q2, l1, l2 ∈ Z, except when q1 = q2 and
l1 = l2, in which case it is equal to 1. Suppose further that the z transform of sq[n]
satisfies equation (3.4), thus we can define uq[n] as the sequence whose z transform is
Uq(z) =Sq(z)
1Mq
∑Mq−1k=0 Sq(ze
−j 2πMk)
then, from the framework developed in Chapter 3 it follows that we can write x[n] as
x[n] =
Q∑q=0
+∞∑i=−∞
〈x[n], uq[n−Mqi]〉uq[n−Mqi]
where uq[n] is the dual of interpolating atom uq[n]. We also know that for any
x(t) ∈ PWTs so that x(nTs) = x[n], 〈x[n], uq[n − Mqi]〉 = 1Ts〈x(t), uq(t − MqTsi)〉
where uq(t) =∑∞
n=−∞ uq[n]sinc( t−nTsTs
). Now, 〈x(t), uq(t−MqTsi)〉 =∫ +∞−∞ x(t)uq(t−
MqTsi)dt =∫ +∞−∞ x(t)uq(−(MqTsi − t))dt = x(t) ∗ uq(−t)(MqTsi). Thus, to get
〈x(t), uq(t−MqTsi)〉 we need to sample an analog filter that implements uq(−t) at rate1
MqTswhen the input of the filter is x(t). If all Mq are equal to M , we have an HFB as
shown in figure 4.1. Discrete Sampling gives us a method to design a particular class
of HBFs, provided that there exists a decomposition of x[n] ∈ l2(Z) as dictated by
equation (4.1). Wavelet Theory gives us a way to do precisely that. In fact, Wavelet
Theory adds the flexibility that all Mq don’t have to be necessarily equal although
they have to be powers of 2. With Discrete Sampling the only difficulty that remains
for having an HFB is the implementation of a set of analog filters, a topic that will
be dealt with in section 4.6.
4.3 Very Brief Overview of Wavelet Theory
I am going to state without proof the main results of Wavelet Theory relevant to
Discrete Sampling. There are several good references on this topic. What I state here
is mostly taken from Stephane Mallat’s book “A Wavelet Tour of Signal Processing”,
Second Edition [28]. Wavelet Theory came about as a tool to represent signals in
![Page 79: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/79.jpg)
4.3. VERY BRIEF OVERVIEW OF WAVELET THEORY 57
L2(R) and l2(Z) in ways that overcame the limitations of the continuous and dis-
crete Fourier Transforms. The Fourier framework is particularly bad for representing
transient signals since it doesn’t represent time location well. Intuitively, the Fourier
framework decomposes signals as a sum of sinusoids. A sinusoid goes from t = −∞ to
t = +∞, or from n = −∞ to n = +∞, oscillating between +1 and −1. A transient
signal, of limited duration, needs a large number of Fourier coefficients because a few
of them would most likely result in infinite duration signals: if you have one Fourier
coefficient you have a single sinusoid, which is of infinite duration; two coefficients
give you a signal which is also of infinite duration with more room to cancel parts of
the signal. Adding coefficients gives you extra degrees of freedom until one can finally
pull off the limited duration signals. There are other ways to think about this, of
course, for instance, that the Fourier transform of a Delta function is a flat spectrum
and that the inverse Fourier transform of a Delta function is a constant signal, but
the conclusion is the same: Fourier analysis doesn’t result in sparse representations
for transient signals. Since the beginning of the XX-th century, mathematicians,
engineers and scientists have tried to come up with alternative representations that
fit particular needs. During the 1980s, Mallat, Daubechies and others unified the
different approaches giving birth to what is known as Wavelet Theory.
4.3.1 Multiresolution Representations
Wavelet bases result from multiresolution decompositions of L2(R). These decompo-
sitions can be discretized to have decompositions of `2(Z). A multiresolution approx-
imation, or decomposition, is a sequence of closed subspaces {Vj}j∈Z of L2(R) that
satisfies the following 6 properties
1. ∀(j, k) ∈ Z2, f(t) ∈ Vj ⇔ f(t− 2jk) ∈ V j
2. ∀j ∈ Z, Vj+1 ⊂ Vj
3. ∀j ∈ Z, f(t) ∈ Vj ⇔ f( t2) ∈ Vj+1
4. limj→+∞ Vj = ∩+∞j=−∞Vj = {0}
![Page 80: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/80.jpg)
58 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
5. limj→−∞ Vj = ∪+∞j=−∞Vj = L2(R)
6. There exists θ such that {θ(t− n)}n∈Z is a Riesz basis of V0
If one thinks about it, and considers point 6 together with points 1, 2 and 3, it’s not
difficult to realize that these multiresolution approximations are sequences of shift
invariant subspaces so that any signal f(t) ∈ L2(R) can be approximated to the
required resolution by projecting it onto the appropriate subspace Vj. The nomencla-
ture used calls 2j the scale parameter while it calls its inverse 2−j the resolution. So,
high resolution means low scale parameter while coarse resolution means high scale
parameter. Perhaps, the most intuitive example of a multiresolution approximation
is the sequence of band-limited subspaces. If one defines Vj as the set of functions
whose Fourier transform has a frequency support included in [− 12j+1 ,
12j+1 ], one gets
a multiresolution approximation. In fact, we see that in this case Vj = PW2j . For
any signal f(t) ∈ L2(R), its orthogonal projection on Vj is the signal whose Fourier
transform in [− 12j+1 ,
12j+1 ] equals the Fourier transform of f(t) but it’s zero elsewhere,
ie, it’s f(t) lowpass filtered in the band [− 12j+1 ,
12j+1 ]. Finally, in this approximation,
θ(t) = sinc(t), so V0 = span{sinc(t−n)}n∈Z. In other words, the space of band-limited
signals of support [−0.5,+0.5], PW1, is V0.
4.3.2 A Multiresolution Approximation is a Collection of
Shift Invariant Subspaces
The following result formalizes what was stated previously.
Theorem 4.1. Suppose we are given a multiresolution approximation {Vj}j∈Z. With
Θ(ω), the Fourier transform of θ(t), we can build the function φ(t) whose Fourier
transform is
Φ(ω) =Θ(ω)√∑+∞
k=−∞ |Θ(ω + 2kπ)|2
Define the set of functions
![Page 81: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/81.jpg)
4.3. VERY BRIEF OVERVIEW OF WAVELET THEORY 59
φj,n(t) =1√2jφ
(t− 2jn
2j
)(n,j)∈Z2
Then, the family φj,n(t) is an orthonormal basis of Vj for all j ∈ Z.
The bottom line is that with this framework we can approximate any f(t) ∈ L2(R) to
any resolution we want by computing the orthogonal projection of f(t) onto Vj; since
we have an orthonormal, shift invariant basis for each subspace Vj, the approxima-
tion of f(t) in Vj is fj(t) =∑∞
n=−∞〈f(t), 1√2jφ(t−2jn
2j
)〉 1√
2jφ(t−2jn
2j
). The function
so defined φ(t) is called the scaling function. In the case of the band-limited mul-
tiresolution approximation, φ(t) = θ(t).
4.3.3 Conjugate Mirror Filters
The multiresolution approximations property 2 implies interesting things about the
scaling function, in particular that 2−1/2φ(t/2) ∈ V1 ⊂ V0, which translates into
1√2φ
(t
2
)=
+∞∑n=−∞
h[n]φ(t− n)
With
h[n] = 〈 1√2φ
(t
2
), φ(t− n)〉
h[n] can be thought of as a discrete time filter. It turns out that this filter plays
a very important role in the whole process. Mallat, in addition to introducing the
concept of multiresolution approximation proved the following theorem.
Theorem 4.2. Suppose we are given an integrable scaling function φ(t) ∈ L2(R),
then the Discrete Time Fourier Transform of h[n], H(ω) satisfies that
∀ω ∈ R, |H(ω)|2 + |H(ω + π)|2 = 2
![Page 82: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/82.jpg)
60 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
with H(0) =√
2. The filters that satisfy these two conditions are called conjugate
mirror filters and were known independently of wavelet theorists. Conversely, sup-
pose we are given a function H(ω), 2π periodic, and continuously differentiable in
a neighborhood of ω = 0, if ∀ω ∈ R, |H(ω)|2 + |H(ω + π)|2 = 2, H(0) =√
2 and
infω∈[−π2,π2
] |H(ω)| > 0, then we can build the Fourier transform Φ(ω) of a scaling
function φ(t) as
Φ(ω) =+∞∏p=1
H(2−pω)√2
4.3.4 Wavelet Decomposition
Since Vj ⊂ Vj−1, let’s define Wj as the orthogonal complement of Vj in Vj−1. Thus
Vj−1 = V j⊕
Wj. Wj is the so called “wavelet space” at resolution 2−j. Wj
contains the information that is lost when a signal in subspace j− 1 is approximated
by its orthogonal projection in subspace j. We have the following result.
Theorem 4.3. Suppose we are given a scaling function φ(t) and its associated con-
jugate mirror filter, h[n]. Let ψ(t) be a function whose Fourier transform, Ψ(ω), is
defined as,
Ψ(ω) =1√2G(ω
2
)Φ(ω
2
)
With,
G(ω) = e−iωH∗(ω + π)
If we define for any scale 2j
ψj,n(t) =1√2jψ
(t− 2jn
2j
)
![Page 83: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/83.jpg)
4.3. VERY BRIEF OVERVIEW OF WAVELET THEORY 61
Then ψj,n(t)n∈Z is an orthonormal basis of Wj. Further, ψj,n(t)(j,n)∈Z2 is an or-
thonormal basis of L2(R). Any f(t) ∈ L2(R) can be written as,
f(t) =∞∑
n=−∞
∞∑j=−∞
〈f(t),1√2jψ
(t− 2jn
2j
)〉 1√
2jψ
(t− 2jn
2j
)
4.3.5 Fast Orthogonal Wavelet Transform
Figure 4.2: DWT Decomposition (top), Inverse (bottom) [28]
Consider the subspace Vj−1 = Vj⊕
Wj. We can define aj[n] = 〈f(t), φj,n(t)〉 and
dj[n] = 〈f(t), ψj,n(t)〉. Mallat showed that the following analysis and reconstruction
formulas hold, using the filters h[n] and g[n] constructed previously,
aj+1[p] =∞∑
n=−∞
h[n− 2p]aj[n]
dj+1[p] =∞∑
n=−∞
g[n− 2p]aj[n]
aj[p] =∞∑
n=−∞
h[p− 2n]aj+1[n] +∞∑
n=−∞
g[p− 2n]dj+1[n]
The first two formulas correspond to the filter bank implementation of the Discrete
![Page 84: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/84.jpg)
62 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.3: WPT Decomposition (top), Inverse (bottom) [28]
Wavelet Transform, DWT, while the third formula is the filter bank implementation
of the Inverse Discrete Wavelet Transform, IDWT.
Introducing the filters g[n] = g[−n] and h[n] = h[−n], figure 4.2 shows the block
diagram of the filter bank implementation of the DWT. In decomposition mode, the
coefficients aj[n] are low pass filtered with h[n] and high pass filtered with g[n]. Then
the output of both filters is downsampled by 2. The downsampled output of the high
pass filter, dj+1[n], is kept, as the wavelet (detail) coefficients of that level, while the
downsampled output of the low pass filter is decomposed anew in the same manner.
When the decomposition is stopped, the downsampled output of both filters is kept:
the downsampled output of g[n], dJ [n], as the wavelet (detail) coefficients at scale 2J ,
the downsampled output of h[n], aJ [n], as the coefficients of the approximation of the
signal with scale 2J . Figure 4.2 also shows how the reverse process, reconstruction or
IDWT, is implemented. It can be shown that a discrete wavelet decomposition with
a tree of depth J − L ≥ 0, gives a decomposition of `2(Z) via an orthonormal basis.
![Page 85: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/85.jpg)
4.3. VERY BRIEF OVERVIEW OF WAVELET THEORY 63
More precisely, given one such decomposition, any x[n] ∈ `2(Z) can be written as,
x[n] =+∞∑l=−∞
aJ [l]φJ [n− 2J−Ll] +J∑
j=L+1
+∞∑l=−∞
dj[l]ψj[n− 2j−Ll] (4.2)
Where the vectors
{φJ [n− 2J−Ll]}l∈Z, {ψj[n− 2j−Ll]}L<j≤J, l∈Z
form an orthonormal basis of `2(Z). It follows that aJ [l] = 〈x[n], φJ [n− 2J−Ll]〉 and
that dj[l] = 〈x[n], ψj[n−2j−Ll]〉. As a corollary the J−L+1 shift invariant subspaces
span{φJ [n−2J−Ll]}l∈Z, span{ψL+1[n−2l]}L<L+1, l∈Z, ... span{ψJ [n−2J−Ll]}J, l∈Z are
mutually orthogonal and the vectors that span each shift invariant subspace form an
orthonormal basis for the corresponding subspace. With the appropriate normaliza-
tion, what really matters is J −L since the above decomposition of `2(Z) starts with
the assumption that aL[n] = x[n].
As an example, say that we take J = 12 and L = 10, equation (4.2) becomes,
x[n] =+∞∑l=−∞
a12[l]φ12[n− 4l] ++∞∑l=−∞
d11[l]ψ11[n− 2l] ++∞∑l=−∞
d12[l]ψ12[n− 4l] (4.3)
which means that l2(Z) = span{φ12[n−4l]}l∈Z⊕
span{ψ12[n−4l]}l∈Z⊕
span{ψ11[n−2l]}l∈Z . Indeed, given a signal x[n] ∈ `2(Z) if we set x[n] = aL[n] = a10[n], we get
through the DWT decomposition recursion a11[n] and d11[n]. We keep d11[n] and
repeat the process with a11[n] to get a12[n] and d12[n], at which point we stop the
recursion. Plugging these coefficients into equation (4.2), we get equation (4.3).
Thus, equation (4.2) implies a decomposition of `2(Z) in a union of mutually orthog-
onal shift invariant subspaces, each of which has an orthonormal generator.
Before we move forward, it must be pointed out that there is also the possibility of
decomposing `2(Z) in a way that both downsampled outputs of ˜h[n] and g[n] are
decomposed recursively, instead of applying the recursion only to the downsampled
output of filter h[n], as the standard DWT does. This alternative way, for both
![Page 86: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/86.jpg)
64 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
decomposition and reconstruction, is shown in figure 4.3. It can be proved that this
decomposition also results in a decomposition of `2(Z) via an orthonormal basis. This
is the so called Wavelet Packet Decomposition, WPD. The WPD is not regularly used
in compression applications because it doesn’t result in significant compression gains
when compared to the DWT. In our case however, it is very interesting because it
matches the canonical Hybrid Filter Bank model with M = 2J−L.
We can now apply the results developed in Chapter 3 for Discrete Sampling to a
wavelet decomposition of l2(Z).
4.4 Discrete Sampling for Continuous Time, Band-
Limited Signals Meets Wavelet Theory
Figure 4.4: HFB Associated with Decomposition Symmlet 8 J=12, L=10
In the previous section about wavelets, it was stated without proof that given a
wavelet multiresolution approximation, we can decompse `2(Z) as
`2(Z) = span{φJ [n− 2J−Ll]}l∈Z⊕ J⊕
j=L+1
span{ψj[n− 2l−Ll]}l∈Z
![Page 87: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/87.jpg)
4.4. DISCRETE SAMPLING FOR PWTS MEETS WAVELET THEORY 65
Figure 4.5: Discrete Equivalent of Figure 4.4
Now, suppose that φJ [n] as well as {ψj[n]}L+1≤j≤J , a total of J − L + 1 discrete
waveforms, are compactly supported and that for each of them the interpolation
condition (3.4) for its z transform is satisfied. Because this decomposition of `2(Z)
fits the model described in section 4.2 as a union of mutually orthogonal shift invariant
subspaces with orthonormal generators, any x[n] ∈ `2(Z) can be written as
x[n] =+∞∑i=−∞
〈x[n], uφJ [n− 2J−Li]〉uφJ [n− 2J−Li]+
J∑j=L+1
+∞∑i=−∞
〈x[n], uψj [n− 2j−Li]〉uψJ [n− 2j−Li]
with uφJ [n] and uφJ [n] the interpolation atom and its respective dual, associated with
φJ [n]. Similarly, {uψj [n]}L+1≤j≤J and {uψj [n]}L+1≤j≤J are the interpolation atoms
and their respective duals, associated with {ψj[n]}L+1≤j≤J .
4.4.1 Example
Let’s illustrate what we have developed so far with an example. The so called Symmlet
8 decomposition with parameters J = 12, L = 10, results in `2(Z) = span{φ12[n −
![Page 88: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/88.jpg)
66 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.6: Symmlet 8 Generators With J=12, L=10
4l]}l∈Z⊕
span{ψ12[n − 4l]}l∈Z⊕
span{ψ11[n − 2l]}l∈Z with the atoms φ12, ψ12 and
ψ11 compactly supported. These atoms are shown in figure 4.6. The corresponding
interpolating atoms, with their respective duals, are shown in figures 4.7, 4.8 and 4.9.
Figure 4.4 shows the block diagram of this example for decomposition and reconstruc-
tion. To verify numerically that this particular setup gives perfect reconstruction, we
used a 32 seconds long segment of the theme “Pageant” from the show KA by Cirque
du Soleil [40]. Our x[n] is obtained by sampling the first channel of the stereo signal
at fs = 44.1Khz with 16 bit amplitude resolution; x[n] has CD audio signal quality.
For simulation purposes, we used the discrete time equivalent of figure 4.4 shown
![Page 89: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/89.jpg)
4.4. DISCRETE SAMPLING FOR PWTS MEETS WAVELET THEORY 67
Figure 4.7: Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With φ12
Figure 4.8: Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ12
![Page 90: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/90.jpg)
68 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.9: Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ11
in figure 4.5: digital filters with impulse responses uφ12 [−n], uψ12 [−n] and uψ11 [−n]
are used in lieu of their respective continuous time filters as well as discrete down-
samplers instead of continuous time samplers. x[n] is 1456128 samples long; thus,
the model x[n] ∈ `2(Z) fits this experiment perfectly. To measure the quality of the
reconstructed signal, we use the metric PSNR defined in equation (3.15), which is
reproduced in equation (4.4) for convenience,
PSNR = 10 log10
(N
max(|x[n]|)2
||x[n]− xrec[n]||2
)(4.4)
As we explained N is the signal’s length. In the case of audio signals in Matlab,
max(|x[n]|) = 1. To gauge the effect of discarding some of the filter outputs, which
would allow us to effectively sample x(t) at a rate lower than Nyquist’s, we define the
discrete signals xφ12 [n] and xφ12ψ12 [n] as,
• xφ12 [n] is the output of the filter whose impulse response is uφ12 [n]. This signal
requires only one forth of the information present in x[n] and one sampler
working at one forth of the Nyquist rate;
![Page 91: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/91.jpg)
4.4. DISCRETE SAMPLING FOR PWTS MEETS WAVELET THEORY 69
• xφ12ψ12 [n] is the signal which results from adding the outputs of the filters whose
impulse responses are uφ12 and uψ12 [n]. This signal requires half the information
present in x[n] but still uses samplers, two of them, working at one fourth of
the Nyquist rate.
The numerical results are shown in table 4.1.
Signal Measured PSNR [dB]xφ12
[n] 40.5xφ12ψ12
[n] 50.7x[n] 136
Table 4.1: Numerical Results for 32s Segment of Pageant
Note that the 136 dB PSNR obtained for x[n] in this case means perfect recon-
struction since x[n]’s amplitude resolution is 16 bits. Indeed one can verify that if
x[n], which was computed using double precision in matlab, is quantized to 16 bits
resolution to obtain x16bit[n], we get ||x16bit[n]− x[n]|| exactly equal to zero. This ex-
ample shows that perfect reconstruction can be obtained not only in theory but also
in practice if one keeps all filter outputs during reconstruction; in other words, if we
keep as many samples as the length of x[n]. But even in such case, all samplers work
at a rate below Nyquist’s. However, table 4.1 also says that we can get an acceptable
reconstruction, both numerically and subjectively, by generating xφ12ψ12 [n] instead of
x[n]. With xφ12 [n] we get artifacts that make it noticeably different from x[n]. To
get a feeling of how these signals differ in the frequency domain, figure 4.10 shows
their magnitude spectra in dB. These spectra have been computed with a 2048 FFT
and smoothed with a moving average filter of length 50 in order to improve their
visualization. The first important thing to note is that all the signals have frequency
components up to fs2
, which means that xφ12 [n] cannot be obtained by just replacing
the filter with impulse response uφ12 [−n] with a low pass filter whose cutoff frequency
is fs4
-in that case the Nyquist frequency would still be fs2
while we can get xφ12 [n]
by sampling at rate fs4
. The second observation is that x[n] and xφ12ψ12 [n] differ in
magnitude mostly in the frequencies close to fs2
, which explains why an untrained
listener cannot differentiate the two.
![Page 92: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/92.jpg)
70 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figu
re4.10:
Sm
ooth
edM
agnitu
de
Sp
ectrum
ofSign
alsx
[n][b
lue],xφ12 [n
][red
]an
dxφ12ψ12 [n
][green
]for
32sSegm
ent
ofT
hem
eP
ageant
![Page 93: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/93.jpg)
4.5. NOISE ANALYSIS 71
4.5 Noise Analysis
Characterizing how electrical noise affects the reconstruction of signals is vital in
practical ADC design. Electrical noise, and other sources of noise that can be modeled
in the same fashion, add in the output of the ADC with the net result of decreasing
the effective resolution of the ADC. A sound understanding of how, and how much,
electrical noise propagates to the output of the ADC is essential in a practical design.
Assuming that we know that we have the noise at the ADCs output under control,
and that such noise can be modeled as additive zero mean Gaussian iid, we can then
compute how this noise corrupts the output of the HBF.
4.5.1 A More Detailed Look at the Discrete Interpolation
Process
Consider the interpolation equation,
x[n] =+∞∑i=−∞
x[iM ]u[n− iM ]
u[n] can be written equivalently, using multirate discrete signal processing as [29][10],
u[n] =M−1∑k=0
uk[n− k]
with uk[n] = u[n+ k] if n is multiple of M and uk[n] = 0 otherwise. The decomposi-
tion of a filter in this manner is called polyphase decomposition. In the context
of interpolating atoms, u0 = δ[n]. In fact, an equivalent way of characterizing an
interpolating atom is that u[n] is an interpolating atom if and only if its polyphase
decomposition results in u0 = δ[n]. Figure 4.12 shows the polyphase decomposition of
uφ12 [n]. Intuitively this means that the actual information of an interpolating signal
is contained in the samples x[nM ] while the other samples x[n] where n 6= nM are
reconstructed from ↑ x[nM ]- upsampling factor equal to M- through the filters uk[n],
![Page 94: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/94.jpg)
72 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.11: Polyphase Decomposition of uφ12 [n], from Wavelet DecompositionSymmlet 8 With J=12, L=10
x[n] =+∞∑i=−∞
x[iM ]u[n− iM ] =+∞∑i=−∞
x[iM ]M−1∑k=0
uk[n− iM − k] =
M−1∑k=0
+∞∑i=−∞
x[iM ]uk[n− iM − k] =
+∞∑i=−∞
x[iM ]δ[n− iM ]+
![Page 95: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/95.jpg)
4.5. NOISE ANALYSIS 73
+∞∑i=−∞
x[iM ]u1[n− iM − 1] + ...+
+∞∑i=−∞
x[iM ]uM−1[n− iM − (M − 1)]
This way of looking at the interpolating process is illustrated in figure 4.12. Let’s call
x0[n] =↑ x[nM ] =∑+∞
i=−∞ x[iM ]δ[n−iM ], x1[n] =↑ x[nM+1] =∑+∞
i=−∞ x[iM ]u1[n−iM − 1], x2[n] =↑ x[nM + 2] =
∑+∞i=−∞ x[iM ]u2[n − iM − 2] and so on. Note that
each of these sequences is the result of a three step process:
i) filtering x[nM ] with filter uk[nM ] (since uk[n] is 0 for n 6= nM).
ii) upsample the sequence obtained in step i) by factor M
iii) delay the signal obtained in step ii) by k to get xk[n]
Therefore x[n] is the sum of M sequences, x[n] = x0[n] + x1[n] + x2[n] + ...+ xM−1[n],
each of which is generaged from ↑ x[nM ] via linear filtering.
Note that in practice it could happen that u[n] is a shifted version of an interpolation
atom that follows this model. In that case uk[nM ] = δ[n] for some k 6= 0 but the
analysis remains valid.
4.5.2 Linear Filtering of a Zero Mean Gaussian iid Process
Before we can analyze how noise affects the fidelity of the reconstructed signal, we
need to consider the filtering of a discrete zero mean Gaussian process since, for
reasons that will be explained later, it is the process that models best the sources of
noise encountered in practice. Suppose that one has a Gaussian iid process w[n] such
that w[n] is distributed as N (0, σ2). Suppose also that we filter this process through
a stable, linear, time invariant filter h[n]. The output of the filter y[n] = w[n] ∗ h[n]
is also a Gaussian process. Although not necessarily independent, y[n] are identically
![Page 96: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/96.jpg)
74 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.12: Interpolation From The Polyphase Decomposition Point of View
distributed as N (0, ||h[n]||2σ2) . In general y[n] is correlated with y[n−i] for i 6= 0. In
words, if one filters a discrete zero mean white Gaussian process with a stable linear
filter, the output is a discrete zero mean Gaussian process whose variance is related
to the variance of the input process through the norm of the filter coefficients. The
output process is not necessarily white. Note that if ||h[n]||2 ≤ 1, the filter does not
amplify the noise, which is a good thing.
4.5.3 Noise Analysis in an HBF Designed with Discrete Sam-
pling
In section 4.4 we showed that there exist wavelet decompositions that allow us to
represent any signal in x[n] ∈ l2(Z) as
x[n] =
Q−1∑q=0
+∞∑i=−∞
〈x[n], uq[n−Mqi]〉uq[n−Mqi]
where {uq[n]}Q−1q=0 are interpolating atoms that span mutually orthogonal shift invari-
ant subspaces and {uq[n]}Q−1q=0 are their respective duals. One such representation was
presented in subsection 4.4.1.〈x[n], uq[n−Mqi]〉 is computed in practice by sampling
at rate 1TMq
the output of an appropriately scaled continuous time filter. Let’s call
oq[i] = 〈x[n], uq[n −Mqi]〉. In a typical HBF setting, the most significant source of
![Page 97: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/97.jpg)
4.5. NOISE ANALYSIS 75
noise is the continuous time thermal noise introduced by the filters, which is additive,
zero mean and Gaussian. As we said in Chapter 1, it can be shown that in track
and hold circuits, as those used by ADCs, the effect of this Gaussian continuous time
noise can be equivalently modeled as a discrete time, zero mean Gaussian iid noise
that is added to oq[i] provided that when we acquire a sample, we wait long enough
to settle the measured sample to a resolution of 6 bit or better. The vast majority
of ADCs work at such resolutions. The variance of the discrete Gaussian iid noise
can then be estimated to be KTCq
in a first order approximation, with Cq the capacitor
of the track and hold circuit, T the temperature in degrees Kelvin and K the Boltz-
mann constant. Note that this allows the designer of the ADC to control the energy
of the noise present at the output of the ADC, although, as explained in Chapter
1, since increasing the size of Cq will make the ADC slower, the value of Cq will be
influenced by a trade off accuracy vs speed. Thus, in practice, instead of measuring
oq[i], we will be measuring oq[i]+wq[i] where we model the noise wq[i] as a discrete iid
process distributed as N (0, σ2q ) with σ2
q = KTCq
. This model means that not only wq[i]
is independent of wq[j] ∀i 6= j, but also that wq1 [i] is independent of wq2 [i] ∀q1 6= q2.
Referring to figure 4.13, which shows the different components of the total noise at
the HBF output w[n], what we are saying is that we assume both “horizontal” and
“vertical” independence. Such assumption is realistic because in practice the noise
present at the output of an analog filter will be statistically independent of the noise
present at output of a different analog filter.
So instead of reconstructing x[n], we’ll reconstruct a noisy version, xw[n],
xw[n] =
Q−1∑q=0
+∞∑i=−∞
{oq[i] + wq[i]}uq[n−Mqi] =
Q−1∑q=0
+∞∑i=−∞
oq[i]uq[n−Mqi] +
Q−1∑q=0
+∞∑i=−∞
wq[i]uq[n−Mqi] =
x[n] +
Q−1∑q=0
+∞∑i=−∞
wq[i]uq[n−Mqi] =
![Page 98: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/98.jpg)
76 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.13: Noise Propagation in a Discrete Sampling HBF
x[n] +
Q−1∑q=0
fq[n] = x[n] + w[n]
Where w[n] =∑Q−1
q=0 fq[n] is a discrete, zero mean Gaussian random process since ∀n,
w[n] is the sum of zero mean Gaussian random variables as shown in figure 4.13. To
compute σ2w[n], let’s consider fq[n] =
∑+∞i=−∞wq[i]uq[n−Mqi]. We see that fq[n] is the
result of filtering ↑ wq[n]- upsampling factor equal to Mq- with the interpolation filter
associated with the shift invariant subspace q. From the polyphase decomposition of
interpolating atoms we know that fq[n] is in fact the sum of Mq sequences, each of
which is obtained by filtering ↑ wq[n] with filter uqk with k = 0, 1, ..,Mq − 1, which
means that fq[n1] is distributed identically as fq[n2] only if n2 = n1 + jMq with
j ∈ Z. From the properties of filtered Gaussian noise, we conclude that the marginal
distribution of fq[n] is N (0, ||uqk||2σ2q ) where k ≡ n mod Mq, thus σ2
fq [n] = ||uqk||2σ2q ,
with k ≡ n mod Mq. We note that σ2fq [n] is Mq periodic.
Let’s now consider the vertical sum. By assumption, for any n1 and any n2, fq1 [n1]
![Page 99: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/99.jpg)
4.5. NOISE ANALYSIS 77
is independent of fq2 [n2] as long as q1 6= q2 even if n1 = n2. Since w[n] =∑Q−1
q=0 fq[n],
σ2w[n] =
∑Q−1q=0 σ
2fq [n]. Given that σ2
fq [n] is Mq periodic, σ2w[n] is Mlcm periodic, where
Mlcm = lcm(M0,M1, ...,MQ−1). Another way to reach this conclusion is to realize
that w[n] is made of Mlcm different processes, wj[n] for 0 ≤ j < Mlcm, that have the
following properties,
• wj[n] = w[n+ j] with σ2wj [n] = σ2
w[j] if n = lMlcm with l ∈ Z
• wj[n] = 0 if n 6= lMlcm with l ∈ Z
• w[n] =∑Mlcm−1
j=0 wj[n− j]
This polyphase decomposition of w[n] in Mlcm sequences makes it easy to see that
σ2w[n] =
∑Q−1q=0 ||uqk||2σ2
q with k ≡ n mod Mq and therefore σ2w[n] is Mlcm periodic.
If ||uqk||2 ≤ 1 ∀k and the uq[n] are aligned such that uq[0] = 1 ∀q, then for a fixed n,∑Q−1q=0 fq[n] is a Gaussian random variable whose variance is bounded by
∑Q−1q=0 σ
2q . If
we still have ||uqk[n]||2 ≤ 1 ∀k but the uq[n] are not aligned, we might be able to get a
better upper bound. In any case, this analysis allows us to bound the variance, thus
the energy, of the noise that corrupts the reconstructed signal. It also gives us an
interesting property to look for in our interpolating atoms, ||uqk[n]||2 ≤ 1 ∀k, which
in turn would imply supq,n uq[n] = 1 given that uqk[n] is δ[n] for some k -most likely
for k = 0.
With our approach we can then estimate the energy of the overall error introduced in
in xw[n]. Consider a signal x[n] of length N such that N is large and N divides Mlcm.
If Mlcm << N this assumption is reasonable. Since the model implicitly implies that
all sources of noise are ergodic, we can estimate the total squared error introduced
by one of the Mlcm sequences that compose w[n], wj[n] as
length(wj[nMlcm])σ2wj [0] =
length(x[n])
Mlcm
σ2wj [0] =
N
Mlcm
σ2w[j]
=N
Mlcm
Q−1∑q=0
||uqk||2σ2q
with k ≡ j mod Mq. By taking j from 0 to Mlcm − 1, the total squared error
introduced in xw[n] can be estimated as
![Page 100: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/100.jpg)
78 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Mlcm−1∑j=0
N
Mlcm
σ2wj [0] =
N
Mlcm
Mlcm−1∑j=0
Q−1∑q=0
||uqk||2σ2q
where k ≡ j mod Mq. Dividing the above expression byN we obtain as the estimated
mean squared error, MSE
MSE =1
Mlcm
Mlcm−1∑n=0
Q−1∑q=0
||uqk||2σ2q (4.5)
where k ≡ j mod Mq. We see that the expression for the MSE estimate is not de-
pendent on N , the length of the signal. The actual MSE is computed as ||x[n]−xw[n]||2N
.
Let’s apply this formula to the example presented in subsection 4.4.1. In this case
N = 1456128, Q = 3, M0 = 2, M1 = 4 and M2 = 4; thus Mlcm = 4. Table 4.2
shows the different ||uqk||2 , where k ≡ j mod Mq, for 0 ≤ j < Mlcm = 4. We
see in the table that we are in a situation where the uq[n] are not aligned. For the
simulation we set σ20 = 10−6, σ2
1 = 2.10−6, σ22 = 3.10−6. Equation (4.5) gives us an
MSE = 4.578.10−6. The measured MSE, obtained as an average of 10 different
runs, was 4.5766.10−6, which is in agreement with the predicted value.
j = 0 j = 1 j = 2 j = 3q = 0 ||u00||2 = 0.7126 ||u01||2 = 1.0000 ||u00||2 = 0.7126 ||u01||2 = 1.0000q = 1 ||u10||2 = 0.7191 ||u11||2 = 0.2760 ||u12||2 = 1.0000 ||u13||2 = 0.3903q = 2 ||u20||2 = 1.0000 ||u21||2 = 0.8981 ||u22||2 = 0.6972 ||u23||2 = 0.7767
Table 4.2: ||uqk||2 , Example of Subsection 4.4.1
4.6 Practical Imprementation of Continuous Time
Filters
The most critical components in an HBF, from a circuit implementation point of view,
are its analog filters. The HBFs designed with Discrete Sampling are no exception.
Each analog filter has to implement a linear time invariant system whose impulse
response is some function u(−t) =∑+∞
n=−∞ u[n]sinc(t−nTsTs
). By construction, this
![Page 101: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/101.jpg)
4.6. PRACTICAL IMPREMENTATION OF CONTINUOUS TIME FILTERS 79
function is non causal, therefore it is impossible to implement it exactly with real
circuits since we only know how to implement causal filters. The usual workaround
when a problem like this appears in engineering is to try to implement a filter whose
impulse response is a delayed and windowed version of the desired impulse response.
This is the approach that we will use here. Once we have overcome this problem we
also need to face the fact that the only causal filters that we know how to implement
are those whose impulse response has a rational Laplace Transform in the form [41],
H(s) =P (s)
Q(s)=b0s
p + b1sp−1 + · · ·+ bp
a0sq + a1sq−1 + · · ·+ aq(4.6)
With the extra requirements that
• q ≥ p, which ensures causality,
• the poles of H(s), ie, the zeros of Q(s), lie on the left hand side of complex
plane, which ensures stability;
once the H(s) function to implement has been settled, circuit designers put it in a cir-
cuit. The thing to keep in mind is that building an analog filter involves two different
stages which are generally decoupled, and therefore, can be addressed independently,
1. Computing the H(s) that implements as closely as possible the filter require-
ments, which are usually expressed in the frequency domain through the transfer
function or in time domain through the impulse response. This is a problem
mathematical in nature.
2. Designing the circuits that implement the approximation to H(s) computed
in the previous item. This is a problem of circuit design. Over the years a
great deal of circuit architectures have been developed to solve this problem;
the actual architecture used will depend on many factors such as the band of
operation, power specifications, tolerance requirements, volume of units, active
vs passive, etc. These requirements will also determine the technology in which
the filter is implemented: printed circuit, integrated circuit, waveguide, etc.
Everything else being equal, the most limiting factor that prevents a given
![Page 102: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/102.jpg)
80 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
H(s) from being implemented accurately is the quality factor Q associated to
each pole-zero of Q(s). For a given point in the complex plane, Q is defined
as the ratio between the absolute value of its imaginary part and the absolute
value of its real part. With the current state of technology, the rule of thumb
is that each Q should be at most 10.
Readers interested in knowing more about the general practice of analog filter design,
which includes how these two problems are solved in most practical situations, are
referred to any good reference on the topic such as [42]. In the rest of the section, I
will focus on how we can accomplish the first task for the analog filters that show up
in an HBF designed with Discrete Sampling, keeping in mind the rule about quality
factors being lower than 10. Once we have the transfer functions that approximate
the analog filters, they can be given to an analog filter engineer for its implementation.
4.6.1 Computing H(s) for Impulse Response u(−t)
Figure 4.14: Idealized Filter Frequency Responses [43]
In classic analog filter design, the specifications for H(s) are given in terms of the
desired frequency response. The ideal frequency responses of the common types of
analog filters used in circuits, shown in figure 4.14, are brick wall and therefore cannot
![Page 103: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/103.jpg)
4.6. PRACTICAL IMPREMENTATION OF CONTINUOUS TIME FILTERS 81
Figure 4.15: Analog Filter Design Parameters [43]
be implemented. During the first half of the XX-th century [44] the problem of how
to approximate these responses with functions that fit the model of equation (4.6) not
only was studied throughly, but, by 1950 several approximations had been developed
analytically: Butterworth, Chebyshev, Bessel and Cauer-also known as elliptical.
These approximations, used to this day together with a mechanical design process
usually implemented in a computer program, allow the circuit designer to obtain the
transfer function H(s) of practically any low pass, band pass, high pass and band
reject filter encountered in practice. The process begins by specifying the magnitude
of the desired frequency response in terms of several filter parameters, such as Amin,
Amax, fc, fstop, shown in figure 4.15 for the case of a low pass filter design. Then, by
following a step by step algorithm, the coefficients for the numerator and numerator
of H(s), as well as the minimum value required for q- called the filter order- to meet
the specifications, are computed. In fact, the design of all these filters, regardless of
their frequency response, is started by providing the filter parameters for a prototype
low pass filter with fc normalized to 1, out of which all other filters are synthesized for
![Page 104: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/104.jpg)
82 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
any response type and arbitrary values of fc and fstop. With this framework, it is not
possible to specify phase requirements, rather, each type of approximation provides
a particular type of phase response. In order to compute the H(s) associated with a
given u(−t) this method is insufficient for two reasons: first, as shown in figure 4.16
for uφ12(−t) from example of subsection 4.4.1, the magnitude spectrum associated
with the Laplace Transform of a typical u(−t), L{uφ12(−t)}(jω), doesn’t fit any of
the standard filter prototypes; second, in our case, we need to be able to specify phase
requirements since we are using the filters with frequency response u(−t) to compute
inner products.
Figure 4.16: Magnitude of L{uφ12(−t)}(jω), Example Subsection 4.4.1
Luckily, there are numerical methods that allow us to find a transfer function in the
form of equation (4.6) that fits a given impulse response, and therefore its frequency
response [45]. These methods can be roughly classified according to whether they try
to approximate h(t), the impulse response of H(s), to the desired impulse response,
in our case a truncated version of u(−t), or whether they try to approximate H(jω),
the frequency response of H(s), to the desired frequency response, in our case U(−f).
These methods are described in great detail in Chapter 5 of [45]. After experimenting
![Page 105: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/105.jpg)
4.6. PRACTICAL IMPREMENTATION OF CONTINUOUS TIME FILTERS 83
with several approaches, which fall in both categories, the one that produced the
best approximation results in our case was the least squares approximation to the
desired frequency response. This method has the inconvenience that, because of the
geometry of the objective function, there are many local minima, so one has to try
several starting points to get a decent result. This proliferation of local minima also
translates in diminishing returns for the accuracy of the approximation with respect
to increases of q.
Least Squares Approximation of U(−f)
Since u(−t) ∈ PWTs, we can compute its Fourier Transform, U(−f), by computing
the Discrete Time Fourier Transform of u[−n] = u(−nTs) , U(−fn), which is a
function of the continuous normalized frequency fn = ffs
, and noticing that U(− ffs
) =
U(−fn) for −12≤ f
fs≤ 1
2and zero otherwise. To simplify things, we will work with
the assumption Ts = 1, which results in u[−n] = uTs=1(−n). Once we have found the
HTs=1(s) whose associated frequency response, HTs=1(jω), approximates UTs=1(−f),
we will scale it, using the scaling property of the Laplace transform, to find the H(s)
whose frequency response, H(jω), approximates U(−f). To perform the least squares
approximation, we use Matlab’s function invfreqs [46]. This function takes as its
most important inputs,
• vector W of length N which contains the normalized angular frequencies in
which the desired frequency response, D(jω) is evaluated. In our caseD(j2πfn) =
U(−fn).
• vector H of length N such that H(k) = D(jW (k)) for 1 ≤ i ≤ N
• the desired degree of the numerator n = p
• the desired degree of the numerator m = q .
Other inputs not detailed above include options about the desired accuracy and num-
ber of iterations. invfreqs then solves the problem
![Page 106: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/106.jpg)
84 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
minb,a
N∑k=1
∣∣∣∣H(k)− P (jW (k))
Q(jW (k))
∣∣∣∣2 (4.7)
Where b is the p + 1 vector such that bi−1 = b(i) for 1 ≤ i ≤ p + 1, a is the q + 1
vector such that ai−1 = a(i) for 1 ≤ i ≤ q + 1 and P (jω), Q(jω) are the frequency
responses associated with P (s) and Q(s) respectively. This problem is solved via the
Gauss-Newton iterative method [47] since the objective function is non linear. The
initial value for the iteration is obtained as the solution of the least squares problem,
minb,a
N∑k=1
|H(k)P (jW (k))−Q(jW (k))|2 (4.8)
which can be solved using linear least squares methods. As an additional feature
invfreqs only produces solutions whose poles lie on the left hand side of the complex
plane, ie, stable solutions.
4.6.2 Design Example
To illustrate the concepts developed in this section, we will present an HBF in which
the analog filters are designed to have impulse responses whose Laplace Transforms
can be expressed in the form of equation (4.6). This example is different from the
example presented in subsection 4.4.1.
The wavelet decomposition used to decompose `2(Z) is Beylkin with a one level
decomposition tree; thus, we have a detail subspace, spanned by ψ[n], and a coarse
approximation subspace, spanned by φ[n]. Their respective atoms uψ[n]-uψ[n] and
uφ[n]-uφ[n] with M = 2 are shown in figures 4.17 and 4.18.
The first thing to decide is how to truncate uψ[−n] and uφ[−n] to facilitate the task of
approximating uψ(−t) and uφ(−n) with filters whose Laplace transform is in the form
of equation (4.6). Remember that at this point we are working with Ts = 1, so the
question can be equivalently formulated for the discrete versions. There is no simple
answer. It’s a matter of trying several different truncated versions to find which one
gives a better approximation. In this example, we decided to make uψ[n] and uφ[n]
![Page 107: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/107.jpg)
4.6. PRACTICAL IMPREMENTATION OF CONTINUOUS TIME FILTERS 85
Figure 4.17: uψ[n] and uψ[n] Associated with ψ[n], Beylkin Wavelet Decomposition2 Subspaces
Figure 4.18: uφ[n] and uφ[n] Associated with φ[n], Beylkin Wavelet Decomposition 2Subspaces
![Page 108: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/108.jpg)
86 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
Figure 4.19: Tuncated uψ[−n] and uφ[−n] for Beylkin Wavelet Decomposition 2 Sub-spaces
Figure 4.20: DuTψ(j2πfn) and DuTφ(j2πfn) for Beylkin Wavelet Decomposition 2Subspaces
![Page 109: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/109.jpg)
4.6. PRACTICAL IMPREMENTATION OF CONTINUOUS TIME FILTERS 87
0.5254s16 − 2.809s15 + 22.55s14 − 79.61s13
+332.5s12 − 815.5s11 + 2205s10 − 3777s9
+7015s8 − 8194s7 + 1.06e004s6 − 8128s5
+7386s4 − 3287s3 + 2040s2 − 334.4s+ 147.9
s18 + 4.558s17 + 53.05s16 + 186.2s15
+1122s14 + 3088s13 + 1.244e004s12 + 2.691e004s11
+7.923e004s10 + 1.336e005s9 + 2.986e005s8 + 3.828e005s7
+6.585e005s6 + 6.125e005s5 + 8.105e005s4 + 4.962e005s3
+4.991e005s2 + 1.55e005s+ 1.156e005
Table 4.3: H1(s)
−0.01146s14 − 0.0369s13 − 0.05885s12 − 1.192s11
+3.053s10 − 15.56s9 + 39.32s8 − 95.45s7
+174s6 − 263.1s5 + 303.3s4 − 263.5s3 + 159.9s2
−61.1s+ 11.04
s16 + 5.231s15 + 29.02s14 + 95.85s13
+289.8s12 + 666.7s11 + 1351s10 + 2236s9
+3195s8 + 3789s7 + 3790s6 + 3096s5
+2043s4 + 1040s3 + 387.3s2 + 93.51s+ 11.04
Table 4.4: H2(s)
![Page 110: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/110.jpg)
88 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
equal to zero for n ≥ 26. Let’s call uTψ[n] and uTφ[n] the truncated versions of uψ[−n]
and uφ[−n] respectively; they are shown in figure 4.19. Once we have these, we need to
compute their frequency response. For that, we compute their Discrete Time Fourier
Transforms, DuTψ(j2πfn) and DuTφ(j2πfn), whose magnitudes are figure 4.20.
These two spectra, both phase and magnitude, are then sampled at 4096 regularly
spaced positions - positive normalized frequencies only. At this point we have W
and H(k) that we can input to invfreqs. As explained earlier, the method of least
squares approximation to a desired frequency response requires to perform several
algorithm executions, each with a different starting point, to find the most accurate
approximation because the objective function has several local minima. In our case
we can do that by trying different values of p and q. In our simulation we obtained
the transfer functions H1(s) and H2(s), that approximate DuTψ(s) and DuTφ(s) re-
spectively, shown in tables 4.3 and 4.4. As we see, p = 16 and q = 18 for H1(s) while
p = 14 and q = 16 for H2(s).
The poles of H1(s) are: −0.1863 + 3.2226j, −0.1863 − 3.2226j, −0.2996 + 2.9563j,
−0.2996−2.9563j, −0.3436+2.6662j, −0.3436−2.6662j, −0.3547+2.3657j, −0.3547−2.3657j, −0.3454+2.0679j, −0.3454−2.0679j, −0.2899+1.7964j, −0.2899−1.7964j,
−0.2083+1.3577j, −0.2083−1.3577j, −0.1550+1.1480j, −0.1550−1.1480j, −0.0962+
0.9031j, −0.0962 − 0.9031j, all of which lie on the left hand side of the complex
plane. We verify that the poles of H2(s) also have this property: −0.1841 + 2.2861j,
−0.1841−2.2861j, −0.2783+2.0259j, −0.2783−2.0259j, −0.2748+1.6794j, −0.2748−1.6794j, −0.2818+1.4638j, −0.2818−1.4638j, −0.3252+1.0224j, −0.3252−1.0224j,
−0.4505+0.7969j, −0.4505−0.7969j, −0.4058+0.4935j, −0.4058−0.4935j, −0.4153+
0.1603j, −0.4153 − 0.1603j. From these poles we compute that the quality factors
required for the implementation of H1(s) are 17.3020, 9.8690,7.7586, 6.6701, 5.9866,
6.1962, 6.5180, 7.4067 and 9.3847. Note that there a total of nine different quality
factors since the eighteen poles come in conjugate pairs. For H1(s) all quality factors
are below 10 except for the quality factor associated with −0.1863 + 3.2226j and
−0.1863− 3.2226j. This means that we might encounter some problems if we decide
to implement this H1(s). The quality factors for H2(s) are 12.4206, 7.2803, 6.1116,
5.1945, 3.1443, 1.7689, 1.2160 and 0.3860. Based on the quality factor alone, H2(s)
![Page 111: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/111.jpg)
4.6. PRACTICAL IMPREMENTATION OF CONTINUOUS TIME FILTERS 89
should be implementable. We will now test, working with normalized Ts = 1, that
the signal obtained from Pageant, introduced in subsection 4.4.1, can be processed
with this HBF.
Figure 4.21: Block Diagram of HBF for Beylkin Wavelet Decomposition 2 Subspaces
Figure 4.22: Discrete Sinc Based Interpolating Filter Used to Resample x(t) at Rate0.25Hz
As shown in figure 4.21, the main difference with the simulations performed in sub-
section 4.4.1 is that instead of simulating analog filters with ideal impulse responses,
we will simulate the response of H1(s) and H2(s) to the continuous time input signal
x(t). To that aim, we will use Matlab’s continuous time linear simulator lsim, which
takes as inputs,
• the system to be simulated. We just need to put in system format filters H1(s)
and H1(s). Since we have their coefficients, Matlab’s tf function does the job,
![Page 112: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/112.jpg)
90 CHAPTER 4. DISCRETE SAMPLING AND HBF’S
• the regularly sampled signal to be simulated. Note that to perform a continuous
time simulation, the input signal needs to be sampled at a frequency several
times higher than fs = 1. A sampling frequency of fs4
= 14Hz is sufficient
in our case. Since x[n] are samples of a function of bandwidth 12Hz, we can
resample x(t) at a rate of 14Hz through the continuous time sinc interpolation
equation, (1.1). We can equivalently filter x[n] = x(nTs) = x(n) with the
discrete filter r[n] such that r[n] = sinc(n4). Since r[n] has infinite duration,
a good approximation can be obtained by using a filter of 161 coefficients, ie,
rt[n] = sinc(n4) for −80 ≤ n ≤ 80, shown in figure 4.22. Thus, we input
x 14[n] = x(n
4) = {↑ x[n]} ∗ rt[n], where the upsampling of x[n] is by a factor
equal to 4,
• the times for which we want the simulation performed. In our case, the times
are 0 + 14i with 0 ≤ i ≤ N − 1, where N is the length of x[n].
The output signal has four times more samples than the length of x[n], thus the sig-
nals provided by lsim have to be downsampled by a factor of 8: factor 4, required
to bring the output to the original sampling rate, multiplied by factor 2, since in our
case M = 2.
Inputting x(t), the signal obtained from Pageant introduced in subsection 4.4.1, we
obtain a PSNR of 56.4dB for x[n] = x(n). If we throw away the information con-
tained in the detail space, thus we keep only the output of H2(s) which is half the
information contained in x[n], we still get a PSNR of 50.3dB for x2[n]. Note that
in terms of quality factors, H2(s) had a better chance to be practical than H1(s), so
depending on the application reconstructing x2[n] might be good enough.
![Page 113: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/113.jpg)
Chapter 5
Conclusions and Future Research
This is a short chapter in which I will discuss my conclusions of three years working
on this research as well as suggestions on how future research works can pick up where
this leaves off.
5.1 Conclusions
I started to work on this research motivated about how the theory of Discrete Sam-
pling, that had been developed by my adviser Brad Osgood and fellow graduate
student William Wu for finite length discrete signals, could be put to practice to
process very long signals, music in particular. Coming from a 2 year background re-
searching on the design of data converters, ADCs and DACs, I wanted to apply it to
sampling generic signals. Music offered the challenge to see whether the theory could
be applied to signals sampled at CD quality rate, 44.1Khz, thus of relatively low
bandwidth up to 22Khz. At that point, I was aware of other techniques that tried
to sample signals at rates below Nyquist, most notably Compressed Sensing. These
techniques however, had important challenges related mostly to practical implemen-
tation. In particular, ADCs based on Compressed Sensing, although they take the
signal samples at rates below the Nyquist rate, require that the analog signal, prior
to taking signal samples, be mixed with, or multiplied by, a digital signal whose rate
is that of Nyquist or higher and whose jitter has great impact in the quality of the
91
![Page 114: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/114.jpg)
92 CHAPTER 5. CONCLUSIONS AND FUTURE RESEARCH
reconstructed signal. Thus, although they effectively sample signals at a lower rate,
the jitter requirements placed on this digital signal made it more difficult to design
general purpose ADCs with Compressed Sensing than with existing techniques. To
me, the fact that Discrete Sampling requires sampling signals at rates strictly below
Nyquist’s was a big motivator. I wanted that any conversion technique that could
come out of Discrete Sampling had in mind the technical limitations of clock jitter and
other effects, such as thermal noise and others described in Chapter 1, that limited
practical data converters.
The first obstacle that I encountered was that the dimension of a normal audio signal
is much higher than the dimension of the matrices that can be processed realistically.
This didn’t prevent me from trying to apply the theory as it was developed at the time
I started. I made many simulations of equation (1.8) with matrices of dimension as
big as 32,768 derived from Wavelet Theory. I wanted to compute several interpolating
matrices that I could use to process long signals. These simulations, run in Matlab,
usually took a whole night in multi CPU, 32 Gb RAM Linux machines. Originally I
thought that I could break the signal into pieces that could be processed independently
by somehow choosing on the fly which interpolating matrix to use for a given segment.
It turned out that this wasn’t a good idea, for the reasons detailed in the introduction
of Chapter 3. One day, while examining the structure of the interpolating matrices
obtained with this method, I realized that their columns where shift invariant, just
as the wavelet matrices were. That’s how I got into thinking about the problem
described in section 3.2. Since it became obvious that interpolating from the signal
samples alone didn’t produce reconstructed signals of good fidelity, I began to study
the problem of orthogonal projections and how that could be done in long signals with
the Discrete Sampling framework. This took me into a deeper study of frames, that I
knew from a class in wavelets offered by the Mathematics Department, Math266, that
produced the results described in section 3.3. A very brief introduction of frame theory
was included in Chapter 2 for completeness purposes. Finally, while investigating how
these results could be put into a working circuit, I found out about HFB’s and realized
that the theory I had developed could be used to obtain a novel way, and probably
more practical than existing methods, of designing them. That’s what Chapter 4 is
![Page 115: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/115.jpg)
5.2. FUTURE RESEARCH 93
about.
My concluding remark is that this has been a fascinating journey in which I have
been able to apply knowledge from different areas of Electrical Engineering and
Mathematics- from ADC design to frames and wavelets- that I have acquired through
all my Stanford years.
5.2 Future Research
One of the many good things about doing interesting research, is that once one
masters the topic at hand, all sorts of new problems pop up in one’s mind that could
extend the research work for many more years. At some point one has to present and
defend one’s work in front of a jury of professors to graduate with a PhD so there are
usually many ways in which the research developed could be extended. In my case, I
see the following three areas that could be explorer further, although I am sure, since
sampling is such an interesting field, that other researchers could find more:
1. Finding decompositions of `2(Z) more adapted to a given particular problem.
What this means is that through this work I have kind of “randomly” chosen
decompositions derived from Wavelet Theory with the only criterion that the
orthonormal atoms that spanned the shift invariant subspaces satisfy the con-
ditions of theorem 3.4. It will be interesting to investigate whether there exists
a more mechanical method of choosing the decomposition so that the signals
of interest for a particular problem require fewer subspaces than those present
in the decomposition. In other words, figuring out a way to design decompo-
sitions for which some subspaces contain negligible information for the signals
of interest. This way, not only the design of the resulting HBF is simplified,
but we can decrease the amount of information sampled to below the Nyquist
rate. In addition, as it happened in the example presented in subsection 4.4.1,
it’s likely that the remaining subspaces could be sampled at rates significantly
below Nyquist. In that example it might be sufficient to recover only signal
xφ12ψ12 [n] which requires half the information required by the Nyquist rate and
where each sampler works at one fourth of the Nyquist rate. The idea would
![Page 116: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/116.jpg)
94 CHAPTER 5. CONCLUSIONS AND FUTURE RESEARCH
be to find a way to do this systematically and not just as a result of trial and
error experimentation.
2. Finding wavelet decompositions that result in simpler practical designs. It is
related to point 1., but different. Using the vocabulary of Compressed Sensing,
point 1. is about finding decompositions of `2(Z) which result in a sparse repre-
sentation in terms of the number of subspaces required to represent the signals
of interest. Here we are concerned with the practicality of the decompositions.
As shown in subsection 4.6.2, finding actual analog filters that approximate the
ones present in an HBF designed with Discrete Sampling is challenging. Maybe
there is a way to impose constraints on the wavelet decomposition such as the
order of the filter required to approximate the frequency response of the reversed
dual, u(−t), or the quality factor of such filter. It would be also interesting to
know if this problem and the problem described in 1. can be solved at the same
time or there is any kind of trade off relating the two.
3. Circuit Implementation. Finally, exploring whether the filters that result from
designing an HBF with Discrete Sampling can be easily implemented with ex-
isting circuit technology and methods. A positive answer to this proposition
could make these HBFs mainstream in the world of ADCs. A good starting
point would be to try the implementation of H2(s) obtained in subsection 4.6.2,
since its order is reasonable, the quality factors of its poles are near or well below
the rule of thumb of 10, and, at least for the audio signal under consideration,
by keeping only its output we can obtain a reasonable reconstruction. It could
be implemented first with discrete components, then, if there is success with
this stage, in an integrated circuit.
Note that I do not mention particular applications amongst these topics because the
spirit of my research has been to find a general framework for designing ADCs that
could be applied widely.
![Page 117: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/117.jpg)
Bibliography
[1] B. Murmann, “EE315B VLSI Data Conversion Circuits Lecture Notes,” Stanford
University, Autumn 2010.
[2] C. Shannon, “Communication in the Presence of Noise,” Proceedings of the IRE,
vol. 37, no. 1, pp. 10 – 21, Jan. 1949.
[3] B. Widrow and I. Kollar, Quantization Noise: Roundoff Error in Digital Com-
putation, Signal Processing, Control, and Communications. Cambridge, UK:
Cambridge University Press, 2008.
[4] W. Wu, “Discrete Sampling: Discrete Generalizations of the Nyquist-Shannon
Sampling Theorem,” Ph.D. dissertation, Stanford University, June 2010.
[5] B. Murmann, “ADC Performance Survey 1997-2011.” [Online]. Available:
http://www.stanford.edu/∼murmann/adcsurvey.html
[6] I. Djbkovic and P. Vaidyanathan, “Generalized Sampling Theorems in Multires-
olution Subspaces,” IEEE Transactions on Signal Processing, vol. 45, pp. 583
–599, March 1997.
[7] P. P. Vaidyanathan and B. Vrcelj, “Biorthogonal Partners and Applications,”
IEEE Transactions on Signal Processing, vol. 49, pp. 1013–1027, Nov. 2001.
[8] B. Vrcelj and P. P. Vaidyanathan, “Fractional Biorthogonal Partners in Channel
Equalization and Signal Interpolation,” IEEE Transactions on Signal Processing,
vol. 51, pp. 1928–1940, July 2003.
95
![Page 118: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/118.jpg)
96 BIBLIOGRAPHY
[9] P. P. Vaidyanathan, “Generalizations of the Sampling Theorem: Seven Decades
After Nyquist,” IEEE Transactions on Circuits and Systems I, vol. 48, pp. 1094–
1109, Sep. 2001.
[10] P.P. Vaidyanathan, Multirate Systems and Filter Banks. Prentice Hall, 1993.
[11] Y. C. Eldar, “Uncertainty Relations for Shift-Invariant Analog Signals,” IEEE
Transactions on Information Theory, vol. 55, pp. 5742–5757, Dec. 2009.
[12] Y. Eldar and T. Michaeli, “Beyond Bandlimited Sampling,” IEEE Signal Pro-
cessing Magazine, vol. 26, pp. 48–68, May 2009.
[13] M. Mishali and Y. C. Eldar, “From Theory to Practice: sub-Nyquist Sampling
of Sparse Wideband Analog Signals,” IEEE Journal of Selected Topics in Signal
Processing, vol. 4, pp. 375–391, April 2010.
[14] ——, “Blind Multiband Signal Reconstruction: Compressed Sensing for Analog
Signals,” IEEE Transactions on Signal Processing, vol. 57, pp. 993–1009, March
2009.
[15] E. Candes, J. Romberg, and T. Tao, “Robust Uncertainty Principles: Exact
Signal Reconstruction from Highly Incomplete Frequency Information,” IEEE
Transactions on Information Theory, vol. 52, pp. 489 – 509, Feb. 2006.
[16] D. Donoho, “Compressed Sensing,” IEEE Transactions on Information Theory,
vol. 52, pp. 1289 –1306, April 2006.
[17] J. A. Tropp, J. N. Laska, M. F. Duarte, J. K. Romberg, and R. G. Baraniuk, “Be-
yond Nyquist: Efficient Sampling of Sparse Bandlimited Signals,” IEEE Trans-
actions on Information Theory, vol. 56, pp. 520–544, Jan. 2010.
[18] M. Unser, “Sampling-50 years After Shannon,” Proceedings of the IEEE, vol. 88,
pp. 569 –587, April 2000.
[19] M. Unser and A. Aldroubi, “A General Sampling Theory for Nonideal Acquisition
Devices,” IEEE Transactions on Signal Processing, vol. 42, pp. 2915 –2925, Nov.
1994.
![Page 119: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/119.jpg)
BIBLIOGRAPHY 97
[20] Y. M. Lu and M. N. Do, “A Theory for Sampling Signals From a Union of
Subspaces,” IEEE Transactions on Signal Processing, vol. 56, pp. 2334–2345,
June 2008.
[21] T. Blumensath and M. E. Davies, “Sampling Theorems for Signals From the
Union of Finite-Dimensional Linear Subspaces,” IEEE Transactions on Infor-
mation Theory, vol. 55, pp. 1872–1882, April 2009.
[22] G. G. Walter, “A Sampling Theorem for Wavelet Subspaces,” IEEE Transactions
on Information Theory, vol. 38, pp. 881–884, March 1992.
[23] S. Haddad, S. Bagga, and W. Serdijn, “Log-domain Wavelet Bases,” IEEE Trans-
actions on Circuits and Systems I, vol. 52, pp. 2023 – 2032, Oct. 2005.
[24] Sandro A. P. Haddad, Wouter A. Serdijn, Ultra Low-Power Biomedical Signal
Processing: An Analog Wavelet Filter Approach for Pacemakers. Springer, 2009.
[25] J. E. Marsden and M. J. Hoffman, Elementary Classical Analysis, Second Edi-
tion. W. H. Freeman and Company, 1993.
[26] O. Christensen, Frames and Bases, An Introductory Course. Birkhauser, 2008.
[27] J. Higgins, Sampling Theory in Fourier and Signal Analysis Foundations. Ox-
ford University Press, 1996.
[28] S. Mallat, A Wavelet Tour of Signal Processing, Second Edition. Academic
Press, 1999.
[29] Alan V. Oppenheim, Ronald W. Schafer and John R. Buck, Discrete Time Signal
Processing Second Edition. Prentice Hall, 1999.
[30] C. Eoff, “The Discrete Nature of the Paley-Wiener Spaces,” Proceedings of The
American Mathematical Society, vol. 123, no. 2, pp. 505–512, 1995.
[31] [Online]. Available: http://www.mathworks.com/help/toolbox/signal/ref/hann.
html
![Page 120: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/120.jpg)
98 BIBLIOGRAPHY
[32] W. Chen, S. Itoh, and J. Shiki, “On Sampling in Shift Invariant Spaces,” IEEE
Transactions on Information Theory, vol. 48, pp. 2802–2810, Oct. 2002.
[33] R. M. Gray, “Toeplitz and Circulant Matrices: A review,” Foundations and
Trends in Communications and Information Theory, vol. 2, pp. 155–239, 2006.
[34] C. Lelandais-Perrault, T. Petrescu, D. Poulton, P. Duhamel, and J. Oksman,
“Wideband, Bandpass, and Versatile Hybrid Filter Bank A/D Conversion for
Software ,” IEEE Transactions on Circuits and Systems I, vol. 56, pp. 1772–
1782, Aug. 2009.
[35] A. Papoulis, “Generalized Sampling Expansion,” IEEE Transactions on Circuits
and Systems, vol. 24, pp. 652 – 654, Nov. 1977.
[36] J. Brown, J., “Multi-Channel Sampling of Low-pass Signals,” IEEE Transactions
on Circuits and Systems, vol. 28, pp. 101 – 106, Feb. 1981.
[37] S. Velazquez, T. Nguyen, and S. Broadstone, “Design of Hybrid Filter Banks for
Analog/Digital Conversion,” IEEE Transactions on Signal Processing, vol. 46,
pp. 956 –967, April 1998.
[38] Z. Song, C. Lelandais-Perrault, and P. Benabes, “Synthesis of Complex Subband
Hybrid Filter Banks A/D Converters Using Adaptive Filters,” in 16th IEEE
International Conference on Electronics, Circuits, and Systems, 2009. ICECS
2009, Dec. 2009, pp. 399 –402.
[39] D. Albuquerque, S. Soldado, J. Vieira, and N. Carvalho, “Cochlear Radio,” in
2010 European Wireless Technology Conference (EuWIT), Sep. 2010, pp. 209
–212.
[40] [Online]. Available: http://www.cirquedusoleil.com/en/shows/ka/default.aspx
[41] B. Murmann, “EE315A VLSI Signal Conditioning Circuits Lecture Notes,” Stan-
ford University, Spring 2011.
![Page 121: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/121.jpg)
BIBLIOGRAPHY 99
[42] R. Schaumann, H. Xiao, and V. Valkenburg, Design of Analog Filters, Second
Edition. Oxford University Press, 2009.
[43] [Online]. Available: http://www.analog.com/library/analogDialogue/archives/
43-09/EDCh%208%20filter.pdf
[44] S. Darlington, “A History of Network Synthesis and Filter Theory for Circuits
Composed of Resistors, Inductors, and Capacitors,” IEEE Transactions onCir-
cuits and Systems, vol. CAS-11, pp. 1–13, Jan. 1984.
[45] A. M. Cohen, Numerical Methods for Laplace Transform Inversion. Springer,
2007.
[46] [Online]. Available: http://www.mathworks.com/help/toolbox/signal/ref/
invfreqs.html
[47] [Online]. Available: http://en.wikipedia.org/wiki/Gauss%E2%80%93Newton
algorithm
![Page 122: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/122.jpg)
100 BIBLIOGRAPHY
![Page 123: DISCRETE SAMPLING IN CONTINUOUS TIME, A DISSERTATIONch771fw4323/... · 2013. 6. 17. · John Pauly, Co-Adviser I certify that I have read this dissertation and that, in my opinion,](https://reader033.vdocuments.us/reader033/viewer/2022060911/60a56c8b2e41072c1a694d77/html5/thumbnails/123.jpg)
Fernando Gomez-Pancorbo
I certify that I have read this dissertation and that, in my opinion, it
is fully adequate in scope and quality as a dissertation for the degree
of Doctor of Philosophy.
(Brad Osgood) Principal Adviser
I certify that I have read this dissertation and that, in my opinion, it
is fully adequate in scope and quality as a dissertation for the degree
of Doctor of Philosophy.
(John Pauly)
I certify that I have read this dissertation and that, in my opinion, it
is fully adequate in scope and quality as a dissertation for the degree
of Doctor of Philosophy.
(Julius Smith)
Approved for the University Committee on Graduate Studies