Frame Permutation QuantizationHa Q. Nguyen, Vivek K Goyal, and Lav R. Varshney

Massachusetts Institute of Technology

Abstract-Frame permutation quantization (FPQ) is a newvector quantization technique using finite frames. In FPQ, avector is encoded using a permutation source code to quantize itsframe expansion. This means that the encoding is a partial order­ing of the frame expansion coefficients. Compared to ordinarypermutation source coding, FPQ produces a greater number ofpossible quantization rates and a higher maximum rate. Variousrepresentations for the partitions induced by FPQ are presentedand reconstruction algorithms based on linear programming andquadratic programming are derived. Reconstruction using thecanonical dual frame is also studied, and several results relateproperties of the analysis frame to whether linear reconstructiontechniques provide consistent reconstructions. Simulations forGaussian sources show performance improvements over entropy­constrained scalar quantization for certain combinations ofvector dimension and coding rate.

I. INTRODUCTION

Redundant representations obtained with frames are playingan ever-expanding role in signal processing due to designflexibility and other desirable properties. One such favorableproperty is robustness to additive noise. This robustness,carried over to quantization noise, explains the success ofboth ordinary oversampled analog-to-digital conversion (ADC)and ~-~ ADC with the canonical linear reconstruction. Butthe combination of frame expansions with scalar quantizationis considerably more interesting and intricate because bound­edness of quantization noise can be exploited in reconstruc­tion [1]-[8] and frames and quantizers can be designed jointlyto obtain favorable performance.

This paper introduces a new use of finite frames in vectorquantization: frame permutation quantization (FPQ). In FPQ,permutation source coding (PSC) [9], [10] is applied to a frameexpansion ofa vector. This means that the vector is representedby a partial ordering of the frame coefficients (Variant I) orby signs of the frame coefficients that are larger than somethreshold along with a partial ordering of the absolute values ofthe significant coefficients (Variant II). FPQ provides a spacepartitioning that can be combined with additional constraintsor prior knowledge to generate a variety of vector quantizers.A simulation-based investigation shows that FPQ outperformsPSC for certain combinations of signal dimensions and codingrates. In particular, improving upon PSC at low rates providesquantizers that perform better than entropy-coded scalar quan­tization (ECSQ) in certain cases [11].

Beyond the exposition of the basic ideas in FPQ, the focusof this paper is on how-in analogy to works cited above­there are several decoding procedures that can sensibly be

This material is based upon work supported by the National ScienceFoundation under Grant No. 0729069.

978-1-4244-7417-2/101$26.00 ©2010 IEEE

used with the encoding of FPQ. One is to use the ordinarydecoding in PSC for the frame coefficients followed by linearsynthesis with the canonical dual; from the perspective offrame theory, this is the natural way to reconstruct. Takinga geometric approach based on consistency yields insteadoptimization-based algorithms. We develop both views andfind conditions on the frame used in FPQ that relate to whetherthe canonical reconstruction is consistent. Along with inspiringnew questions about finite frames and being of interest forcompression when a signal has already been acquired, FPQmay have impact on sensor design. Sensors that operate atlow power and high speed by outputting orderings of signallevels rather than absolute levels have been demonstrated andare a subject of renewed interest [12], [13].

This paper is a condensed version of [14]. Among theexcisions are several references, proofs, developments for auniform source, and all further mentions of Variant II permu­tation codes and variable-rate coding. Preliminary results onFPQ were mentioned briefly in [15].

II. BACKGROUND

We assume fixed-rate coding and the conventional squared­error fidelity criterion Ilx - xl1 2 between source x and repro­duction i: Some statements assume a known source distribu­tion over which performance is measured in expectation.

A. Vector Quantization

A vector quantizer is a mapping from an input x E jRN

to a codeword x from a finite codebook C. Without loss ofgenerality, a vector quantizer can be seen as a compositionof an encoder a : jRN ----t I and a decoder f3 : I ----t jRN,

where I is a finite index set. The encoder partitions jRN intoIII regions or cells {a- 1(i )}iEI ' and the decoder assigns areproduction value to each cell. Having R bits per componentmeans III = 2N R.

For any codebook, the encoder a that minimizes Ilx - xl1 2

maps x to the nearest element of the codebook. The partitionis thus composed of convex cells. Since the cells are convex,reproduction values are optimally within the correspondingcells-whether to minimize mean-squared error distortion,maximum squared error, or some other reasonable function ofsquared error. Reproduction values being within correspondingcells is formalized as consistency:

Definition 2.1: The reconstruction x = f3(a(x)) is calleda consistent reconstruction of x when a(x) = a(x) (orequivalently f3(a(x)) = x). The decoder f3 is called consistentwhen f3(a(x)) is a consistent reconstruction of x for all x.

B. Permutation Source Codes

A permutation source code is a vector quantizer inwhich codewords are related through permutations. Permu­tation codes were originally introduced as channel codes bySlepian [16]. They were then applied to a specific sourcecoding problem, through the duality between source encodingand channel decoding, by Dunn [9] and developed in greatergenerality by Berger et ale [10], [17], [18].

Let J.1;1 > J.1;2 > ... > J.1;K be real numbers, and letnl, n2, ... , n« be positive integers that sum to N (an (or­dered) integer partition of N). The initial codeword of thecodebook C has the form

Codebook Optimization: Assume that x is random and thatthe components of x are i.i.d. Let el 2:: e2 2:: ... 2:: eN denotethe order statistics of random vector x = (Xl, ... , XN) andTIl 2:: Tl2 2:: ... 2:: TIN denote the order statistics of randomvector [z] ~ (IXII, ... , IXNI).l For a given initial codewordXinit, the per-letter distortion of an optimally-encoded PSC isgiven by

D = N- 1E [L~l L£EIi (e£ - JLi)2] (4)

where Iis are the sets of indexes generated by the integerpartition:

1For consistency with earlier literature on PSCs, we are reversing the usualsorting of order statistics [19].

The matrix F E jRMxN with kth row equal to <P'k is called theanalysis frame operator. Equivalent to (7a) in matrix form is

The analysis of [17] shows that when N is large, the optimalpartition gives performance equal to ECSQ of x. Performancedoes not strictly improve with increasing N; permutationcodes outperform ECSQ for certain combinations ofblock sizeand rate [11].

(6)

(7b)

(5a)

etc. (5b)

{I, 2, ... ,nl}'

{nl + l,nl + 2, ... ,nl + n2},

These distortions can be deduced simply by examining whichcomponents of x are mapped to which elements of Xinit.

Optimization of (4) over both {ni}~l and {J.1;i}~1 subjectto (3) is difficult, partly due to the integer constraint of the par­tition. However, given an integer partition (nl' n2, ... , n«),the optimal initial codeword can be determined easily fromthe means of the order statistics:

where IN is the N x N identity matrix.A frame is called a tight frame if the frame bounds can be

chosen to be equal. A frame is an equal-norm frame if allof its vectors have the same norm. If an equal-norm frame isnormalized to have all vectors of unit norm, we call it a unit­norm frame. A unit-norm tight frame (UNTF) must satisfyF*F = (MjN)IN .

C. Frame Definitions and Classifications

In this paper, we use frame expansions only for quantizationusing PSCs, which rely on order relations of real numbers.Therefore we limit ourselves to real finite frames.

Definition 2.3: A set of N-dimensional vectors, q> ={<Pk}~l C jRN, is called aframe if there exist a lower framebound, A > 0, and an upper frame bound, B < 00, such that,for all x E jRN,

Allxl1 2 <L~l I(x, <Pk) 12 <Bllxl1 2. (7a)

where each J.1;i appears n, times. When Xinit has this form,we call it compatible with (nl' tia, ... , ti«). The codebook isthe set of all distinct permutations of Xinit. The number ofcodewords in C is thus given by the multinomial coefficient

L = N!. ( )nl! n2! ... nK! 2

The permutation structure of the codebook enables low­complexity nearest-neighbor encoding [10]: map x to thecodeword x whose components have the same order as x; inother words, replace the nl largest components of x with J.1;1,

the n2 next-largest components of x with J.1;2, and so on. Sincethe complexity of sorting a vector of length N is O(N log N)operations, the encoding complexity for PSC is much lowerthan with an unstructured source code and only O(log N)times higher than scalar quantization.

The per-component rate is defined as

R = N- Ilog2L. (3)

Under certain symmetry conditions on the source distribution,all codewords are equally likely so the rate cannot be reducedby entropy coding.

Partition Properties: The partition induced by a PSC is com­pletely determined by the integer partition (nl' n2, ... , n K ).Specifically, the encoding mapping can index the permutationP that places the nl largest components of x in the firstnl positions (without changing the order within those nlcomponents), the n2 next-largest components of x in the nextn2 positions, and so on; the J.1;iS are actually immaterial. Thisencoding is placing all source vectors x such that Px is n­descending in the same partition cell, defined as follows.

Definition 2.2: Given an ordered integer partition n =(nl' n2, ... , nK) of N, a vector in jRN is called n-descendingif its nl largest entries are in the first nl positions, its n2

next-largest components are in the next n2 positions, etc.The property of being n-descending is to be descending up tothe arbitrariness specified by the integer partition n.

Because this is nearest-neighbor encoding for some code­book, the partition cells must be convex. Furthermore, multi­plying x by any nonnegative scalar does not affect the encod­ing, so the cells are convex cones. We develop a convenientrepresentation for the partition in Section III.

Xinit = (J.1;I,.'" J.1;1, J.1;2,···, J.1;2,···, J.1;K,···, J.1;K), (1)~nl~ ~n2~ ~nK~

and for odd N by

Definition 2.4: The real harmonic tight frame (HTF) of Mvectors in lRN is defined for even N by

where k = 0,1, ... , M - 1. The modulated harmonic tightframes are defined by

(11)

III. FRAME PERMUTATION QUANTIZATION

FPQ is simply PSC applied to a frame expansion.

[ -~ ] x ~ [ t~ ~ ~ ],where the inequalities are elementwise.

D. Reconstructionfrom Frame Expansions

A central use of frames is to formalize the reconstructionof x E lRN from the frame expansion Yk = (x, <Pk), k =1, 2, ... , M, or estimation of x from degraded versions ofthe frame expansion. Using the analysis frame operator wehave Y = Fx, and (7) implies the existence of at least onelinear synthesis operator G such that GF = IN. A framewith analysis frame operator G* is then said to be dual to q>.

The frame condition (7) also implies that F*F is invertible,so the Moore-Penrose inverse (pseudo-inverse) of the frameoperator Ft = (F* F) -1F* exists and is a valid synthesisoperator. Using the pseudo-inverse for reconstruction hasseveral important properties including an optimality for mean­squared error (MSE) under assumptions of uncorrelated zero­mean additive noise and linear synthesis. Reconstruction usingFt is called canonical reconstruction and the correspondingframe is called the canonical dual. In this paper, we use theterm linear reconstructionfor reconstruction using an arbitrarylinear operator.

When Y is quantized to '0, it is possible for the quantizationnoise '0 - Y to have mean zero and uncorrelated components;this occurs with subtractive dithered quantization [23] orunder certain asymptotics [24]. In this case, the optimality ofcanonical reconstruction holds. However, it should be notedthat even with these restrictions, canonical reconstruction isoptimal only among linear reconstructions.

When nonlinear construction is allowed, quantization noisemay behave fundamentally differently than other additivenoise. The key is that a quantized value gives hard constraintsthat can be exploited in reconstruction. For example, supposethat '0 is obtained from Y by rounding each element to thenearest multiple ofa quantization step size ~. Then knowledgeof 'Om is equivalent to knowing

u» E ['Ok - ~~, 'Ok + ~~]. (10)

Geometrically, (x, <Pk) = 'Ok - ~~ and (x, <Pk) = 'Ok + ~~are hyperplanes perpendicular to <Pk, and (10) expresses thatx must lie between these hyperplanes. Using the upper andlower bounds on all M components of Y, the constraints onx imposed by '0 are readily expressed as [2]

A. Encoder Definition and Canonical Decoding

Definition 3.1: A frame permutation quantizer with anal-ysis frame F E lRM x N, integer partition m(m1, m2, ... , m«), and initial codeword 'Oinit compatiblewith m encodes x E lRN by applying a permutation sourcecode with integer partition m and initial codeword 'Oinit to Fx.The canonical decoding gives x = Ft '0, where '0 is the PSC

(9)for k = 1,2, ... ,M,

* f2 [ 1 2k1r 4k1r (N - l)k1r¢k+l = VN v'2,cos M ,cos M , ... ,cos M '

. 2k1r . 4k1r . (N - 1)k1r]SIll M ' SIll M , ... , SIll M '

(8b)

where '"Y = 1 or '"Y = -1 (fixed for all k).Classification of frames is often up to some unitary equiv­

alence. Holmes and Paulsen [20] proposed several types ofequivalence relations between frames. In particular, for twoframes in lRN

, q> = {<Pk}~l and W = {'l/Jk}~l' we say q>and Ware

* f2 [ k-n 3k1r (N - l)k1r<Pk+1 = VN cos M'cOS M , ... ,COS M '

. kat . 3k1r . (N - 1)k1r]SIll M,SIll M , ... .sm M (8a)

(i) type I equivalent if there is an orthogonal matrix U suchthat 'l/Jk = U<Pk for all k;

(ii) type II equivalent if there is a permutation a(·) on{I, 2, ... , M} such that 'l/Jk = <Pa(k) for all k; and

(iii) type III equivalent if there is a signfunction in k, 8(k) =±1 such that 'l/Jk = 8(k)<Pk for all k.

Another important subclass ofUNTFs is defined as follows:Definition 2.5 ([21], [22]): A UNTF q> = {<Pk}~l C

lRN is called an equiangular tight frame (ETF) if there existsa constant a such that I(<PR, <Pk) I = a for all 1 ::; e< k ::; M.ETFs are sometimes called optimal Grassmannianframes or2-uniformframes.

In our analysis of FPQ, we will find that restricted ETFs­where the absolute value constraint can be removed from Defi­nition 2.5-play a special role. In matrix view, a restricted ETFsatisfies F*F = (M/N)IN and FF* = (1 - a)IM + aJM,where JM is the all-l s matrix of size M x M. The followingproposition specifies the restricted ETFs for the codimension-lcase.

Proposition 2.6: For M = N +1, the family ofall restrictedETFs is constituted by the Type I and Type II equivalents ofmodulated HTFs.

Finally, the following property of modulated HTFs in theM = N + 1 case will be very useful.

Proposition 2. 7: If M = N +1 then a modulated harmonictight frame is a zero-sum frame, Le., each column of theanalysis frame operator F sums to zero.

(12)

(16)

(15c)

(15b)

D(m) =

and express all of (14) for one fixed i as D}m) Z 2:Omimi+l xr­Continuing our recursion, it only remains to gather the

inequalities (14) across i E {I, 2, ... , K - I}. Let

D(m)1

D(m)2

which has

D(m)K-l

is an m, x M differencing matrix. Allowing f to vary acrossI i + 1, we define the mimi+l x M matrix

D~m)~,Mi+l

D~m)~,Mi+2

(13)

reconstruction of y. We sometimes use the triple (F, m, Yinit)

to refer to such an FPQ.The result ofthe encoding can be expressed as a permutation

P from the permutation matrices of size M. The permutationis such that P Fx is m-descending. For uniqueness in therepresentation P chosen from the set of permutation matrices,we can specify that the first ml components of Py are kept inthe same order as they appeared in y, the next m2 componentsof Py are kept in the same order as they appeared in y, etc.Then P is in a subset 9(m) of the M x M permutationmatrices and

Notice that the encoding uses the integer partition m but notthe initial codeword Yinit. The PSC reconstruction of y isP- 1Yinit , so the canonical decoding gives FtP- 1Yinit .

The size of the set 9(m) is analogous to the codebook sizein (2), and the per-component rate of FPQ is thus defined as

-1 M!R = N log2 .

ml!m2! ... mK!

for every k E E, and f E I j •

By transitivity, considering every i < j gives redundantinequalities. Taking only j = i +1, we obtain a full description

(17)D(m)PFx 2: o.

C. Consistent Reconstruction Algorithms

The constraints (17) specify unbounded sets. To be ableto decode FPQs in analogy to [2, Table I], we require someadditional constraints. Two examples are developed in [14]: asource x bounded to [-!, !]N (e.g., having an i.i.d. uniformdistribution over [-!, !]) or having an i.i.d. standard Gaussiandistribution. The Gaussian case is recounted here.

Suppose x has i.i.d. Gaussian components with mean zeroand unit variance. Since the source support is unbounded,something beyond consistency must be used in reconstruction.Here we use a quadratic program to find a good bounded,consistent estimate and combine this with the average valueof Ilxll.

The problem with using (17) combined with maximizationof minimum slackness alone (without any additional bounded­ness constraints) is that for any purported solution, multiplyingby a scalar larger than 1 will increase the slackness of all theconstraints. Thus, any solution technique will naturally andcorrectly have Ilxll -+ 00. Actually, because the partition cellsare convex cones, we should not hope to recover the radialcomponent of x from the partition. Instead, we should onlyhope to recover a good estimate of x/llxll.

To estimate the angular component x/llxll from the parti­tion, it would be convenient to maximize minimum slacknesswhile also imposing a constraint of Ilxll = 1. Unfortunately,this is a nonconvex constraint. It can be replaced by Ilxll ::; 1

rows. The property of Z being m-descending can be expressedas D(m)z 2: OL(m)xM.

Now we can apply these representations to FPQ. Since PFxis m-descending, consistency is simply expressed as

Omix(M-Mi) ] Z

I m i x 1 Omix(M-f) ] Z

[OmixMi-l I m i

2: [Omi x(f-l)

D}7) = [OmixMi_l I mi OmiX(.e-Mi-1) -lmi x 1 OmixM-.e]

(15a)

B. Expressing Consistency Constraints

Suppose FPQ encoding of x E lRN with frame F E lRM x N,

integer partition m = (ml' m2, ... , mtc), and initial codewordYinit compatible with m results in permutation P E g(m). Wewould like to express constraints on x that are specified by(F, m, Yinit, P). This will provide an explanation of the par­titions induced by FPQ and lead to reconstruction algorithmsin Section III-C.

Knowing that a vector is m-descending is a specificationof many inequalities. Recall the definitions of the index setsgenerated by an integer partition given in (5), and use thesame notation with nkS replaced by mkS. Then z being m­descending implies that for any i < i,

Zk 2: Zf for every k E Ii, f E I i+1 with i = 1, ... , K - 1.(14)

For one fixed (i, f) pair, (14) gives IIiI= m; inequalities, onefor each k E Ii. These inequalities can be gathered into anelementwise matrix inequality as

because slackness is proportional to II xII. This suggests theoptimization

maximize 8 subject to Ilxll < 1 and D(m)PFx :2: 81L (m ) x 1.

Denoting the x at the optimum by xang ' we still need to choosethe radial component, or length, of x.

For the N(O, IN) source, the mean length is [25]

J27r ~--~E[llxllJ = (3(N/2, 1/2) ~ y'N - 1/2.

We can combine this with xang to obtain a reconstruction x.This method is described explicitly in [14, Alg. 3].

IV. CONDITIONS ON THE CHOICE OF FRAME

In this section, we provide necessary and sufficient condi­tions so that a linear reconstruction is also consistent. We firstconsider a general linear reconstruction, x = Ry, where R issome N x M matrix and y is a decoding of the PSC of y.We then restrict attention to canonical reconstruction, whereR = Ft. For each case, we describe all possible choices of a"good" frame F, in the sense of the consistency of the linearreconstruction.

A. Arbitrary Linear Reconstruction

We begin by introducing a useful term.Definition 4.1: A matrix is called column-constant when

each column of the matrix is a constant. The set of all M x Mcolumn-constant matrices is denoted :1.

We now give our main results for arbitrary linear recon­struction combined with FPQ decoding of an estimate of y.

Theorem 4.2: Suppose A = FR = aIM + J for somea :2: 0 and J E :1. Then the linear reconstruction x = Ryis consistent with Variant I FPQ encoding using frame F,an arbitrary integer partition and an arbitrary Variant I initialcodeword compatible with it.

The key point in the proof of Theorem 4.2 is showing thatthe inequality

(18)

where A = F R, holds for every integer partition m and everyinitial codeword Yinit compatible with it. It turns out that theform of matrix A given in Theorem 4.2 is the unique form thatguarantees that (18) holds for every pair (m, Yinit). In otherwords, the condition on A that is sufficient for any integerpartition m and any initial codeword Yinit compatible with itis also necessary for consistency for every pair (m, Yinit).

Theorem 4.3: Consider FPQ using frame F with M :2: 3. Iflinear reconstruction x = Ry is consistent with every integerpartition and every initial codeword compatible with it, thenmatrix A = F R must be of the form aIM + J, where a :2: 0and J E:1.

B. Canonical Reconstruction

We now restrict the linear reconstruction to use the canon­ical dual; Le., R is restricted to be the pseudo-inverse Ft =(F*F) -1F*. The following corollary characterizes the non­trivial frames for which canonical reconstructions are consis­tent.

Corollary 4.4: Consider FPQ using frame F with M > Nand M :2: 3. For canonical reconstruction to be consistent withevery integer partition and every initial codeword compatiblewith it, it is necessary and sufficient to have M = N + 1 andA = FFt = 1M - ltJM, where JM is the M x Mall-Ismatrix.

We continue to add more constraints to frame F. Byimposing tightness and unit norm on our analysis frame, wecan progress a bit further from Corollary 4.4 to derive theform of FF*.

Corollary 4.5: Consider FPQ using unit-norm tight frameF with M > Nand M :2: 3. For canonical reconstructionto be consistent for every integer partition and every initialcodeword compatible with it, it is necessary and sufficient tohave M = N + 1 and

1 1 1-N -If1 1FF*=

-N -N(19)

1 1 1-N -N

Recall that a UNTF that satisfies (19) is a restricted ETF.Therefore Corollary 4.5 together with Proposition 2.6 gives usa complete characterization of UNTFs that are "good" in thesense of canonical reconstruction being consistent.

Corollary 4.6: Consider FPQ using unit-norm tight frameF with M > Nand M :2: 3. For canonical reconstructionto be consistent for every integer partition and every initialcodeword compatible with it, it is necessary and sufficient forF to be a modulated RTF or a Type I or Type II equivalent.

V. SIMULATIONS

In this section, we provide simulations to demonstrate someproperties of FPQ and to demonstrate that FPQ can giveperformance better than ECSQ and ordinary PSC for certaincombinations of signal dimension and rate. For every datapoint shown, the distortion represents a sample mean estimateof N- 1E[llx - x11 2] over at least 106 trials. Testing was donewith exhaustive enumeration of the relevant integer partitions.This makes the complexity of simulation high, and thusexperiments are only shown for small Nand M. Recall theencoding complexity of FPQ is low, O( M log M) operations.The decoding complexity is polynomial in M, and in someapplications it could be worthwhile to precompute the entirecodebook at the decoder. Thus much larger values of NandM than used here may be practical.

Gaussian source. Let x have the N(O, IN) distribution.Figure 1 summarizes the performance of FPQ with decodingusing the algorithm described in Section III-C. Also shown arethe performance ofentropy-constrained scalar quantization and

- ECSQ--- RD

o M =5o M =6

M =7

o

---------

oo .o

o Q

o

O.

\\\\\\\

\\

\\

\\

\\

\\

\\

\ -,-,

-,"-,,­

<,

..... """"

0.1

0.3

0.2

0.9

0.8

0.7

c 0.6o:e.9 0.5(/)

0 0.4

- ECSQ--- RD

o M =4o M =5v M =6

M =7

---------

o

o

o

o v

\\\\

\\\

\\

\\

\\

\\

\\

\\

\\

\ -,-,

<,<,

..... """"

0.1

0.7

0.9

0.8

0.3

0.2

c 0.6o:e.9 0.5(/)

0 0.4

0.5Rate

1.5 2 2.5 0.5Rate

1.5 2 2.5

(a) N = 4 (b) N = 5

Fig. 1. Performance of Variant I FPQ on an i.i.d. N(o, 1) using modulated harmonic tight frames ranging in size from N to 7. Performance of PSC isnot shown because it is equivalent to FPQ with M = N for this source. Also plotted are the performance of entropy-constrained scalar quantization and thedistortion-rate bound.

REFERENCES

N. T. Thao and M. Vetterli, "Deterministic analysis of oversampled AID[16]

[l]conversion and decoding improvement based on consistent estimates," [17]IEEE Trans. Signal Process., vol. 42, no. 3, pp. 519-531, Mar. 1994.

[2] V. K. Goyal, M. Vetterli, and N. T. Thao, "Quantized overcomplete [18]expansions in IFtN : Analysis, synthesis, and algorithms," IEEE Trans.Inform. Theory, vol. 44, no. 1, pp. 16-31 , Jan. 1998. [19]

[3] Z. Cvetkovic, "Resilience properties of redundant expansions underadditive noise and quantization," IEEE Trans. Inform. Theory, vol. 49, [20]no. 3, pp. 644-656, Mar. 2003.

[4] S. Rangan and V. K. Goyal, "Recursive consistent estimation with [21]bounded noise," IEEE Trans. Inform. Theory, vol. 47, no. 1, pp. 457-464, Jan. 2001.

[5] J. 1. Benedetto, A. M. Powell, and 6. Yilmaz, "Sigma- Delta (I;Ll) [22]quantization and finite frames," IEEE Trans. Inform. Theory, vol. 52,no. 5, pp. 1990-2005, May 2006.

[6] A. M. Powell, "Estimation algorithms with noisy frame coefficients," in [23]Proc. Wavelets XII, part of SPIE Optics & Photonics, vol. 6701, SanDiego, CA, Aug. 2007. [24]

[7] B. G. Bodmann and V. I. Paulsen, "Frame paths and error bounds forsigma-delta quantization," Appl. Comput. Harmon. Anal., vol. 22, no. 2,pp. 176-197 , Mar. 2007. [25]

[8] B. G. Bodmann and S. P. Lipshitz, "Randomly dithered quantization andsigma-delta noise shaping for finite frames," Appl. Comput. Harmon.Anal., vol. 25, no. 3, pp. 367- 380, Nov. 2008. [26]

the distortion-rate bound. Of course, the distortion-rate boundcan only be approached with N -+ 00; it is not presented asa competitive alternative to FPQ for N = 4 and N = 5.

We have not provided an explicit comparison to ordinaryPSC because, due to rotational-invariance of the Gaussiansource, FPQ with any orthonormal basis as the frame isidentical to PSC. (The modulated harmonic tight frame withM = N is an orthonormal basis.) PSC and FPQ are sometimesbetter than ECSQ; increasing M gives more operating pointsand a higher maximum rate;2 and M = N +1 seems especiallyattractive.

[9] J. G. Dunn, "Coding for continuous sources and channels," Ph.D.dissertation, Columbia Univ., New York, 1965.

[10] T. Berger, F. Jelinek, and J. K. Wolf, "Permutation codes for sources,"IEEE Trans. Inform. Theory, vol. IT-18, pp. 160-169, Jan. 1972.

[II] V. K. Goyal, S. A. Savari, and W. Wang, "On optimal permutationcodes," IEEE Trans. Inform. Theory, vol. 47, pp. 2961-2971 , Nov. 2001.

[12] V. Brajovic and T. Kanade, "A VLSI sorting image sensor: Globalmassively parallel intensity-to-time processing for low-latency adaptivevision," IEEE Trans. Robot. Automat., vol. 15, no. I, pp. 67-75, Feb.1999.

[13] X. Guo, X. Qi, and J. G. Harris, "A time-to-first-spike CMOS imagesensor," IEEE Sensors 1., vol. 7, no. 8, pp. 1165-1175, Aug. 2007.

[14] H. Q. Nguyen, V. K. Goyal, and L. R. Varshney, "Frame permutationquantization," arXiv:0909.l599vl [cs.IT]., Sep. 2009.

[15] L. R. Varshney and V. K. Goyal, "Ordered and disordered sourcecoding," in Proc. UCSD Workshop Inform. Theory & Its Applications,La Jolla, CA, Feb. 2006.D. Slepian, "Permutation modulation," Proc. IEEE, vol. 53, no. 3, pp.228-236, Mar. 1965.T. Berger, "Optimum quantizers and permutation codes," IEEE Trans.Inform. Theory, vol. IT-18, pp. 759-765, Nov. 1972.--, "Minimum entropy quantizers and permutation codes," IEEETrans. Inform. Theory, vol. IT-18, pp. 149-157, Mar. 1982.H. A. David and H. N. Nagaraja, Order Statistics, 3rd ed. Wiley-Interscience, 2003.R. B. Holmes and V. I. Paulsen, "Optimal frames for erasures," LinearAlgebra Appl. , vol. 377, pp. 31-51 ,2004.T. Strohmer and R. W. Heath, "Grassmanian frames with application tocoding and communications," Appl. Comput. Harmon. Anal., vol. 14,no. 3, pp. 257-275, 2003.M. A. Sustik, J. A. Tropp, I. S. Dhillon, and R. W. Heath, "On theexistence of equiangular tight frames," Linear Algebra Appl., vol. 426,pp. 619-635, 2007.R. M. Gray and T. G. Stockham, Jr., "Dithered quantizers," IEEE Trans.Inform. Theory, vol. 39, no. 3, pp. 805-812, May 1993.H. Viswanathan and R. Zamir, "On the whiteness of high-resolutionquantization errors," IEEE Trans. Inform. Theory, vol. 47, no. 5, pp.2029-2038, Jul. 2001.D. 1. Sakrison, "A geometric treatment of the source encoding of aGaussian random variable," IEEE Trans. Inform. Theory, vol. IT-14, pp.481-486, May 1968.H. Q. Nguyen, L. R. Varshney, and V. K. Goyal, "Concentric permutationsource codes," arXiv:0909.0704vl [cs.IT]., Sep. 2009.

2A discussion of the density of PSC rates is given in [26, App. B].