Cm~lput. &c Elect. Engng, Vol. I, pp. 545 SS0. Pergamon Press, 1974. Printed in Great Britain.

IMPORTANCE SAMPLING IN COMPUTER SIMULATION OF SIGNAL PROCESSORS

V. GREGERS HANSEN Raytheon Company, Wayland, Massachusetts 01778, U.S.A.

(Received5 March 1974)

Ahstract--Use of the variance reducing technique of importance sampling in a Monte Carlo simulation can make it possible to estimate very low error probabilities. The fundamental principle of this technique is discussed and examples are presented which demonstrate the feasibility of improving the simulation efficiency by many orders of magnitude.

I . I N T R O D U C T I O N

A Monte Carlo simulation on a digital computer is frequently the only feasible procedure for obtaining quantitative data on the performance of signal processors in communicat ion and radar systems.

A fundamental limitation of the Monte Carlo method is the restricted number of repeti- tions of the basic simulation that can be carried out with an acceptable computat ional effort. This number determines the accuracy of the performance estimates obtained and limits the extent of any desired parametric investigations. Several approaches can be followed in attempts to increase the efficiency of a computer simulation. These may be to replace parts of the simulation by analytical results, to find more efficient computer algorithms*, or to employ one of several variance-reducing techniques to improve the efficiency of the Monte Carlo simulation[l].

In the evaluation of the detection performance of radar processors or digital communica- tion systems, it is often required to be able to estimate very low error probabilities. This requirement is difficult to satisfy in a direct Monte Carlo simulation where the smallest probability which can be estimated is of the order N~ t where N R is the number of repeti- tions used. In the past this problem has been by-passed by performing as many repetitions as possible at a point of higher error probability and then somehow extrapolating the results obtained towards the lower error probabilities of interest.

In this paper it is described how the technique of importance sampling ([1], Chap. 5) can be applied to this class of problems to overcome the limitations of a restricted number of repetitions of the simulation.

* More rapid methods of generating pseudo random numbers is an area which has been the subject of con- siderable effort [2].

545

( A L E Vol, ] No. 4 -F

546 V. GREGERS HANSEN

X I

X 2

D

= S i g n a l

p r o c e s s o r

iw

, .[comoo,oto,]

I z : g ( x f . . . . x N) Thresho ld V

Fig. 1. General representation of detection problem.

= H3

• H 0

2. P R I N C I P L E O F I M P O R T A N C E S A M P L I N G

The fundamental principle of the importance sampling technique, as used to improve the efficiency of a computer simulation of a signal processor, can best be described with reference to a rather general detection situation as illustrated in Fig. 1. The input to the signal processor consists of a set of random variables (and constants) which is written in vector notation as x = xt . . . . . XN. The .statistical properties of x are known and given by the joint probability density function (pdf) po(X) under hypothesis H 0 and pdx) under hypothesis H~.i" The signal processor is characterized by the transformation z = g(x) and the choice between the two hypotheses is based on comparison of z with the threshold V. In many practical detectors the transformation g(x) is so complex that an analytical determination of the distribution of the output z is precluded.

Then to determine the probability of an error of the first (or second) kind:]: (or in radar terminology, the false alarm probability) a direct Monte Carlo simulation can be used, in which inputs x are generated according to p0(x) using pseudo-random numbers in the computer. Each specific input x~ is then transformed into the corresponding output z~ = g(xi), and after N R repetitions the probability of error is estimated as

1 NR

= ~ i = , ~" u(z, - V) (1)

where u( ) is the unit step function. Thus the probability is estimated as the ratio between the number of occurrences of the event z~ ) V and the total number of repetitions N R. The important limitation of such a direct Monte Carlo simulation is that the smallest probability that can be estimated is of the order ~ ~ (NR)- t, corresponding to a single event in the N R repetitions. Thus, if estimation of low values o fe is desired a corresponding large number of repetitions is required.

To overcome this limitation, importance sampling can be introduced into the simulation by replacing the true pdf, po(x), by a modified pdf, p~(x), which is chosen so that inputs x, for which a large output z results, are generated with an increased probability[l]. With the simulation modified in this way the estimate of e has to be computed as

1 ~l po(xi) = u ( z , - v). (2)

i- For simplicity the discussion will be restricted to binary detection.

,* The procedure is completely symmetrical for determining the probability of an error of the second kind (probability of false dismissal).

Importance sampling in computer simulation of signal processors 547

This equation differs from (1) by the ratio of the likelihoods of obtaining a specific x~ without and with importance sampling. It is easily shown that use of (2) ensures that the estimate of ~ is unbiased as is the case for a direct simulation[l].

When importance sampling is used no theoretical limits are imposed on the accuracy of the estimates which can be obtained by simulation. The practical choice of a pdf, p*(x), which leads to a significant improvement in efficiency is, however, not always straight- forward. The most important consideration is that the occurrences of z~ >/ V, which are forced to take place as a result of the choice of p~(x), must correspond to inputs x~ which are the most likely in terms of the true pdf, p0(x), when considering the set of all inputs, x, for which z ~ V. Due to the complexity of the transformation g(x) the choice of p~(x) has to be made on the basis of qualitative considerations. The ensuing improvement in simulation efficiency then has to be judged on the basis of the results obtained.

3. TWO EXAMPLES

Two examples will be presented to illustrate the large improvement in simulation efficiency that can be realized through use of the importance sampling technique. In the first example a non-parametric rank test known as the "Spearman rho" is considered. The second example represents a radar signal processing method used to ensure a constant false alarm rate at the detector output when the power level of the input noise is unknown.

When the Spearman rho test is used, the input, x, is a permutation of the integers 1, 2 . . . . . N. Under hypothesis H 0 the permutat ion is completely random, whereas under

t.O

,o[-

la IO 2 ~ ~ l ~, , ' - , ,v: ,6

. t k,~ \ x % N,~: I000

iO.~_ "~ \ ® Exoc~-

-- k \ ~ \ \ point s

~" io.~_ \ \ ~' ~ k Gouss an

~" ,o 5 _ ~ \\\

,o t \\\ to" - L I

1300 1400 1500 1600

Threshold, V

Fig. 2. Probability of error for Spearman rho test as obtained by a Monte Carlo simulation using importance sampling.

548 V. GREGERSHANSEN

i0 -~

I0 ~ I

{3

°

~_ , 0 -< _

iO- ' _

o

tL lO-,e

I0 ': t I0 '~

1300

N:16 Np = 1000 ® E x o c ¢

p o i n t s

1 ~400 1500 Threshold, V

1600

Fig. 3. Probability of error for Spearman rho test as obtained by a Monte Carlo simulation using importance sampling.

H 1 an increasing trend is present in the data. The Spearman rho test for deciding between these hypotheses is [3].

N

z = ~ i "xi (3) i=1

Curves of probability of error versus decision threshold, when hypothesis H o is true, were obtained by a Monte Carlo simulation using the importance sampling technique[4]. Typical results are shown in Figs. 2 and 3 for N = 16, Na = 1000, and different pdfs p*(x). The importance sampling was introduced in this case by choosing p*(x), such that an increasing trend was present in the input data x. The points shown as circles in Fig. 3 represent the extreme tail of the error distribution which can be theoretically determined. The agreement between the results obtained by simulation and the theoretical points is excellent and confirms the validity of the approach in this case. It is se~n from these curves that the use of importance sampling has increased the efficiency of the simulation by up to l0 orders of magnitude (10 billions).

The radar processing technique which is used for the second example is sometimes referred to as a "cell-averaging CFAR (constant false alarm rate)" detector[7]. In this case we assume a situation where there are N + l inputs, xl . . . . , xN+ t, to the signal processor in Fig. i. Under hypothesis H o we have

N + I

po(x) = l-I p,(xi) (4) i = 1

Importance sampling in computer simulation of signal processors 549

whereas under H 1

N

p l ( x ) = p.tx~+ 1) I-I p.(x~). (5) i = 1

The pdf p,,(x) is known except for a scale factor and ps(x) is such that under H 1, x~+ 1 is stochastically larger than under H o. The following test statistic then has the property of being scale-invariant under hypothesis H o :

"X:N+ 1 Z - I T (6)

i=1

Thus for a given shape ofp,(x) the curve of false alarm probability versus decision threshold is not affected by an arbitrary positive scale factor. For the specific case of an exponential pdf

{ ; e x p ( - c . x ) x > ~ 0

p,,(x) = otherwise (7)

where c is the scale factor the probability of error, when H 0 is true is

( e = 1 + (8)

I'0;

i I

,o

h e o r e t i c o l c u r v e

I 0

" ~ N =32

. ~e = I 0 0 0

I0" Q)

I0 - -

Q- i 0 5 - -

i0 6 - -

I0" : 0 5 IO 15

T h r e s h o l d , V

Fig. 4. Probability of error for cell-averaging CFAR test. Points shown obtained by Monte Carlo simulation using importance sampling.

550 V. GREGERS HXNSE~

This theoretical curve is shown in Fig. 4 in full line whereas the points were obtained by Monte Carlo simulation using the importance sampling technique introduced by scaling the pdf for the input t x~+ 1 so that

p~tx) = I7 p.tx~) * .p , , (x .+,) (9) i=1

where

I /XN+II p*(XN+ ,) = ~ p , l ~ - ]" (10)

The simulation results are in good agreement with the theoretical curve and the improve- ment in efficiency is again many orders of magnitude. Other examples of the use of impor- tance sampling technique are presented in [6] and [7].

4. CONCLUSIONS

The use of the technique of importance sampling to improve the efficiency of Monte Carlo simulations of signal processor in communica t ions and radar has been described. Two examples were presented which demonst ra te the feasibility of improving the simula- tion efficiency by many orders of magnitude.

It is concluded that the importance sampling technique is an effective tool which should be considered in evaluations of low-probabil i ty events where the Monte Carlo technique appears to be the best or only procedure for obtaining quanti tat ive results on expected performance.

REFERENCES

I. J. M. Hammersley and D. C. Handscomb, Monte Carlo Methods. Methuen, London (1964). 2. C. M. Rader et al., Bell Sys. Tech. J. 49, 2303-2310 (1970). 3. J. Hajek, Nonparametric Statistics. Holden-Day, San Francisco (1969). 4. V. G. Hansen, IEEE Trans. Information Theory IT-16, 309-318 (May 1970). 5. H. M. Finn, RCA Rev.Vol. 30 414-464 (September 1968). 6. V. G. Hansen, IEEE Trans. Information Theory IT-IS, 664-667 (September 1972). 7. V. G. Hansen and H. R. Ward, IEEE Trans. Aerospace Electronic Systems AES-,8, 648-.652 (September 1972).

"t" A scaling of the inputs x I . . . . . xN by a factor smaller than one was found to be unnecessary for the value of N considered here.