randomnumbergeneration

EMSE 273- Discrete System Simulation

Lecture Notes by J.R. van Dorp 1Source: Discrete Event Simulation by Banks Carson and Nelson Chapter 7

7. RANDOM NUMBER - GENERATION

7.1 Properties of Random Numbers

A sequence of random numbers , must have two important statisticalV V " 8properties: and Uniformity Independence

Each V V Y! "3 should be an independent realization (or sample) from a variable:

x0 1

f(x)

1

x0 1

f(x)

1

f(x)f(x)

1

IV B.B B l '! # #" " "# "!Z +



Consequences of Uniformity and Independence Properties:

1. If the interval is divided into subinterval of equal length, the expected! " 8number of observations in each interval is where is the total number ofR8 Robservations

2. The probability of observing a value in a particular interval is independent ofthe previous values drawn.

7.2 Generation of Pseudo-Random Numbers

"Pseudo" means false, so false random numbers are being generated. If themethod is known the numbers cannot be truly random. The goal of anygeneration scheme is to the real properties of a uniform distribution and"imitate"independence as closely as possible.



Problems that can occur with Pseudo Random Numbers:

1. Non-Uniformity

2. Numbers are discrete valued and not continuous on ! "

3. The mean of generated numbers IV "#

4. The variance of generated numbers Z +



Usually, pseudo-random numbers are generated by a computer as part of thesimulation. Numerous methods are available. In selecting a routine, there are anumber of important considerations.

1. The routine should be fast

2. The routine should be platform independent and portable between differentprogramming languages.

3. The routine should have a sufficiently long cycle (much longer than therequired number of samples).

4. The random numbers should be replicable. Usefull for debugging and variancereduction techniques.

5. Most importantly, the routine should closely approximate the ideal statisticalproperties.



Inventing techniques that seem to generate random number is easy; inventing techniques that really produce sequences that appear to independent,

uniformly distributed is incredibly difficult.

7.3 Techniques for Generating Random Numbers

The linear congruential method is most widely known. Will also report anextension that yield sequences with a longer period.

7.3.1. Linear Congruential Method

Generates a sequence using\ \ \ " # $

\ +\ -79.7 ("3" 3 ,



where is called the , is the , is the\ + -! seed variable constant multiplierincrement, and is the .7 modulus

When , is referred to as the - ! - ! (" mixed congruential method( ).multiplicative congruential method

The selection of the valeus for and drastically affect the statistical+ - 7 \!properties and the cycle length. Variations of are quite common in computer("packages.

EXAMPLE 7.1: and .\ #( + "( - %$ 7 "!!!Here integer values will be between and because of the . Note! ** modulus 7that random integers are being generated rather than random numbers. Theseintegers should be uniformly distributed. Convert to numbers in by! "normalizing with :modulus 7

V (#\733



\ #(!\ "( #( %$79. "!! &!#79. "!! #"V !!#" #"!!\ "( # %$79. "!! ((79. "!! ((#V !((# ("!!etc.

Of primary importance is uniformity and statistical independence. Of secondaryimportance is and within the sequencemaximum density maximum period

V 3 " # 3,

Note that, the sequence can only take values in:

! "7 #7 7 "7 "

Thus is discrete rather than continuous.V3



This is easy to fix by choosing large modulus . Values such as and7 7 # "$"7 #%) are in common use in generators appearing in many simulationlanguages). Maximum Density and Maximum period can be achieved by theproper choice of , , and .+ - 7 \!

For a power of , say , and , the longest period is7 # 7 # - ! T 7 #, ,achieved provided is a relative prime to (that is the greates common- 7factor of and is ), and where is an integer.- 7 " + " % 5 5

For a power of , say , and , the longest period7 # 7 # - !,T 7% # \,# ! is achieved provided the seed is odd and the multiplier+ $ ) 5 5 ! " , for some .

For is a prime number and , the longest period is7 - ! T 7 "achieved provided the multiplier has the property that the smallest integer+5 + " 7 5 7 " such that is divisible by is .5



EXAMPLE 7.2: , and and .+ "" 7 # '% - ! \ " # $ %' !

EXAMPLE 7.3: , and .+ "* 7 "! "!! - ! \ '$# !

EXAMPLE 7.4: , .+ ( "')!( 7 # " - !& $"This one is in actual use and extensively tested. Conditions ensure a periodT 7 " \ By changing the user can control repeatibility of sequence!

k=0 Xk Rk Gap0 12345671 1422014746 0.0006 0.66162 456328559 0.6622 0.44973 849987676 0.2125 0.18334 681650688 0.3958 0.07845 1825340118 0.3174 0.53266 1687465831 0.8500 0.06427 1569179335 0.7858 0.05518 2097898185 0.7307 0.24629 1988278849 0.9769 0.051010 9584176 0.9259 0.9214



Average 0.499Variance 0.086Average Gap 0.002

Over 500 Samples

Emperical CDF of LCG Generator over 500 Samples

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1.000

0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000

r

C

D

F

Em perical CDF Uniform CDF



7.3.2. Combined Linear Congruential Method

Random Number Generator of Example 7.4. with a period of

# " # "!$" *

is no longer adequate with current advancements of computing powers. Area ofnew research is deriving generators with longer periods (See, L'Ecuyer [1998]).This research uses the following important result.

If are independent discrete valued random variables (not[ [ [3" 3# 35necesarrily identical distributed), but one of them, say , is uniformly[3"distributed on the integers , then! 7 #"

[ [ 79. 7 "3 34 "4"

5 is uniformly distributed on the integers ! 7 ""



1. Let \3" 3# 35\ \ 5 - ! be the i-th output from different multiplicative congruential generators, where first generator has modulus and the7"multiplier chosen so that the period is .+ 7 "" "

2. The first generator then produces that are approximately uniform\3"distributed on " 7 ""

3. Hence, are approximately uniform distributed on [ \ " ! 7 #3" 3" "

L'Ecuyer suggest combined generators of the form

\ 3 4"

54"

34 " " \ 79. 7 "

with

V \ !\ !3

\7 37 "7 3

3"""



Note that the " \ "4" 3" coefficient implicitly perform the subtraction . Forexample, if ,5 #

" \ " " \ " " \! " 4"3" 3# 344"

#Maximum possible period for such a generator is:

T 7 "

#

#3"

55

5"

which is achieved by the following generator.



EXAMPLE 7.5:1. Selected seed Selected seed \ " #"%(%)$&'# \ " #"%(%)$$$*)" ! #!,

2 Evaluate each individual generator

\ %!!"% \ 79. #"%(%)$&'$"4" "4

\ %! \ 79. #"%(%)$$$**#4" #4692

3. Set

\ \ \ 79. #"%(%)$&'#4" "4" #4"

4. Return

V \ !\ !4"

\#"%(%)$&'$ 4#"%(%)$&'##"%(%)$&'$ 4

4"5. Set and go to Step 2.4 4 "



ARENA RANDOM NUMBER GENERATOR:

E "%!$&)!E )"!(#)E 79. %#*%*'(!)(8 8# 8$

F ('"#F "$(!&)*F 79. %#*%*%%%%$8 8" 8$^ E F 79. %#*%*'(!)(8 8 8

Y ^ !^ !8

^8%#*%*'(!)) 8%#*%*'(!)(%#*%*'(!)) 8

Seed = a six-vector of first three s, sE F8 8



7.4 Tests for Random Numbers

Desirable Properties: Uniformity and Independence1. Uses the Kolmogorov-Smirnic or the Chi-Square Test to Frequency Test:compare the distribution of the set of numbers to a uniform distribution.2. Test the runs up and down or the runs above and below the meanRuns Test:by comparin the actual values to expected values. The Statistics for comparisonis the chi-square test.3. Test the correlation between numbers and compares theAutocorrelation Test:sample correlation to the expected correlation of zero.4. Counts the number of digits that appear between repetitions of aGap Test:particular digit and then uses the Kolmogorov Smirnov test to compare with theexpected sized of the gaps.5. Treat numbers grouped together as a poker hand. Then the handsPoker Test:obtained are compared to what is expected using the chi-square test.

First Entry test for Uniformity. Second through Fifth Entry tests for Independence.



7.4.1. Frequency Tests

1. The Kolmogorov-Smirnov TestTest compares a continuous cdf to an emperical cdf , of the sample ofJB W BRR observations.

Step 1: Determine Order Statistics

V V V ($" # R

Define:

W B 3 " R "! B V

V B V" B V

R

"3R 3 3"

R

Step 2: Compute viaH Q+B lJB W Bl

B R



R(i) R(i+1)R(i-1)

(i-2)/n

i/n

(i+1)/n

(i-1)/n

F(x)=xUniform CDF

H Q+B V H Q+B V " 3 R " 3 R

3 3 "R R

3 3 ,

H Q+B H H



Step 3: Compute for significant level or (for !"! !!& !!" R $&

H H H "## "$' "'$R R R

!"! !!& !!" Step 4: Test Hypothesis

H H W B JBH H W B JB Accept: No Difference between and Reject: No Difference between and RR



2. The Chi-square TestThe chi-square test uses the sample statistic

;!#3"

83 3

#

3 S I I

where is the observed number in the -th class, is the expected number inS 3 I3 3the -th class, and is the number of classes.3 8

Step 1: Determine Order Statistics

V V V ($" # R

Step 2: Divided Range in equidistant intervals , such thatV V 8 + , R " 3 3each interval has at least observations.&

Step 3: Calculate for 3 " R

S R W , W + I R J, J+ 3 R 3 R 3 3 3 3



Step 4: Calculate

;!#3"

83 3

#

3 S I I

Step : Determine for significant level , & ;8"#

; ;; ;! 8"# # R! 8"# # R W B JB W B JB Accept: No Difference between and Reject: Difference between and 7.4.2. Runs Test

0.08 0.09 0.230.11 0.16 0.180.02 0.09 0.300.12 0.13 0.29

Both KS and Chi-Square test would accept the above sample as from a uniform,but if we look at the sample above column wise we see that each column isstrictly larger than the other. Hence, independence may be an issue in that case.



1. Runs up and Runs DownThe runs test examines the arrangement of numbers in a sequence to test thehypothesis of independence. A run is defined as a succesion of similar eventspreceded and follow by a different event. Length of the run is the number ofevents that occur in the run

EXAMPLE: Series of coin Tosses

LXXLLXXXLX

Run 1 has length 1. Run 2 has lenght 2, Run 3 has length 2,Run 4 has length 3, Run 5 has length 1, Run 6 has length 1.

TWO CONCERNS in a Runs Test:1. The number of Runs 2. The length of the Runs.

An up run is a sequence of numbers which is succeeded by a larger number. Adown run is a sequence of numbers each of which is succeeded by a smallernumber.



0.87 - 0.08 + 0.08 +0.15 + 0.18 + 0.93 -0.23 + 0.23 + 0.15 +0.45 + 0.36 + 0.96 -0.69 - 0.42 + 0.26 +0.32 - 0.55 + 0.84 -0.3 - 0.63 + 0.28 +

0.19 + 0.72 + 0.79 -0.24 - 0.89 + 0.36 +0.18 + 0.91 0.570.65 +0.82 +0.93 -0.22 +0.81

Runs 8 Runs 1 Runs 9Runs up 4 Runs up 1 Runs up 5Runs Down 4 Runs Down 0 Runs Down 4

Example 1 Example 2 Example 3

Example 2 and Example 3 are two extremes. If is the number of observation,Rthe maximum number of runs is and the minimum number of runs is . If R " " +is the number of runs in an independent sequence of uniform numbers, the meanand the variance of is given by+



. 5+ #+ #R " "'R #*

$ *!

For , the distribution of is well approximated by a Normal distribution,R #! +R . 5+ #+ .

STEP 1: Calculate number of runs +

STEP 2: Calculate

^ + + #R " "'R #*$ *!.

5+

+

STEP 3: Determine acceptence region from a distributionD D R! " # #

STEP 4:

^ D D ^ D D # ## # Accept: IndependenceReject: Independence



2. Runs above and below the meanAcceptance of the indepence following the previous test is necessary but notsufficient condition. An independent sequence should not contain long runs withobservations above the mean and below the mean.

Let the total number of runs (above or below the mean) and be the number, 8"of observations above the mean and be the number of observations below the8#mean. Note that, the maximum number of runs is and the minimumR 8 8" #number of runs is "

Given and 8 8" #

. 5, " # " # " ##, # #8 8 " #8 8 #8 8 RR # R R "

and if or is approximately normal distributed.8 #! 8 #! ," #

STEP 1: Calculate number of runs ,



STEP 2: Calculate

^ , + .

5,

,

#8 8R #

"

#8 8 #8 8 RR R"

" #

" # " ##

STEP 3: Determine acceptence region from a distributionD D R! " # #

STEP 4:

^ D D ^ D D # ## # Accept: IndependenceReject: Independence



Homework 1:Implement the in these notes for a randomThe Kolmogorov-Smirnov Test sample of 100 numbers generated by the RAND() function in EXCEL and test thehypothesis whether we accept or reject uniformity of the random numbersgenerated by the RAND() function.

Homework 2:Implement the in these notes for a random sample of 100Chi-square Test numbers generated by the RAND() function in EXCEL and test the hypothesiswhether we accept or reject uniformity of the random numbers generated by theRAND() function. Use 10 bins (or classes).

randomnumbergeneration

Documents

discrete event simulation

false random numbers

pseudo random numbers

van dorp

banks carson

nelson chapter

random number generation7

mean of generated numbers