statistics, data analysis, and simulation – ss 2015 · dr. michael o. distler statistics, data...
TRANSCRIPT
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 1 / 25
Mainz, May 7, 2015
Statistics, Data Analysis, andSimulation – SS 2015
08.128.730 Statistik, Datenanalyse undSimulation
Dr. Michael O. Distler<[email protected]>
2. Random Numbers
2.1 Why random numbers:
SimulationSamplingNumerical analysisComputer programmingDecision makingCryptographyAestheticsRecreation
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 2 / 25
2.2 Number representation
unsigned integernibble 4 bit 0 . . . 15
byte, char 8 bit 0 . . . 255short word 16 bit 0 . . . 65 535
word, int 32 bit 0 . . . 4 294 967 295long word 64 bit 0 . . . 1.8446744 · 1019
signed integernibble 4 bit −8 . . . 7
byte, char 8 bit −128 . . . 127short word 16 bit −32 768 . . . 32 767
word, int 32 bit −2 147 483 648 . . . 2 147 483 647floating-point format
float (32-bit)double (64-bit)
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 3 / 25
Binary floating-point format
sign exponent (8 bits) fraction (23 bits)
02331
0 0 1 1 1 1 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 = 0.1562530 22 (bit index)
single precision double precisionwidth 32 bits 64 bits
exponent 8-bit 11-bitfraction 23-bit 52-bit
furthest from zero ±3.403 · 1038 ±1.798 · 10308
closest to zero ±1.175 · 10−38 ±2.225 · 10−308
denormalized ±1.401 · 10−45 ±4.941 · 10−324
gap ±1.192 · 10−7 ±2.220 · 10−16
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 4 / 25
2.3 Random Number Generators
1927 L.H.C. Tippett published 40,000 random digits“taken at random from census reports”
1939 M.G. Kendall and B. Babington-Smith produceda table of 100,000 random digitsusing a mechanical random number generator
1946 John von Neumann first suggested the “middle-square”method. His idea was to take the square of the previousrandom number and to extract the middle digits;for example, if we are generating 10-digit numbers:rj+1 = (r2
j div 100,000)mod 10,000,000,000r0 = 5772156649, r2
0 = 33317 7923805949︸ ︷︷ ︸r1=7923805949
09201
Von Neumann’s original “middle-square method” has actuallyproved to be a comparatively poor source of random numbers.The danger is that the sequence tends to get into a rut, a shortcycle of repeating elements. For example, if zero ever appearsas a number of the sequence, it will continually perpetuateitself.
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 5 / 25
Random Number Generators
There is another fairly obvious objection to the “middle-square”technique: how can a sequence generated in such a way berandom, since each number is completely determined by itspredecessor? The answer is that this sequence isn’t random,but it appears to be.Sequences generated in a deterministic way such as this areusually called pseudo-random or quasi-random sequences inthe technical literature, but here we will make a distinction:
pseudo-random numbers should pass the samestatistical tests as true random numbers doquasi-random numbers or sequences can betransformed like random numbers but should only be usedfor MC integration.
Xj+1 = f (Xj ,Xj−1, . . . ,X1)
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 6 / 25
2.3.1 The Linear Congruential Method
By far the most popular random number generators in usetoday are special cases of the following scheme, introduced byD.H. Lehmer in 1949.We choose four “magic numbers”:
m, the modulus; 0 < m.a, the multiplier; 0 ≤ a < m.c, the increment; 0 ≤ c < m.
X0, the starting value; 0 ≤ X0 < m.The desired sequence of random numbers is then obtained bysetting
Xn+1 = (a Xn + c)mod m, n ≥ 0
For example, the sequence obtained when m = 10 andX0 = a = c = 7 is
7,6,9,0,7,6,9,0, . . .
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 7 / 25
The Linear Congruential Method Generator (LCG)
The following theorem makes it easy to tell if the maximumperiod is achieved. The proof can be found in: Donald E.Knuth: The Art of Computer Programming, Vol. 2
Theorem: The linear congruential sequence defined by m, a,c, and X0 has period length m if and only if
i) c is relatively prime to m; (“teilerfremd”)ii) b = a− 1 is a multiple of p, for every prime p dividing m;iii) b is a multiple of 4, if m is a multiple of 4.
Try m = 16, a = 5, c = 3, X0 = 0:
Xn+1 = (5 Xn + 3)mod 16
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 8 / 25
LCGs parameters in common use
Source m a c output bitsNumerical Recipes 232 1664525 1013904223Borland C/C++ 232 22695477 1 bits 30..16 in rand(),
bits 30..0 in lrand()glibc (used by GCC) 231 1103515245 12345 bits 30..0ANSI C: Watcom, . . . 231 1103515245 12345 bits 30..16Borland Delphi, Virtual Pascal 232 134775813 1 bits 63..32 of (seed * L)Microsoft Visual/Quick C/C++ 232 214013 2531011 bits 30..16
(343FD16) (269EC316)Microsoft Visual Basic (≤v6) 224 1140671485 12820163
(43FD43FD16) (C39EC316)MMIX by Donald Knuth 264 6364136223846793005
1442695040888963407VAX’s MTH$RANDOM,old versions of glibc 232 69069 1Java’s java.util.Random 248 25214903917 11 bits 47...16LC53 in Forth 232 − 5 232 - 333333333 0
Source: Wikipedia
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 9 / 25
2.3.2 Multiplicative congruential method
If c = 0, the generator is often called a multiplicativecongruential generator, or Lehmer RNG.
Xn+1 = (a Xn)mod m
Advantage: faster algorithmDisadvantage: no Zero, possibly shorter period
Definition: When a is relatively prime to m, the smallest integerλ for which aλ mod m = 1 is conventionally called the order of amodulo m. Any such value of a that has the maximum possibleorder modulo m is called a primitive element modulo m.
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 10 / 25
Multiplicative congruential generators
Let λ(m) denote the order of a primitive element, i.e., themaximum possible order, modulo m.
λ(2) = 1λ(4) = 2λ(2e) = 2e−2, e > 2λ(pe) = pe−1 (p − 1), prime p > 2
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 11 / 25
Multiplicative congruential generators
Theorem: The number a is a primitive element modulo pe ifand only if
1 pe = 2, a is odd;or pe = 4, a mod 4 = 3;or pe = 8, a mod 8 = 3,5,7;or p = 2, e ≥ 4, a mod 8 = 3,5;
2 p is odd, e = 1, a 6= 0(modulo p), anda(p−1)/q 6= 1(modulo p) for any prime divisor q of p − 1;
3 p is odd, e > 1, a satisfies (2), and a(p−1)/q 6= 1(modulo p2)
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 12 / 25
Multiplicative congruential generators
Theorem: The maximum period possible when c = 0 is λ(m).This period is achieved if
1 X0 is relatively prime to m;2 a is a primitive element modulo m
Note that we can obtain a period of length m − 1 if m is prime;this is just one less than the maximum length, so for all practicalpurposes such a period is as long as we want.
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 13 / 25
Techniques for producing numbers from variousdistributions
The inverse transform sampling method:
Let X be a random variable whose distribution can bedescribed by the cumulative distribution function F .We want to generate values of X which are distributedaccording to this distribution.
The inverse transform sampling method works as follows:
1 Generate a random number u from the standard uniformdistribution in the interval [0,1].
2 Compute the value x such that F (x) = u.3 Take x to be the random number drawn from the
distribution described by F.
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 14 / 25
Inverse transform sampling
Generator gives:
0 ≤ Xn < m ↪→ 0 ≤ Xn
m′< 1
Uniform distribution: U(0,1)
Transformation:
f (x)dx = U(0,1)du
CDF:∫ x
−∞f (t)dt = F (x) = u
x = F (−1)(u)
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 15 / 25
Acceptance-Rejection Method
Suppose we want to generate samples from a density f definedon some set X.
Let g be a density on X from which we know how to generatesamples and with the property that
f (x) ≤ cg(x)
for some constant c.
1 generate X from distribution g.2 generate U from U(0,1).3 If (U ≤ f (X)
cg(x))
return Xotherwise
go to Step 1.The acceptance-rejection method for sampling from density fuses candidates from density g.
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 16 / 25
press any key
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 17 / 25
2.4. Qualität von Generatoren
2.4.1 SpektraltestBilde Paare aus benachbarten Zahlen
(xj , xj+1) j = 0,1, . . . ,n − 1
Darstellung als Punkte in einem 2dim kartesischenKoordinatensystem:
a = 3, m = 7 : 1,3,2,6,4,5,1, . . .
(1,3), (3,2), (2,6), (6,4), (4,5), (5,1)
Punkte eines MLCG bilden regelmäßiges Gitter.
Warum? Im Wertebereich 0 ≤ xj < m gibt es m2 Zahlenpaare.MLCG liefert aber nur m − 1 Zahlenpaare
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 18 / 25
Beispiele:
0
1
2
3
4
5
6
0 1 2 3 4 5 6
x i+
1
xi
Spektraltest
a=3 m=7
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 19 / 25
Beispiele:
0
10
20
30
40
50
60
70
80
90
0 10 20 30 40 50 60 70 80 90
x i+
1
xi
Spektraltest
a=29 m=97
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 20 / 25
Beispiele:
0
10
20
30
40
50
60
70
80
90
0 10 20 30 40 50 60 70 80 90
x i+
1
xi
Spektraltest
a=23 m=97
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 21 / 25
Beispiele:
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
x i+
1
xi
Spektraltest
a=29 m=97
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 22 / 25
Beispiele:
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
x i+
1
xi
Spektraltest
a=23 m=97
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 23 / 25
Umrechnung auf Gitter 0 ≤ xjm < 1.
Voll besetztes Gitter hat Linienabstand d = 1m
Unser Gitter hat bei gleichmäßiger Verteilung bestenfalls:
d ≈ m−1/2 für 2 Dimensionen
Ungleichmäßige Abstände:
d � m−1/2
Theoretische Überlegungen liefern obere Grenzen für diekleinstmöglichen Gitterabstände in t Dimensionen:
dt ≥ d∗t = ct m−1/t
c2 = 4√
3/4, c3 = 6√
1/2, c4 = 4√
1/2, c5 = 2−0.3
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 24 / 25
2.4.2 Test auf gleichmäßige Verteilung
Das Intervall [0,1] wird in k gleiche Unterintervalle derLänge 1/k unterteilt.N Zufallszahlen werden erzeugt.Ni fallen in das Unterintervall i .∑
Ni = N, 〈Ni〉 = Nk ,
k∑i=1
(Ni − N/k)2
N/k= χ2
sollte einer χ2-Verteilung mit (k − 1) Freiheitsgradenfolgen.
Dr. Michael O. Distler <[email protected]> Statistics, Data Analysis, and Simulation – SS 2015 25 / 25