lecture iii: review of classic quadratic variation …jrojo/pasi/lectures/calderon lec3.pdf · rice...
TRANSCRIPT
![Page 1: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/1.jpg)
Chris Calderon, PASI, Lecture 3
Lecture III: Review of Classic Quadratic Variation Results and Relevance to Statistical Inference in
Finance
Christopher P. CalderonRice University / Numerica Corporation
Research Scientist
![Page 2: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/2.jpg)
Chris Calderon, PASI, Lecture 3
OutlineI Refresher on Some Unique Properties of Brownian Motion
II Stochastic Integration and Quadratic Variation of SDEs
III Demonstration of How Results Help Understand ”Realized Variation” DiscretizationError and Idea Behind “Two Scale Realized Volatility Estimator”
![Page 3: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/3.jpg)
Chris Calderon, PASI, Lecture 3
Part I
Refresher on Some Unique Properties of Brownian Motion
![Page 4: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/4.jpg)
Chris Calderon, PASI, Lecture 3
OutlineFor Items I & II Draw Heavily from:
For Item III Highlight Results from:
Protter, P. (2004) Stochastic Integration and Differential Equations, Springer-Verlag, Berlin.Steele, J.M. (2001) Stochastic Calculus and Financial Applications, Springer, New York.
Barndorff-Nielsen, O. & Shepard, N. (2003) Bernoulli 9 243-265.Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
![Page 5: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/5.jpg)
Chris Calderon, PASI, Lecture 3
Assuming Some Basic Familiarity with Brownian Motion (B.M.)
• Stationary Independent Increments
• Increments are Gaussian
• Paths are Continuous but “Rough” / “Jittery”(Not Classically Differentiable)
• Paths are of Infinite Variation so “Funny” Integrals Used in Stochastic Integration, e.g.
Bt −Bs ∼ N (0, t− s)
tR0
BsdBs =12 (B
2t − t)
![Page 6: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/6.jpg)
Chris Calderon, PASI, Lecture 3
Assuming Some Basic Familiarity with Brownian Motion (B.M.)
The Material in Parts I and II are “Classic”Fundamental & Established Results but Set the Stage to Understand Basics in Part III.
The References Listed at the Beginning Provide a Detailed Mathematical Account of the Material I Summarize Briefly Here.
![Page 7: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/7.jpg)
Chris Calderon, PASI, Lecture 3
A Classic Result Worth Reflecting On
ti = iTN
Implies Something Not Intuitive to Many Unfamiliar with Brownian Motion …
limN→∞
N
i=1(Bti −Bti−1)2 = T
![Page 8: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/8.jpg)
Chris Calderon, PASI, Lecture 3
0 1
0.8
1
1.2
1.4
1.6
1.8
2
t
Zt
Fix Terminal Time, “T”and Refine Mesh by Increasing “N”.
Discretely Sampled Brownian Motion Paths
Shift IC to Zero if “Standard Brownian Motion”Desired
![Page 9: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/9.jpg)
Chris Calderon, PASI, Lecture 3
0 1
0.8
1
1.2
1.4
1.6
1.8
2
t
Zt
Fix Terminal Time, “T”and Refine Mesh by Increasing “N”.
Discretely Sampled Brownian Motion Paths
Suppose You are Interested in Limit of:k(t)
i=1
(Bti −Bti−1)2
k(t) = max{i : ti+1 ≤ t}
ti ≡ i TNδ ≡ T
N
![Page 10: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/10.jpg)
Chris Calderon, PASI, Lecture 3
Different Paths, Same Limit
0 1
0.8
1
1.2
1.4
1.6
1.8
2
t
Z t
θ1
Bt(ω1)
Bt(ω2)
[Proof Sketched on Board]
limN→∞
k(t)Pi=1(Bti −Bti−1)2 = t
![Page 11: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/11.jpg)
Chris Calderon, PASI, Lecture 3
Different Paths, Same Limit
0 1
0.8
1
1.2
1.4
1.6
1.8
2
t
Z t
θ1
Compare To Central Limit Theorem for “Nice”iid Random Variables.
Bt(ω1)
Bt(ω2)
[Proof Sketched on Board]
limN→∞
k(t)Pi=1(Bti −Bti−1)2 = t
![Page 12: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/12.jpg)
Chris Calderon, PASI, Lecture 3
Different Paths, Same Limit
0 1
0.8
1
1.2
1.4
1.6
1.8
2
t
Z t
θ1
Compare/Contrast to Properly Centered and Scaled Sum of “Nice” iid Random Variables:
Bt(ω1)
Bt(ω2)
SN ≡NPi=1
(θ(ωi)−μ)√N
SNL→ N (0, σ2)
limN→∞
NPi=1(Bti −Bti−1)2 ≡ lim
N→∞
NPi=1Yi = T
![Page 13: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/13.jpg)
Chris Calderon, PASI, Lecture 3
Continuity in Time
0 1
0.8
1
1.2
1.4
1.6
1.8
2
t
Z t
θ1
Above is “Degenerate” but Note Time Scaling Used in Our Previous Example Makes Infinite Sum Along a Path Seem Similar to Computing Variance of Independent RV’s
SN ≡NPi=1
(θ(ωi)−μ)√N
SNL→ N (0, σ2)
limN→∞
NPi=1(Bti −Bti−1)2 ≡ lim
N→∞
NPi=1Yi = T
![Page 14: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/14.jpg)
Chris Calderon, PASI, Lecture 3
Let 0 < t ≤ Tπn = {0 ≤ t1 ≤ . . . ti ≤ . . . ≤ tn = t}
P ≡ All Finite Partitionsof [0, t]
B.M. Paths Do Not Have Finite Variation
limn→∞
maxi<n
(ti+1 − ti) = 0
![Page 15: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/15.jpg)
Chris Calderon, PASI, Lecture 3
B.M. Paths Do Not Have Finite Variation
V[0,t](ω) = supπ∈P ti∈π
|Bti+1 −Bti |
Suppose That:
t = limN→∞
NPi=1(Bti −Bti−1)2
Then:P(V[0,t] <∞) > 0
≤ limN→∞
supti∈πN
|Bti −Bti−1 |NPi=1
|Bti −Bti−1 |
![Page 16: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/16.jpg)
Chris Calderon, PASI, Lecture 3
B.M. Paths Do Not Have Finite Variation
t = limN→∞
NPi=1(Bti −Bti−1)2
= 0Tends to Zero Due to a.s. uniform continuity of B.M. Paths
t > 0, so Probability of Finite Variation Must be Zero
≤ limN→∞
supti∈πN
|Bti −Bti−1 |NPi=1
|Bti −Bti−1 |
≤ limN→∞
supti∈πN
|Bti −Bti−1 |V[0,t]
![Page 17: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/17.jpg)
Chris Calderon, PASI, Lecture 3
Part II
Stochastic Integration and Quadratic Variation of SDEs
![Page 18: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/18.jpg)
Chris Calderon, PASI, Lecture 3
Stochastic Integration
P(V[0,t] <∞) = 0
“Finite Variation Term” “Infinite Variation Term”
X(ω, t) =
t
0
b(ω, s)ds+
t
0
σ(ω, s)dBs
![Page 19: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/19.jpg)
Chris Calderon, PASI, Lecture 3
Stochastic Integration
P(V[0,t] <∞) = 0
“Infinite Variation Term” Complicates Interpretting Limit as Lebesgue Integral
X(ω, t) =
t
0
b(ω, s)ds+
t
0
σ(ω, s)dBs
σ Xti(Bti) |Bti+1 −Bti |
![Page 20: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/20.jpg)
Chris Calderon, PASI, Lecture 3
Stochastic Differential Equations (SDEs)
Written in Abbreviated Form:
X(ω, t) =
t
0
b(ω, s)ds+
t
0
σ(ω, s)dBs
dX(ω, t) = b(ω, t)dt+ σ(ω, t)dBt
![Page 21: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/21.jpg)
Chris Calderon, PASI, Lecture 3
Stochastic Differential Equations Driven by B.M.
Adapted and Measurable Processes
dXt = btdt+ σtdBtI will use this shorthand for the above (which is itself shorthand for stochastic integrals…)
dX(ω, t) = b(ω, t)dt+ σ(ω, t)dBt
![Page 22: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/22.jpg)
Chris Calderon, PASI, Lecture 3
Stochastic Differential Equations Driven by B.M.
dXt = 0dt+ 1dBt
dXt = b(Xt, t)dt+ σ(Xt, t)dBt
dXt = btdt+ σtdBtGeneral Ito Process
Diffusion Process
Repackaging Brownian Motion
![Page 23: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/23.jpg)
Chris Calderon, PASI, Lecture 3
Quadratic Variation for General SDEs (Jump Processes Readily Handled)
“Realized Variation”
Quadratic Variation: Defined if the limit taken over any partition exists with the mesh going to zero.
RVπN (t) ≡NPi=1(Xti −Xti−1)2
:={ Vt}
![Page 24: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/24.jpg)
Chris Calderon, PASI, Lecture 3
Stochastic Integration
“Finite Variation Term” “Infinite Variation Term”
Y (ω, t) =
t
0
b0(ω, s)ds+
t
0
σ0(ω, s)dXs
Quadratic Variation of “X” Can Be Used to Define Other SDEs (A “Good Stochastic Integrator”)
![Page 25: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/25.jpg)
Chris Calderon, PASI, Lecture 3
Quadratic Variation Comes Entirely From Stochastic Integral
i.e. the drift (or term in front of “ds” makes) no contribution to the quadratic variation
[quick sketch of proof on board]
X(ω, t) =
t
0
b(ω, s)ds+
t
0
σ(ω, s)dBs
![Page 26: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/26.jpg)
Chris Calderon, PASI, Lecture 3
Stochastic Integration
P(V[0,t] <∞) = 0
“Infinite Variation Term” Can Now Be Dealt With If One Uses Q.V.
X(ω, t) =
t
0
b(ω, s)ds+
t
0
σ(ω, s)dBs
σ Xti(Bti) |Bti+1 −Bti |
![Page 27: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/27.jpg)
Chris Calderon, PASI, Lecture 3
Quadratic Variation and “Volatility”
Xt =
t
0
bsds+
t
0
σsdBs
[X,X]t ≡ Vt =t
0
σ2sdt
Typical Notation for QV
![Page 28: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/28.jpg)
Chris Calderon, PASI, Lecture 3
Quadratic Variation and “Volatility”
[X,X]t := X2t − 2
t
0
Xs−dXs
[X,X]t ≡ Vt =t
0
σ2sdt
For Diffusions
More Generally for Semimartingales(Includes Jump Processes)
![Page 29: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/29.jpg)
Chris Calderon, PASI, Lecture 3
Part III
Demonstration of How Results Help Understand ”Realized Variation”Discretization Error and Idea Behind “Two Scale Realized Volatility Estimator”
![Page 30: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/30.jpg)
Chris Calderon, PASI, Lecture 3
Most People and Institutions Don’t Sample Functions (Discrete
Observations)
RVπN (t) ≡NPi=1(Xti −Xti−1)2
[X,X]t
Assuming Process is a Genuine Diffusion, How Far is Finite N Approximation From Limit?
![Page 31: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/31.jpg)
Chris Calderon, PASI, Lecture 3
Discretization Errors and Their Limit Distributions
Barndorff-Nielsen, O. & Shepard, N. (2003) Bernoulli 9 243-265.
Paper Below Derived Explicit Expressions for Variance for Fairly General SDEs Driven by B.M.
N 1/2¡RVπN (t) − [X,X ]t
¢|σt L→ N (0, C
tR0
σ4sds)
![Page 32: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/32.jpg)
Chris Calderon, PASI, Lecture 3
Discretization Errors and Their Limit Distributions
Barndorff-Nielsen, O. & Shepard, N. (2003) Bernoulli 9 243-265.
Their paper goes over some nice classic moment generating function results for wide class of spot volatility processes (and proves things beyond QV)
N 1/2¡RVπN (t) − [X,X ]t
¢|σt L→ N (0, C
tR0
σ4sds)
![Page 33: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/33.jpg)
Chris Calderon, PASI, Lecture 3
Discretization Errors and Their Limit Distributions
Barndorff-Nielsen, O. & Shepard, N. (2003) Bernoulli 9 243-265.
One Condition That They Assume:δ ≡ t
N
limN→0
δ1/2N
i=1
|σ2ti+δ − σ2ti | = 0
![Page 34: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/34.jpg)
Chris Calderon, PASI, Lecture 3
Discretization Errors and Their Limit Distributions
Good for Many Stochastic Volatility Models … BUT Some Exceptions are Easy to Find (See Board)
δ ≡ t
Barndorff-Nielsen, O. & Shepard, N. (2003) Bernoulli 9 243-265.
N
limN→0
δ1/2N
i=1
|σ2ti+δ − σ2ti | = 0
![Page 35: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/35.jpg)
Chris Calderon, PASI, Lecture 3
What Happens When One Has to Deal with “Imperfect” Data?
Suppose “Price” Process Is Not Directly Observed But Instead One Has:
yti = Xt1 + ²i
Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
![Page 36: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/36.jpg)
Chris Calderon, PASI, Lecture 3
What Happens When One Has to Deal with “Imperfect” Data?
yti = Xt1 + ²i
Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
Due to Fact that Real World Data Does Not Adhere to Strict Mathematical Definition of a Diffusion
![Page 37: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/37.jpg)
Chris Calderon, PASI, Lecture 3
What Happens When One Has to Deal with “Imperfect” Data?
If “observation noise” sequence is i.i.d. (and independent of X) and it is very easy to see problem of estimating quadratic variation (see demostration on board).
yti = Xt1 + ²i
Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
![Page 38: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/38.jpg)
Chris Calderon, PASI, Lecture 3
What Happens When One Has to Deal with “Imperfect” Data?
If “observation noise” sequence is i.i.d. (and independent of X) and it is very easy to see problem of estimating quadratic variation (see demostrationon board). Surprisingly These Stringent Assumption Model Many Real World Data Sets (e.g. See Results from Lecture 1)
yti = Xt1 + ²i
Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
![Page 39: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/39.jpg)
Chris Calderon, PASI, Lecture 3
Simple Case “Bias” Estimate
Compute Realized Variation of Y and Divide by 2N
Then Obtain (Biased) Estimate by Subsampling
Finally Aggregate Subsample Estimates and Remove Bias
Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
![Page 40: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/40.jpg)
Chris Calderon, PASI, Lecture 3
Simple Case “Bias Corrected” Estimate
Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
\[X,X]T = [Y, Y ](avg)T − Ns
N [Y, Y ](all)T
Result Based on “Subsampling”
![Page 41: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/41.jpg)
Chris Calderon, PASI, Lecture 3
Chasing A (Time Series) Dream Obtaining a “Consistent” Estimate with Asymptotic Error Bounds
Zhang, L.. Mykland, P. & Ait-Sahalia,Y. (2005) JASA 100 1394-1411.
Result Above Valid for Simple Observation Noise but Results (by Authors Below) Have Been Extended to Situations More General Than iid Case.
N1/6( \[X,X]T [X,X]T )|σt L→ N³0, C1(VAR(²))
2 +C2TR0
σ4sds´
![Page 42: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/42.jpg)
Chris Calderon, PASI, Lecture 3
Force Force
DNA Melting
Tensions Important in Fundamental Life Processes Such As: DNA Repair and DNA Transcription
Single-Molecule Experiments Not Just a Neat Toy:
They Have Provided Insights Bulk Methods Cannot
![Page 43: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/43.jpg)
Chris Calderon, PASI, Lecture 3
Time Dependent DiffusionsStochastic Differential Equation (SDE):
“Measurement” Noise
Find/Approximate: “Transition Density” / Conditional Probability Density
For Frequently Sampled Single-Molecule Experimental Data, “Physically Uninteresting Measurement Noise” Can Be Large Component of Signal.
dzt = b(zt, t;Θ)dt+ σ(zt;Θ)dWt
yti = zti + ²ti
p(zt, |ys;Θ)
![Page 44: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/44.jpg)
Chris Calderon, PASI, Lecture 3
A Word from the Wise (Lucien Le Cam)• Basic Principle 0: Don’t trust any principle• Principle 1: Have clear in your mind what it is you want
to estimate…
• Principle 4: If satisfied that everything is in order, try first a crude but reliable procedure to locate the general area your parameters lie.
• Principle 5: Having localized yourself by (4), refine the estimate using some of your theoretical assumptions, being careful all the while not to undo what you did in (4).
…
Le Cam L.. (1990) International Statistics Review 58 153-171.
![Page 45: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/45.jpg)
Chris Calderon, PASI, Lecture 3
Example of Relevance of Going Beyond iid Setting
Calderon, J. Chem. Phys. (2007).
Watching Process at Too Fine of Resolution Allows “Messy” Details to Be Statistically Detectable
![Page 46: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/46.jpg)
Chris Calderon, PASI, Lecture 3
Example of Relevance of Going Beyond iid Setting
X-E
[X|X
]t+
dtt
t+dt
r
r ≡ Xn − EPθi [Xn|Xn−1]
Calderon, J. Chem. Phys. (2007).
![Page 47: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/47.jpg)
Chris Calderon, PASI, Lecture 3
Lecture IV: A Selected Review of Some Recent Goodness-of-Fit Tests Applicable to Levy Processes
Christopher P. CalderonRice University / Numerica Corporation
Research Scientist
![Page 48: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/48.jpg)
Chris Calderon, PASI, Lecture 3
OutlineI Compare iid Goodness of Fit Problem to Time Series Case
II Review a Classic Result Often Attributed to Rosenblatt (The Probability Integral Transform)
III Highlight Hong and Li’s “Omnibus” Test
IV Demonstrate Utility and Discuss Some Other Recent Approaches and Open Problems
![Page 49: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/49.jpg)
Chris Calderon, PASI, Lecture 3
Outline
Primary References (Item II & III):
Diebold, F. Hahn J., & Tay, A. (1999) Review of Economics and Statistics 81 661-663.Hong, Y. and Li, H. (2005) Review of Financial Studies 1837-84.
![Page 50: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/50.jpg)
Chris Calderon, PASI, Lecture 3
Goodness-of-Fit Issues .
−5 0 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
p(x)
True and ParametricDensity
![Page 51: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/51.jpg)
Chris Calderon, PASI, Lecture 3
−5 0 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
P(x
)Goodness-of-Fit in Random Variable
Case (No Time Evolution)
−5 0 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
xp(
x)
![Page 52: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/52.jpg)
Chris Calderon, PASI, Lecture 3
−5 0 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
p(x)
−5 0 5 10 150
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
p(x)
Goodness-of-Fit Issues
![Page 53: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/53.jpg)
Chris Calderon, PASI, Lecture 3
−5 0 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
p(x)
−5 0 5 10 150
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
p(x)
−5 0 50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
P(x
)
−5 0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
P(x
)
Goodness-of-Fit in Random Variable Case (No Time Evolution)
![Page 54: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/54.jpg)
Chris Calderon, PASI, Lecture 3
−5 0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
x
P(x
)
P1
P2 F2
F1
Note:
Truth and approximate distribution changing with (known) time index
Time series only get “one”sample from each evolving distribution
Goodness-of-Fit in Time Series
![Page 55: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/55.jpg)
Chris Calderon, PASI, Lecture 3
Time Series: Residual and Extensions
Problem with Using Low Order Correlation Shown In Lecture 2.
Can we Make Use of All Moments?
ri ≡ Xi − Eθ[Xi|Xi−1]
![Page 56: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/56.jpg)
Chris Calderon, PASI, Lecture 3
Time Series: Residual and Extensions
Problem with Using Low Order Correlation Shown In Lecture 2.
Can we Make Use of All Moments?
ri ≡ Xi − Eθ[Xi|Xi−1]
Zi =XiR−∞
p(X|Xi−1; θ0)dXYES. With the Probability Integral Transform (PIT)
Also Called Generalized Residuals
![Page 57: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/57.jpg)
Chris Calderon, PASI, Lecture 3
Test the Model Against Observed Data:
Time Series (Noisy, Correlated, and Possibly Nonstationary)
θ
Probability Integral Transform / Rosenblatt Transform /“Generalized Residual”
Zi ∼ U [0, 1]If Assumed Model Correct, Residuals i.i.d. with Known Distribution:: Formulate Test Statistic::
Q = H({Z1, Z2, . . . ZT })
Zi = G(Xi;Xi−1, θ)
{X1, X2, . . . XN}
![Page 58: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/58.jpg)
Chris Calderon, PASI, Lecture 3
Simple Illustration of PIT[Specialized to Markov Dynamics]
Start time series random variables characterized by :
Introduce (strictly increasing) transformation:
Transformation introduces “new” R.V. => New distribution
“Truth” density in “X” (NOTE: no parameter)
PIT transformation:
Want expression for “new” density in terms of “Z” Monotone Function of X
Xn ∼ P(Xn|Xn−1)Zn = h(Xn; θ)
Zn ∼ F(Zn|Zn−1; θ)p(Xn|Xn−1) ≡ dP(Xn|Xn−1)
dXn
Zn ≡ h(Xn) :=XnR−∞
f(X 0|Xn−1; θ)dX 0
dF(Zn|Zn−1;θ)dZn
= dP(Xn|Xn−1)dXn
dXn
dZn
= dP(Xn|Xn−1)dXn
1dZndXn
= p(Xn|Xn−1) 1f(Xn|Xn−1;θ)
?= 1
![Page 59: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/59.jpg)
Chris Calderon, PASI, Lecture 3
Q(j) ≡³h(N − j)M1(j)− hA0
´/V
120
gj(z1, z2) ≡1
N − jNX
τ=j+1
Kh(z1, Zτ )Kh(z2, Zτ−j).
M1(j) ≡1Z0
1Z0
³gj(z1, z2)− 1
´2dz1dz2.
Location (Depends on Bandwidth)
Scale
![Page 60: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/60.jpg)
Chris Calderon, PASI, Lecture 3
Q(j) ≡³h(N − j)M1(j)− hA0
´/V
120
gj(z1, z2) ≡1
N − jNX
τ=j+1
Kh(z1, Zτ )Kh(z2, Zτ−j).
M1(j) ≡1Z0
1Z0
³gj(z1, z2)− 1
´2dz1dz2.
![Page 61: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/61.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−1.7
−1.6
−1.5
−1.4
−1.3
−1.2
−1.1
−1
−0.9
−0.8
Time [ns]
yStationary Time Series [No Time Dependent Forcing]
![Page 62: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/62.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−1.7
−1.6
−1.5
−1.4
−1.3
−1.2
−1.1
−1
−0.9
−0.8
Time [ns]
yStationary Time Series?
![Page 63: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/63.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−1.7
−1.6
−1.5
−1.4
−1.3
−1.2
−1.1
−1
−0.9
−0.8
Time [ns]
yStationary Time Series?
![Page 64: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/64.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−1.7
−1.6
−1.5
−1.4
−1.3
−1.2
−1.1
−1
−0.9
−0.8
Time [ns]
yDynamic Rules “Changing” Over Time
(Simple 1D SDE Parameters “Evolving” )
![Page 65: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/65.jpg)
Chris Calderon, PASI, Lecture 3
“Subtle” CouplingCalderon, J. Phys. Chem. B (2010).
![Page 66: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/66.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−1.7
−1.6
−1.5
−1.4
−1.3
−1.2
−1.1
−1
−0.9
−0.8
Time [ns]
yEstimate “Simple” SDE in Local Time Window. Then Apply Hypothesis Tests To Quantify Time of Validity
dΦt = (A+ BΦt)dt+ (C +DΦt)dWt
![Page 67: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/67.jpg)
Chris Calderon, PASI, Lecture 3
Estimate Model in Window “0”; Keep Fixed and Apply to Other Windows
![Page 68: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/68.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−1.7
−1.6
−1.5
−1.4
−1.3
−1.2
−1.1
−1
−0.9
−0.8
Time [ns]
yWindow 0 Window 2 Window 6 Window 8
![Page 69: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/69.jpg)
Chris Calderon, PASI, Lecture 3
Each Test Statistic Uses ONE Short PathCalderon, J. Phys. Chem. B (2010).
![Page 70: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/70.jpg)
Chris Calderon, PASI, Lecture 3
Stationary Time Series?
Subtle Noise Magnitude Changes due Unresolved Degrees of Freedom (Check Validity of a “Born-Oppenheimer” Type Proxy)
dΦt = (A+ BΦt)dt+ (C +DΦt)dWt
![Page 71: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/71.jpg)
Chris Calderon, PASI, Lecture 3
Stationary Time Series?
Subtle Noise Magnitude Changes due Unresolved Degrees of Freedom (Check Validity of a “Born-Oppenheimer” Type Proxy)
Evolving “X” Influences Higher Order Moments of “Y”
![Page 72: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/72.jpg)
Chris Calderon, PASI, Lecture 3
Stationary Time Series?
Subtle Noise Magnitude Changes due Unresolved Degrees of Freedom (Check Validity of a “Born-Oppenheimer” Type Proxy)
![Page 73: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/73.jpg)
Chris Calderon, PASI, Lecture 3
Estimate “Simple” SDE in Local Time Window. Then Apply Hypothesis Tests To Quantify Time of Validity
![Page 74: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/74.jpg)
Chris Calderon, PASI, Lecture 3
Window 0 Window 2 Window 6 Window 8
![Page 75: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/75.jpg)
Chris Calderon, PASI, Lecture 3
“Subtle” Noise Magnitude ChangesCalderon, J. Phys. Chem. B (2010).
![Page 76: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/76.jpg)
Chris Calderon, PASI, Lecture 3
Frequentist vs. Bayesian Approaches
• Concept of “Residual” Unnatural in Bayesian Setting. All Information Condensed into Posterior (Time Residual Can Help in Non-ergodic Settings)
• Uncertainty Information Available in Both Under Ideal Models with Heavy Computation (Bayesian Methods Need “Prior” Specification)
• Test Against Many Alternatives in a FrequentistSetting (Instead of a Handful of Candidate Models as is Done in Bayesian Methods)
![Page 77: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/77.jpg)
Chris Calderon, PASI, Lecture 3
Several IssuesEvaluate Z at Estimated Parameter (Their Limit Distribution Requires Root N Consistent Estimator to Get [Asymptotic ] Critical Values)
Asymptopia is a Strange Place [Multiplying by Infinity Can Be Dangerous When Numerical Proxies are Involved]
In Nonstationary Case, Initial Condition Distribution May Be Important but Often Unknown.
![Page 78: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/78.jpg)
Chris Calderon, PASI, Lecture 3
Testing Surrogate Models
−4 −2 0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Q
F(Q)
OD Δ t=0.15 psOD Δ t=0.30 psOD Δ t=0.45 psOU Δ t=0.15 psOU Δ t=0.30 psOU Δ t=0.45 psNull
(a)
One Times Series Reduces to One Test Statistic. Empirical CDF Summarizes Results from 100 Time Series
∆t =0.15ps
∆t =0.30ps0.45 ps
Hong & Li, Rev. Financial Studies, 18 (2005). Chen, S, Leung, D. & Qin,J., Annals of Statistics, 36(2008). Ait-Sahalia, Fan, Peng, JASA (in press) Calderon & Arora, J. Chemical Theory & Computation 5 (2009).
![Page 79: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/79.jpg)
Chris Calderon, PASI, Lecture 3
A Sketch of Local to Global
or
Fitting Nonlinear SDEs Without A Priori Knolwedge of Global
Parametric Model
![Page 80: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/80.jpg)
Chris Calderon, PASI, Lecture 3
Local Time Dependent Diffusion ModelsCalderon, J. Chem. Phys. 126 (2007).
Approximate Locally by Low Order Polynomials, e.g.
Can use Physics-based Proxies e.g. Overdamped Langevin Dynamics
dzt = b(zt, t;Θ)dt+ σ(zt;Θ)dWt
b(z) ≈A+Bz + f(t)σ(z) ≈C +Dz
σ(z) ≈C +Dzb(z) ≈(C +Dz)2/(2kBT )(A+Bz + f(t))
![Page 81: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/81.jpg)
Chris Calderon, PASI, Lecture 3
Maximum Likelihood
For given model & discrete observations, find the maximum likelihood estimate:
Special case of Markovian Dynamics:
“Transition Density” (aka Conditional Probability Density)
θ ≡ maxθ p(z0;Θ)p(z1|z0; θ) . . . p(zT |zT−1; θ)
θ ≡ maxθ p(z0, . . . , zT ; θ)
![Page 82: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/82.jpg)
Chris Calderon, PASI, Lecture 3
After Estimating Acceptable (Not Rejected)
Local Models, What’s Next?
• Quality of Data: “Variance” (MC Simulation in Local Windows)
• Then Global Fit (Nonlinear Regression)
![Page 83: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/83.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
Time
State Dependent Noise Would “Fool” Wavelet Type MethodsZ
(Sta
te S
pace
)
Note: Noise Magnitude Depends on State
And the State is Evolving in a Non-Stationary Fashion
dZt = k(vpullt− Zt)dt+ σ(Zt)dWt
![Page 84: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/84.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
0 0.5 1 1.5 20
0.5
1
1.5
2
Time
Z (S
tate
Spa
ce)
σ(Z)
Z (State Space)
Note: Noise Magnitude Depends on State
And the State is Evolving in a Non-Stationary Fashion
![Page 85: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/85.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
0 0.5 1 1.5 20
0.5
1
1.5
2
Time
Z (S
tate
Spa
ce)
Z (State Space)
σ(Z)
Local Time Window
Note: Noise Magnitude Depends on State
And the State is Evolving in a Non-Stationary Fashion
![Page 86: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/86.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
Time
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
σ(Z)Local Time Window
Corresponding State Window
![Page 87: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/87.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
Time
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Global Nonlinear Function of Interest (the “truth”)
σ(Z)
![Page 88: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/88.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
Time
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Point Estimate Comes From Local Parametric Model (Parametric Likelihood Inference Tools Available)
σ(Z)
![Page 89: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/89.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
Time
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Point Estimate Comes From Local Parametric Model (Parametric Likelihood Inference Tools Available)
σ(Z)
σ(z) ≈ C +D(z − z0)
C
![Page 90: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/90.jpg)
Chris Calderon, PASI, Lecture 3
Maximum Likelihood
For given model & discrete observations, find the maximum likelihood estimate:
Special case of Markovian Dynamics:
“Transition Density” (aka Conditional Probability Density)
θ ≡ maxθ p(z0;Θ)p(z1|z0; θ) . . . p(zT |zT−1; θ)
θ ≡ maxθ p(z0, . . . , zT ; θ)
![Page 91: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/91.jpg)
Chris Calderon, PASI, Lecture 3
0
0.5
1
1.5
2
00.511.522.530 0.25 0.5 0.75 10
0.5
1
1.5
2
Time
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Zo
Noisy Point Estimates (finite discrete time series sample uncertainty)
Spatial Derivative of Function of Interest ∂σ(Z)/∂Zσ(Z)
σ(z) ≈ C +D(z − z0)
C D
![Page 92: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/92.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Idea Similar in spirit to J. Fan, Y. Fan, J. Jiang, JASA 102(2007) except uses fully parametric MLE (no local kernels) in discrete local state windows
Time
From Path to Point
σ(Z)
C
![Page 93: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/93.jpg)
Chris Calderon, PASI, Lecture 3
0 0.25 0.5 0.75 10
0.5
1
1.5
2
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Validity of Local Models Can Be Tested Using Generalized Residuals Available Using Transition Density with Local Parametric Approach
σ(Z)
HOWEVER: Not Willing To Assume Stationarity in Windows. (Completely Nonparametric and Fourier Analysis Problematic). “Subject Specific Function Variability” of Interest
![Page 94: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/94.jpg)
Chris Calderon, PASI, Lecture 3
Noisy Point Estimates(Obtained Using Local Maximum Likelihood)
Transform
“X vs t” Data to
“X vs Y=F(X)”
Estimate Nonlinear Function via Regression
0
0.25
0.5
0.75
10 0.5 1 1.5 2
Z oZ (State Space)
0 0.5 1 1.5 20
0.5
1
1.5
2
2.5
Function of Interest
Tim
eσ(Z)
![Page 95: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/95.jpg)
Chris Calderon, PASI, Lecture 3
0 0.5 1 1.5 20
0.5
1
1.5
2
2.5
3
0
0.25
0.5
0.75
10 0.5 1 1.5 2
Z oZ (State Space)
Spatial Derivative of Function of Interest
Tim
e
Noisy Point Estimates(Obtained Using Local Maximum Likelihood)
Transform
“X vs t” Data to
“X vs Y’=G(X)”
Estimate Nonlinear Function via Regression
∂σ(Z)/∂Z
![Page 96: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/96.jpg)
Chris Calderon, PASI, Lecture 3
0 0.5 1 1.5 20
0.5
1
1.5
2
2.5
0
0.25
0.5
0.75
10 0.5 1 1.5 2Z (State Space)
Tim
e
Use Noisy Diffusive Path to Infer Regression Data
σ(Z)
(1)
(3)
{zm, C, D}Mm=1
(2)
![Page 97: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/97.jpg)
Chris Calderon, PASI, Lecture 3
Same Ideas Apply to Drift Function:
Can Also Entertain Simultaneous Regression:
{zm, A, B, C, D}Mm=1
{zm, A, B}Mm=1
![Page 98: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/98.jpg)
Chris Calderon, PASI, Lecture 3
0 0.5 1 1.5 20
0.5
1
1.5
2
2.5
SplineFunction Estimate
0
0.25
0.5
0.75
10 0.5 1 1.5 2Z (State Space)
Function of Interest
Tim
e
Variability of Between Different Functions Gives Information About Unresolved Degrees of Freedom Reliable Means for Patching Windowed Estimates Together is Desirable
{zm, σ(zm), dσ(z)dz |zm}Mm=1σ(Z)
![Page 99: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/99.jpg)
Chris Calderon, PASI, Lecture 3
Open Mathematical Questions for Local SDE Inference
How Can One Efficiently Choose “Optimal” Local Window Sizes” ?
Hypothesis Tests Checking Markovian Assumption with Non-Stationary Data? (“Omnibus” goodness-of-fit tests of Hong and Li [2005] based on probability integral transform currently employed)
![Page 100: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/100.jpg)
Chris Calderon, PASI, Lecture 3
Gramicidin A: Commonly Studied Benchmark36,727 Atom Molecular Dynamics Simulation
z
Potassium Ion
Explicitly Modeled Water Molecules
Gramicidin A Protein
Explicitly Modeled Lipid Molecules
Compute Work: Integral of “Force time Distance”
![Page 101: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/101.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−10
0
10
20
30
40
50
τ [ns]
W [k
T]
PMF and Non-equilibrium Work
Variability Induced byConformation Initial Condition (variability between curves)
and
“Thermal” Noise (Solvent Bombardment, Vibrational Motion, etc. quantified by tube width)
Nonergodic Sampling:
Pulling “Too Fast”Molecular Dynamics Trajectory “i”
SDE Trajectory “Tube i”
![Page 102: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/102.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−10
0
10
20
30
40
50
τ [ns]
W [k
T]
PMF and Non-equilibrium WorkMolecular Dynamics Trajectory “i”
SDE Trajectory “Tube i” Distribution at
Final Time is non-Gaussian and can be viewed asMIXTURE of distributions
![Page 103: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/103.jpg)
Chris Calderon, PASI, Lecture 3
Importance of Tube VariabilityMixture of Distributions
Thermal Noise
Conformational Noise
![Page 104: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/104.jpg)
Chris Calderon, PASI, Lecture 3
Diffusion Coefficient / Effective Friction State DependentOther Kinetic Issues
NOTE: Noise Intensity Differs in the Two Sets of Trajectories
![Page 105: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/105.jpg)
Chris Calderon, PASI, Lecture 3
3.00 2.25 1.50 0.75 010
20
30
40
50
60
λ[A]
W[k
T]
3.00 2.25 1.50 0.75 0.00
10
20
30
40
50
60
λ[A]
W[k
T]
PMF and Non-equilibrium WorkMolecular Dynamics Trajectory “j”
SDE Trajectory “Tube j”
Molecular Dynamics Trajectory “Tube i”
[each tube starts with same q’s but different p’s]
![Page 106: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/106.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1
0.8
1
1.2
1.4
1.6
1.8
2
t
Zt
Use one path, discretely sampledat many time points, to obtain acollection of θs (one for each path)
θ3
θ1
θ2θ4
θ5
Vertical lines reprentobservation times
Pathwise ViewEnsemble Averaging Problematic Due to Nonergodic / Nonstationary Sampling
Collection of Models Another Way of Accounting for “Memory”
θi
![Page 107: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/107.jpg)
Chris Calderon, PASI, Lecture 3
Potential of Mean Force SurfaceHummer & Szabo, PNAS 98 (2001).
Minh & Adib, PRL 100 (2008).
Several issues arise when one attempts to use this surface to approximate dynamics in complex systems.
![Page 108: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/108.jpg)
Chris Calderon, PASI, Lecture 3
0 0.2 0.4 0.6 0.8 1−10
0
10
20
30
40
50
τ [ns]
W [k
T]
PMF and Non-equilibrium WorkMolecular Dynamics Trajectory “i”
SDE Trajectory “Tube i” Distribution at
Final Time is non-Gaussian and can be viewed asMIXTURE of distributions
![Page 109: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/109.jpg)
Chris Calderon, PASI, Lecture 3
z[A]
U[k
T]
−15 −10 −5 0 5 10 150
5
10
15
20
25
Allen et al. (Ref. 27)Bastug et al. (Ref. 28)FR (Ref. 16)FR (w/ SPA Data)SPA
PMF of a “Good” CoordinateCalderon, Janosi & Kosztin, J. Chem. Phys. 130 (2009).
PMF and Confidence Band Computed Using 10 Stretching & Relaxing Noneqall-atom MD Paths
PMFsComputed Using Umbrella Sampling
![Page 110: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/110.jpg)
Chris Calderon, PASI, Lecture 3
Penalized Splines(See Ruppert, Wand, Carroll’s Text: Semiparametric Regression, Cambridge
University Press, 2003.)
Observed or Inferred Data
Spline Basis with K<<m (tall and skinny design matrix)
where β ≡ (η, ζ)“Fixed Effects”“Random Effects”
Observation Error
y = {f(x1), . . . , f(xm), ∂f(x1), . . . , ∂f(xm)}+ ²,
yi = η0 + η1xi . . . ηpxpi +
KXj=1
ζjBj(xi),
![Page 111: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/111.jpg)
Chris Calderon, PASI, Lecture 3
Penalized Splines(See Ruppert, Wand, Carroll’s Text: Semiparametric Regression, Cambridge
University Press, 2003.)
Observed or Inferred Data
Spline Basis with K<<m (tall and skinny design matrix)
P-Spline Problem (Flexible Penalty)
where
y = {f(x1), . . . , f(xm), ∂f(x1), . . . , ∂f(xm)}+ ²,
yi = η0 + η1xi . . . ηpxpi +
KXj=1
ζjBj(xi),
ky − Cβk22 + αkDβk22 β ≡ (η, ζ)
![Page 112: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/112.jpg)
Chris Calderon, PASI, Lecture 3
Penalized Splines Using Derivative Information
Poorly Conditioned Design Matrix
CPuDI :=
⎛⎜⎜⎜⎜⎜⎜⎜⎜⎝
1 x1 . . . xp1 (κ1 − x1)p+ . . . (κK − x1)p+
......
...1 xm . . . x
pm (κ1 − xm)p+ . . . (κK − xm)p+
0 1 . . . pxp−11 p(κ1 − x1)p−1+ . . . p(κK − x1)p−1+...
......
0 1 . . . pxp−1m p(κ1 − xm)p−1+ . . . p(κK − xm)p−1+
⎞⎟⎟⎟⎟⎟⎟⎟⎟⎠.
![Page 113: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/113.jpg)
Chris Calderon, PASI, Lecture 3
Pick Smoothness: Solve Many Least Squares Problems
P-Spline Problem (Flexible Penalty)
Choose with Cost Function (GCV). This object has nice “mixed model” interpretation as ratio of observation variance / random effects variance
Selection Requires Traces and Residuals for Each Candidate
α
α
ky − Cβk22 + αkDβk22
![Page 114: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/114.jpg)
Chris Calderon, PASI, Lecture 3
PSQR Penalized Splines using QR(Calderon, Martinez, Carroll, Sorensen)
Allows Fast and Numerically Stable P-spline Results. Exactly Rank Deficient “C” Can Be Treated
Can Process Many Batches of Curves (Facilitates Solving Many GCV Type Optimization Problems).
Data Mining without User Intervention
(e.g. Closely Spaced and/or Overlapping Knots Are Not Problematic)
Let Regularization Be Handled By Built In Smoothness Penalty (Do Not Introduce Extra Numerical Regularization Steps).
![Page 115: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/115.jpg)
Chris Calderon, PASI, Lecture 3
Demmler Reinsch Basis Approach(Popular Method For Efficiently Computing Smoothing Splines)
Forms Eigenbasis using Cholesky Factor of
Traces and Residuals for Each Candidate Can Easily be Obtained in Vectorized Fashion
Squaring a Matrix is Not a Good Idea (Subtle and Dramatic Consequences)
αCTC
![Page 116: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/116.jpg)
Chris Calderon, PASI, Lecture 3
Avoid ad hoc Regularization
Analogous Existing Methods Do Not Use Good Numerics
e.g. if Cannot be CholeskyFactored, ad hoc solution:
Find Factor of
Instead and Solving that Penalized Regression Problem Originally Posed
Others use SVD truncation in Combination with Regularization (Again Not Problem Posed)
CTC + ηD
CTC
![Page 117: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/117.jpg)
Chris Calderon, PASI, Lecture 3
Basic Sketch of Our Approach
(1) Factor C via QR (Done Only Once)
(2) Then Do SVD on Penalized Columns
(3) For Given Find (Lower Dimensional) QR and Exploit Banded Matrix Structure. Solve Original Penalized Least Squares Problem with CHEAP QR
(4) Repeat (3) and Choose µS√αI
¶Banded “R”Like Matrix
α
α
![Page 118: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/118.jpg)
Chris Calderon, PASI, Lecture 3
QR and Least Squares
Minimizing Vector Equivalent to Solution of
Efficient QR Treatment?
Exploit P-Spline Problem Structure
Some in Statistics use QR, but Combine Penalized Regression with SVD Truncation
°°°°°µ
C√αD
¶β −
µy0
¶°°°°°2
2
.
A ≡µ
C√αD
¶(CTC + αDTD)β=CT y
ATAβ = AT (yT , 0)T
![Page 119: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/119.jpg)
Chris Calderon, PASI, Lecture 3
PSQR (Factor Steps)1. Obtain the QR decomposition of C = QR.
2. Partition result above as: QR = (QF , QP )
µRF11 R120 RP22
¶.
3. Obtain the SVD of RP22 = USVT .
4. Form the following:
Q = (QF , QPU), V =
µI 00 V T
¶,
R =
µRF11 R12V0 S
¶≡µR11 R120 R22
¶,
b =
µQT y0
¶≡
⎡⎣µbFbP¶
0
⎤⎦.
![Page 120: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/120.jpg)
Chris Calderon, PASI, Lecture 3
PSQR (Solution Steps)
5. For each given α (and/or DP ) form: eDα =√αDPV and fWα =
µSeDα
¶.
6. Obtain the QR decomposition fWα = Q0R0.
7. Form c = (R0)−1(Q0)TµbP
0
¶.
8. Solve βaα =
µR−111 (b
F − R12c)V c
¶.
![Page 121: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/121.jpg)
Chris Calderon, PASI, Lecture 3
PSQR (Efficiency)
So if
Then problem reduces to finding x minimizing
Orthogonal Matrix Close to “R”
µSV T
DP
¶=
µS
DPV
¶V T ≡
µR22DPV
¶V T ,
DP = diag(1, . . . 1)D = diag(0, . . . , 0, 1, . . . 1)
°°°°°µ
S√αV
¶x−
µbP
0
¶°°°°°2
2
=
°°°°°µI, 00, V
¶µS√αI
¶x−
µbP
0
¶°°°°°2
2
.
![Page 122: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/122.jpg)
Chris Calderon, PASI, Lecture 3
PSQR and Givens Rotations
Orthogonal Matrix Close to “R”
Givens Rotations to Finalize QR
Applying Rotations to RHS Yields:
°°°°°µ
S√αV
¶x−
µbP
0
¶°°°°°2
2
=
°°°°°µI, 00, V
¶µS√αI
¶x−
µbP
0
¶°°°°°2
2
.
DP = diag(1, . . . 1)
(√Λ)−1SbP
Where R =√Λ
![Page 123: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/123.jpg)
Chris Calderon, PASI, Lecture 3
Subtle Errors
![Page 124: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/124.jpg)
Chris Calderon, PASI, Lecture 3
PuDI Design Matrix (One Curve at a Time)
Sparsity Pattern of TPF Basis QR Needed in Step 1 (Before Penalty Added)
![Page 125: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/125.jpg)
Chris Calderon, PASI, Lecture 3
TPF: Free and Penalized Blocks
Last K Columns
CPuDI :=
⎛⎜⎜⎜⎜⎜⎜⎜⎜⎝
1 x1 . . . xp1 (κ1 − x1)p+ . . . (κK − x1)p+
......
...1 xm . . . x
pm (κ1 − xm)p+ . . . (κK − xm)p+
0 1 . . . pxp−11 p(κ1 − x1)p−1+ . . . p(κK − x1)p−1+...
......
0 1 . . . pxp−1m p(κ1 − xm)p−1+ . . . p(κK − xm)p−1+
⎞⎟⎟⎟⎟⎟⎟⎟⎟⎠.
![Page 126: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/126.jpg)
Chris Calderon, PASI, Lecture 3
PuDI Design Matrix (Batches of Curves)
SparsityPattern of TPF Basis QR Needed in Step 1 (Before Penalty Added)
![Page 127: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/127.jpg)
Chris Calderon, PASI, Lecture 3
Screened Spectrum
![Page 128: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/128.jpg)
Chris Calderon, PASI, Lecture 3
Simple Basis Readily Allows For Useful Extensions
(e.g. Generalized Least Squares)
![Page 129: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/129.jpg)
Chris Calderon, PASI, Lecture 3
0
0.5
1
1.5
2
00.511.522.53
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Zo
Function of Interest (the “truth”)Noisy Point Estimates (finite discrete time series sample uncertainty)
Spatial Derivative of Function of Interest
²i∂σ(Z)/∂Zσ(Z)
²i
![Page 130: Lecture III: Review of Classic Quadratic Variation …jrojo/PASI/lectures/CALDERON Lec3.pdf · Rice University / Numerica Corporation ... Properties of Brownian Motion. Chris Calderon,](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b7179fa7f8b9a95348bcd5b/html5/thumbnails/130.jpg)
Chris Calderon, PASI, Lecture 3
0
0.5
1
1.5
2
00.511.522.53
Z (S
tate
Spa
ce)
0
0.5
1
1.5
2
00.511.52
Zo
Spatial Derivative of Function of Interest
²i∂σ(Z)/∂Zσ(Z)
²i
Point Estimates Noise Distribution Depends on Window and Function Estimated (Quantifying and Balancing These Can Be Important); Especially for Resolving “Spiky” Features.