probability theory presentation 10
Post on 10-Apr-2018
215 Views
Preview:
TRANSCRIPT
-
8/8/2019 Probability Theory Presentation 10
1/51
BST 401 Probability Theory
Xing Qiu Ha Youn Lee
Department of Biostatistics and Computational BiologyUniversity of Rochester
October 7, 2010
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
2/51
Outline
1 Introduction to functional analysis
2 Convergence of Sequence of Measurable Functions
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
3/51
Motivation (I)
Functional analysis is in some sense the linear algebra ofmeasurable functions/random variables. Youve already
seen that linear combinations of r.v.s are r.v.s.
The usual linear algebra deals with finite dimensional
vectors. In general, random variables are inherently infinite
dimensional.
For an Euclidean space, all linear transformations can be
expressed as matrix multiplications in a basis system.
There is also a way to define a (infinite) basis system (and
coordinates) for a functional space. So lineartransformations of r.v.s can be expressed in this basis
system explicitly.
It turns out, all linear transformations are integrals in a
basis system.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
4/51
Motivation (I)
Functional analysis is in some sense the linear algebra ofmeasurable functions/random variables. Youve already
seen that linear combinations of r.v.s are r.v.s.
The usual linear algebra deals with finite dimensional
vectors. In general, random variables are inherently infinite
dimensional.For an Euclidean space, all linear transformations can be
expressed as matrix multiplications in a basis system.
There is also a way to define a (infinite) basis system (and
coordinates) for a functional space. So lineartransformations of r.v.s can be expressed in this basis
system explicitly.
It turns out, all linear transformations are integrals in a
basis system.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
5/51
Motivation (I)
Functional analysis is in some sense the linear algebra ofmeasurable functions/random variables. Youve already
seen that linear combinations of r.v.s are r.v.s.
The usual linear algebra deals with finite dimensional
vectors. In general, random variables are inherently infinite
dimensional.For an Euclidean space, all linear transformations can be
expressed as matrix multiplications in a basis system.
There is also a way to define a (infinite) basis system (and
coordinates) for a functional space. So lineartransformations of r.v.s can be expressed in this basis
system explicitly.
It turns out, all linear transformations are integrals in a
basis system.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
6/51
Motivation (I)
Functional analysis is in some sense the linear algebra ofmeasurable functions/random variables. Youve already
seen that linear combinations of r.v.s are r.v.s.
The usual linear algebra deals with finite dimensional
vectors. In general, random variables are inherently infinite
dimensional.For an Euclidean space, all linear transformations can be
expressed as matrix multiplications in a basis system.
There is also a way to define a (infinite) basis system (and
coordinates) for a functional space. So lineartransformations of r.v.s can be expressed in this basis
system explicitly.
It turns out, all linear transformations are integrals in a
basis system.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
7/51
Motivation (I)
Functional analysis is in some sense the linear algebra ofmeasurable functions/random variables. Youve already
seen that linear combinations of r.v.s are r.v.s.
The usual linear algebra deals with finite dimensional
vectors. In general, random variables are inherently infinite
dimensional.For an Euclidean space, all linear transformations can be
expressed as matrix multiplications in a basis system.
There is also a way to define a (infinite) basis system (and
coordinates) for a functional space. So lineartransformations of r.v.s can be expressed in this basis
system explicitly.
It turns out, all linear transformations are integrals in a
basis system.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
8/51
Motivation (II)
The functional norm will act as vector length, andsometimes we can even define an inner product between
two vectors. Consequently two r.v.s may have an angle
between them; they may be orthogonal to each other.
Many important mathematical concepts, such as continuity,
convergence, and completeness, can be derived from thenorm of a functional space.
Unlike n-dim Euclidean vector spaces, norms defined on
an infinite functional space are not equivalent. Depending
on different norms, we have different functional spaces.
Lp() spaces, 1 p are the most importantfunctional spaces for studying probability theory.
Other spaces, such as the Sobolev spaces are useful for
nonparametric regression, functional analysis, SDE, etc.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
9/51
Motivation (II)
The functional norm will act as vector length, and
sometimes we can even define an inner product between
two vectors. Consequently two r.v.s may have an angle
between them; they may be orthogonal to each other.
Many important mathematical concepts, such as continuity,
convergence, and completeness, can be derived from thenorm of a functional space.
Unlike n-dim Euclidean vector spaces, norms defined on
an infinite functional space are not equivalent. Depending
on different norms, we have different functional spaces.
Lp() spaces, 1 p are the most importantfunctional spaces for studying probability theory.
Other spaces, such as the Sobolev spaces are useful for
nonparametric regression, functional analysis, SDE, etc.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
10/51
Motivation (II)
The functional norm will act as vector length, and
sometimes we can even define an inner product between
two vectors. Consequently two r.v.s may have an angle
between them; they may be orthogonal to each other.
Many important mathematical concepts, such as continuity,
convergence, and completeness, can be derived from thenorm of a functional space.
Unlike n-dim Euclidean vector spaces, norms defined on
an infinite functional space are not equivalent. Depending
on different norms, we have different functional spaces.
Lp() spaces, 1 p are the most importantfunctional spaces for studying probability theory.
Other spaces, such as the Sobolev spaces are useful for
nonparametric regression, functional analysis, SDE, etc.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
11/51
Motivation (II)
The functional norm will act as vector length, and
sometimes we can even define an inner product between
two vectors. Consequently two r.v.s may have an angle
between them; they may be orthogonal to each other.
Many important mathematical concepts, such as continuity,
convergence, and completeness, can be derived from thenorm of a functional space.
Unlike n-dim Euclidean vector spaces, norms defined on
an infinite functional space are not equivalent. Depending
on different norms, we have different functional spaces.
Lp() spaces, 1 p are the most importantfunctional spaces for studying probability theory.
Other spaces, such as the Sobolev spaces are useful for
nonparametric regression, functional analysis, SDE, etc.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
12/51
Motivation (II)
The functional norm will act as vector length, and
sometimes we can even define an inner product between
two vectors. Consequently two r.v.s may have an angle
between them; they may be orthogonal to each other.
Many important mathematical concepts, such as continuity,
convergence, and completeness, can be derived from thenorm of a functional space.
Unlike n-dim Euclidean vector spaces, norms defined on
an infinite functional space are not equivalent. Depending
on different norms, we have different functional spaces.
Lp() spaces, 1 p are the most importantfunctional spaces for studying probability theory.
Other spaces, such as the Sobolev spaces are useful for
nonparametric regression, functional analysis, SDE, etc.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
13/51
Lp-space
(,F, ) is a measurable space.
For p 1, we define Lp(,F, ) (in short, Lp) to be thespace of -measurable functions such that
fp =
|f|
p
d
1p
< .
Special case: random variables with finite mean (L1);
random variables with finite variance (L2).
Another special case: L(), the space of all almostsurely bounded r.v.s:
f = limp
fp = ess sup
|f(x)|.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
14/51
Lp-space
(,F, ) is a measurable space.
For p 1, we define Lp(,F, ) (in short, Lp) to be thespace of -measurable functions such that
f
p =
|f
|
pd1p
N.
Completeness. A functional space X is complete if every
Cauchy sequence converges to a member in X.
Lp spaces are complete.
Implication: if a sequence of r.v.s X1, X2, . . . satisfies
limn,mE|Xn Xm|p = 0, then there must be a r.v. X to
which Xn converges, and X Lp() as well. So say if Xn
have finite variances, X must have finite variance as well.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
24/51
Basic properties (II)
A norm induces a distance: distp(f,g) = f gp. Withdistance we can define Cauchy sequence. f1, f2, . . . is aCauchy sequence (relative to the given distance) if > 0,there exists N N, such that
distp(fn, fm) < , n, m> N.
Completeness. A functional space X is complete if every
Cauchy sequence converges to a member in X.
Lp spaces are complete.
Implication: if a sequence of r.v.s X1, X2, . . . satisfies
limn,mE|Xn Xm|p = 0, then there must be a r.v. X to
which Xn converges, and X Lp() as well. So say if Xn
have finite variances, X must have finite variance as well.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
25/51
Basic properties (II)
A norm induces a distance: distp(f,g) = f gp. Withdistance we can define Cauchy sequence. f1, f2, . . . is aCauchy sequence (relative to the given distance) if > 0,there exists N N, such that
distp(fn, fm) < , n, m> N.
Completeness. A functional space X is complete if every
Cauchy sequence converges to a member in X.
Lp spaces are complete.
Implication: if a sequence of r.v.s X1, X2, . . . satisfies
limn,mE|Xn Xm|p = 0, then there must be a r.v. X to
which Xn converges, and X Lp() as well. So say if Xn
have finite variances, X must have finite variance as well.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
26/51
Basic properties (II)
A norm induces a distance: distp(f,g) = f gp. Withdistance we can define Cauchy sequence. f1, f2, . . . is aCauchy sequence (relative to the given distance) if > 0,there exists N N, such that
distp(fn, fm) < , n, m> N.
Completeness. A functional space X is complete if every
Cauchy sequence converges to a member in X.
Lp spaces are complete.
Implication: if a sequence of r.v.s X1, X2, . . . satisfies
limn,mE|Xn Xm|p = 0, then there must be a r.v. X to
which Xn converges, and X Lp() as well. So say if Xn
have finite variances, X must have finite variance as well.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
27/51
Dense subset/approximation
For simplicity, assume = R.
Recall Q is dense in R. Dense subsets in Lp:
set of simple functions;set of continuous functions;set of smooth functions (functions with arbitraryderivatives).set of polynomials. (checkout the Bernstein polynomialsfrom Wikipedia)
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
28/51
Dense subset/approximation
For simplicity, assume = R.
Recall Q is dense in R. Dense subsets in Lp:
set of simple functions;set of continuous functions;set of smooth functions (functions with arbitraryderivatives).set of polynomials. (checkout the Bernstein polynomialsfrom Wikipedia)
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
29/51
Dense subset/approximation
For simplicity, assume = R.
Recall Q is dense in R. Dense subsets in Lp:
set of simple functions;set of continuous functions;set of smooth functions (functions with arbitraryderivatives).set of polynomials. (checkout the Bernstein polynomialsfrom Wikipedia)
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
30/51
Dense subset/approximation
For simplicity, assume = R.
Recall Q is dense in R. Dense subsets in Lp:
set of simple functions;set of continuous functions;set of smooth functions (functions with arbitraryderivatives).set of polynomials. (checkout the Bernstein polynomialsfrom Wikipedia)
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
31/51
Dense subset/approximation
For simplicity, assume = R.
Recall Q is dense in R. Dense subsets in Lp:
set of simple functions;set of continuous functions;set of smooth functions (functions with arbitraryderivatives).set of polynomials. (checkout the Bernstein polynomialsfrom Wikipedia)
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
32/51
Dense subset/approximation
For simplicity, assume = R.
Recall Q is dense in R. Dense subsets in Lp:
set of simple functions;set of continuous functions;set of smooth functions (functions with arbitraryderivatives).set of polynomials. (checkout the Bernstein polynomialsfrom Wikipedia)
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
33/51
Basis
A basis (e1,e2, . . . , en) of n-dim linear space (notnecessarily orthogonal):
1 ei are linearly independent;2 every X X can be written as a linear combination of
(e1,e2, . . . ,en). X =n
i=1 xiei.
For a Banach space:1 ei are linearly independent;2 every X X can be written as
X =
i=1
xiei,
this summation is understood as a limit.
Example: Taylor expansion + smooth function
approximation of an Lp([0, 1],B,L) function.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
34/51
Basis
A basis (e1,e2, . . . , en) of n-dim linear space (notnecessarily orthogonal):
1 ei are linearly independent;2 every X X can be written as a linear combination of
(e1,e2, . . . ,en). X =n
i=1 xiei.
For a Banach space:1 ei are linearly independent;2 every X X can be written as
X =
i=1
xiei,
this summation is understood as a limit.
Example: Taylor expansion + smooth function
approximation of an Lp([0, 1],B,L) function.
Qiu, Lee BST 401
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
35/51
Basis
A basis (e1,e2, . . . , en) of n-dim linear space (not
necessarily orthogonal):1 ei are linearly independent;2 every X X can be written as a linear combination of
(e1,e2, . . . ,en). X =n
i=1 xiei.
For a Banach space:1 ei are linearly independent;2 every X X can be written as
X =
i=
1
xiei,
this summation is understood as a limit.
Example: Taylor expansion + smooth function
approximation of an Lp([0, 1],B,L) function.
Qiu, Lee BST 401
B i
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
36/51
Basis
A basis (e1,e2, . . . , en) of n-dim linear space (not
necessarily orthogonal):1 ei are linearly independent;2 every X X can be written as a linear combination of
(e1,e2, . . . ,en). X =n
i=1 xiei.
For a Banach space:1 ei are linearly independent;2 every X X can be written as
X =
i=
1
xiei,
this summation is understood as a limit.
Example: Taylor expansion + smooth function
approximation of an Lp([0, 1],B,L) function.
Qiu, Lee BST 401
B i
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
37/51
Basis
A basis (e1,e2, . . . , en) of n-dim linear space (not
necessarily orthogonal):1 ei are linearly independent;2 every X X can be written as a linear combination of
(e1,e2, . . . ,en). X =n
i=1 xiei.
For a Banach space:1 ei are linearly independent;2 every X X can be written as
X =
i=
1
xiei,
this summation is understood as a limit.
Example: Taylor expansion + smooth function
approximation of an Lp([0, 1],B,L) function.
Qiu, Lee BST 401
B i
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
38/51
Basis
A basis (e1,e2, . . . , en) of n-dim linear space (not
necessarily orthogonal):1 ei are linearly independent;2 every X X can be written as a linear combination of
(e1,e2, . . . ,en). X =n
i=1 xiei.
For a Banach space:1 ei are linearly independent;2 every X X can be written as
X =
i=
1
xiei,
this summation is understood as a limit.
Example: Taylor expansion + smooth function
approximation of an Lp([0, 1],B,L) function.
Qiu, Lee BST 401
B i
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
39/51
Basis
A basis (e1,e2, . . . , en) of n-dim linear space (not
necessarily orthogonal):1 ei are linearly independent;2 every X X can be written as a linear combination of
(e1,e2, . . . ,en). X =n
i=1 xiei.
For a Banach space:1 ei are linearly independent;2 every X X can be written as
X =
i=
1
xiei,
this summation is understood as a limit.
Example: Taylor expansion + smooth function
approximation of an Lp([0, 1],B,L) function.
Qiu, Lee BST 401
Inner product and Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
40/51
Inner product and Hilbert space
A complete normed linear space such as Lp
is called aBanach space.
A Hilbert space H is a Banach space with a inner productf, g : H H R which satisfies1
1 Bilinearity: aX+ bY, Z = aX, Z + bY, Z.
2 X, Y = Y, X.2
3 X, X 0 and X, X = 0 iff X = 0.
An inner product induces a norm: X :=
X, X. But anorm in general can not be extended to an inner product.
L2 is a Hilbert space and the only Hilbert space among Lp
spaces. Its inner product: X, Y2 = EXY =XYd.
1R should be replaced by C for spaces of complex valued functions.2For complex Hilbert spaces, X, Y = Y, X, where is complex
conjugate.
Qiu, Lee BST 401
Inner product and Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
41/51
Inner product and Hilbert space
A complete normed linear space such as L
p
is called aBanach space.
A Hilbert space H is a Banach space with a inner productf, g : H H R which satisfies1
1 Bilinearity: aX+ bY, Z = aX, Z + bY, Z.
2 X, Y = Y, X.2
3 X, X 0 and X, X = 0 iff X = 0.
An inner product induces a norm: X :=
X, X. But anorm in general can not be extended to an inner product.
L2 is a Hilbert space and the only Hilbert space among Lp
spaces. Its inner product: X, Y2 = EXY =XYd.
1R should be replaced by C for spaces of complex valued functions.2For complex Hilbert spaces, X, Y = Y, X, where is complex
conjugate.
Qiu, Lee BST 401
Inner product and Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
42/51
Inner product and Hilbert space
A complete normed linear space such as L
p
is called aBanach space.
A Hilbert space H is a Banach space with a inner productf, g : H H R which satisfies1
1 Bilinearity: aX+ bY, Z = aX, Z + bY, Z.
2 X, Y = Y, X.2
3 X, X 0 and X, X = 0 iff X = 0.
An inner product induces a norm: X :=
X, X. But anorm in general can not be extended to an inner product.
L2 is a Hilbert space and the only Hilbert space among Lp
spaces. Its inner product: X, Y2 = EXY =XYd.
1R should be replaced by C for spaces of complex valued functions.2For complex Hilbert spaces, X, Y = Y, X, where is complex
conjugate.
Qiu, Lee BST 401
Inner product and Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
43/51
Inner product and Hilbert space
A complete normed linear space such as L
p
is called aBanach space.
A Hilbert space H is a Banach space with a inner productf, g : H H R which satisfies1
1 Bilinearity: aX+ bY, Z = aX, Z + bY, Z.
2 X, Y = Y, X.2
3 X, X 0 and X, X = 0 iff X = 0.
An inner product induces a norm: X :=
X, X. But anorm in general can not be extended to an inner product.
L2 is a Hilbert space and the only Hilbert space among Lp
spaces. Its inner product: X, Y2 = EXY =XYd.
1R should be replaced by C for spaces of complex valued functions.2For complex Hilbert spaces, X, Y = Y, X, where is complex
conjugate.
Qiu, Lee BST 401
Inner product and Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
44/51
Inner product and Hilbert space
A complete normed linear space such as L
p
is called aBanach space.
A Hilbert space H is a Banach space with a inner productf, g : H H R which satisfies1
1 Bilinearity: aX+ bY, Z = aX, Z + bY, Z.
2 X, Y = Y, X.2
3 X, X 0 and X, X = 0 iff X = 0.
An inner product induces a norm: X :=
X, X. But anorm in general can not be extended to an inner product.
L2 is a Hilbert space and the only Hilbert space among Lp
spaces. Its inner product: X, Y2 = EXY =XYd.
1R should be replaced by C for spaces of complex valued functions.2For complex Hilbert spaces, X, Y = Y, X, where is complex
conjugate.
Qiu, Lee BST 401
Inner product and Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
45/51
Inner product and Hilbert space
A complete normed linear space such as L
p
is called aBanach space.
A Hilbert space H is a Banach space with a inner productf, g : H H R which satisfies1
1 Bilinearity: aX+ bY, Z = aX, Z + bY, Z.
2 X, Y = Y, X.2
3 X, X 0 and X, X = 0 iff X = 0.
An inner product induces a norm: X :=
X, X. But anorm in general can not be extended to an inner product.
L2 is a Hilbert space and the only Hilbert space among Lp
spaces. Its inner product: X, Y2 = EXY =XYd.
1R should be replaced by C for spaces of complex valued functions.2For complex Hilbert spaces, X, Y = Y, X, where is complex
conjugate.
Qiu, Lee BST 401
Inner product and Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
46/51
Inner product and Hilbert space
A complete normed linear space such as L
p
is called aBanach space.
A Hilbert space H is a Banach space with a inner productf, g : H H R which satisfies1
1 Bilinearity: aX+ bY, Z = aX, Z + bY, Z.
2 X, Y = Y, X.2
3 X, X 0 and X, X = 0 iff X = 0.
An inner product induces a norm: X :=
X, X. But anorm in general can not be extended to an inner product.
L2 is a Hilbert space and the only Hilbert space among Lp
spaces. Its inner product: X, Y2 = EXY =XYd.
1R should be replaced by C for spaces of complex valued functions.2For complex Hilbert spaces, X, Y = Y, X, where is complex
conjugate.
Qiu, Lee BST 401
Properties of a Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
47/51
Properties of a Hilbert space
With an inner product, we can define orthogonality. X isorthogonal to Y if X, Y = 0.
Also the angel between two vectors: cos := X, YXY .
A Hilbert space is a Banach space, so it has a basis. We
can go one step further: a separable Hilbert spaces has anorthonormal basis (e1,e2, . . .) such that: a) (ei) is a basis;b) ei = 1; c) ei, ej = 0. Given an orthonormal basis,every X X can be expressed as:
X =i=1
X, eiei.
Qiu, Lee BST 401
Properties of a Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
48/51
Properties of a Hilbert space
With an inner product, we can define orthogonality. X isorthogonal to Y if X, Y = 0.
Also the angel between two vectors: cos := X, YXY .
A Hilbert space is a Banach space, so it has a basis. We
can go one step further: a separable Hilbert spaces has anorthonormal basis (e1,e2, . . .) such that: a) (ei) is a basis;b) ei = 1; c) ei, ej = 0. Given an orthonormal basis,every X X can be expressed as:
X =i=1
X, eiei.
Qiu, Lee BST 401
Properties of a Hilbert space
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
49/51
Properties of a Hilbert space
With an inner product, we can define orthogonality. X isorthogonal to Y if X, Y = 0.
Also the angel between two vectors: cos := X, YXY .
A Hilbert space is a Banach space, so it has a basis. We
can go one step further: a separable Hilbert spaces has anorthonormal basis (e1,e2, . . .) such that: a) (ei) is a basis;b) ei = 1; c) ei, ej = 0. Given an orthonormal basis,every X X can be expressed as:
X =i=1
X, eiei.
Qiu, Lee BST 401
Applications
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
50/51
Applications
The first n-terms provides a good approximation of X:
Xn
i=1
X, eiei =
i=n+1
qX, eiei =
i=n+1
X, ei 0.
This approximation is the foundation of nonparametricregression (splines are n-term approximations of an
unknown regression function in an abstract Hilbert space),
Fourier analysis, wavelet analysis, PDE, and much more.
We can define projections in a Hilbert space. A projection
to a Hilbert subspace M X breaks X into two parts,X = ProjMX+ X
. ProjMX Mhas the smallest distancewith X. This is the theoretic foundation of regression
theory.
Qiu, Lee BST 401
Applications
http://find/http://goback/ -
8/8/2019 Probability Theory Presentation 10
51/51
pp cat o s
The first n-terms provides a good approximation of X:
Xn
i=1
X, eiei =
i=n+1
qX, eiei =
i=n+1
X, ei 0.
This approximation is the foundation of nonparametricregression (splines are n-term approximations of an
unknown regression function in an abstract Hilbert space),
Fourier analysis, wavelet analysis, PDE, and much more.
We can define projections in a Hilbert space. A projection
to a Hilbert subspace M X breaks X into two parts,X = ProjMX+ X
. ProjMX Mhas the smallest distancewith X. This is the theoretic foundation of regression
theory.
Qiu, Lee BST 401
http://find/http://goback/
top related