spectral theory for subnormal operators › ... › s0002-9947-1979-0542882-2.pdf · spectral...
TRANSCRIPT
TRANSACTIONS OF THEAMERICAN MATHEMATICAL SOCIETYVolume 255, November 1979
SPECTRAL THEORY FOR SUBNORMAL OPERATORS
BY
R. G. LAUTZENHEISER
Abstract. We give an example of a subnormal operator T such that
C \ a(T) has an infinite number of components, int(o(7T)) has two compo-
nents U and V, and T cannot be decomposed with respect to U and V. That
is, it is impossible to write T = 7", © T2 with o(T¡) = Ü and o(T¿ = V.
This example shows that Sarason's decomposition theorem cannot be ex-
tended to the infinitely-connected case.
We also use Mlak's generalization of Sarason's theorem to prove theorems
on the existence of reducing subspaces. For example, if X is a spectral set
for T and K c X, conditions are given which imply that T has a nontrivial
reducing subspace 911 such that oiT]^) c K. In particular, we show that if
T is a subnormal operator and if I" is a piecewise C2 Jordan closed curve
which intersects o-( T) in a set of measure zero on T, then T = Tx © T2 with
<r(T,) c o(T) n ext(r) and a(T2) c o-(T) n int(r).
Introduction. In this paper operator will always refer to a bounded linear
transformation from a Hubert space % into DC. An open problem in operator
theory which has been the subject of many papers in recent years is the
invariant subspace problem. That is, does an arbitrary operator T have a
nontrivial invariant subspace? One approach to solving this problem utilizes
the spectrum of the operator. For example, if a(T), the spectrum of T, is the
union of two nonempty disjoint compact sets Kx and K2, then the Riesz-Dun-
ford functional calculus may be applied to show that T has invariant
subspaces 91L, such that a(T\GM,j) = K¡ for i = 1, 2. Since neither set K¡ is
equal to o{T), <Dlt, must be nontrivial. The Jordan canonical form of a matrix
illustrates the success of this approach when the dimension of % is finite.
This approach suggests the more general question: for which nonempty
compact sets K does there exist an invariant subspace 91L for T so that
o(T\?R,) = K1 If o(T) is finite, then the answer is every compact subset of
a(T). On the other hand, if T is the unilateral shift, then every nontrivial
invariant subspace <D1L of T has the property that a(r|91c) = o(T). Note that
if it is possible to give an affirmative answer for a nonempty set K which is
not o(T), then the corresponding subspace 91L is necessarily nontrivial.
The purpose of this paper is to investigate the above question when certain
conditions are imposed on the operator T. We shall consider compact
Received by the editors April 17, 1978 and, in revised form, October 20, 1978.
AMS (MOS) subject classifications (1970). Primary 47A15, 47A25, 47B20; Secondary 47A10.Key words and phrases. Spectral set, subnormal operator, Gleason part, peak set.
© 1979 American Mathematical Society
0002-9947/79/0000-OS1 3/SO5.O0
301
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
302 R. G. LAUTZENHEISER
spectral sets X for T which have a nontrivial "algebraic" decomposition. That
is, instead of decomposing X topologically, we shall identify X with the
maximal ideal space of R(X) and then try to answer the question when K is
the closure of a nontrivial Gleason part for R(X). This generalizes a result of
D. Sarason [27], in which he found conditions so that if K is the closure of a
component of int X, then the answer is yes. In §3, we give an example which
shows the limitations and, at the same time, leads to a generalization of
Sarason's result.
An important class of operators for which the invariant subspace problem
remains open consists of the subnormal operators. An operator T is subnor-
mal if it has a normal extension or, equivalently, T= A^l^lt where N is
normal and 911 is an invariant subspace for N. Later we shall see that the
best examples of operators to which our results apply are those whose spectra
are spectral sets. Subnormal operators have this property and are essentially
the only known examples to which our results may be applied. We shall be
most interested in finding conditions on AT, n K2 so that if T is subnormal
and a(T) = Kx u K2, then T has reducing subspaces 911, such that a(T\G]tj)
= K¡,i= 1, 2. A subspace 91L is reducing for T if 911 is invariant for both T
and T* or, equivalently, if P is the projection onto <9lt, then PT = TP. If 911
is reducing for T, then T may be written as T = 7, © T2 where Tx = Tl'DIt
and T2 = T\'?St-L. In this paper, the notation T = Tx @ T2 will not neces-
sarily imply that T has a nontrivial reducing subspace. For instance, Tx might
be trivial which means that it is the 0-operator on the 0-dimensional Hubert
space and T2 would then be equal to T.
2. Spectral sets and Sarason's theorem. If T is an operator on % and A1 is a
closed set containing a(T), then X is called a spectral set for 7" if ||/(r)|| <
\\f\\x for all rational functions/with poles off X. J. von Neumann introduced
spectral sets in [32] and proved that the unit disk is a spectral set for all
contractions. He also proved that o(T) is equal to the intersection of all
spectral sets for T. C. Berger [5] has improved this result by showing that the
intersection of all compact spectral sets is a(T). In this paper we shall only
consider compact spectral sets.
Even though an operator T possesses many compact spectral sets, it is
usually quite difficult to exhibit any other than the disk about the origin of
radius ||7"J|. However, under certain conditions, it is possible to show that
a(T) is a spectral set for T. For instance, using the spectral theorem, it is easy
to show the spectrum of a normal operator is a spectral set. In [3], S. K.
Berberian noted that a(T) is a spectral set for T if and only if ||/(7^|| is the
spectral radius of f(T) for all rational functions / with poles off a(T). This
follows directly from the spectral mapping theorem for rational functions.
This result and the fact that the spectrum of the minimal normal extension of
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
SPECTRAL THEORY FOR SUBNORMAL OPERATORS 303
a subnormal operator is contained in the spectrum of the subnormal operator
(see Lemma (3.2)) imply that the spectrum of a subnormal is a spectral set. If,
however, T is assumed to be only hyponormal (7T* < T*T), then a(T) need
not be a spectral set for T (see Clancey [8] or Wadhwa [33] for an example).
If X is compact, then R(X) will denote the uniform closure of the rational
functions with poles off X. If A' is a spectral set for T, then it is possible to
associate an operatorfiT) with each/ e R(X). That is, if/ e R(X) and {/„}
is a sequence of rational functions converging to /, then the sequence of
operators {f„(T)} has a limit in $(3C), the operators on %, and we denote
this limit by/(T). It is also easy to see that fiX) is a spectral set for f(T). In
case X = a(T), then fio(T)) = o(f(T)) and the correspondence between /
and/(7") establishes an isometric isomorphism from R(X) into % (%).
The following two well-known results illustrate the connection between
spectral sets and reducing subspaces and also give an indication as to the type
of results which follow. One theorem involves a topological condition on X,
and the other, a condition on the algebra R(X).
Theorem (2.1) (Williams [37]). If X = Xx u X2 is a spectral set for T where
Xx and X2 are disjoint and compact, then T = Tx © T2 with o(T¡) c X¡,
i = 1,2. In particular, if o(T) n X¡ 9* 0 for i = 1, 2, then T has a nontrivial
reducing subspace.
Theorem (2.2) (von Neumann [32]). If X is a spectral set for T and
R(X) = C(X), then T is normal.
Both theorems may be derived from the following lemma.
Lemma (2.3) (Berger [4], Foais. [12], Lebow [20]). If X is a spectral set for
T, then for each x,y e %, there exists a complex Borel measure p supported
ondX suchthat \\^J < \\x\\ \\y\\, px¡x > Owith |fa,J = \\x\\\and
(f(T)x,y)=ff(z)d^
for all f in R(X).
Proof. Since R(X) is isometrically isomorphic to R(X)\dX, the map
fr^*(fiT)x,y) defines a continuous linear functional on R(X)\dX. Continuity
follows from the assumption that X is a spectral set for T. Extending the map
to C(dX) by the Hahn-Banach theorem while preserving the norm and then
applying the Riesz representation theorem gives the desired measure.
To prove Theorem (2.1), note that Xx, the characteristic function of X„ is
in R(X) and then use Lemma (2.3) to show that Xx(T) is a projection and
that a( Txx ( T)) c X¡. Theorem (2.2) follows by observing that/(z) = z is in
R(X) and that/(T) = T*. That is,
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
304 r. G. LAUTZENHEISER
(f(T)x, X) = fzd^x = fz d^x = (T*X, X).
Since/(T) commutes with T, Tis normal.
In 1965, D. Sarason proved the following remarkable theorem relating the
components of int X to reducing subspaces of T.
Theorem (2.4) (Sarason [27]). Let X be a spectral set for T such that C\X
is connected. If Gx, G2 . . . are the components of int X, then
71- JV0Í 2 ®Tj\
where N is normal with o(N) c dX and G¡ is a spectral set for T¡. In particular,
o(Tj) C GV
Sarason's proof depends entirely on the existence of a normal dilation of T.
That is, if C \ X is connected and X is a spectral set for T, then there exists a
normal operator S defined on a larger Hubert space such that a(S) c dX and
fiT) = Pf(S)\% for all/in R(X). The operator P is the projection onto %.
See Lebow [20] for a proof. Since it is not known whether normal dilations
exist even in the simple case when X is an annulus (see Arveson [2]), to
generalize Theorem (2.4) to more complicated sets would require different
techniques. In §3, we give an example to show that it cannot be generalized to
include sets X whose complement has an infinite number of components.
However, if C \ X has a finite number of components, then the theorem is
true. This will be covered in §4.
Decomposition theorems such as Theorem (2.4) are quite useful when
trying to show that T has nontrivial reducing subspaces. For instance, if N is
nontrivial, then T surely has a nontrivial reducing subspace. That is, it is well
known that normal operators have an abundance of reducing subspaces. If N
is trivial, then to use the decomposition theorem, we must show that at least
two of the Tjs are nontrivial. The operator T¡ will, in fact, be nontrivial if
a(T) n G¡ ¥= 0. To see this, first consider the following lemma.
Lemma (2.5). If X¡ is a spectral set for 7) G <S (%¡) for i = 1, 2, 3, . . . and
X = \JfL\X¡ is compact, then X is a spectral set for T = 2°11 © Tt.
Proof. Since ||7;|| < sup{|z| \z G Xj) < sup{|z| | \z\ G X}, it follows that
Tis an operator on % = 2°li © 3C,-. If/is a rational function with poles off
X, then ||/(Z;.)|| < \\f\\x¡ < \\f\\x for i = 1, 2, . . . . Therefore ||/(r)|| =
sup{||/(7;.)||} < H/lljr and X is a spectral set for T.
Note that since X is a spectral set for T, o(T) c X. In the special case that
X¡ = a(T¡), then we have that a(T) = U,°liO-(7)). In case there are only a
finite number of operators, then it is easy to show that a(T) = U "^xa(Tj).
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
SPECTRAL THEORY FOR SUBNORMAL OPERATORS 305
Proposition (2.6). Let the notation be as in Theorem (2.4). If a{T) n G, ¥=
0, then T¡ is nontrivial and if a(í)£U ,°liG,> then N is nontrivial.
Proof. If Tt were trivial, then
o(T) = o(A0uo(2 © Tj)-
But the previous lemma implies
4 2 © T,) C U G,..
Therefore, for A G a(r) n G,, it follows that A G o(JV) or X G U^.G,, but
this is impossible since a(A^) c öA" and G, n Uy7yGy=0 . Thus 7) is non-
trivial.
If a{T) ¡ZUr.iG, then by Lemma (2.5), a(r) =* a(2°°=, © Tj). Since
a(T) = a(Af)Uai J ®t\,
it follows that JV is nontrivial.
If X is the union of two closed disks whose intersection is a point, A' is a
spectral set for T, and a(T) intersects both components of int X, then we see
that T has a nontrivial decomposition T = Tx © T^. (The normal operator
may be absorbed into the operators Tx and T2.) When X has this simple form,
it is possible to use other techniques to show that T has nontrivial invariant
subspaces (see Stampfli [29]). However, if A' is a cornucopia (i.e., a disk with a
ribbon which winds around and clusters on the boundary of the disk) with
the same properties as above, then again it follows that T has a nontrivial
decomposition T= 7, © 72. Other known techniques for producing reducing
subspaces seem to fail in this situation. It appears that J. G. Stampfli was the
first to observe that Theorem (2.4) may be used in this case.
3. An example. In this section we give an example to show that Theorem
(2.4) cannot be extended to the case when C \ X has an infinite number of
components. The example is a subnormal operator T satisfying:
(i) C \ a(T) has an infinite number of components,
(ii) int o(T) has two components U and V, and
(iii) T cannot be written as Tx © T2 with a(Tx) c î/and a(Tj) c V.
In fact, T satisfies a somewhat stronger property than (iii): namely, for
each nontrivial invariant subspace <91t of T, o(T) c ct(7"|'D1L). In order to
define T and show that it has the desired properties, we need to introduce
some terminology and recall a few facts.
If W is an open subset of C whose boundary is a rectifiable Jordan curve T
and <p is a conformai mapping from A= {z\ \z\ < 1} onto W, then <p has a
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
306 R. G. LAUTZENHEISER
one-to-one continuous extension to A which is absolutely continuous on 9A. If
E is a measurable subset of T, then the arc length of E is Sw-\E)W(e")\ dt.
Since <p'(e") ¥= 0 a.e. on 3A, it follows that a measurable set E c T has arc
length zero if and only if <p~l(E) has measure zero. This is standard material
and may be found in the books by Duren [11] or Tsuji [30].
The concept of a representing measure will also be needed. If X is compact
and a G A', then m is a representing measure for evaluation at a on R(X) if m
is a probability measure supported on 3A and fia) = // dm for all / G R(X).
It is easy to show that each a El X has a representing measure and if R(X) is
a Dirichlet algebra (i.e., Re(J?(A')|3A') is dense in CR(dX)), then each a & X
has a unique representing measure. If C \ X is connected, then it is well
known that R(X) is a Dirichlet algebra.
Lemma (3.1). Let W and <p be as above, and fix an element z0 = re'9 in A. If
m is the unique representing measure for evaluation at <p(z0) on R( W), then
(i) iff G L\m), thenf° <p G L\Pr{9 - t) dt/2ir) and
ffdm = ff o <p{eu)Pr{9 - t) dt/2m
where Pr is the Poisson kernel,
(ii) m and arclength on T are mutually absolutely continuous, and
(iii) (F. and M. Riesz theorem on T) if f G Hl(m) = Ll(m)-closure of
R(W), then either fiz) =£ 0 a.e. [m] or f(z) = 0 a.e. [m].
Proof. If u is the continuous image of the measure Pr(9 — t)dt/2ir under
<p, then for/ G L'(«),/ ° <p G L\Pr(9 - i)dt/2-n) and
ffdu=f(f°<p)(ei>)Pr(9-t)£
(see [17, p. 180]). For/ G R( W), f ° <p G R(K) and
ffdu = /(/ o <p)(eu)P,(i - i) ^ - (/o <p)(z0) = /(,p(z0)).
Since w is the unique representing measure for evaluation at <p(z0), m = u.
Therefore (i) is true. Since Pr(ß — t)dt/2ir and the arc length on 3A are
mutually absolutely continuous and m is the continuous image of Pr(9 —
t)dt/2-n under <p, (ii) also follows.
To prove (iii), suppose / G Hl(m) and fiz) = 0 for all z G E where
m(E) > 0. Let {/„} be a sequence in R(W) converging to/in Lx(m). By (i),
flfn -f\dm= f{\fn - f] o <p)(e»)P,(ê -t)£
= /|/n°<P-/°<p|(e")^(0-O¿-
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
SPECTRAL THEORY FOR SUBNORMAL OPERATORS 307
Therefore, f ° <p G 3C1 = L\dt/2tr)-c\osuTe of R(A) and by the classical F.
and M. Riesz theorem (see Hoffman [18, p. 52]), it follows that/ ° <p(e") ¥= 0
a.e. on 3A. Hence fiz) =£ 0 a.e. [m].
The following lemma is useful when trying to locate the spectrum of a
subnormal operator if the spectrum of its minimal normal extension is
known. For a proof, see Halmos [16, problems 157 and 158].
Lemma (3.2). If T is a subnormal operator and N is its minimal normal
extension, then a(N) c o(T). Furthermore, if G is a component of C\ o(N),
then either G c o(T) or G n a(T) = 0.
We are now in a position to define T and show that it possesses the desired
properties. Let A be a Cantor set of positive measure in the interval [—1, 1].
Delete from A the open disks whose diameters are the components of
[—1, 1] \ A and let X denote the resulting set. This set is usually referred to as
the "string of beads". Let U and V be the components of int X and fix points
a G U and b G V. Let ma be the representing measure for evaluation at a on
R{U) and mb be the representing measure for evaluation at b on R(V).
If m = ma + mb and % is the L2(w)-closure of R(X), then define T:
% -» OC by ( Tf)(z) = zfiz). The operator T is subnormal and its minimal
normal extension is Mz, multiplication by z on L2(m). Since ma and mb are
mutually absolutely continuous with respect to arclength on 3f7 and dV,
respectively, it follows that o{Mj) = dX.
We now wish to show that o(T) = A. If À G X then multiplication by
l/(z — A) defines an operator on % which is clearly the inverse of T — XI.
This implies that o(T) c X. If a and b are in o(T), then it follows from
Lemma (3.2) that o(T) = X. Suppose a G o(T). Since T — ai is invertible,
there is an element/ G % such that (T - al)fiz) = (z - a)fiz) = 1 a.e. [m].
If {/„} is a sequence in R(X) which converges to/ in L2(m), then {/„} also
converges to /in L2(mj). Therefore,
0 = j(z - a)f„ dma -^¡{z - a)fdma = j\ dma = 1.
Since this is an obvious contradiction, we may conclude that a G o(T).
Replacing a by b in the above argument implies b is also in o(T). Thus
X = a(T) and since T is subnormal, A' is a spectral set for T.
It remains to show that T satisfies property (iii). Let 9IL be a nontrivial
invariant subspace for T and suppose there exists « G U such that a &
o-(r|91t). Let/be an arbitrary element of 91L. Since (7" - w/)91t = 911, for
each n = 1, 2, . . . , there exists an element f„ G 91L such that (z — u)"fn(z)
= /(z) a.e. [m]. We shall now consider the functions /„ and / as elements of
Hl(ma). Each h G Hl(ma) induces an analytic function honU defined by
h(z) = ¡h dmz. Since (z - u>)"fn(z) = /(z), it follows that / has a zero of
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
308 R. G. LAUTZENHEISER
arbitrary order at <o. Therefore, fiz) = 0 for all z G. U, and hence, / = 0 a.e.
[ma]. In particular,/ = 0 a.e. [ma] on the Cantor set A.
If we now consider/as an element of H\mb), then it follows from (iii) of
Lemma (3.1) that/ = 0 a.e. [mb]. That is, mb(A) > 0 and/ = 0 a.e. [mb] on A.
Therefore, / = 0 a.e. [m]. But this implies that 9IL is trivial which gives a
contradiction. Therefore U c a(T\^M.) and replacing a by b and U by V, it
also follows that V c a(T|91l). Since we now have A c a(7|91L), T must
satisfy condition (iii).
4. Gleason parts. If A is a function algebra on a compact set X, then A/¿,
the space of maximal ideals of A, can be identified with the nonzero
multiplicative linear functionals defined on A. Since each element of MA has
norm one, it follows that ||<p - i|>|| < 2 for <p, ̂ G MA. Furthermore, A.
Gleason [15] discovered that ||<p — \¡i\\ < 2 defines an equivalence relation on
MA and the equivalence classes are referred to as the Gleason parts for A. If
an equivalence class contains precisely one point, then it is called a trivial
Gleason part. The following lemma gives a useful criterion for determining
whether two elements <p, \p in MA are in the same Gleason part. Its proof, as
well as the general theory of function algebras, may be found in the books by
A. Browder [7] and T. Gamelin [13].
Lemma (4.1). Two elements <p and \p in MA are in different Gleason parts if
and only if there exists a sequence {/,} of functions in A such that ||/J| < 1,
<p(/„)^l andW„)^0.
The maximal ideal space of R(X) consists precisely of point evaluations on
X and, thus, may be identified with the set X. The Gleason parts for R(X)
then form a decomposition of X. D. Wilken [35] proved that each nontrivial
Gleason part has positive 2-dimensional Lebesgue measure, which im-
mediately implies that there is at most a countable number of nontrivial
Gleason parts for R(X). The trivial Gleason parts correspond to the peak
points for R(X). That is, those points p for which there exists a function
/ G R(X) such that fip) = 1 and |/(x)| < 1 for all x ¥=p.
In [21], Mlak generalized Sarason's theorem by proving a decomposition
theorem for operator-valued representations of arbitrary function algebras.
The proof used function algebraic techniques and the decomposition was
with respect to the Gleason parts. He later considered the representation of
R(X) into ® (%) when A" is a spectral set for T and gave a direct generaliza-
tion of Theorem (2.4). See Theorem (4.2) below. Our intention is to show how
this result can be used to produce reducing subspaces and also to locate the
spectra of various operators. The proof of Mlak's theorem may be found in
Mlak's paper [22] or the author's thesis [19]. We should also mention that
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
SPECTRAL THEORY FOR SUBNORMAL OPERATORS 309
Seever [28] has recently given a proof of Mlak's theorem using second dual
techniques.
Theorem (4.2). Let X be a spectral set for T and let Gx, G2, . . . be the
nontrivial Gleason parts for R(X). Then
T= N ©I2 ®tA
where N is normal with o(N) c SA', and G, is a spectral set for T¡ (in particular,
o(T) c G~).
Notice that von Neumann's theorem is a special case since by a theorem of
Bishop [6], R(X) = C(X) if and only if each point of A' is a peak point. Also
Sarason's theorem follows from the fact that if C \ X has a finite number of
components, then the nontrivial Gleason parts for R(X) are precisely the
components of int A (see Ahern and Sarason [1] or Gamelin [13, p. 149]).
Recall that the operator T given in §3 has the property that a(T) is a
spectral set for T and T cannot be decomposed with respect to the two
components of int a(T). In view of Theorem (4.2), this is not so surprising
since even though int a(T) has more than one component, there is only one
nontrivial Gleason part for R(o(T)). This follows by observing that the two
measures ma and mb are not mutually orthogonal.
D. Wilken [36] conjectured that if G is a nontrivial Gleason part and
x G G, then either x G G or x is a peak point for R(X). An affirmative
answer to this conjecture was recently obtained by M. S. Melnikov [39].
Using Melnikov's result, we have the following corollary to Theorem (4.2).
Corollary (4.3). If X is a spectral set for T and a(T) intersects at least two
nontrivial Gleason parts for R(X), then T has a nontrivial reducing subspace.
Proof. Suppose T does not have a nontrivial reducing subspace. The
decomposition for T in Theorem (4.2) is simply T = 7) for some i. Now
suppose a nontrivial Gleason part G intersects o(T). It then follows that
G n <r(r) c o(T) - a(Tj) C G„
which implies G n G¡ ¥= 0. It now follows from Melnikov's result that G
must be G¡. Therefore, at most one nontrivial Gleason part can intersect o(T).
As previously mentioned, each a G A has a unique representing measure if
R(X) is a Dirichlet algebra. See [14] for conditions which imply that R(X) is
a Dirichlet algebra. One of the geometric conditions is that 3A" is connected
and the diameters of the components of C \ X are bounded away from zero.
Using this, it is now easy to give examples in which C \ X has an infinite
number of components and the nontrivial Gleason parts for R(X) are
precisely the components of int(A).
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
310 R. G. LAUTZENHEISER
5. Peak sets for R(X). In this section we use Theorem (4.2) to show how
peak sets of R(X) are related to reducing subspaces of T. A proper subset M
of A" is a peak set for R(X) if there exists an/ G R(X) such that/(x) = 1 for
all x G M and |/(jc)| < 1 for all x G X \ M. If M = {a} is a peak set for
R(X), then a is a peak point for R(X). A theorem of Bishop states that
R(X) = C(A) if and only if every point of A" is a peak point for R(X) (see
Gamelin [13, p. 54]).
Theorem (5.1). If X is a spectral set for T and M is a peak set for R(X),
then T = R © S with a(R) c M and a(S) cA" \ M. Furthermore, M and
X \ M are spectral sets for R and S, respectively.
Proof. Let/ be a function in R(X) which is identically 1 on M and whose
modulus is strictly less than 1 off M. If a G M and b G X \ M, then
f(a) —* 1 and f(b) —» 0 as n —» oo. Thus, by Lemma (4.1), a and b are in
different Gleason parts. If Gx, G2, . . . are the nontrivial Gleason parts for
R(X), then each G, is contained entirely in M or entirely in X \ M. Let
A = {/: G, c A/} and B = {i: G, C X \ M).By Theorem (4.2), T = N © (2°°-1 © T¡) where TV is normal with o(N) c
3A and G, is a spectral set for 7). It follows from Lemma (2.5) that M is a
spectral set for 2,E/< © T¡ and A \ M is a spectral set for 2,eB © 3TJ« Since
N is normal, it may be decomposed as the direct sum of two normal operators
Nx and N2 such that o(Nx) c M and a(N2) czX \ M. If
* - #i © f 2 © ^ ) and S = N2 © ( 2 0 7)1
then Ä and 5 have the desired properties.
Corollary (5.2). Let X = o(T) be a spectral set for T and let M be a peak
set for R(X). If M ¡Z.X \ M, then T has a nontrivial reducing subspace.
Proof. Let R and S be as in the previous theorem. Since a(T) = a(R) u
a(S), it is impossible for T to equal R or S. Hence both R and S are
nontrivial operators.
Recall that an operator T is completely nonnormal if T has no reducing
subspaces 911 such that T|91L is normal. Clancey and Putnam [9] have
characterized the compact sets which are the spectra of completely nonnor-
mal operators whose spectrum is a spectral set. In particular, they proved that
if X = a(T) is a spectral set for 7 and there exists an open disk D which
intersects K with the property that R(X n D) = C(A" n D), then T = R ©
S where R is a normal operator. We shall show that this result follows from
Theorem (4.2) and that it is possible to choose R so that o(R) =X n D. The
first part of the proof is similar to the proof given by Clancey and Putnam
since both rely on the following theorem.
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
SPECTRAL THEORY FOR SUBNORMAL OPERATORS 311
Theorem (5.3) (Bishop's Localization Theorem). /// is continuous on X
and each z G X has a neighborhood N(z) such that f G R(X n N(x)), then
f G R(X).
For a proof, see Gamelin [13, p. 51].
Theorem (5.4). Let X = a(T) be a spectral set for T. If there exists an open
disk D such that X n D * 0 and R(X n D) = C(A n D), then T= R © S
where R is normal, a(R) = X n D, and a(S) C X \ D.
Proof. We first observe that if X G X n D, then X is a peak point for
R(X). To show this, choose e so that B(X; 2e), the open disk about X of radius
2e, is contained in D. Now construct a continuous function/on X such that
/(A) = 1, / is identically 0 on X n B(X; e) and |/(x)| < 1 for all x in X not
equal to X. It is easy to see that the hypotheses of Theorem (5.3) are satisfied.
Hence,/ G R(X) and, thus, A is a peak point for R(X).
If G„ G2, . . . are the nontrivial Gleason parts for R(X) and X G A n D,
then it follows that X G U," ,G,.. Let 7 = 2V © (2°°-i © ^-) be the decom-
position in Theorem (4.2). Since a(S°l, 0 T¡) C(J°1XG¡, we must have
X n D c a(JV). If E is the spectral measure for N, then Ä = NE(X n 7>) is
normal, a(Ä) ein A and, since D is open, A" n D c o(Ä). Therefore, if
S = ATJ?(Jr\D)©j 2 ©3r/l
then 7" = Ä © S, a(R) =X n 7> and a(5) c A" \ D.
6. Operators with thin spectra. If X is the "string of beads" example
discussed in §3 and if the linear measure of the intersection of X and the
x-axis is zero, then the two components of the interior of X belong to
different Gleason parts for R(X). This, in fact, is a special case of the more
general result: if X is compact, if T is a piecewise C2 closed Jordan curve
which intersects A in a set of measure zero on T, and a and b are in different
components of the complement of T, then a and b are in different Gleason
parts for R(X). In fact, X intersect the closure of the unbounded component
in a peak set for R(X). This result will provide sufficient conditions in terms
of the spectrum for a subnormal operator to have a nontrivial reducing
subspace.
We need the concept of analytically negligible in order to prove the above
result. A set E is analytically negligible if each function which is continuous
on C and analytic on an open set U can be approximated uniformly on C by
functions which are continuous on C and analytic on U u E. Examples of
analytically negligible sets and the relationship between analytic negligibility
and continuous analytic capacity may be found in Gamelin [13, Chapter
VIII]. Vitushkin [31] has shown that a piecewise C2 curve is analytically
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
312 R. G. LAUTZENHEISER
negligible. Davie and Oksendal have proved the following theorem which
relates analytic negligibility and rational approximation.
Theorem (6.1) (Davie and Oksendal [10]). Let K and M be compact sets
such that 3AT n dM is analytically negligible. Iff G C(K u M),f\K G R(K),
andf\M G R(M), then f G R(K u M).
If T is a closed Jordan curve, then the bounded and unbounded compo-
nents of the complement of T will be denoted by Tx and T2, respectively.
Theorem (6.2). If X is compact and T is a piecewise C2 closed Jordan curve
which intersects X in a set of measure zero on T, then X n T2 is a peak set for
R(X). In particular, points in X n T, and X n T2 will be in different Gleason
parts.
Proof. Let M = fx n X and K =T2n X. Since the measure of X n T is
zero, it follows from a theorem of Rudin (see Hoffman [18, p. 8]), that there
exists a continuous function g defined on Tx such that g is analytic on Tx,
g(z) = 1 for z G X n T, and |g(z)| < 1 for z G X n T. If we define/ on X
by/(z) = g(z) for z G M and/(z) - I for z € K, then/ G C(X),f\M = g G
R(M), and/ljf G R(K). Furthermore, dK n 3M is a subset of a piecewise C2
curve and, hence, is analytically negligible. It now follows from Theorem (6.1)
that/ G R(X). Thus, K is a peak set for R(X).
Theorem (6.3). Let X be a spectral set for T.IfT is a piecewise C2 Jordan
curve which intersects X in a set of measure zero on T, then T = R © S with
a(R) c X n r2 and a(S) cX n Tx.
Proof. Follows directly from Theorem (6.1) and Theorem (5.1).
Corollary (6.4). Let T be a subnormal operator. If T is a piecewise C2
Jordan curve which intersects o(T) in a set of measure zero on T, a(T) nF,^
0, and a(T) n T2 ¥= 0, then T has a nontrivial reducing subspace.
Proof. By the above theorem, T = R © 5 with a(R) c a(T) n T2 and
a(S) ca(7) n T,. Since a(T) n T, =£ 0, i = 1, 2, it is impossible for either
R or S to be trivial. Therefore, T has a nontrivial reducing subspace.
Remarks. C. R. Putnam has obtained results similar to ours for subnormal
operators. His approach differs from ours in that he uses the spectral measure
of the minimal normal extension and also employs a "projection" operator
technique. For more details, see the papers [24]-[26].
I would like to thank the referee for pointing out that Scott Brown [40]
recently solved the invariant subspace problem for subnormal operators. His
result has been generalized by Jim Agler [41] to operators whose spectra are
spectral sets.
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
spectral theory for subnormal operators 313
References
1. P. R. Ahem and D. Sarason, On some hypo-Dirichlet algebras of analytic functions, Amer. J.
Math. 89 (1967), 932-941.2. W. Arveson, Subalgebras of C*-algebras. II, Acta Math. 128 (1972), 271-308.3. S. K. Berberian, A note on operators whose spectrum is a spectral set, Acta Sei. Math.
(Szeged) 27 (1966), 201-203.4. C A. Berger, A strange dilation theorem. Notices Amer. Math. Soc. 12 (1965), 590.
5._, A brief note on the existence of spectral sets, preprint.
6. E. Bishop, A minimal boundary for function algebras, Pacific J. Math. 9 (1959), 629-642.
7. A. Browder, Introduction to function algebras, Benjamin, New York, 1969.
8. K. F. Clancey, Examples of non-normal seminormal operators whose spectra are not spectral
sets, Proc. Amer. Math. Soc. 24 (1970), 797-800.
9. K. F. Clancey and C R. Putnam, Normal parts of certain operators, J. Math. Soc. Japan 24
(1972), 198-203.10. A. M. Davie and B. K. Oksendal, Rational approximation on the union of sets, Proc. Amer.
Math. Soc. 29 (1971), 581-584.11. P. L. Duren, Theory of Hp spaces, Academic Press, New York, 1970.
12. C. Foias, Some applications of spectral sets. I, Harmonic-spectral measure, Acad. R. P.
Romîne Stud. Cere. Mat. 10 (1959), 365-401; English transi., Amer. Math. Soc. Transi. (2) 61
(1967), 25-62.13. T. W. Gamelin, Uniform algebras, Prentice-Hall, Englewood Cliffs, N. J., 1969.
14. T. W. Gamelin and J. Garnett, Pointwise bounded approximation and Dirichlet algebras, J.
Functional Analysis 8 (1971), 360-404.
15. A. Gleason, Function algebras in Seminar on Analytic functions. Vol. II, Institute for
Advanced Study, Princeton, N. J., 1957, pp. 213-226.16. P. R. Halmos, A Hilbert space problem book, Van Nostrand, Princeton, N. J., 1967.
17. E. Hewitt and K. Stromberg, Real and abstract analysis, Springer, New York, 1965.
18. K. Hoffman, Banach spaces of analytic functions, Prentice-Hall, Englewood Cliffs, N. J.,
1962.19. R. G. Lautzenheiser, Spectral sets, reducing subspaces, and function algebras, Ph. D. Thesis,
Indiana University, 1973.
20. A. Lebow, On von Neumann's theory of spectral sets, J. Math. Anal. Appl. 7 (1963), 64-90.
21. W. Mlak, Decompositions and extensions of operator valued representations of function
algebras, Acta Sei. Math. (Szeged) 30 (1969), 181-193.
22._, Partitions of spectral sets, Ann. Polon. Math. 25 (1972), 273-280.
23. C. R. Putnam, The spectra of subnormal operators, Proc. Amer. Math. Soc. 28 (1971),
473-477.24. _, Invariant subspaces of certain subnormal operators, J. Functional Analysis 17
(1974), 263-273.25._, Generalized projections and reducible subnormal operators, Duke Math. J. 43 (1976),
101-108.26._, Peak sets and subnormal operators, Illinois J. Math. 21 (1977), 388-394.
27. D. Sarason, On spectral sets having connected complement, Acta Sei. Math. (Szeged) 26
(1965), 289-299.28. G. L. Seever, Operator representations of uniform algebras. I, preprint.
29. J. G. Stampfli, A local spectra theory for operators. IV. Invariant subspaces, Indiana Univ.
Math. J. 22 (1972), 159-167.
30. M. Tsuji, Potential theory, Maruzen, Tokyo, 1959.
31. A. G. Vitushkin, Analytic capacity of sets andproblems in approximation theory, Uspehi Mat.
Nauk 22 (1967), 141-199; Russian Math. Surveys 22 (1967), 139-200.
32. J. von Neumann, Eine Spektraltheorie für allgemeine Operatoren eines imitaren Raumes,
Math. Nachr. 4 (1951), 258-281.33. B. L. Wadhwa, A hyponormal operator whose spectrum is not a spectral set, Proc. Amer.
Math. Soc. 38 (1973), 83-85.
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use
314 R. G. LAUTZENHEISER
34. J. Wermer, Dirichlet algebras, Duke Math. J. 27 (1960), 373-382.35. D. Wilken, Lebesgue measure for parts of R{X), Proc. Amer. Math. Soc. 18 (1967), 508-512.
36._, The support of representing measures for R(X), Pacific J. Math. 26 (1968), 621-626.
37. J. P. Williams, Minimal spectral sets of compact operators, Acta Sei. Math. (Szeged) 28
(1967), 93-106.38. L. Zalcman, Analytic capacity and rational approximation, Lecture Notes in Math., vol. 50,
Springer-Verlag, Berlin, 1968.39. M. S. Melnikov, On the Gleason parts of the algebra R(X), Mat. Sb. 101 (143) (1976), No. 2
(10), 293-300. (Russian)40. S. W. Brown, Some invariant subspaces for subnormal operators, preprint.
41. Jim Agler, An invariant subspace theorem, preprint.
Department of Mathematics, Rose-Hulman Institute of Technology, Terre Haute,
Indiana 47803
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use