Inequalities for Unitarily Invariant Norms and Bilinear MatrixProductsRoger A. Horn and Roy MathiasThe Johns Hopkins University, Baltimore, Maryland 21218, USAYoshihiro NakamuraResearch Institute for Applied Electricity, Hokkaido University, Sapporo 060, JapanMay 4, 2001AbstractWe give several criteria that are equivalent to the basic singular value majorization inequality(1.1) that is common to both the usual and Hadamard products. We then use these criteriato give a uni�ed proof of the basic majorization inequality for both products. Finally, weintroduce natural generalizations of the usual and Hadamard products and show that althoughthese generalizations do not satisfy the majorization inequality, they do satisfy an importantweaker inequality that plays a role in establishing their submultiplicativity with respect to everyunitarily invariant norm.1 IntroductionIn this paper we consider three results that are true for both the Hadamard product and theusual product, and in an attempt to better understand these results, study their generalizations tofamilies of bilinear products that contain these two familiar products.LetMm;n denote the space ofm-by-n complex matrices, and de�ne Mn �Mn;n. The Hadamardproduct of A;B 2 Mm;n is A � B � [aijbij ] 2 Mm;n. We use � : Mm1;n1 �Mm2;n2 ! Mm3;n3 todenote a bilinear function, which we interpret as a `product'. Given A 2Mm;n let �i(A) denote theith largest singular value of A for i = 1; : : : ;minfm;ng. A norm k � k on Mm;n is unitarily invariantif kUAV k = kAk for all A 2 Mm;n and all unitary U 2 Mm and V 2 Mn; a norm k � k on Mn isunitary similarity invariant if kUAU�k = kAk for all A 2 Mn and all unitary U 2 Mn. A normk �kQ onMm;n is said to be a Q-norm if there is some unitarily invariant norm k �k onMn such thatkAkQ = kA�Ak1=2 for all A 2 Mm;n [2]; notice that a Q-norm is necessarily unitarily invariant. Amatrix P 2Mm;n is called a partial isometry if its singular values are all either 0 or 1.After some preliminary general remarks about bilinear products, in Theorem 3.1 we present1

several equivalent conditions for a bilinear product � to satisfy the basic inequalitykXi=1 �i(A �B) � kXi=1 �i(A)�i(B) for all A 2Mm1 ;n1 ; B 2Mm2;n2 ; k = 1; : : : ; q; (1.1)where q � minfm1; m2; m3; n1; n2; n3g. The inequality (1.1) is satis�ed by both the Hadamardproduct [8, Lemma 1] and the usual product [5]. Although the proofs of (1.1) for the two productsin [8] and [5] are very di�erent, using the characterization in Theorem 3.1 one can readily verify(1.1) for both the usual product and the Hadamard product by similar arguments.Next we introduce two families of partitioned products, (3.8) and (3.9), that contain theusual product and Hadamard product as special cases, and show that the usual product and theHadamard product are the only members of these two families for which the basic inequality (1.1)holds.In [10] (1.1) was used to prove that, for all A and B of appropriate dimensions and all unitarilyinvariant norms, both the usual product and the Hadamard product satisfy an inequality of Cauchy-Schwarz type: kA �Bk2 � kA�Ak kB�Bk: (1.2)In [8], (1.1) was used to proveTheorem 1.1 Let k � k be a unitarily invariant norm on Mn, and let � be either the usual productor the Hadamard product. Then the following are equivalent :(a) kAk � �1(A) for all A 2Mn(b) kAk � �(A) for all A 2Mn(c) kA �Bk � kAk kBk for all A;B 2MnWe show that both (1.2) and Theorem 1.1 hold for the two families of partitioned products (3.8)ann (3.9), even though the full set of inequalities (1.1) does not hold for these products.For simplicity of notation we consider only the square case mi = ni = n in the remainder ofthis paper; the general rectangular case follows easily from this by adding suitable zero blocks.The following result of Ky Fan ([4, Theorem 1] or [9, Chapter 3]) will be useful.Lemma 1.2 Let A 2Mn. ThenkXi=1 �i(A) = maxfjtr PAj : P 2Mn; rank (P ) � k; �1(P ) � 1g k = 1; : : : ; n: (1.3)2 Some General Remarks on Bilinear ProductsAn unknown matrix A = [aij ] 2 Mn is completely determined by the values of tr AX for allX 2Mn. Indeed, it su�ces to know these values only for the n2 matrices X = Eij , i; j = 1; : : : ; n,where all entries of Eij are zero except for an entry 1 at the intersection of row i and j; one checks2

that tr AEij = aij . It follows from this observation that a given bilinear product � on Mn iscompletely determined by the values of tr (A �B)C for all A;B;C 2 Mn. In particular, if �1 and�2 are two given bilinear products on Mn, then A �1 B = A �2 B for al A;B 2 Mn if and only iftr A(�1B)C = tr (A �2 B)C for all A;B;C 2Mn.Associated with any bilinear product � on Mn is another bilinear product �R characterized bythe adjoint-like identitytr (A�RB)C � tr A(B � C) for all A;B;C 2Mn: (2.1)The notation �R is intended to remind us that the parentheses and position of the bilinear productare moved one position to the right in (2.1). The same process associates with �R another bilinearproduct (�R)R, which we denote simply by �RR:tr (A�RRB)C � tr A(B�RC) = tr (B�RC)A = tr B(C �A) for all A;B;C 2Mn: (2.2)The construction of new bilinear products in this way stops at this point however, as one �nds thattr (A �RRR B)C = tr B(C�RA) = tr (C�RA)B = tr C(A �B) = tr (A �B)Cfor all A;B;C 2Mn, so �RRR = �.It is easy to identity �R and �RRfor several familiar bilinear products �. For example, when �is the usual matrix product, we �nd thattr (A�RB)C =; tr A(B �C) = tr ABC = tr (A �B)C;so A�RB = AB; it follows that A�RRB = AB as well. Thus, when � is the usual matrix productthe �R and �RR are also the usual matrix product.When � is the Hadamard product �, it is straightforward to verify that tr (A�RB)C = tr A(B �C) = tr (A �BT )C, andtr (A �RR BC = tr A(B �R C) = tr (B �R C)A = tr (BC �A) = tr (AT �B)C:Thus, A �R B = A �BT and A �RR B = AT �B.Another simple bilinear product is given by A �B � ABT . One veri�es easily in this case thatA�RB = BTAT and A�RRB = ATB.A fourth bilinear product is given by A �B � UAV BW , where U; V;W 2Mn are given. In thiscase, tr (A�RB)C = tr A(B � C) = tr A(UBV CW ) = tr (WAUBV )C;so A�RB = WAUBV ), a bilinear product of the same form as �, but with the three structuralmatrices U; V andW shifted cyclically one position to the right. It follows that a�RRB = VAWBU .This last product is of particular interest because any bilinear product � on Mn may be repre-sented as A �B = NXk=1UkAVkBWk for all A;B 2Mn (2.3)3

for some N � n6 and given Uk; Vk;Wk 2 Mn, k = 1; : : : ; N . This is clear from the fact thatEijAElmBEpq = ajlbmpEiq. If a representation of the form (2.3) is known for a given bilinearproduct �, then �R and �RR are given byA �R B = NXk=1WkAUkBVk and A �RR B = NXk=1VkAWkBUk for all A;B 2Mn: (2.4)3 Main ResultsTheorem 3.1 Let � :Mn�Mn !Mn be given bilinear product. Then the following are equivalent:(a) Pki=1 �i(A �B) �Pki=1 �i(A)�i(B) for all k = 1; : : : ; n, and all A;B 2Mn.(b) jtr P (Q �R)j � minfrank P; rank Q; rank Rg for all partial isometries P;Q;R 2Mn.(c) jtr P (Q�R)j � 1 for all partial isometries P;Q;R 2Mn with minfrank P; rank Q; rank Rg =1.(d) For all A;B 2Mn , �1(A �B) � �1(A)�1(B): (3.1)nXi=1 �i(A �B) � �1(A) nXi=1 �i(B) (3.2)and nXi=1 �i(A �B) � �1(B) nXi=1 �i(A): (3.3)(e) For any A;B 2Mn �1(A �B) � �1(A)�1(B) (3.4)�1(A�RB) � �1(A)�1(B) (3.5)and �1(A�RRB) � �1(A)�1(B); (3.6)where �R and �RR are de�ned by (2.1) and (2.2).Proof: Lemma 1.2 shows that (a) implies (b). By the linearity of the trace function, and the factthat any partial isometry is the sum of rank one partial isometries, (c) implies (b). However, (c) isa special case of (b), so (b) and (c) are equivalent. Now we show that (b) implies (a).Suppose that � is a bilinear product for which (b) holds, and let A;B be given. De�ne�i � �i(A)� �i+1(A) � 0 i = 1; : : : ; n�1�i � �i(B)� �i+1(B) � 0 i = 1; : : : ; n�14

and �n = �n(A) � 0, �n = �n(B) � 0. ThennXi=j �i = �j(A) and nXi=j �i = �j(B):By the singular value decomposition [7, Theorem 7.3.4], we may writeA = nXi=1 �iQi and B = nXi=1 �iRi;where Qi and Pi are rank i partial isometries. Now let P be a given rank k partial isometry andcompute jtr P (A �B)j = j tr P [( nXi=1 �iQi) � ( nXj=1�jRj)] j= j tr P ( nXi;j=1�i�jQi �Rj) j� nXi;j=1 jtr P (�i�jQi �Rj)j� nXi;j=1�i�jminfi; j; kg= kXl=1 ( nXi=l �i)( nXj=l �j)= kXl=1 �l(A)�l(B):The inequality (a) now follows from Lemma 1.2. We have shown that (a), (b) and (c) are equivalent.It is clear that (a) implies (d). Now let us show that (d) implies (e). The inequality (3.4) is thesame as (3.1). To deduce (3.5) we use (3.2) to compute�1(A�RB) = maxfjtr (A�RB)Cj : C 2Mn; rank C = 1; �1(C) � 1g= maxfjtr A(B � C)j : C 2Mn; rank C = 1; �1(C) � 1g� maxf�1(A) nXi=1 �i(B � C) : C 2Mn; rank C = 1; �1(C) � 1g� maxf�1(A)�1(B) nXi=1 �i(C) : C 2Mn; rank C = 1; �1(C) � 1g= �1(A)�1(B):The inequality (3.6) follows in the same way from (3.3).Finally we show that (e) implies (c). Let P;Q;R 2Mn be partial isometries with minfrank P; rank Q; rank Rg =1. If rank P = 1 then jtr P (Q �R)j � �1(Q �R) � �1(Q)�1(R) = 15

by Lemma 1.2 and (3.4). If rank Q = 1 (respectively, rank R =1) then proceed in the same wayusing the de�nition of �R (respectively, �RR) and (3.6) (respectively, (3.5)).It is now easy to use Theorem 3.1 (e) to verify the inequality (1.1) for the usual product andthe Hadamard product. This is immediate for the usual product since in this case both �R and �RRare the usual product and we have the well known inequality �1(AB) � �1(A)�1(B). Since �1(�)is also submultiplicative with respect to the Hadamard product � (see [12] for the original proof or[6] for a variety of di�erent proofs), we have �1(A �B) � �1(A)�1(B),sigma1(A�RB) = �1(A �BT ) � �1(A)�1(BT ) = �1(A)�1(B);and �1(A�RB) = �1(AT �B) � �1(AT )�1(B) = �1(A)�1(B):Now that we have a common proof of the basic inequality (1.1) for the usual product and theHadamard product it is natural to look for a common generalization of these two results. For thisreason we considered two natural families of products that contain both the usual product and theHadamard product.Let p; q be positive integers, let n = pq, and let A;B 2Mn be partitioned asA = [Aij ]pi;j=1; B = [Bij ]pi;j=1 with Aij ; Bij 2Mq: (3.7)De�ne the products 21 and 22 on Mn byA21B = [AijBij ]pi;j=1 (3.8)and A22B = [ pXk=1Aik �Bkj ]pi;j=1: (3.9)If p = 1 then 21 is the usual product and 22 is the Hadamard product, while if q = 1 then 21 isthe Hadamard product and 22 is the usual product.Although 21 and 22 look rather di�erent, they are essentially the same in the context of unitarysimilarity invariant norms. The following result can be proved by direct computation.Lemma 3.2 Let p and q be given integers, let n = pq, and let A;B 2 Mn be partitioned as in(3.7). Then A22B = V ((V �AV )21(V �BV ))V � (3.10)where V 2Mn is the permutation matrix corresponding to the permutation�(p(k� 1) + j) = q(j � 1) + k j = 1; : : : ; p k = 1; : : : ; q:6

Using the identity (3.10) and the unitary similarity invariance of the singular values, it is clearthat the basic inequality (1.1) holds for the product 21 if and only if it holds for 22(with respectto the partitioning pq = n). Consider the following two matrices, where p = q = 2 and n = pq = 4.A = 0BBB@ 1 0 0 10 0 0 00 0 0 01 0 0 11CCCA ; B = 0BBB@ 1 0 0 00 0 1 00 1 0 00 0 0 11CCCA :Then the singular values of A are 2; 0; 0; 0 and those of B are 1; 1; 1; 1, while those ofA21B = 0BBB@ 1 0 1 00 0 0 00 0 0 00 1 0 11CCCAare p2;p2; 0; 0. Thus, the inequality (1.1) fails to hold for the product 21 (and hence also for 22)for k = 2; 3; 4, and there is no hope of being able to prove (1.1) for these general products exceptin the cases p = 1 or q = 1, which are exactly the usual and Hadamard products.Despite the failure of the products 21 and 22 to satisfy the basic inequalities (1.1), there issomething more that can be said about them. The inequality (1.2) of Cauchy-Schwarz type and theequivalences in Theorem 1.1 for the usual and Hadamard products have been strongly associatedin the literature with the basic inequality (1.1). Nevertheless, both of these results are correct forthe general products 21 and 22, which do not satisfy (1.1). Before giving a proof of this assertion,we introduce two preliminary results. For X 2Mm;n we denote jX j � (X�X)1=2.The following result is part of Theorem 2.3 of [11].Lemma 3.3 Suppose that the block matrixA = � L XX� M �is positive semide�nite, where X 2Mm;n. Then, L and M are positive semide�nite andk jX jp k2 � kLpk kMpk (3.11)for all p > 0, all k = 1; : : : ; minfm;ng, and all unitarily invariant norms k � k.Lemma 3.4 Let A;B 2 Mn, let n = pq, and let F = [Fij ]pi;j=1, where Fij 2 Mq is the matrix ofall ones if i = j and is the zero matrix if i 6= j. Then(a) kA � Fk � kAk for any unitary similarity invariant norm k � k.(b) The block matrix � (AA�) � F (A21B)(A21B)� (B�B) � F � (3.12)is positive semide�nite. 7

Proof: The assertion in (a) is proved in [3]; A � F is a convex combination of (diagonal) unitarysimilarities of A.To prove (b), take any vectors x = [xj ]pj=1 and y = [yj ]pj=1, with xj ; yj 2 Cq. Then, using theCauchy-Schwarz inequality inequality twice, we havejy�(A21B)xj2 = j pXi;j=1 y�iAijBijxj j2= j pXi;j=1(A�ijyi)�(Bijxj )j2� f pXi;j=1 kA�ijyik2kBijxjk2 g2� f pXi;j=1 kA�ijyik22gf pXi;j=1 kBijxjk22 g= f pXi=1 y�i ( pXj=1AijA�ij)yi gf pXj=1 x�i ( pXi=1B�ijBij)xi g= y�[(AA�) � F ]y � x�[(B�B) � F ]x:The assertion (b) now follows from the criterion in [7, Theorem 7.7.7 (a)].We can now state and prove the analogs of (1.2) and Theorem 1.1 for the partitioned products21 and 22.Theorem 3.5 Let n = pq, let � denote either of the products 21 or 22, let F 2Mn be de�ned asin Lemma 3.4, let k � k be any given unitarily invariant norm on Mn, and let k � kQ be any Q-normon Mn. ThenkA �Bk2 � k(AA�) � Fk k(B�B) � Fk � kA�Ak kB�Bk for all A;B 2Mn (3.13)and kA �BkQ � �1(A)kBkQ and kA �BkQ � �1(B)kAkQ for all A;B 2Mn: (3.14)Furthermore, the following are equivalent(a) kAk � �1(A) for all A 2Mn(b) kAk � �(A) for all A 2Mn(c) kA �Bk � kAk kBk for all A;B 2MnProof: To prove (3.13) for 21, apply (3.11) with p = 1 to the positive semide�nite matrix (3.12)and use (a) in Lemma 3.4:kA21Bk2 � k(AA�) � Fk k(B�B) � kAA�k kB�Bk = kA�Ak kB�Bk:8

To prove the second inequality in (3.14), let k � k be the unitarily invariant norm associated withk � kQ. Because the matrix in (3.12) is positive semide�nite we have [1, Lemma 2]A21B = ((AA)� � F )1=2C((B�B) � F )1=2for some contraction C. Thus(A21B)�(A21B) = ((B�B) � F )1=2C�((AA)� � F )C((B�B) � F )1=2:Since kXYZk � �1(X) kY k �1(Z) for any unitarily invariant norm we havekA21Bk2Q = k(A21B)�(A21B)k= k((B�B) � F )1=2C�((AA)� � F )C((B�B) � F )1=2k� �1(((B�B) � F )1=2C�) k(AA)� � Fk �1(C((B�B) � F )1=2)� �1(((B�B) � F )1=2) k((AA)� � F )k �1(((B�B) � F )1=2)= �1(((B�B) � F )) k((AA)� � F )k� �1(B�B)kAA�k= �21(B)kAk2Q:For the last inequality we have used Lemma 3.4 (a) with the two unitarily invariant norms �1(�)and k � k. To prove the �rst inequality in (3.14), apply the preceding argument to (A21B)(A21B)�and use the fact that, for any unitarily invariant norms onMn XX� has the same norm asX�X forany X 2Mn since both products have the same singular values. The inequalities (3.13) and (3.14)for 21 now imply the same inequalities for 22 by using (3.10) and the unitary invariance of k � k.The equivalence of (a) and (b) is the same as in Theorem 1.1. To show that (c) implies (a), letA = B = E11, the n-by-n matrix whose 1;1 entry is 1 and all other entries are 0, and notice thatE11 �E11 = E11. Then (c) gives kE11k � kE11k2, so kE11k � 1. Now use the fact that a unitarilyinvariant norm is a Schur-convex function of the singular values of its argument [7, Corollary 7.4.47]to conclude that kAk � k�1(A)E11k = �1(A)kE11k � �1(A). Conversely, if (a) holds then we canuse (3.13), Theorem 1.1 for the usual product, and the fact that every unitarily invariant norm isself adjoint to obtainkA �Bk2 � kA�Ak kB�Bk � kA�k kAk kB�k kBk = kAk2kBk2:Taking square roots gives (c).In particular, Theorem 3.5 (a,c) shows that 21and 22are submultiplicative with respect to thespectral norm, a general result that includes spectra norm submultiplicativity for both the usualand Hadamard product.Although the products 21 and 22 both satisfy the Cauchy-Schwarz inequality (3.13), and soin particular are submultiplicative with respect to the spectral norm, the products �R and �RRassociated with them are not even are not even submultiplicative with respect to the spectralnorm. We will only prove this statement for the products �R and �RR associated with 21. (Note9

that it is clear from the equivalence of (a) and (e) in Theorem 3.1 that at least one on �R and �RR isnot spectral norm submultiplicative.) By a straightforward computation we have a representationof 21 in the form (2.3) A21B = pXi;j=1P iiAP jiBP jj ;where P ij 2 Mpq denotes the p� p block matrix with i; j block equal to Iq and all other blocks 0(i.e. P ij = Eij Iq with Eij 2Mp). Now using (2.4) we haveA�RB = pXi;j=1P jjAP iiBP ji and A�RRB = pXi;j=1P jiAP jjBP ii:and hence that A�RB and A�RRB are block diagonal with diagonal blocks given by(A�RB)ii = pXk=1AikBik and (A�RRB)ii = pXk=1AkiBki:Using these formulae with p = q = 2 one can see that forA = 0BBB@ 1 0 0 10 0 0 00 0 0 00 0 0 01CCCA and B = 0BBB@ 1 0 0 00 0 1 00 0 0 00 0 0 01CCCAwe have A�RB = BT �RRAT = 0BBB@ 2 0 0 00 0 0 00 0 0 00 0 0 01CCCA ;from which it is evident that neither �R nor �RR are submultiplicative with respect to the spectralnorm.Acknowledgement We are grateful to Prof. T. Ando for pointing out that the inequality (1.1) isfalse for 21.References[1] T. Ando, R. A. Horn, and C. R. Johnson. The singular values of a Hadamard product: Abasic inequality. Lin. Multilin. Alg., 21:345{65, 1987.[2] R. Bhatia. Perturbation inequalities for the absolute value map in norm ideals of operators.J. Operator Theory, 19:129{36, 1988.[3] C. Davis. Various averaging operations onto subalgebras. Illinois J. Math., 3:523{53, 1959.[4] K. Fan. Maximum properties and inequalities for the eigenvalues of completely continuousoperators. Proc. Nat. Acad. Sci., 37:760{6, 1951.10

[5] A. Horn. On the singular values of a product of completely continuous operators. Proc. Nat.Acad. Sci., 36:374{5, 1950.[6] R. A. Horn. The Hadamard product. In Matrix Theory and Applications. American Mathe-matical Society, Providence, to appear.[7] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, New York,1985.[8] R. A. Horn and C. R. Johnson. Hadamard and conventional submultiplicativity for unitarilyinvariant norms on matrices. Lin. Multilin. Alg., 20:91{106, 1987.[9] R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, NewYork, 1989.[10] R. A. Horn and R. Mathias. An analog of the Cauchy-Schwarz inequality for Hadamardproducts and unitarily invariant norms. to appear in SIAM J. Matrix Anal. Appl.[11] R. A. Horn and R. Mathias. Cauchy-Schwarz inequalities associated with positive semide�nitematrices. to appear in Linear Algebra Appl.[12] J. Schur. Bermerkungen zur Theorie der beschr�ankten Bilinearformen mit unendlich vielenVer�anderlichen. J. f�ur Reine und Angewandte Mathematik, 140:1{28, 1911.

11