Schur product theorem

In mathematics, particularly in linear algebra, the Schur product theorem states that the Hadamard product of two positive definite matrices is also a positive definite matrix. The result is named after Issai Schur[1] (Schur 1911, p. 14, Theorem VII) (note that Schur signed as J. Schur in Journal für die reine und angewandte Mathematik.[2][3])

Proof

Proof using the trace formula

For any matrices ${\displaystyle M}$ and ${\displaystyle N}$, the Hadamard product ${\displaystyle M\circ N}$ considered as a bilinear form acts on vectors ${\displaystyle a,b}$ as

${\displaystyle a^{*}(M\circ N)b=\operatorname {tr} \left(M^{\textsf {T}}\operatorname {diag} \left(a^{*}\right)N\operatorname {diag} (b)\right)}$

where ${\displaystyle \operatorname {tr} }$ is the matrix trace and ${\displaystyle \operatorname {diag} (a)}$ is the diagonal matrix having as diagonal entries the elements of ${\displaystyle a}$.

Suppose ${\displaystyle M}$ and ${\displaystyle N}$ are positive definite, and so Hermitian. We can consider their square-roots ${\displaystyle M^{\frac {1}{2}}}$ and ${\displaystyle N^{\frac {1}{2}}}$, which are also Hermitian, and write

${\displaystyle \operatorname {tr} \left(M^{\textsf {T}}\operatorname {diag} \left(a^{*}\right)N\operatorname {diag} (b)\right)=\operatorname {tr} \left({\overline {M}}^{\frac {1}{2}}{\overline {M}}^{\frac {1}{2}}\operatorname {diag} \left(a^{*}\right)N^{\frac {1}{2}}N^{\frac {1}{2}}\operatorname {diag} (b)\right)=\operatorname {tr} \left({\overline {M}}^{\frac {1}{2}}\operatorname {diag} \left(a^{*}\right)N^{\frac {1}{2}}N^{\frac {1}{2}}\operatorname {diag} (b){\overline {M}}^{\frac {1}{2}}\right)}$

Then, for ${\displaystyle a=b}$, this is written as ${\displaystyle \operatorname {tr} \left(A^{*}A\right)}$ for ${\displaystyle A=N^{\frac {1}{2}}\operatorname {diag} (a){\overline {M}}^{\frac {1}{2}}}$ and thus is strictly positive for ${\displaystyle A\neq 0}$, which occurs if and only if ${\displaystyle a\neq 0}$. This shows that ${\displaystyle (M\circ N)}$ is a positive definite matrix.

Case of M = N

Let ${\displaystyle X}$ be an ${\displaystyle n}$-dimensional centered Gaussian random variable with covariance ${\displaystyle \langle X_{i}X_{j}\rangle =M_{ij}}$. Then the covariance matrix of ${\displaystyle X_{i}^{2}}$ and ${\displaystyle X_{j}^{2}}$ is

${\displaystyle \operatorname {Cov} \left(X_{i}^{2},X_{j}^{2}\right)=\left\langle X_{i}^{2}X_{j}^{2}\right\rangle -\left\langle X_{i}^{2}\right\rangle \left\langle X_{j}^{2}\right\rangle }$

Using Wick's theorem to develop ${\displaystyle \left\langle X_{i}^{2}X_{j}^{2}\right\rangle =2\left\langle X_{i}X_{j}\right\rangle ^{2}+\left\langle X_{i}^{2}\right\rangle \left\langle X_{j}^{2}\right\rangle }$ we have

${\displaystyle \operatorname {Cov} \left(X_{i}^{2},X_{j}^{2}\right)=2\left\langle X_{i}X_{j}\right\rangle ^{2}=2M_{ij}^{2}}$

Since a covariance matrix is positive definite, this proves that the matrix with elements ${\displaystyle M_{ij}^{2}}$ is a positive definite matrix.

General case

Let ${\displaystyle X}$ and ${\displaystyle Y}$ be ${\displaystyle n}$-dimensional centered Gaussian random variables with covariances ${\displaystyle \left\langle X_{i}X_{j}\right\rangle =M_{ij}}$, ${\displaystyle \left\langle Y_{i}Y_{j}\right\rangle =N_{ij}}$ and independent from each other so that we have

${\displaystyle \left\langle X_{i}Y_{j}\right\rangle =0}$ for any ${\displaystyle i,j}$

Then the covariance matrix of ${\displaystyle X_{i}Y_{i}}$ and ${\displaystyle X_{j}Y_{j}}$ is

${\displaystyle \operatorname {Cov} \left(X_{i}Y_{i},X_{j}Y_{j}\right)=\left\langle X_{i}Y_{i}X_{j}Y_{j}\right\rangle -\left\langle X_{i}Y_{i}\right\rangle \left\langle X_{j}Y_{j}\right\rangle }$

Using Wick's theorem to develop

${\displaystyle \left\langle X_{i}Y_{i}X_{j}Y_{j}\right\rangle =\left\langle X_{i}X_{j}\right\rangle \left\langle Y_{i}Y_{j}\right\rangle +\left\langle X_{i}Y_{i}\right\rangle \left\langle X_{j}Y_{j}\right\rangle +\left\langle X_{i}Y_{j}\right\rangle \left\langle X_{j}Y_{i}\right\rangle }$

and also using the independence of ${\displaystyle X}$ and ${\displaystyle Y}$, we have

${\displaystyle \operatorname {Cov} \left(X_{i}Y_{i},X_{j}Y_{j}\right)=\left\langle X_{i}X_{j}\right\rangle \left\langle Y_{i}Y_{j}\right\rangle =M_{ij}N_{ij}}$

Since a covariance matrix is positive definite, this proves that the matrix with elements ${\displaystyle M_{ij}N_{ij}}$ is a positive definite matrix.

Proof of positive semidefiniteness

Let ${\displaystyle M=\sum \mu _{i}m_{i}m_{i}^{\textsf {T}}}$ and ${\displaystyle N=\sum \nu _{i}n_{i}n_{i}^{\textsf {T}}}$. Then

${\displaystyle M\circ N=\sum _{ij}\mu _{i}\nu _{j}\left(m_{i}m_{i}^{\textsf {T}}\right)\circ \left(n_{j}n_{j}^{\textsf {T}}\right)=\sum _{ij}\mu _{i}\nu _{j}\left(m_{i}\circ n_{j}\right)\left(m_{i}\circ n_{j}\right)^{\textsf {T}}}$

Each ${\displaystyle \left(m_{i}\circ n_{j}\right)\left(m_{i}\circ n_{j}\right)^{\textsf {T}}}$ is positive semidefinite (but, except in the 1-dimensional case, not positive definite, since they are rank 1 matrices). Also, ${\displaystyle \mu _{i}\nu _{j}>0}$ thus the sum ${\displaystyle M\circ N}$ is also positive semidefinite.

Proof of definiteness

To show that the result is positive definite requires further proof. We shall show that for any vector ${\displaystyle a\neq 0}$, we have ${\displaystyle a^{\textsf {T}}(M\circ N)a>0}$. Continuing as above, each ${\displaystyle a^{\textsf {T}}\left(m_{i}\circ n_{j}\right)\left(m_{i}\circ n_{j}\right)^{\textsf {T}}a\geq 0}$, so it remains to show that there exist ${\displaystyle i}$ and ${\displaystyle j}$ for which corresponding term above is non-negative. For this we observe that

${\displaystyle a^{\textsf {T}}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{\textsf {T}}a=\left(\sum _{k}m_{i,k}n_{j,k}a_{k}\right)^{2}}$

Since ${\displaystyle N}$ is positive definite, there is a ${\displaystyle j}$ for which ${\displaystyle n_{j}\circ a\neq 0}$ (since otherwise ${\displaystyle n_{j}^{\textsf {T}}a=\sum _{k}(n_{j}\circ a)_{k}=0}$ for all ${\displaystyle j}$), and likewise since ${\displaystyle M}$ is positive definite there exists an ${\displaystyle i}$ for which ${\displaystyle \sum _{k}m_{i,k}(n_{j}\circ a)_{k}=m_{i}^{\textsf {T}}(n_{j}\circ a)\neq 0.}$ However, this last sum is just ${\displaystyle \sum _{k}m_{i,k}n_{j,k}a_{k}}$. Thus its square is positive. This completes the proof.

References

1. Schur, J. (1911). "Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen". Journal für die reine und angewandte Mathematik. 1911 (140): 1–28. doi:10.1515/crll.1911.140.1. S2CID 120411177.
2. Zhang, Fuzhen, ed. (2005). The Schur Complement and Its Applications. Numerical Methods and Algorithms. 4. doi:10.1007/b105056. ISBN 0-387-24271-6., page 9, Ch. 0.6 Publication under J. Schur
3. Ledermann, W. (1983). "Issai Schur and His School in Berlin". Bulletin of the London Mathematical Society. 15 (2): 97–106. doi:10.1112/blms/15.2.97.

External links

• Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen at EUDML