Covariance operator

In probability theory, for a probability measure P on a Hilbert space H with inner product ${\displaystyle \langle \cdot ,\cdot \rangle }$, the covariance of P is the bilinear form Cov: H × H → R given by

${\displaystyle \mathrm {Cov} (x,y)=\int _{H}\langle x,z\rangle \langle y,z\rangle \,\mathrm {d} \mathbf {P} (z)}$

for all x and y in H. The covariance operator C is then defined by

${\displaystyle \mathrm {Cov} (x,y)=\langle Cx,y\rangle }$

(from the Riesz representation theorem, such operator exists if Cov is bounded). Since Cov is symmetric in its arguments, the covariance operator is self-adjoint (the infinite-dimensional analogy of the transposition symmetry in the finite-dimensional case). When P is a centred Gaussian measure, C is also a nuclear operator. In particular, it is a compact operator of trace class, that is, it has finite trace.

Even more generally, for a probability measure P on a Banach space B, the covariance of P is the bilinear form on the algebraic dual B^#, defined by

${\displaystyle \mathrm {Cov} (x,y)=\int _{B}\langle x,z\rangle \langle y,z\rangle \,\mathrm {d} \mathbf {P} (z)}$

where ${\displaystyle \langle x,z\rangle }$ is now the value of the linear functional x on the element z.

Quite similarly, the covariance function of a function-valued random element (in special cases is called random process or random field) z is

${\displaystyle \mathrm {Cov} (x,y)=\int z(x)z(y)\,\mathrm {d} \mathbf {P} (z)=E(z(x)z(y))}$

where z(x) is now the value of the function z at the point x, i.e., the value of the linear functional ${\displaystyle u\mapsto u(x)}$ evaluated at z.