C.4 The covariance matrix

The terms of the covariance matrix $V_{ij}$ are defined by:

$\begin{equation} V_{ij} = \left< \left(u_ i - u^0_ i\right) \left(u_ j - u^0_ j\right) \right> \label{eqn:def_ covar} \end{equation}$

(C.8)

Its leading diagonal terms may be recognised as equalling the variances of each of our $n_\mathrm {u}$ variables; its cross terms measure the correlation between the variables. If a component $V_{ij} > 0$ , it implies that higher estimates of the coefficient $u_ i$ make higher estimates of $u_ j$ more favourable also; if $V_{ij} < 0$ , the converse is true.

It is a standard statistical result that $\mathbf{V} = (-\mathbf{A})^{-1}$ . In the remainder of this section we prove this; readers who are willing to accept this may skip onto Section C.5.

Using $\Delta u_ i$ to denote $\left(u_ i - u^0_ i\right)$ , we may proceed by rewriting Equation () as:

	$\displaystyle V_{ij}$	$\displaystyle =$	$\displaystyle \idotsint _{u_ i=-\infty }^{\infty } \Delta u_ i \Delta u_ j \mathrm{P}\left( \mathbf{u} \| \left\{ \mathbf{x}_ i, f_ i, \sigma _ i \right\} \right) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u}$		(C.9)
	$\displaystyle$	$\displaystyle =$	$\displaystyle \frac{ \idotsint _{u_ i=-\infty }^{\infty } \Delta u_ i \Delta u_ j \exp (-Q) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} }{ \idotsint _{u_ i=-\infty }^{\infty } \exp (-Q) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} } \nonumber$

The normalisation factor in the denominator of this expression, which we denote as $Z$ , the partition function, may be evaluated by $n_\mathrm {u}$ -dimensional Gaussian integration, and is a standard result:

	$\displaystyle Z$	$\displaystyle =$	$\displaystyle \idotsint _{u_ i=-\infty }^{\infty } \exp \left(\frac{1}{2} \Delta \mathbf{u}^\mathbf {T} \mathbf{A} \Delta \mathbf{u} \right) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u}$		(C.10)
	$\displaystyle$	$\displaystyle =$	$\displaystyle \frac{(2\pi )^{n_\mathrm {u}/2}}{\mathrm{Det}(\mathbf{-A})} \nonumber$

Differentiating $\log _ e(Z)$ with respect of any given component of the Hessian matrix $A_{ij}$ yields:

$\begin{equation} -2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e(Z) \right] = \frac{1}{Z} \idotsint _{u_ i=-\infty }^{\infty } \Delta u_ i \Delta u_ j \exp (-Q) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} \end{equation}$

(C.11)

which we may identify as equalling $V_{ij}$ :

$\displaystyle \label{eqa:v_ zrelate} V_{ij}$	$\displaystyle =$	$\displaystyle -2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e(Z) \right]$	(C.12)
$\displaystyle$	$\displaystyle =$	$\displaystyle -2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e((2\pi )^{n_\mathrm {u}/2}) - \log _ e(\mathrm{Det}(\mathbf{-A})) \right] \nonumber$
$\displaystyle$	$\displaystyle =$	$\displaystyle 2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e(\mathrm{Det}(\mathbf{-A})) \right] \nonumber$

This expression may be simplified by recalling that the determinant of a matrix is equal to the scalar product of any of its rows with its cofactors, yielding the result:

$\begin{equation} \frac{\partial }{\partial A_{ij}} \left[\mathrm{Det}(\mathbf{-A})\right] = -a_{ij} \end{equation}$

(C.13)

where $a_{ij}$ is the cofactor of $A_{ij}$ . Substituting this into Equation () yields:

$\begin{equation} V_{ij} = \frac{-a_{ij}}{\mathrm{Det}(\mathbf{-A})} \end{equation}$

(C.14)

Recalling that the adjoint $\mathbf{A}^\dagger$ of the Hessian matrix is the matrix of cofactors of its transpose, and that $\mathbf{A}$ is symmetric, we may write:

$\begin{equation} V_{ij} = \frac{-\mathbf{A}^\dagger }{\mathrm{Det}(\mathbf{-A})} \equiv (-\mathbf{A})^{-1} \end{equation}$

(C.15)

which proves the result stated earlier.