MultivariateNormalDegenerate#

class liesel.distributions.mvn_degen.MultivariateNormalDegenerate(loc, prec, rank=None, log_pdet=None, validate_args=False, allow_nan_stats=True, name='MultivariateNormalDegenerate', tol=1e-06)[source]#

Bases: Distribution

A potentially degenerate multivariate normal distribution.

Provides the alternative constructor from_penalty() and sampling via sample().

This is a simplified code-based illustration of how the log-probability for an array x is evaluated:

xc = x - loc
log_prob = -0.5 * (rank * np.log(2*np.pi) - log_pdet) -0.5 * (xc.T @ prec @ xc)
Parameters:
  • loc (Any) – The location (= mean) vector.

  • prec (Any) – The precision matrix (= a pseudo-inverse of the variance-covariance matrix).

  • rank (Union[Any, int, None]) – The rank of the precision matrix. Optional. (default: None)

  • log_pdet (Union[Any, float, None]) – The log-pseudo-determinant of the precision matrix. Optional. (default: None)

  • validate_args (bool) – Python bool, default False. When True, distribution parameters are checked for validity despite possibly degrading runtime performance. When False, invalid inputs may silently render incorrect outputs. (default: False)

  • allow_nan_stats (bool) – Python bool, default True. When True, statistics (e.g., mean, mode, variance) use the value NaN to indicate the result is undefined. When False, an exception is raised if one or more of the statistic’s batch members are undefined. (default: True)

  • name (str) – Python str, name prefixed to Ops created by this class. (default: 'MultivariateNormalDegenerate')

  • tol (float) – Numerical tolerance for determining which eigenvalues of the distribution’s precision matrices should be treated as zeros. Used in rank and log_pdet, if they are computed by the class. Also used in sample(). (default: 1e-06)

Notes

  • If they are not provided as arguments, rank and log_pdet are computed based on the eigenvalues of the precision matrix prec. This is an expensive operation and can be avoided by specifying the corresponding arguments.

  • When you draw samples from the distribution via sample(), it is always necessary to compute the eigendecomposition of the distribution’s precision matrices once and cache it, because sampling requires both the eigenvalues and eigenvectors.

Details on sampling

To draw samples from a denegerate multivariate normal distribution, we 1) draw standard normal samples with mean zero and variance one, 2) transform these samples to have the desired covariance structure, and 3) add the desired mean.

The main problem is to find out how we have to transform the standard normal samples in step 2. Say that we have a singular \((m \times m)\) precision matrix \(P\). We can view it as the generalized inverse of a variance-covariance matrix \(\Sigma\). We can obtain \(\Sigma\) by finding the eigenvalue decomposition of the precision matrix, i.e.

\[P = QA^+Q^T,\]

where \(Q\) is the orthogonal matrix of eigenvectors and \(A^+\) is the diagonal matrix of eigenvalues. Note that, if the precision matrix is singular, then \(\text{diag}(A^+)\) contains zeroes. Now we take the inverse of the non-zero entries of \(\text{diag}(A^+)\), while the zero entries remain at zero, resulting in a matrix \(A\). We can now write

\[\Sigma = Q A Q^T.\]

Now we can go through the three steps in detail. We first draw a vector of the desired length \(z \sim N(0, I)\) from a standard normal distribution. \(I\) is the identity matrix of appropriate dimension. Next, we transform the sample by applying \(x = Q A^{1/2}z\), such that \(\text{Cov}(x) = \Sigma\):

\[ \begin{align}\begin{aligned}\text{Cov}(x) & = Q A^{1/2} I (A^{1/2})^T Q^T \\\ & = Q A Q^T \\\ & = \Sigma.\end{aligned}\end{align} \]

In the last step, we add the desired mean \(\mu\) to \(x\). Note that the distribution is not a proper distribution on \(\mathbb{R}^m\), where \(m\) refers to the number of columns and rows of \(P\). Any vector in the null space of \(P\) can be added to any \(x \in \mathbb{R}^m\) without changing the density. The samples generated using the procedure described above are orthogonal to the null space of \(P\).

Methods

This section contains does not show inherited methods. Please refer to the documentation of the parent class for documentation on inherited methods.

from_penalty(loc, var, pen[, rank, ...])

Alternative constructor based on a penalty matrix and an inverse smoothing parameter.

Attributes

This section contains does not show inherited attributes. Please refer to the documentation of the parent class for documentation on inherited attributes.

eig

Eigenvalues and eigenvectors of the distribution's precision matrices.

loc

Locations.

log_pdet

Log-pseudo-determinants of the distribution's precision matrices.

prec

Precision matrices.

rank

Ranks of the distribution's precision matrices.