ML EPS REG

ML_EPS_REG = [real]
Default: ML_EPS_REG = 1E-14 

Description: Initial value for the threshold of the eigenvalues of the covariance matrix in the evidence approximation.

---

This threshold is used to determine which eigenvalues $\lambda_{k}$ of the covariance matrix $\mathbf{\Phi}^{\mathrm{T}}\mathbf{\Phi}/\sigma^{2}_{\mathrm{v}}$ are used in the optimization of the regularization parameters $\sigma^{2}_{\mathrm{w}}$ and $\sigma^{2}_{\mathrm{v}}$ determined by the following equations

$\sigma^{2}_{\mathrm{w}}=\frac{|\mathbf{\bar{w}}|^{2}}{\gamma},$

$\sigma^{2}_{\mathrm{v}}=\frac{|\mathbf{T}-\mathbf{\phi}\mathbf{\bar{w}}|^{2}}{M-\gamma},$

$\gamma=\sum\limits_{k=1}^{N_{\mathrm{B}}} \frac{\lambda_{k}}{\lambda_{k}+1/\sigma^{2}_{\mathrm{w}}}$.

All eigenvalues satisfying $\lambda_{i} / \lambda_{\mathrm{max}}$ > ML_EPS_REG are contributing by the above equations.

The parameter ML_EPS_REG is not necessarily kept constant throughout the calculation. If at any point in the iterations of the Evidence Approximation of the square of the quadratic norm of errors (eigth column of REGR/REGRF in ML_LOGFILE) gets too big (more than 1.2 times larger than before) then ML_EPS_REG is doubled.

The seventh entry of REGR/REGRF in the ML_LOGFILE shows the ratio of the regularization ($\sigma_{v}^{2}/ \sigma_{w}^{2}$) and the largest eigenvalue. Usually this number is a number with many varying digits. If this number becomes a "well rounded" number (e.g. 1.00000000E-14), it is an indication that the cap for the current ML_EPS_REG is reached. That means that regularization becomes crucial.

Related Tags and Sections

ML_LMLFF, ML_IALGO_LINREG, ML_IREG, ML_SIGV0, ML_SIGW0

Examples that use this tag

---