ML EPS REG

ML_EPS_REG = [real]
Default: ML_EPS_REG = 1E-14 

Description: Initial value for the threshold of the eigenvalues of the covariance matrix in the evidence approximation.

---

This threshold is used to determine which eigenvalues $\lambda _{k}$ of the covariance matrix $\mathbf {\Phi } ^{\mathrm {T} }\mathbf {\Phi } /\sigma _{\mathrm {v} }^{2}$ are used in the optimization of the regularization parameters $\sigma _{\mathrm {w} }^{2}$ and $\sigma _{\mathrm {v} }^{2}$ determined by the following equations

$\sigma _{\mathrm {w} }^{2}={\frac {|\mathbf {\bar {w}} |^{2}}{\gamma }},$

$\sigma _{\mathrm {v} }^{2}={\frac {|\mathbf {T} -\mathbf {\phi } \mathbf {\bar {w}} |^{2}}{M-\gamma }},$

$\gamma =\sum \limits _{k=1}^{N_{\mathrm {B} }}{\frac {\lambda _{k}}{\lambda _{k}+1/\sigma _{\mathrm {w} }^{2}}}$.

All eigenvalues satisfying $\lambda _{i}/\lambda _{\mathrm {max} }$ > ML_EPS_REG are contributing by the above equations.

The smaller the value of ML_EPS_REG the the smaller the error of the fit. But at the same time the effects of overfitting increase. We determined empirically that the default value of 1E-14 is a safe value in most cases.

If at any point in the iterations of the Evidence Approximation of the square of the quadratic norm of errors (eigth column of REGR/REGRF in ML_LOGFILE) gets too big (more than 1.2 times larger than before) then ML_EPS_REG is doubled.

The seventh entry of REGR/REGRF in the ML_LOGFILE shows the ratio of the regularization ($\sigma _{v}^{2}/\sigma _{w}^{2}$) and the largest eigenvalue. Usually this number is a number with many varying digits. If this number becomes a "well rounded" number (e.g. 1.00000000E-14), it is an indication that the cap for the current ML_EPS_REG is reached. That means that regularization becomes crucial.

Related Tags and Sections

ML_LMLFF, ML_IALGO_LINREG, ML_IREG, ML_SIGV0, ML_SIGW0

Examples that use this tag

---