ML IALGO LINREG

ML_IALGO_LINREG = [integer]
Default: ML_IALGO_LINREG = 1 

Description: This tag determines the algorithm that is employed to solve the system of linear equations in the ridge regression method for machine learning.

---

In the ridge regression method for machine learning one needs to solve for the unknown weights $\mathbf{w}$ minimizing

$|| \mathbf{Y} - \mathbf{\Phi} \mathbf{w} || \rightarrow \mbox{min}$

For more details please see here.

The following options are available to solve for $\mathbf{w}$:

• ML_IALGO_LINREG=1: Bayesian linear regression (see here). Recommended for NSW$\ge$1. Usable with ML_MODE = TRAIN, SELECT, and REFITBAYESIAN.
• ML_IALGO_LINREG=2: QR factorization. Usable with ML_MODE = REFIT and REFITBAYESIAN.
• ML_IALGO_LINREG=3: Singular value decomposition. Usable with ML_MODE = REFIT and REFITBAYESIAN.
• ML_IALGO_LINREG=4: Singular value decomposition with Tikhonov regularization. Usable with ML_MODE = REFIT and REFITBAYESIAN.

For on the fly learning ML_MODE = TRAIN and reselection of local reference configurations ML_MODE = SELECT, it is strictly necessary to use Bayesian regression (ML_IALGO_LINREG=1), since uncertainty estimates are only available for Bayesian regression.

Refitting: Although the above modes result in an ML_FFN file that could be used for production runs, we strongly advise to refit the ML_ABN files. For that copy the ML_ABN file to the ML_AB file and use ML_MODE= REFIT (if Bayesian error estimation is required during production runs ML_MODE= REFITBAYESIAN is an option, but at the cost of significatnly slower calculation time). This mode employs ML_IALGO_LINREG=4 by default.

From ML_IALGO_LINREG>1, ML_IALGO_LINREG=3 and 4 are the most tested approaches and we use ML_IALGO_LINREG=4 routinely before employing a machine learned force field. ML_IALGO_LINREG=4 gives more stable force fields and better fitting accuracy than ML_IALGO_LINREG=3, due to the regularization term employed.

ML_IALGO_LINREG=4 dramatically improves the condition number of the fitting compared to ML_IALGO_LINREG=1 since it directly uses the design matrix. In contrast ML_IALGO_LINREG=1 requires to use the covariance matrix (square of the design matrix), which effectively doubles the condition number. However, ML_IALGO_LINREG=4 needs significantly more memory than ML_IALGO_LINREG=1 (at least twice that much). Please always monitor the memory estimates in ML_LOGFILE! It should be also noted that ML_IALGO_LINREG=4 is computationally somewhat more demanding than ML_IALGO_LINREG=1, but it typically requires between a few minutes and an hour. So usually the extra cost is negligible compared to the original training.

Related tags and articles

ML_LMLFF, ML_MODE, ML_W1, ML_WTOTEN, ML_WTIFOR, ML_WTSIF

Examples that use this tag

---