Processing math: 100%

CP2K Open Source Molecular Dynamics

User Tools

Site Tools

Trace:
howto:pao-ml

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
howto:pao-ml [2018/01/25 00:23] – [Tuning the PAO Optimization] oschuetthowto:pao-ml [2024/01/03 13:19] (current) – oschuett
Line 1: Line 1:
-====== How to run simulations with PAO-ML ====== +This page has been moved to: https://manual.cp2k.org/trunk/methods/machine_learning/pao-ml.html
- +
-PAO-ML stands for Polarized Atomic Orbitals from Machine Learning. It uses machine learning to generate geometry adopted small basis sets. It also provides exact ionic forces. The scheme can serve as an almost drop-in replacement for conventional +
-basis sets to speedup otherwise standard DFT calculations. The method is similar to semi-empirical models based on minimal basis sets, but offers improved accuracy and quasi-automatic parameterization. However, the method is still in an early stage - so use with caution. For more information see: [[doi>10.3929/ethz-a-010819495]]. +
- +
-===== Step 1: Obtain training structures ===== +
- +
-The PAO-ML scheme takes a set of training structures as input. For each of these structures, the variational PAO basis is determined via an explicit optimization. The training structures should be much smaller than the target system, but large enough to contain all the //motifs// of the larger system. For liquids a good way to obtain structures is to run an MD of a smaller box. +
- +
-===== Step 2: Calculate reference data in primary basis ===== +
- +
-Choose a primary basis set, e.g. ''DZVP-MOLOPT-GTH'' and perform a full [[inp>CP2K_INPUT/FORCE_EVAL/DFT/LS_SCF|LS_SCF]] optimization. You should also enable [[inp>FORCE_EVAL/DFT/LS_SCF.html#RESTART_WRITE| RESTART_WRITE]] to save the final density matrix. It can be used to speed up the next step significantly. +
- +
-===== Step 3: Optimize PAO basis for training structures ===== +
- +
-Choose a [[inp>FORCE_EVAL/SUBSYS/KIND.html#PAO_BASIS_SIZE | PAO_BASIS_SIZE]] for each atomic kind. Good results can already be optained with a minimal basis sets. Slightly larger-than-minimal PAO basis sets can significantly increase the accuracy. However, they are also tougher to optimize and machine learn. +
- +
-Most of the PAO settings are in the [[inp>FORCE_EVAL/DFT/LS_SCF/PAO | PAO]] sections: +
-<code> +
-&PAO +
-  EPS_PAO    1.0E-7                    ! convergence threshold of PAO optimization +
-  MAX_PAO    10000                     ! minimal PAO basis usually converge withing 2000 steps. +
-   +
-  MAX_CYCLES 500                       ! tunning parameter for PAO optimization scheme +
-  MIXING     0.5                       ! tunning parameter for PAO optimization scheme +
-  PREOPT_DM_FILE primay_basis.dm       ! restart DM from primary basis for great speedup +
-   +
-  LINPOT_REGULARIZATION_DELTA 1E-6     !!!! Critical parameter for accuracy vs learnability trade-off !!!! +
-   +
-  LINPOT_REGULARIZATION_STRENGTH 1E-3  ! rather insensitive parameter, 1e-3 works usually +
-  REGULARIZATION 1.0E-3                ! rather insensitive parameter, 1e-3 works usually +
-   +
-  PRECONDITION YES                     ! not important, don't touch  +
-  LINPOT_PRECONDITION_DELTA 0.01       ! not important, don't touch  +
-  LINPOT_INITGUESS_DELTA 1E+10         ! not important, don't touch +
- +
-  &PRINT +
-    &RESTART +
-      BACKUP_COPIES 1                  ! write restart files, just in case +
-    &END RESTART +
-  &END PRINT +
-&END PAO +
-</code> +
- +
-Settings for individual atomic kinds are in the [[inp>FORCE_EVAL/SUBSYS/KIND | KIND ]] section: +
-<code> +
-&KIND H +
-  PAO_BASIS_SIZE 1    ! set this to at least the minimal basis size +
-  &PAO_POTENTIAL +
-    MAXL 4            ! 4 works usually +
-    BETA 2.0          ! 2 work usually, but is worth exploring in case of accuracy or learnability issues. +
-  &END PAO_POTENTIAL +
-&END KIND +
-</code> +
- +
-==== Tuning the PAO Optimization ===== +
-Find the optimal PAO basis poses an intricate minimization problem, because the rotation matrix U and the Kohn-Sham +
-matrix H have to be optimized in a self-consistent manner. In order to speedup the optimization, the Kohn-Sham matrix is only updated occasionally while most time is spend on optimizing U. This alternating scheme is controlled by two input parameters: +
-  * The frequency with which H is recalculated is determined by [[inp>FORCE_EVAL/DFT/LS_SCF/PAO#list_MAX_CYCLES|MAX_CYCLES]].  +
-  * Overshooting during the U optimization is damped via [[inp>FORCE_EVAL/DFT/LS_SCF/PAO#list_MIXING|MIXING]]. +
- +
-The progress of the PAO optimization can be tracked from lines that start with ''PAO| step''. The columns have the following meaning: +
-<code> +
-             step-num             energy          conv-crit. step-length   time +
- PAO| step   1121                 -186.164843303  0.227E-06  0.120E+01     1.440 +
-</code> +
- +
-  * The step number counts the number of energy evaluation, ie. the number of U matrices probed. It can increase with different intervals, when the [[inp>FORCE_EVAL/DFT/LS_SCF/PAO/LINE_SEARCH#list_METHOD|ADAPTive]] line-search method is used. When the step number reaches [[inp>FORCE_EVAL/DFT/LS_SCF/PAO#MAX_PAO|MAX_PAO]] then the optimization is terminated prematurely.  +
-  * The energy is the quantity that is optimized. It contains **only the first order term** of the total energy, ie. Tr[HP], but shares the same variational minima. It furthermore contains the contributions from the various regularization terms. +
-  * The convergence criterion is the norm of the gradient normalized by system size. It is compared against [[inp>FORCE_EVAL/DFT/LS_SCF/PAO#EPS_PAO|EPS_PAO]] to decided if the PAO optimization has converged. The overall optimization is terminated if this convergence criterion is reached within two steps after updating the Kohn-Sham matrix. +
-  * The step length is the outcome of the line search. It should be of order 1. If it starts to behave erratically towards the end of the optimization, this indicates that further optimization is hindered by numerical accuracy e.g. from [[inp>FORCE_EVAL/DFT/LS_SCF#list_EPS_FILTER|EPS_FILTER]] or [[inp>FORCE_EVAL/DFT/LS_SCF#list_EPS_SCF|EPS_SCF]]. +
-  * The time is the time spend on this optimization step in seconds. This number can varry accordingly to the number of performed lines search steps. +
- +
-===== Step 4: Optimize machine learning hyper-parameters ===== +
- +
-For the simulation of larger systems the PAO-ML scheme infers new PAO basis sets from the training data. For this two heuristics are employed: A [[https://en.wikipedia.org/wiki/Feature_(machine_learning) | descriptor ]] and an inference algorithm. Currently, only one simple descriptor and [[https://en.wikipedia.org/wiki/Gaussian_process| Gaussian processes]] are implemented. However, this part offers great opportunities for future research. +
- +
-In order to obtain good results from the learning machinery a small number of so-called [[https://en.wikipedia.org/wiki/Hyperparameter | hyperparameters]] have to be carefully tuned for each application. For the current implementation this includes the [[inp>FORCE_EVAL/DFT/LS_SCF/PAO/MACHINE_LEARNING.html#GP_SCALE| GP_SCALE]] and the descriptor's [[inp>FORCE_EVAL/SUBSYS/KIND/PAO_DESCRIPTOR.html#BETA | BETA ]] and [[inp>FORCE_EVAL/SUBSYS/KIND/PAO_DESCRIPTOR.html#SCREENING | SCREENING]]. +
- +
-For the optimization of the hyper-parameter exists no gradient, hence one has to use a derivative-free method like the one by [[https://en.wikipedia.org/wiki/Powell%27s_method| Powell]]. A versatile implementation is e.g. the [[src>cp2k/tools/scriptmini/ | scriptmini ]] tool. A good optimization criterion is the variance of the energy difference wrt. the primary basis across the training set. Alternatively, atomic forces could be compared. Despite the missing gradients, this optimization is rather quick because it only performs calculations in the small PAO basis set. +
- +
-===== Step 5: Run simulation with PAO-ML ==== +
- +
-Most of the PAO-ML settings are in the [[inp>FORCE_EVAL/DFT/LS_SCF/PAO/MACHINE_LEARNING | PAO/MACHINE_LEARNING]] sections: +
-<code> +
-&PAO +
-  MAX_PAO 0                  ! use PAO basis as predicted by ML, required for correct forces +
-  PENALTY_STRENGTH 0.0       ! disable penalty, required for correct forces +
-   +
-  &MACHINE_LEARNING +
-    GP_SCALE 0.46            !!! critical tuning parameter - depends also on descriptor settings !!! +
-    GP_NOISE_VAR 0.0001      ! insensitive parameter +
- +
- +
-    METHOD GAUSSIAN_PROCESS  ! only implemented method - opportunity for future research +
-    DESCRIPTOR OVERLAP       ! only implemented method - opportunity for future research +
-    PRIOR MEAN               ! try once ZERO - makes usually no difference +
-    TOLERANCE 1000.0         ! disable check for max variance of GP prediction +
-     +
-    &TRAINING_SET +
-          ../training/Frame0000/calc_pao_ref-1_0.pao +
-          ../training/Frame0100/calc_pao_ref-1_0.pao +
-          ../training/Frame0200/calc_pao_ref-1_0.pao +
-          ! add more ... +
-    &END TRAINING_SET +
-  &END MACHINE_LEARNING +
-&END PAO +
-</code> +
- +
-Settings for individual atomic kinds are again in the [[inp>FORCE_EVAL/SUBSYS/KIND | KIND ]] section: +
-<code> +
-&KIND H +
-  PAO_BASIS_SIZE 1      ! use same settings as for training +
-  &PAO_POTENTIAL +
-    MAXL 4              ! use same settings as for training +
-    BETA 2.0            ! use same settings as for training +
-  &END PAO_POTENTIAL +
-   +
-  &PAO_DESCRIPTOR +
-     BETA   0.16        !!! important ML hyper-parameter !!! +
-     SCREENING 0.66     !!! important ML hyper-parameter !!! +
-     WEIGHT 1.0         ! usually not needed when BETA and SCREENING are choose properly +
-  &END PAO_DESCRIPTOR +
-&END KIND +
-</code> +
- +
- +
- +
-===== Debugging accuracy vs learnability trade-off ===== +
- +
-When optimizing the PAO reference data in Step 3 one has to make a trade-off between accuracy and learnability. Good learnability means that similar structures leads to similar PAO parameters. In other words the PAO parameters should depend smoothly on the atomic positions. In general, the settings presented above should yield good results. However, if problems arise in the later machine learning steps, this might be the culprit. +
- +
-Unfortunately, there is not yet a simple way to assess learnability. One way to investigate is to create a set of structures along a reaction coordinate, e.g. a dimer dissociation. One can then plot the numbers from the ''Xblock'' in the ''.pao'' files vs. the reaction coordinate. +
- +
-The most critical parameters for learnability are [[inp>FORCE_EVAL/DFT/LS_SCF/PAO.html#LINPOT_REGULARIZATION_DELTA | LINPOT_REGULARIZATION_DELTA]] and the potential's [[inp>FORCE_EVAL/SUBSYS/KIND/PAO_POTENTIAL.html#BETA | BETA]].+
howto/pao-ml.1516839793.txt.gz · Last modified: 2020/08/21 10:15 (external edit)