dev:regtesting
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
dev:regtesting [2015/05/29 10:29] – [Adding and Resetting Tests] 130.60.136.205 | dev:regtesting [2021/06/02 13:14] – tmueller | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== CP2K Regression Testing ====== | ====== CP2K Regression Testing ====== | ||
- | CP2K comes with over 2500 test input files (located in [[src>cp2k/tests]]) which serve as both examples on how to use the many features in CP2K and also as a method for developers to test modifications and extensions to CP2K. In order to reduce the chance of bugs being introduced into the code, and ensure that all parts of the code are working, we recommend that all developers | + | CP2K comes with over 3000 test input files (located in [[src> |
==== Dashboard ==== | ==== Dashboard ==== | ||
- | A number of regtests are run automatically by various members of our community. The results of these tests are collected centrally at the [[http:// | + | A number of regtests are run automatically by various members of our community. The results of these tests are collected centrally at the [[http:// |
==== Code Coverage ==== | ==== Code Coverage ==== | ||
- | We aim that the regression test suite covers all the functionality of CP2K. For this purpose we regularly create [[http:// | + | We aim that the regression test suite covers all the functionality of CP2K. For this purpose we regularly create [[http:// |
===== How does it work? ===== | ===== How does it work? ===== | ||
- | The regression test suite is run using the [[src>cp2k/tools/ | + | The regression test suite is run using the [[src> |
It performs the following tasks: | It performs the following tasks: | ||
- | * update to the current SVN version (including new tests) | ||
* performs a realclean build of the source | * performs a realclean build of the source | ||
* executes a list of tests | * executes a list of tests | ||
Line 21: | Line 20: | ||
===== Running the regtests ===== | ===== Running the regtests ===== | ||
+ | |||
+ | ==== Step 0: make based testing ==== | ||
+ | |||
+ | * If you are able to build and run cp2k on the local machine the easiest way is to start the regtesting by running '' | ||
+ | * Be careful about the value of '' | ||
+ | * If this fails (e.g. on batch systems), continue with Step 1 else go to interpretation. | ||
==== Step 1: Preparation ==== | ==== Step 1: Preparation ==== | ||
- | * You must be able to build and run cp2k on the given machine, the rest should be ' | ||
* Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | * Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' | ||
- | * Checkout | + | * Clone a version of cp2k into '' |
* Set up the arch files so that you can cleanly build cp2k (test this) | * Set up the arch files so that you can cleanly build cp2k (test this) | ||
- | ==== Step 2: Configuration | + | ==== Step 2: Running |
+ | < | ||
+ | $ tools/ | ||
+ | Usage: do_regtest [OPTION] | ||
+ | Run the CP2K regression test suite | ||
+ | Example: do_regtest -c my_regtest.conf | ||
- | The // | + | General: |
+ | -h, -help, --help | ||
+ | -c, -config FILE read any of the following | ||
+ | -cp2kdir PATH | ||
+ | -dirout PATH root path of output folder (default: current working directory). | ||
+ | -mpiexec EXE name of the executable to run mpi-ranks. Default: mpiexec. | ||
- | ^ Config Variable ^ Default ^ Meaning | + | Build: |
- | | '' | + | -version VERSION |
- | | '' | + | -arch ARCH ARCH passed to make. Default: Linux-x86-64-gfortran. |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
+ | Runtime: | ||
+ | -mpiranks NRANKS | ||
+ | -ompthreads NTHREADS | ||
+ | -maxtasks NPROCS | ||
+ | -maxbuildtasks NPROCS | ||
+ | -jobmaxtime SECONDS | ||
+ | -farming | ||
- | ==== Step 3: Execution ==== | + | Testing: |
- | Normally, you just have to call the // | + | |
+ | | ||
+ | -skiptest | ||
+ | -skipunittest | ||
+ | -skipdir TESTDIR | ||
+ | -restrictdir TESTDIR | ||
+ | -retest | ||
- | < | + | Exit codes: |
- | $ ./cp2k/tools/regtesting/do_regtest -config < | + | |
+ | 3 | ||
+ | 4 build errors | ||
+ | 5 | ||
+ | 6 | ||
+ | 7 | ||
+ | 8 | ||
+ | |||
+ | For more information visit: <http://cp2k.org/dev:regtesting> | ||
</ | </ | ||
- | However, the // | + | ==== Step 3: Interpretation ==== |
- | ^ Command Line Switch | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | | '' | + | |
- | + | ||
- | The // | + | |
- | + | ||
- | ^ Exit Code | + | |
- | | 0 | clean exit with testing | + | |
- | | 1 | problem with svn update | + | |
- | | 3 | problem with realclean | + | |
- | | 4 | build errors | + | |
- | | 5 | problem with retest option - no TEST directory with latest test results found | | + | |
- | | 6 | problem with retest option - no error summary exists in the last TEST directory | | + | |
- | | 7 | reference directory is locked | + | |
- | | 8 | ctrl-C (SIGINT) and various other signals trapped | + | |
- | | 100 | no svn changes since last run - clean exit without testing | + | |
- | + | ||
- | + | ||
- | ==== Step 4: Interpretation ==== | + | |
A test results can be any of the following: | A test results can be any of the following: | ||
^ Test Result | ^ Test Result | ||
Line 93: | Line 94: | ||
===== Adding and Resetting Tests===== | ===== Adding and Resetting Tests===== | ||
- | The test-suite is fully controlled by the following files in the [[src>cp2k/tests]] directories | + | The test-suite is fully controlled by the following files in the [[src> |
^ File Name ^ Content | ^ File Name ^ Content | ||
| '' | | '' | ||
| '' | | '' | ||
- | | '' | + | | '' |
| '' | | '' | ||
+ | ===== Run with sbatch ===== | ||
+ | The '' | ||
+ | |||
+ | What you need: | ||
+ | |||
+ | * '' | ||
+ | * a CP2K source tree with a built CP2K | ||
+ | |||
+ | ==== Instructions ==== | ||
+ | |||
+ | The way the regtest script works is that it goes through all the directories (for example '' | ||
+ | |||
+ | Also, '' | ||
+ | |||
+ | Append the following to your '' | ||
+ | |||
+ | <code bash> | ||
+ | CP2K_BASE_DIR="/ | ||
+ | CP2K_TEST_DIR=" | ||
+ | # CP2K_REGTEST_SCRIPT_DIR="" | ||
+ | |||
+ | CP2K_ARCH=" | ||
+ | CP2K_VERSION=" | ||
+ | |||
+ | # the following is the default, adjust if you want to run single tests with more than 2 ranks/tasks | ||
+ | NTASKS_SINGLE_TEST=2 | ||
+ | NNODES_SINGLE_TEST=1 | ||
+ | SRUN_CMD=" | ||
+ | |||
+ | # the following should be sufficiently generic: | ||
+ | |||
+ | mkdir -p " | ||
+ | cd " | ||
+ | |||
+ | cp2k_rel_dir=$(realpath --relative-to=" | ||
+ | # srun does not like `-np`, override the complete command instead: | ||
+ | export cp2k_run_prefix=" | ||
+ | |||
+ | " | ||
+ | -arch " | ||
+ | -version " | ||
+ | -nobuild \ | ||
+ | -mpiranks ${NTASKS_SINGLE_TEST} \ | ||
+ | -ompthreads ${OMP_NUM_THREADS} \ | ||
+ | -maxtasks ${SLURM_NTASKS} \ | ||
+ | -cp2kdir " | ||
+ | |& tee " | ||
+ | |||
+ | # the above will output both to the slurm-*.out as well as a log file, | ||
+ | # if you want only the log file replace the `|& tee` with a `>& | ||
+ | |||
+ | # More options: | ||
+ | # -farming | ||
+ | # -retest | ||
+ | </ | ||
+ | |||
+ | A complete '' | ||
+ | |||
+ | <code bash> | ||
+ | #!/bin/bash -l | ||
+ | #SBATCH --time=01: | ||
+ | #SBATCH --nodes=4 | ||
+ | #SBATCH --ntasks-per-node=128 | ||
+ | #SBATCH --cpus-per-task=2 | ||
+ | #SBATCH --ntasks-per-core=1 | ||
+ | |||
+ | # More SBATCH options: | ||
+ | # If you need 512GB memory nodes (otherwise only 256GB guaranteed): | ||
+ | # #SBATCH --mem=497G | ||
+ | # To run on the debug queue (max 10 nodes, 30 min): | ||
+ | # # | ||
+ | |||
+ | set -o errexit | ||
+ | set -o nounset | ||
+ | set -o pipefail | ||
+ | |||
+ | export MPICH_OFI_STARTUP_CONNECT=1 | ||
+ | export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK} | ||
+ | export OMP_PROC_BIND=close | ||
+ | export OMP_PLACES=cores | ||
+ | |||
+ | source " | ||
+ | |||
+ | module load cpeGNU | ||
+ | module load \ | ||
+ | cray-fftw \ | ||
+ | ELPA/ | ||
+ | libxsmm/ | ||
+ | libxc/5.1.3 \ | ||
+ | Libint-CP2K/ | ||
+ | gcc/10.2.0 | ||
+ | |||
+ | # Let the user see the currently loaded modules in the slurm log for completeness: | ||
+ | module list | ||
+ | |||
+ | CP2K_BASE_DIR="/ | ||
+ | CP2K_TEST_DIR=" | ||
+ | |||
+ | CP2K_ARCH=" | ||
+ | CP2K_VERSION=" | ||
+ | |||
+ | NTASKS_SINGLE_TEST=2 | ||
+ | NNODES_SINGLE_TEST=1 | ||
+ | SRUN_CMD=" | ||
+ | |||
+ | # to run tests across nodes (to check for communication effects), use: | ||
+ | # NNODES_SINGLE_TEST=4 | ||
+ | # SRUN_CMD=" | ||
+ | |||
+ | # the following should be sufficiently generic: | ||
+ | |||
+ | mkdir -p " | ||
+ | cd " | ||
+ | |||
+ | cp2k_rel_dir=$(realpath --relative-to=" | ||
+ | # srun does not like `-np`, override the complete command instead: | ||
+ | export cp2k_run_prefix=" | ||
+ | |||
+ | " | ||
+ | -arch " | ||
+ | -version " | ||
+ | -nobuild \ | ||
+ | -mpiranks ${NTASKS_SINGLE_TEST} \ | ||
+ | -ompthreads ${OMP_NUM_THREADS} \ | ||
+ | -maxtasks ${SLURM_NTASKS} \ | ||
+ | -cp2kdir " | ||
+ | | ||
+ | </ | ||
+ | |||
+ | ==== Test parallelization via Farming ==== | ||
+ | |||
+ | Since the startup time of '' | ||
+ | |||
+ | Simply add the option '' | ||
+ | |||
+ | ==== Minimal directory setup ==== | ||
+ | |||
+ | If you want to test a precompiled executable there is a minimal directory layout you have to reproduce to run the regtest: | ||
+ | |||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | An example if your HPC center uses EasyBuild to provide the CP2K package: | ||
+ | |||
+ | < | ||
+ | cp2k-prebuilt | ||
+ | ├── data -> / | ||
+ | ├── exe | ||
+ | │ └── prebuilt -> / | ||
+ | └── tests | ||
+ | </ | ||
+ | |||
+ | and then update the variables as follows: | ||
+ | |||
+ | <code bash> | ||
+ | CP2K_BASE_DIR="/ | ||
+ | CP2K_TEST_DIR=" | ||
+ | CP2K_REGTEST_SCRIPT_DIR="/ | ||
+ | |||
+ | CP2K_ARCH=" | ||
+ | CP2K_VERSION=" | ||
+ | </ |
dev/regtesting.txt · Last modified: 2023/10/19 14:03 by krack