dev:regtesting
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
dev:regtesting [2014/05/30 10:22] – Add link to regtest file krack | dev:regtesting [2020/08/21 10:15] – external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ==== CP2K Regression Testing ===== | + | ====== CP2K Regression Testing |
- | === Regression Testing Script === | + | CP2K comes with over 3000 test input files (located in [[src> |
- | CP2K comes with over 2400 test input files (located in '' | + | ==== Dashboard ==== |
+ | A number of regtests are run automatically by various members of our community. The results of these tests are collected centrally at the [[http:// | ||
- | The regression test suite is run using the '' | + | ==== Code Coverage ==== |
+ | We aim that the regression test suite covers all the functionality of CP2K. For this purpose we regularly create [[http://www.cp2k.org/ | ||
- | In addition, when new features are added to CP2K, please add new tests which cover those features and commit them to SVN along with the code. | + | ===== How does it work? ===== |
- | === Automated Regression Tester === | + | The regression test suite is run using the [[src> |
+ | It performs the following tasks: | ||
+ | * performs a realclean build of the source | ||
+ | * executes a list of tests | ||
+ | * compares the results (outputs) with those of the last known result (reference) | ||
+ | * produces a summary | ||
- | An automated regression test service is available at http:// | ||
- | We plan to extend | + | ===== Running |
- | === CP2K Code Coverage | + | ==== Step 0: make based testing ==== |
- | We aim that the regression test suite covers all the functionality of CP2K. | + | * If you are able to build and run cp2k on the local machine |
+ | * Be careful about the value of '' | ||
+ | * If this fails (e.g. on batch systems), continue with Step 1 else go to interpretation. | ||
- | === Results of the Automatic Regression Tester | + | ==== Step 1: Preparation ==== |
- | * [[http://sourceforge.net/p/cp2k/ | + | * Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a while) so make it something like '' |
+ | * Clone a version of cp2k into '' | ||
+ | * Set up the arch files so that you can cleanly build cp2k (test this) | ||
- | [[http://cp2k.web.psi.ch/ | + | ==== Step 2: Running ==== |
+ | < | ||
+ | $ tools/regtesting/do_regtest --help | ||
+ | Usage: do_regtest [OPTION] | ||
+ | Run the CP2K regression test suite | ||
+ | Example: do_regtest -c my_regtest.conf | ||
+ | |||
+ | General: | ||
+ | -h, -help, --help | ||
+ | -c, -config FILE read any of the following configuration switches from FILE. | ||
+ | -cp2kdir PATH | ||
+ | -dirout PATH root path of output folder (default: current working directory). | ||
+ | -mpiexec EXE name of the executable to run mpi-ranks. Default: mpiexec. | ||
+ | |||
+ | Build: | ||
+ | -version VERSION | ||
+ | -arch ARCH ARCH passed to make. Default: Linux-x86-64-gfortran. | ||
+ | -quick | ||
+ | -nobuild | ||
+ | |||
+ | Runtime: | ||
+ | -mpiranks NRANKS | ||
+ | -ompthreads NTHREADS | ||
+ | -maxtasks NPROCS | ||
+ | -maxbuildtasks NPROCS | ||
+ | -jobmaxtime SECONDS | ||
+ | -farming | ||
+ | |||
+ | Testing: | ||
+ | -noreset | ||
+ | -shard ISHARD NSHARDS | ||
+ | -skiptest | ||
+ | -skipunittest | ||
+ | -skipdir TESTDIR | ||
+ | -restrictdir TESTDIR | ||
+ | -retest | ||
+ | |||
+ | Exit codes: | ||
+ | 0 clean exit with testing | ||
+ | 3 | ||
+ | 4 build errors | ||
+ | 5 | ||
+ | 6 | ||
+ | 7 | ||
+ | 8 | ||
+ | |||
+ | For more information visit: <http:// | ||
+ | </ | ||
+ | |||
+ | ==== Step 3: Interpretation ==== | ||
+ | A test results can be any of the following: | ||
+ | ^ Test Result | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | |||
+ | The last two outcomes generally mean that a bug has been introduced, which requires investigation. | ||
+ | Since regtesting only yields information relative to a previously known result, it is most useful to do a regtest | ||
+ | |||
+ | |||
+ | ===== Adding and Resetting Tests===== | ||
+ | The test-suite is fully controlled by the following files in the [[src> | ||
+ | |||
+ | ^ File Name ^ Content | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
+ | | '' | ||
- | === Latest commits === | ||
- | {{rss> |
dev/regtesting.txt · Last modified: 2023/10/19 14:03 by krack