ElmerSolver cmake test case how-to 
==================================
Juhani Kataja / CSC, 2014/11, 2017/1
Peter Råback /CSC, 8/2020

The aim of this document is to give some background on testing and describe how
to create a minimalistic test problem using cmake/ctest. 

Background
----------
The test cases are used to check for possible bugs, and to ensure consistency
and backward compatibility. For that purpose they are currently around 700
(situation 8/2020) cases. Most of these are minimalistic but there are also
computationally more expensive cases.

The tests are located in directory

$ELMERSRC/fem/tests

Developers and other are encouraged to provide minimal test cases for new or
untested features. This ensures the feature will be available also in the
future versions of Elmer.


Running test cases
------------------
When you have compiled the code you can run all the case with
>ctest

It is may be a good idea to only run the small tests with two
processes, for example
>ctest -L quick -j2

There are many labels besided "quick" that can be used. 

Test cases can be also run manually and used as starting point for more
complicated cases. Look at "runtests.cmake" to get a hint how the mesh is
created and execute the simulation usually just saying "ElmerSolver" in
the test directory.


Creating a ctest from scratch
-----------------------------

1)    Create a subdirectory in "fem/tests/<testname>" named after the test

2)    Define your analysis write your <sifname>.sif file, mesh files, f90-files
      etc. necessary to run the test analysis ELMERSOLVER_STARTINFO.
      
      In order to test a norm resulting from a solver, add the lines
################################################################################
Reference Norm = Real <norm>
Reference Norm Tolerance = Real <tol>
################################################################################
      to the <sifname>.sif file inside the solver block to be tested.

      Equivalently, top-level directives
################################################################################
Solver N :: Reference Norm = Real <norm>
Solver N :: Reference Norm Tolerance = Real <tol>
################################################################################
      may be used.
      

3)    Create CMakeLists.txt file in the test directory containing. 
################################################################################
INCLUDE(test_macros)

CONFIGURE_FILE(<sifname>.sif <sifname>.sif COPYONLY)

ADD_ELMERTEST_MODULE(<testname> <modulename> <modulesource.f90>)

file(COPY ELMERSOLVER_STARTINFO <file_1> <file_2> ... <file_n> DESTINATION
"${CMAKE_CURRENT_BINARY_DIR}/")

ADD_ELMER_TEST(<testname>)
################################################################################

4)    Create runtest.cmake containing
################################################################################
include(${TEST_SOURCE}/../test_macros.cmake)
execute_process(COMMAND <command-name> <arg1> <arg2> ...)
RUN_ELMER_TEST()
################################################################################

5)    To run this test, compile Elmer and run "ctest -R <testname>" in the build
      directory (This will run all test matching <testname>)


Parallel tests and labels
-------------------------

The command ADD_ELMER_TEST() accepts following optional arguments

ADD_ELMER_TEST(<testname> NPROC <n> LABELS <label_1> <label_2> ... <label_m>)

where n is the number of MPI processes and <label_i> is a label associated with the tests.

Having parallel test requires partitioned mesh as well. That can be automatically done in
the runtest.cmake file with the `execute_process` cmake command:

execute_process(COMMAND ${ELMERGRID_BIN} 2 2 <meshdir> -metiskway #np -nooverwrite)

This will execute ElmerGrid with arguments "2 2 <meshdir> -metiskway #np -nooverwrite".


