In order to assure that the preprocessor and building scripts work as intended, every script comes with a collection of tests.
Unit Testing Suites
The test suite reside in the subdirectory bin
and carry the filename prefix test_
. Every test that comes with reference files come with an additional subdirectory testing/test_name-of-your-sub
.
The tests are run on GitLab after each change in /bin
and /testing
, see CI/CD and the configuration in .gitlab-ci.yml
for details.
Test Template
Each test has 5 stages:
- Prepare generic variables.
- Run the script to be tested (with different combinations of input arguments.
- Compare the output with a reference file usually via a
diff
command. - Check whether all files where created using
if [ -f ...]
. - Exit with exit code
1
if script has failed or continue with next test.
Generic Variables
In order to modularize tests, the following generic variables are used:
Name | Description | Example Value |
---|---|---|
$RESEARCH_TESTING |
The base directory for this test | $RESEARCH_ROOT/testing/test_preproc_wg |
$LOGFILE |
The name of the logfile for this test | $RESEARCH_ROOT/log/preproc_wg.log |
$SCRIPT |
The basic command that is tested | $RESEARCH_ROOT/bin/preproc_wg.sh |
Plain template
Use this template in order to register new tests; replace NN
and Description
accordingly:
#!/bin/bash
# test_my-routine.sh
echo "--------------------------------------------------"
echo "Running tests in $0"
echo "--------------------------------------------------"
RESEARCH_TESTING=
LOGFILE=
SCRIPT=
rm -rf $LOGFILE && mkdir -p $RESEARCH_ROOT/log && touch $LOGFILE
#BEGIN Test NN: Description
T=NN
#echo "##################################################"
echo "Test $T: Description"
#echo "##################################################"
# preparation
# run script
#check results
DIFF=
#echo "--------------------------------------------------"
if [ "$DIFF" != "" ] && [ -f "" ]
then
echo "Test $T: failed"
exit 1
else
echo "Test $T: passed."
fi
#END Test 1
#...
#don't modify below, these should be the last lines (errors exit earlier)
echo "--------------------------------------------------"
echo "ALL $T tests in $0 PASSED."
exit 0
Concrete Example
From test_preproc_wg.sh
:
#BEGIN Test 1: Preprocessor for Working Group
T=1
#echo "##################################################"
echo "Test $T: Preprocessor for Working Group only .md file"
#echo "##################################################"
# preparation
RESEARCH_BUILD=$RESEARCH_TESTING/build
rm -rf $RESEARCH_TESTING/build
mkdir -p $RESEARCH_BUILD
REF_FILE=$RESEARCH_TESTING/aperiodSchr-preproc-wg.md
cd $RESEARCH_TESTING
# run script
$RESEARCH_ROOT/bin/preproc_wg.sh aperiodSchr.md >> $LOGFILE 2>&1
#check results
DIFF=$(diff $RESEARCH_TESTING/build/aperiodSchr.md $RESEARCH_TESTING/aperiodSchr-preproc-wg.md)
#echo "--------------------------------------------------"
if [ "$DIFF" != "" ] && [ -f "$RESEARCH_BUILD/aperiodSchr.md" ]
then
echo "Test $T: failed"
exit 1
else
echo "Test $T: passed."
fi
#END Test 1