Getting started with unit testing of SAS programs

Terminology

Test Macro: SAS Code that needs to be tested.

Test Case: A call of a test macro in a controlled environment to test a single(!) feature of the test macro.

Assert: A macro that ensures the adherence to a single feature of the test macro.

Test Scenario: The collection of all test cases for one or more test macro. A scenario covers all test cases for a specific business case (see examples scenario database_test.sas)

Test driver: A program that represents a test scenario.

Installation

Prerequisites

Windows or Linux or experimental AIX
SAS® 9.4 in English or German

Download and extract ZIP-file

Download the current SASUnit-Version from GitHub or as legacy from sourceforge.net/projects/sasunit/
Choose the release file appropriate for your operating system.
Extract the downloaded ZIP-file at your preferred location. For example C:\sasunit.

After installation your folder structure should look like this:
SASUnit folder

example: includes a sample program to practice SASUnit and to ensure the proper functionality of SASUnit (Operational Qualification). Inside the example folder there is a predefined example to test SASUnit and to learn what SASUnit is all about.
saspgm: includes the macro code for the Unit Testing Framework

Example bin folder

bin: Includes SASUnit setup scripts depending on which operating system you are running. Currently there is only support for SAS® 9.4 SASUnit setup scripts will create different shells scripts for running the SASUnit-code depending on which version of SAS® and which operating system you are running. For more details take a look here

Example dat folder

dat: Contains test- and reference data that is used in the SASUnit examples.

Example <\language> folder

<language>: Contains SASUnit outputs and configuration files refer to the results page. The SASUnit documentation is stored in subdirectories.
<language>\doc: Contains the SASUnit- and code documentation including links that refer to the results page. The SASUnit documentation is stored in subdirectories.
<language>\logs: Contains log files of the sasunit suite (run_all.sas) <language>\scn_logs: Contains log files of all the scenarios <language>\testdb: Contains all SAS files of SASUnit (tsu, scn, cas, tst)

Example <language>\doc folder

css: contains the css styles used in SASUnit
images: contains icons used in the SASUnit documentation
js: contains java scripts for sorting tables ans tree view functionality
pgmDoc: contains the SASUnit program documentation
tempDoc: contains the output of SAS to be later convereted to HTML
testDoc: contains the SASUnit test documentation

Configuration

The process of configuring and running SASUnit setup scripts is already discussed here

Starting test suite

Running SASUnit examples

First let the SASUnit examples script run so you can have a look at the generated example files and the documentation. You find the script files under sasunit/example/bin.

Double click on the filename that fits your needs.
For a first run I suggest to use sasunit.<SASVersion>.<OS>.<Language>.overwrite.fast.cmd.
The current folder must be the folder where the scripts reside, this is always the case when you start a script from Windows explorer.
A console and a batch version of SAS® will start. If you start one of the above script files only the changed test scenarios will be executed.
The inffix .overwrite. will execute all test scenarios independent of any changes made in the test macros.

SASUnit now checks the example code, which you find in sasunit\example\saspgm. This folder contains a program and testcase for each example. The testcase is always named *_test.sas at the end.

SASUnit Test Documentation

Once the script has finished you will find the results in the folder sasunit\example\doc.

Have a look at the documentation. Depending on the language settings you chose you double click on the equivalent HTML-file. Your standard browser will open and forward you to the main page of SASUnit Test Documentation.

The Tab „Test Scenarios“ shows an overview of the different tests, where you can find them and when they were executed.

In SASUnit you see at a glance if your code is correct. The checkbox shows the results.
- test turned out correct

- test needs a manual check

- test result is not correct

When clicking on the failed scenario (No. 010) you can see that the test scenario contains 6 tests and one of them failed. This is also the tab “Test cases” which lists all test cases for each test scenario with detailed information and a link to the log file when clicking on the Last Run.

You will get a detailed description of every executed assertion when you click on a test scenario and then on a test case. Here for example you find an overview of test scenario 4 and test case 3. As you can see it shows which assertion was executed, some details about the ran test and the expected and actual results.

When clicking on the last tab you will find an overview of all units that where tested in this test scenario. There is a Column “Test Coverage” – this is only shown when using SAS9.3 or higher.

The Column “Test Coverage” gives you an overview of how much code is covered. For example when looking at the Unit under Test boxplot.sas one can see that not all of the code is covered. When clicking on the 97 you will get coloured code with a colour legend.

Applies only to SASunit 1.5 or earlier:

There are two additional command scripts for Doxygen operation. They can be used for generation of program documentation and they extract program comments and write them to HTML files. You need to install Doxygen (www.doxygen.org) on your system in order to use them.

• doxygen.cmd is used on Windows
• doxygen.sh is used on Linux

Note for Linux users: You need to give execution rights to the LINUX script files. Use "chmod a+x ./sasunit.*.sh".

Logfiles

Each SASUnit Session generates its own logfiles and stores them in the coeesponding folder under <language>.

SASUnit test suite log files in Folder <language>\logs

In the folder <language>\logs you can find three log files:

• log4sasunit_run_all.log This log is written by the SASUnit test suite. This log combines controller and report generation
• log4sasunit_suite.log This log holds aggregated information of the SASUnit runs like message from %init_sasunit and whhich scenarios are started.
• log4sasunit_asserts.log The last log file is experimental and contains messages from all asserts. This is the first step towards running SASUnit without test data base.

SASUnit test scenarios log files in Folder <language>\scn_logs

This folder cotains many log files starting with 000.log. It contains messages of the test for spawning SAS sessions. Each scneario can have multiple log files (examples for scenario 1).

• 001.log The pverall log of that scenario
• 001_nnn.log Each test case has it's own log file.
• 001.tcg File with the information for the test coverage

Here is the overview of the scenario process with log file names:

Trouble Shooting

If a script does not run, i.e. shows errors in the command prompt window, try the following steps:

• Make sure that the path to sas.exe and sas config are correctly specified in the setup script.
• When errors or warnings in the SAS log are indicated, first try to open the test report by means of the proper html file in example\doc. If it opens, there is a link to run_all.log on the main page. Otherwise open example\<language>\logs\log4sasunit_run_all.log. Errors or warnings might result from special configuration of your SAS or OS installation.
• We have a support forum at https://github.com/HMS-Analytical-Software/SASUnit/issues.

SASUnit Macros

Controller Macros

• SASUnit

  • initSASUnit
  • runSASUnit
  • reportSASUnit

• Testscenarios

  • initScenario
  • initTestcase
  • endTestcall
  • endTestcase
  • endScenario

initSASUnit
Initialization of a test suite that may comprise several test scenarios.
An existing test repository is opened or a new test repository is created.

runSASUnit Invokes one or more test scenarios.
Procedure:

• Check whether test repository was already initialized with %initSASUnit, if not: End.
• Determination of the test scenarios to be invoked.
• For every test scenario:
• Check whether it already exists in the test repository.
• if yes: Check whether the test scenario was changed since last invocation.
• if no: Creation of the test scenario in the test repository.
• In case the test scenario is new or changed: The test scenario is executed in an own SAS session which is initialized by _scenario.sas. All test results are gathered in the test repository.

reportSASUnit Creation of a test report.

initScenario Start of a new test scenario, necessary for interactive mode . internally:

• Insertion of relevant data into the test repository
• Setting of flag g_inScenario

initTestcase Start of a new test case that comprises an invocation of a program under test and one or more assertions.
internally:

• Insertion of relevant data into the test repository
• Redirection of SAS log
• Setting of flag g_inTestcase
• Setting of flag g_inTestcall

endTestcall Ends an invocation of a program under test.
internally:

• Ensure sequence
• End redirection of SAS log
• Reset ODS destinations
• Re-Setting of flag g_inTestcall

endTestcase Ends a test case. Result and finish time are added to the test repository. internally:

• Re-Setting of flag g_inTestcase
• Updating test database

endScenario Start of a new test scenario necessary for interactive mode

• Re-Setting of flag g_inScenario
• Updating test database

Asserts

assertColumns
Check whether there are differences between the values of the columns of two sas data sets (PROC COMPARE).
The values of the two data sets are considered to match each other if all columns existing in data set i_expected are also existing in data set i_actual. Optionally one can define a deviation for numerical values so that all corresponding values can be deviating from each other less than a maximal deviation of i_fuzz (Caution: this corresponds to the parameter 'criterion' of PROC COMPARE, the parameter 'fuzz' has a different meaning in the context of PROC COMPARE)

assertEquals
Check whether there are differences between the value of a macro variable and an expected value.
The values can be character string or numerical. Optionally one can define a deviation for numerical values so that the values can be deviating from each other less than a maximal deviation of i_fuzz.

assertForeignKey
Checks whether a foreign key relationship between the columns of two data sets exists.
This assert supports simple and composite keys. The number of specified columns in parameters i_mstKey and i_lookupKey must be the same and columns have to be in the same order. If more than one column is specified the parameter i_cmpKeyLen has to be provided containing the number of columns. Eventually needed renaming of key variables takes place automatically.

assertLibrary
Check whether all files are identical in the libraries i_expected and i_actual.
The comparison report is created later, as PROC REPORT does not support ODS Document.

assertLog
Check whether errors or warnings appear in the log.
If number of errors and warnings does not appear in the log as expected, the check of the assertion will fail.

assertLogMsg
Check whether a certain message appears in the log.
If the message does not appear in the log as expected, the check of the assertion will fail. If i_not is set to 1, the check of the assertion will fail in case the message is found in the log.

assertManual
assertManual serves as placeholder for manual Tasks.

assertPerformance
Check whether runtime of the testcase is below or equal a given limit.

assertPrimaryKey
Checks whether a set of columns can be used as primary key for the data set.

assertRecordCount
This assert checks whether a certain number of records exist in a data set specified by parameters i_libref and i_memname.
Furthermore a where condition can be specified (if not specified set to 1) as well as the number of expected records in the data set that meet the given where condition.

assertRecordExists
Check whether at least one record exists which satisfies a certain WHERE condition.

assertReport
Check whether a report file exists and was created during the current SAS session.
It is possible to write an instruction into the test protocol indicating the need to perform a manual check of the report. Writes an entry into the test repository indicating the need to perform a manual check of the report and copies the report and a given report template (optional).

assertRowExpression
Checks if all observations meet a given WHERE expression.

assertTableExists
Check whether a certain data set, view or catalogue exists.
Setting the optional parameter i_not to 1 allows to test whether a certain data set, view or catalogue does not exist.
Step 1: Check whether library has been assigned successfully.
Step 2: Check for existence with exist function

assertTrue
Check whether a condition is true.

Getting started

Write a simple test

Add two new SAS® files to the folder sasunit\example\saspgm.
Call them for our example getfirstnobs_test.sas and getfirstnobs.sas.
The test will check if your SAS® program writes the first 5 observations of a SAS® table to a new table.

Open the test file getfirstnobs_test.sas in your preferred SAS® development environment and write the following code into it:

%initScenario(i_desc=Test scenario for getFirstNObs.sas);

%initTestCase (i_object = getFirstNObs.sas, i_desc = %STR(My first Testscenario))

DATA work.expected;
   SET SASHELP.class (obs = 5);
RUN;


%getFirstNobs (i_input = SASHELP.class, i_output = work.output)

                        
%endTestcall()

%assertColumns (  i_actual     =  work.output
                , i_expected   = work.expected
                , i_desc       = compare estimated values) 
%assertLog()

%endTestcase()

%endScenario()

Paste the following code to the getFirstNObs.sas file

%Macro getFirstNObs ( i_input =, i_output = );

   DATA &i_output.;
      SET &i_input. (obs = 5);
   RUN;

%Mend getFirstNobs;

Execute the test

After you have written and saved your macro and test code, start the batch file again.
SASUnit now checks for all the test files in the saspgm folder and executes new and changed tests.

Look at the results

So now you can refresh your browser showing the SASUnit Test Documentation and will find the new generated test in the tab “test scenarios”. Here you can see that No. 009 is our newly written SASUnit test and that it works correctly.

When you click on No. 009 and then on the first (and only) test case you will see more details for this test case and that each assertion succeeded.

SASUnit uses a built-in documentation feature for program documentatio since release 1.6 though no DoxyGen resources are used any more and there is no need to install DoxyGen anymore. SASUnit supports some DoxyGen tags and has a similar behaviour. A list of supported tags is onclude in the How to section (read more...).

Please make sure to leave a blank line after your tag because a DoxyGen-Tag ends at a blank line or a new tag. For more information about how to use DoxyGen and further tags refer to DoxyGen Documentation.

For SASUnit 1.3 and higher you do not need to set a DoxyGen tag. If nothing is set the Documentation will show the name of the test scenario as shown as follows for test scenario 2:

Creating your own project

​1. Create a folder structure that includes the following folder:

• bin
• a folder for your data, maybe with subfolders for test and reference data
• a folder for your macro code
• a folder for your tests
• a folder for the documentation

This is what your folder structure could be like:

​2. Copy all needed script files to your bin folder

• Adjust paths in the sasunit.setup.9.4.cmd shell script
  Adjust the values as described here.

​3. Copy or generate macros in your macros folder. These macros can either be the code to be tested or supplementary code. Generate tests in your tests folder.
​4. Copy the file run_all.sas from example\saspgm to your tests folder
​5. Adjust autocall paths in the run_all file

Now you can start the script file and have a look at Documentation output when clicking on myOwnProject\testreport\rep\index.html

Back to User's Guide