-
Notifications
You must be signed in to change notification settings - Fork 0
Release Notes Version 2.1
KLandwich edited this page Feb 14, 2024
·
2 revisions
- Discontinuity of SAS9.3
- Due to SAS and Microsoft strategy the only SAS version supported under Windows 10 ist SAS 9.4. In conequence we needed to cut off support for SAS 9.3 under Windows
- For Linux environments we switched to CentOS and decided for simplicity of maintenance to cut off SAS 9.3 too.
- You will still be able to run SASUnit with SAS 9.3 on both platforms, but we will focus on SAS 9.4 for development and testing.
- SASUnit now provides a setup script
- To simplify the setup process of SASUnit there is an setup script available.
- It's the only file you need to adjust with your personal environment. It will then created multiple start scripts for SASUnit with your personal project.
- SASunit uses log4sas specified in a logconfig.xml
- Now it is possible to use SASUnit in projects that also use an logconfig.xml
- Messaging was completely change to log4sas
- There are numerous log files
- the traditional run_all.log
- a new aggregated log file for the test suite
- a new aggregated log file for all scenarios
- a new log file containing all assert and the result. It has the same content that is written to the test data base. So this can be a prototype for running SASUnit without test data base.
- Paths within SASUnit have "/" as separator and are not quoted. No double quotes around paths with blanks for Windows nor a "\ " under Linux.
- This facilitates implementing SASUnit features. Only when communicating with the operating system the paths will be adjusted.
- Support standard linux paths
- Under Linux SASUnit can now be splitted across etc, opt, var "as usual".
- Tidy up HTML folder
- HTML folder now has subfolders to restrict the number of files in the root folder of the documentation
- User contribution by Chuck Castillo
- New coveniance assertion assertTrue
- Assert as place holder for manual Tasks
- New assertion assertManual
- New layout of target.tsu
- For each new parameter we use in SASUnit we had to change the layout (e.g. variables) of this table.
- The table is now implemented as a key-value-pair for each paramter
- Introducing new parameters will not change the layout anymore.
- assertRecordCount
- assertRecordCount now always shows the counted observatios. This is espacially usefull when working with a where expression.
- assertLogMsg
- assertLogMsg converted the passed string to uppercase to eliminate case sensitivity. If there are regEx modifiers in string they where also converted to uppercase, which may result in a different meaning.
- Therfore a new parameter was introduced for case sensitivity. The uppercase conversion was replaced be adding "/i" to the regEx string.
- Test coverage of SASUnit self tests
- For each macro there is at least an "empty" test scenario resulting in statement coverage.
- This will guide future creation of additional self tests
- Conveniance change for test cases that will run on both platforms
- For assertImage and assertText there are script files that are dependent on the operating system.
- To run these asserts on platforms alike an independent scenario was created to cover simple functionality of the script files.
- This is the first release with separate download packages per operating system
- We did that because SASUnit needed to have write access to it's own macros (due to Windows CR/LF under linux).
- This is now obsolete, because we fix CR/LF while building the download package
- If you have any problems getting your appropiate download package then use underneath link
- We did that because SASUnit needed to have write access to it's own macros (due to Windows CR/LF under linux).
Fixed bugs:
- When running under hungarian WINDOWS the response of dir command was not recognized correctly
- missing folder tst when running SASUnit interactively in a "bare" environment
- G_ERROR not found in checkSpawning.sas
- Issue with autocall syntax in SAS 9.2 under linux
- SASUnit now can handle calls with no scenarios in search path
- Program documentation: Links to examinee documentation in page Additonal Information are now functional
- Improved performance of test coverage. The creation of the html pages is now faster
- Program description in \brief and \details tag now can be up to 1000 characters
- Support for NOXCMD environments is completed in interactive mode.
- Links in documentation are rendered correctly in both calls, with and without program documentation.
- Source codes of examinees and test scenarios are copied to documentation folder to keep the documented version.
- Test scenarios program files may now be located outside autocall folders.
- Macros names that are contained in other macro names are now handled correctly.
- e.g. Macro1 and Macro1_14 are treated seperately.
Fixed bug: "" is not a valid option vale for SYSIN. Occurred when
- you use multiple calls to runsasunit (perhaps with disjunct paths)
- you changed the examinee only and none of the corresponding scenarios
- you do NOT use OVERWRITE
Support for test driven development
- For a detailed overview check this documentation.
- German version is here.
- SASUnit can be used in Enterprise Guide for interactive and fast development of test scenarios.
- Some features can only be used in the traditional batch processing.
- assertLog and assertLogMsg cannot scan the log because it is redirected to Enterprise Guide.
- A compact overview report is created at the end.
- Test data base will be updated by interactive runs.
- Support for interactive environments with NOXCMD..
- initscenario.sas
- Starts a scenario. Used in interactive environments. May be neceassary in batch runs too.
- endscenario.sas * Ends a scenario and triggers compact report. Used in interactive environments. May be neceassary in batch runs too.
- _checkrunenvironment.sas
- checks operating system and sas version against the requirments of the current implementation
- _noxcmd_dir.sas
- Same functionality as _dir but without XCMDs
- _readenvmetadata.sas * Checks which run time environemnt is used * Display manager * Enterprise Guide * SAS Studio * Jupyter Notebooks
- Up to 30 autocall path are available
- Especially when you use multiple toolboxes and need to test them in SASUnit then 10 autocall paths are not enough
None.
- New stunning feature
- SASUnit now ships with a new layout. Take a look.
- If I doesn't please you - then just make your own layout
- SASUnit now ships with a new layout. Take a look.
- New page with instructions to create your own SASUnit style
- Fixed typo in shell scripts for continuous integration
- Error with built-in SAS macros like %cmpres
- Due to incorrect implementation of SAS autocall paths in _runProgramSpawned these error could occur.
- Fixed by correct implementation
- In some cases SASUnit couldn't find the source code of the macro and therefore the description of the test scenario was missing.
- Fixed using a default from the test database
- Incorrect behaviour of assertText and assertImage
- If one results is missing, the assert now grants access to the existing result
- Home page of SASUnit test documentation now shows last runtime in two forms:
- Summary of run time of all scenarios
- Complete execution time of last SASUnit run including number of executed scenarios
- Rearranged and enhanced documentation on sourceforge
- Documentation home page
- Goal was to provide quicker and easier access to User's Guide and Getting started
- System requirements
- Now there is a separate page for system requirements.
- New section "How to und Best Practices"
- Collection of information around SASUnit, SASUnit PlugIn and Jenkins
- Will be extended in the future
- Includes known page "Best Practices configring SASUnit".
- Documentation home page
- There's a new chapter in SASUnit Users' Guide: Best Practice configuring SASUnit
- SASUnit has now a built-in feature for program documentation and no longer needs DoxyGen.
- Program documentation is now integrated into SASUnit
- There's a new subtree in the tree view that contains the program documentation
- SASUnit only supports "" as starting character for documentation tags. "@" is NOT supported
- There's a new subtree in the tree view that contains the program documentation
- Three new asserts
- assertText
- Compares two text files with an external shell script
- Under Windows the assert uses FC.exe
- Under Linux the diff command from bash shell is used
- Compares two text files with an external shell script
- assertImage
- Compares to images. It can mark and highlight differences between images or count the number of differing pixels
- Under Windows ImageMagick is used
- Version 6.9.1 or higher is needed. Earlier versions do not deliver a proper return code
- Under Linux ImageMagick is also used
- Version 6.6.9-7 or higher is needed. Earlier versions do not deliver a proper return code
- Under Windows ImageMagick is used
- Compares to images. It can mark and highlight differences between images or count the number of differing pixels
- assertExternal
- A generic assert that calls an shell script and evaluates the return code of the shell script. assertText and assertImage are two specific implementations of assertExternal.
- assertText
- When using SASUnit under Windows in languages other than German or English, SASUnit couldn't find the examinees
- This was due to the evaluating the result of the DIR-Command.
- Now the evaluation used language independent strings
- Under some circumstances paths inside SASUnit were not evaluated correctly when mixing absolute and relative paths in %initSASUnit
- Inside SASUnit now path are always absolute paths
- There's a check for consistency of values out of shell variables and corresponding SASUnit macro variables
- Under Win2008 Server R2 there sometimes was a general I/O Failure when executing test scenarios
- This problem is fixed
- Getting started section is completed in a first version (See Getting started with unit testing of SAS programs)
- A video covering all topics of Getting started is hosted on YouTube (Video: SASUnit Getting Started)
- SASUnit still ships with a DoxyGen configuration file, that is bound to DoxyGen Version 1.8.2
- SASUnit does no longer use DoxyGen resources
- Cross reference check for altered macros
- New parameter i_crossref turns on a cross reference check within %run_sasunit.
- This cross reference should reveal side effects in an early state of development.
- For each altered test candidate cross reference now checks dependent scenarios.
- If a scenario uses altered test candidates then this scenario is started too.
- Beside this core functionality the gathered calling sequence is used to display new reports in the Section "unit under test" that is cross reference report
- Cross reference report
- Two reports are generated per macro
- Which macros are called by this one
- Which macros are calling this one
- D3 technology (under BSD license) is used to render a tree of the calling sequence
- Two reports are generated per macro
- Output of OS commands is captured an routed to the SAS log file
- Tree view of SASUnit is now purely written in CSS, inspired by "The CSS Ninja tree view example" and showing the result in a color icon for the node
- For scenarios, test cases asserts and programs the appropriate result is displayed in a color icon
- SASUnit script files for CI are compatible with the new SASUnit Jenkins plugin that will be released soon.
- HTML pages are now rendered in UTF-8 encoding.
- Detection of date and time values is now more stable
- If running on windows a query to reg.exe is submitted to get current date/time format
- Log when displayed as HTML-page contained misleading text due to SAS quotation
- Table for scenario overview is now sortable. We use table sorter by Christian Bach under GPL license. JQuery Framework is under MIT license.
Support for new SAS version 9.4
No support for SAS 9.1.3 anymore due to usage of HTML4
- It is needed because we use the new inline formatting syntax, which is correctly nestable
- HTML pages are now created via SAS ODS using HTML4-Destination. There is no need to change your SAS session settings.
- The only HTML-Pages not created via SAS ODS are the log pages due to links for errors and warnings and the tree view html page.
There are several new assert:
- assertForeignKey Checks whether a foreign key relationship between the columns of two data sets exists.
- assertPrimaryKey Checks whether a set of columns can be used as primary key for the data set.
- assertRecordCount This assert checks whether a certain number of records exist in a data set specified by parameters i_libref and i_memname.
- assertRecordExists Check whether at least one record exists which satisfies a certain WHERE condition.
- assertRowExpression Check whether at least one record exists which satisfies a certain WHERE condition.
- assertTableExists Check whether a certain data set, view or catalogue exists.
Reworked assertion framework
- New asserts no longer require changes to overall macros. Now there is a defined interface.
- Each assert must implement the following macros:
- assert<assertName>
- This macro is called by the SASUnit user to validate conditions
- _render_assert<assertName>act
- This macro renders the actual column of the detail page for each test case. _renderDataColumn can be called in a default implememtation. See _render_assertEqualsact.sas for details
- _render_assert<assertName>exp
- This macro renders the expected column of the detail page fo each test case. renderDataColumn can be called in a default implememtation. See _render_assertEqualsExp.sas for details
- assert<assertName>
- Each assert may implement the following macro
- _render_assert<assertName>rep
- This macro renders html pages that are referenced by the corrresponding act- and exp-macros. (e.g. Comparison results for assertColumns)
- _render_assert<assertName>rep
Asserts now print the result of the test to the logfile.
- This feature is available when you specify i_verbose=1 in initTestcase.
- Each failed assert writes an error message to the logfile. It has the following Syntax:
- ERROR(SASUNIT): <error message>
- Each passed assert writes a NOTE to the log.
- This message can be modified by any assert. If you don't specify a message then a default message indicating failure or success is generated.
SASUnit will generate a XML-file containing JUnit-output by default.
- The xml file is stored in the rep folder as any other reporting result.
- Each scenario is treated as a test suite in Junit
- Each testcase is mapped to a Junit test case
- Only the first failed assert per test case is mapped to a Junit failure
- Each testcase is mapped to a Junit test case
SASUnit example project covers all new asserts
- There is a new scenario "Test for building a database".
This scenario deals with the creation of a data base and are checking constraints after and prior joining some tables to a combined dataset
It consists of two testcases
- Basedatasets
- Combineddataset
SASUnit ships with a DoxyGen configuration file, that is bound to DoxyGen Version 1.8.2
Several further enhancement included in release 1.3
- Slight changes to table headers
- Prior to execution of a scenario all created files from last execution are deleted
- New parameter i_include and i_exclude for assertColumns
- SASUnit utility macros now have shorter names
- Adjusted check for LINUX-os
- SASUnit now works correctly if paths contain blanks. This is true now for WINDOWS AND LINUX
- Detection of date and time values is now more stable
There is a new option to activate test coverage reporting.
- Using the test coverage reports it is easier to assess how much source code is covered by the test cases. Use this feature to discover which parts of a unit under test are never executed during tests. More about test coverage assessment see Patrick Warnat's paper on PhUSE 2010.
- The macro initsasunit now offers the new option i_ testcoverage. Setting this parameter to a value of 1 activates generation of test coverage reporting. This is now the default operation mode for all command scripts in the bin folder.
- In the SASUnit report, in the tab Units under Test, a new column ‘test coverage’ can be found, showing the calculated test coverage percentage for every unit under test. The percentage is calculated as the number of source code lines executed at least once during all tests of a unit divided by the number of all lines of the unit except non-contributing lines (e.g. empty lines or comment lines). The test coverage percentage figure is a html hyperlink targeting a colored html representation of the source code file. In this html representation, for every source code line it is shown whether the line was executed in the tests or not.
- This feature is only supported in combination with SAS 9.3.
- Test coverage assessment is implemented using the SAS option MCOVERAGE. Unfortunately, we discovered that the MCOVERAGE – Option is not working perfectly (SAS 9.3 TS1M2). In some cases executable source code is marked as non-contributing code. In very seldom cases, executed code is not marked as executed code. We have reported these findings to SAS Institute and they have confirmed this behavior. It is classified as an error to be corrected in the future. However, the available test coverage assessment is properly working in most cases and thus it is already good enough to be helpful.
- Corrected error in displaying the status of scenarios. If an error is detected in the scenario log then the scenario is now marked as erroneous. This is currently only available on the "Test Scenarios" page. It will be integrated into the "Units under Test" page in one of the next releases.
- This is the first release of SASUnit where project management has been fully hosted on SourceForge, so this is a starting point for faster release cycles and for better integration of the HMS development team and the user community in the future.
- Support for different environments has been reorganized, there now is a pair of command files for each scenario, one for complete execution of the test suite and one for incremental execution of changed programs only.
- Projects can now be moved without the need to change command files.
- Command files now pause when there is an error or a warning.
- Testing procedures have been improved by the means of more unit tests and a detailed test plan, also across different environments.
- The main page now includes more information about the execution environment (SAS version, configuration, user id etc.).
- The SASUnit logo has been changed to the green OK symbol and it now links to SourceForge.
- There is a new assertion for performance tests (assertperformance.sas).
- Example regression_test.sas has been enhanced for better comprehensibility (see regression.xls) and is now also available under SAS on Linux.
- Added support for testcases that need to redirect logfiles. Macro variables g_logfile and g_printfile hold the redirected paths used by SASUnit.
- Errors resulting from special characters in parameters of SASUnit macros have been fixed.
- Missing style sheet in assertreport has been added.
- Errors in the scenario log are now being highlighted in the scenario details report.
- Scenarios without a valid test case are reported as well.
- Missing test scenario programs are reported as well.
- Problems with English settings of Microsoft Windows have been solved.
- Column "number of test cases" on overview page for units under test wrongly showed number of checks, this has been fixed and column "number of checks" has been added.
- Warning and error messages have been eliminated from run_all.log.
- Fixed issue where %sysexec call to SAS for running of test scenarios failed because the call string became too long.
- Some additional minor bugs have been solved.
You can get the most current version of SASUnit and also the full self testing environment from the svn repository. Note that the svn head revision might not be fully tested.
Klaus Landwich (project management, development) Chuck Castillo (develoment assertTrue) Anne Kirchmann (documentation) Dr. Patrick Rene Warnat (development test coverage) Johannes Lang (development) Nikolas Aggelidakis (Linux development) Bernhard Braun (testing and develoment) Leonhard Rohr (develoment) Oliver Thieleman (develoment) Grischa Pfister (project management) Roland Mannshardt (project management) Andreas Mangold (project management)
Back to [User's Guide](User's-Guide)