-
Notifications
You must be signed in to change notification settings - Fork 1
Testing
The fuc group decided a goal of 90% test coverage for the project.
Integration tests are tests which test the whole compiler chain: Lexer -> Parser -> Semantic Analyser -> IR Generator -> LLVM Backend.
The integration tests can be found in fuc/code/test. Due to the limitations of JUnit, in order to be able to use the example programes in different test classes, they are centralized in ExampleProgs.java. The example programs comprise of the examples from common/examples (loaded from there) plus some addition test programs. TestBase.java provides the base class for all integration tests.
There are two kinds of integration tests:
-
Compilation Tests
-
Runtime Tests
The compilation tests test if the example program compiles through all stages of the compiler, producing some kind of LLVM IR Code in the end. These tests only test if the compiler runs through without producing errors, not for the correctness of the resulting LLVM IR Code.
If the example programs are erroneous, they are testet for producing the expected error messages.
The compilation tests are included in the default run-tests ant target (and thereby also included in the travis CI testing).
The runtime tests compile the example programs and execute the resulting LLVM IR Code with lli. The results of the execution are then checked for the correct output (print statements) and exit code (return statement). The runtime tests hereby provide limited correctness testing (no proof of correctness of course!).
The runtime tests need a working LLVM installation (>= version 3.2). They are not included by default, but have their own ant target: ant run-tests-runtime.
Every commit pushed to the remote master branch is tested with Travis CI. If something is pushed it will trigger an ant run-all-tests and send an email to the commit author (email taken from git) if something failed.
On the main github page (in README.md) is a png that displays the current build status of the master branch. The testing can be seen in action an the Travis CI status page for fuc.
The cross tests test for the interchangeability of the moduls from the [fuc] and the [javabite] compiler.
For the tests to work, the javabite moduls have to be placed into the code/dist directory as jar-files. The javabite jars can be obtained be running ant buildCompiler in the javabite repository.
The crosstests have their own ant target: run-tests-cross.
For milestone 2, only cross tests switching the lexer are implemented. The tests use the javabite lexer and the rest of the fuc front- and backend to perform the tests.