2Payment_terminal_acceptance_testing.tex

%!TEX root = thesis.tex

\chapter{Payment Terminal Acceptance Testing}
\label{chapter:Payment terminal acceptance testing} 

When developing software with agile methodologies for payment terminals, i.e for an embedded system, testing is a crucial part of the process. The earlier the defects and errors in the software are detected, the lower the cost and needed effort will be for correcting those \citep{myers2011art}.

Test environment that can be used in acceptance testing of payment terminals has several challenges to tackle and matters related to physical and technical aspects of the payment terminals have to be considered. This Chapter discusses the background of these challenges. Open source technologies were also preferred by the customer. Therefore, a section discusses the benefits obtained by using open source software and hardware in acceptance testing environment for payment terminals. Chapter also discusses the different approaches for acceptance testing as well as how should the test suites be defined in order to make them understandable and reusable.

\section{Benefits of Open Source Solutions}
\label{section:Open source}

When designing an automated acceptance testing environment from scratch, evaluation and availability of different possible components play a significant role in terms of development speed and cost. Suitability of one individual software subsystem is hard to determine just based on a manual or documentation of the product. Software has to be evaluated in terms of functionality, stability and performance. In addition, different software decisions have to be compatible with each other. Software components might also need some modification to suit the needs of the intended environment. All this applies to the hardware parts as well.

Open source software provides an advantage on these matters over closed source products as the source code is easily available \citep{morgan2007benefits}. As open source software can be accessed free of charge, a component can be easily evaluated by trying out whether they work for the purpose or not. The evaluation can also include an analysis about how easily the open source product can be modified to suit the needs. This especially is hard to achieve with commercial closed source products as the source code is not available.

According to \cite{paulson2004empirical}, open source projects usually have fewer defects than closed source projects. Defects are found and fixed rapidly as they are reported openly to the open source community. If a defect is found during evaluation of the product, it can also be corrected by the user. By doing this, the user can contribute to the project. This, on the other hand, is hardly never possible with closed software. \cite{paulson2004empirical} also state that open source projects foster more creativity than closed source counterparts. This means that number of functions added over time is higher in open source projects. When using the product in some new field of use, this can be a great advantage as user can report desired features to the community and it can be added relatively quickly if the feature is considered needed by the community.

Open-source hardware means that details and plans of the product and its parts are commonly available \citep{rubow2008open,acosta2009open}. This allows that parts can be manufactured and modified by anyone with knowledge and skills to suit individual needs. When detailed part descriptions are available, multiple manufacturers can fabricate the actual parts. This creates competition and therefore usually lowers the price of individual hardware components.

As the overall security of the payment terminals is a high priority, use of open source technologies is seen as an effort to fulfill this requirement. Open source products provide transparency to the actual users and therefore support growing trust amongst customers.

\section{Common Characteristics Between Payment Terminals}
\label{section:payment terminals}

When designing automated test environment for different kinds of payment terminals, different physical and technical features have to be taken into account. Environment has to be able to manipulate different types of payment terminals and test structure has to be designed to adapt to the needs of different software and their different versions running on the payment terminals.

Majority of payment terminals share some common characteristics as they are made for same purpose: handling card payments. Scope of this thesis is to propose a testing environment for those payment terminals that share three main features: a keyboard, a screen and a card slot. Different types of terminals are visualized in Figure~\ref{fig:terminals} and Figure~\ref{fig:izettle} below.

Screens of the payment terminals differ in terms of size, placement and type. Test environment has to take into account different screen placements and it has to support both black and white (BW) and colored displays.

Keyboards of payment terminals share majority of keys together as number keys are needed for entering a personal identification number (PIN) code and accept- and decline-buttons are needed for accepting and canceling the payment. Keyboard layouts, however, differ between different manufacturers and even amongst different models of the same manufacturer.

Location of the chip card slot is usually on the lower side of the payment terminal or on top of the screen of the payment terminal. Research done within this master's thesis is limited to those terminals that have the chip card slot at the lower side of the payment terminal as this simplifies the hardware needed for test environment. This is described more in depth in section~\ref{section:Proposed hardware}. This study is also limited to chip card readers and therefore, magnetic stripe readers and near field communication (NFC) payments are not addressed.

\begin{figure}
\centering
\begin{subfigure}{0.5\textwidth}
  \centering
  \includegraphics[height=6.8cm]{images/terminal1_1.png}
  \caption{Terminal 1}
  \label{fig:sub1}
\end{subfigure}%
\begin{subfigure}{0.5\textwidth}
  \centering
  \includegraphics[height=6.8cm]{images/terminal1_2.png}
  \caption{Terminal 2}
  \label{fig:sub2}
\end{subfigure}
\caption{Two examples of payment terminals from different manufacturers. Image for subfigure \ref{fig:sub1}: \citep{image1}}
\label{fig:terminals}
\end{figure}

\begin{figure}[ht]
  \begin{center}
    \includegraphics[width=7cm]{images/izettle.png}
    \caption{Example of a payment terminal which attaches to a smart phone.}
    \label{fig:izettle}
  \end{center}
\end{figure}

\section{Different Approaches for Test Automation}
\label{section:test automation approaches}

According to \cite{broekman2003testing}, testing of embedded systems and embedded system software can be very different depending on what kind of system is under testing. Mobile phones have to be tested in a very different manner than for example cruise control system in cars. Nevertheless, some general guidelines and similarities exists and should be followed.

Testing of a payment terminal software in an automated way can be viewed at different levels. Most abstract classification can be seen if the testing is divided into two levels: white box testing and black box testing. White box testing is a methodology where the source code is investigated and test cases are written to test the internal logic of the program. Black box testing, on the other hand, concentrates only on the inputs and the outputs of the software. Everything between those is not in the field of interest as black box testing only focuses on whether the right input produces the wanted output. \citep{nguyen2001testing,myers2011art}

\cite{khan2012comparative} distinguishes these methodologies clearly from each other by stating that white box testing is a process wher,e full knowledge of source code is needed in order to write the tests. Black box testing is described in a way that only the inputs and outputs of the application has to be known and black box testing has no or only little relevance to internal works of the program \citep{pressman2005software}. Black box testing methodologies can be thus seen to apply for testing of working product against the initial requirements of the software.

\cite{huizinga2007automated}, on the other hand, presents that test automation can be divided into several layers that are unit testing, integration testing, system testing and acceptance testing. Unit testing is defined to cover testing of a single unit of the software's source code e.g. individual methods and functions of the software. Integration testing is described as a testing phase to verify that different parts of the software work together as a group. System testing is described as being a testing phase where hardware and software is integrated and tested to meet the requirements of the system. This can however include simulated data. Acceptance testing is represented as highest abstraction level of this classification as it ensures that the final product meets its acceptance criteria defined by the customers.

However, these classifications are not mutually exclusive as both white box and black box testing methodologies can be applied to all levels of testing. For example, when implementing unit tests for a software, individual methods are commonly being tested in terms of whether a certain input produces a right output. This can be seen to follow the black box testing methodology if the methods tested are simple and small enough. Correspondingly, acceptance tests can be used to validate whether the system meets the business requirements and for this, knowledge about the business logic is needed \citep{haugset2008automated}. This, on the other hand, can be seen to follow the white box testing methodology.

As black box testing is based on the external exceptions and behavior of the software \citep{khan2012comparative}, required acceptance testing of the payment terminal software can be seen to follow this methodology. Intended automated testing of the payment terminals seems to also follow the acceptance testing phase of the division made by \cite{khan2012comparative}.

\cite{Ramler} divides the general architecture of an embedded system into three parts. In this classification the human machine interface (HMI) is the top layer. This is followed by the software running on the device and the lowest level are the hardware components of the machine which can be accessed through different analog and digital interfaces. As this master's thesis addresses only the acceptance testing of one instance of an embedded system and as it only has to verify whether the system fulfills its acceptance testing requirements, these abstraction levels can be overlooked. System under test (SUT) can be viewed at a level where only the inputs and the outputs of the system are considered important. Also for this purpose, the black box testing methodology seems to be the appropriate testing manner.

Acceptance testing of a payment terminal software can be seen as a testing phase where the UI of the device and the use cases of the device are tested at the final production level, i.e. through using the real buttons of the device under test and observing that the expected messages can be seen through the screen of the same device. This can be seen as an effort to automate a real human user using the payment terminal.

\section{Test Suite Syntax}
\label{section:test suite syntax}

Test suite syntax plays a significant role in an automated acceptance testing environment of payment terminals in terms of test readability, reusability and adaptivity. When building an automated acceptance testing environment, the tests should be understandable enough that the whole development team and all of the project's stakeholders can easily adopt to the test syntax.

According to the well recognized guidelines of test automation by \cite{snakeoil}, test automation and the process that it automates should be kept carefully separated. Test automation should be built in a form that it is easy to review and distinct from the process that it automates. These guidelines should be taken into account also when determining a suitable test framework and test suite syntax.

When evaluating suitable test automation frameworks, it should be recognized that simplicity is a key factor of successful test automation. Software projects usually involve some sort of quality assurance (QA) or even a separated QA team. Projects also tend to involve fair amount of people with no technical background or programming skills and yet their responsibilities can still involve guaranteeing the quality of the software. \cite{just_enough} recognize that high level test languages help to share the knowledge amongst the people that are responsible for the product. Sharing information and knowledge amongst the project's stakeholders helps achieving the objectives of test automation and builds up the morale amongst the people that are involved.

\cite{lowell2003successful} state that acceptance tests should be easy as possible to write or otherwise people working with the project will not write the tests as the task is seen unpleasant. In order to cope with changing requirements or updated features, the tests should be easy to maintain as people have to be able to update them even if they have been written by someone else. For this reason, the test cases should be human readable and understandable also to non-technical people. Test steps should be self explanatory and unambiguous.

Test cases in acceptance testing of a payment terminal contain relatively high amount of repetition, for example, test step of inserting a PIN code is the same whether right or wrong PIN code is inserted or whether the test case would validate a credit or debit payment. For this reason, test case syntax should be as modular as possible in order to allow reuse of keywords with different parameters. Easily reusable keywords also allows fast creation of new test cases.

Tests can essentially be written in some conventional programming language, for example Java or Python, or by using some higher level language. There are many widely used test frameworks available for conventional programming languages, for example jUnit for Java \citep{junit}. This, however, requires programming experience to some extent in order to be able to understand and modify existing tests or write new ones. This would mean the usage of conventional programming language would be opposing the guideline for writing the tests as understandable as possible and therefore it would be opposing the best practices of automated acceptance testing. On the other hand, test case syntax must be versatile enough to accommodate different kinds of testing scenarios and needs. Efficient use of variables must be possible and for example use of different kinds of loop structures must be supported. This leads to a situation where the abstraction level of the test cases has to be considered carefully.

\begin{figure}[ht]
  \begin{center}
    \includegraphics[width=\textwidth]{images/junit_example.png}
    \caption{Example of a jUnit test case that tries to login to website.}
    \label{fig:junit}
  \end{center}
\end{figure}
\FloatBarrier

In addition to the test frameworks utilizing the use of some conventional programming language for test cases, there are also couple of well-recognized tests frameworks available that use a more natural language for writing the tests. These frameworks usually use the same libraries for interacting with the system under test as more low-level frameworks, but they allow a higher-level syntax in the actual test scripts. One popular example of this kind of higher level test framework is Cucumber \citep{cucumber}. Cucumber is an open source acceptance test framework that utilizes behavior-driven development (BDD) style. Cucumber uses Gherkin language that is designed to be human readable without previous knowledge of programming \citep{gherkin}. This means that also non-technical personnel involved with the project can understand the test cases.

\begin{figure}[ht]
  \begin{center}
    \includegraphics[width=9cm]{images/cucumber_example.png}
    \caption{Example of a simple Cucumber test scenario and use of Gherkin language.}
    \label{fig:cucumber}
  \end{center}
\end{figure}
\FloatBarrier

Another good example of a higher level test framework is Robot Framework (RF). RF is a generic keyword-driven test automation framework that allows creation of human readable test cases \citep{robotframework}. Reusability and extendibility of high-level keywords is also made relatively easy \citep{stresnjak2011usage}. \cite{Rfuserguide} also outlines that RF has a highly modular software architecture allowing it to be easily connected to any kind of SUT by using different test libraries.

Example of a Robot Framework test case can be seen in Figure~\ref{fig:robot_example} below. It is easy to see the intended test case execution by looking at the test case. This will be the goal for the environment proposed later on in this master's thesis.

\begin{figure}[ht]
  \begin{center}
    \includegraphics[width=\textwidth]{images/robot_example.png}
    \caption{Example of a simple Robot Framework test suite \citep{robotframework}.}
    \label{fig:robot_example}
  \end{center}
\end{figure}
\FloatBarrier