discussion.tex

\chapter{Discussion}
\label{cha:discussion}

In chapter \ref{cha:results} we layed out the results of our work. In this chapter we now want to discuss the results of the user study as well as the usage of the library for developers. Additionally this chapter offers insight into some architectural consideration and lessons learned.

\section{User Study}
\label{sec:discussion-usability}

A system such as the one developed in course of this thesis can be evaluated in many ways: The performance of the software can be quantified by measuring response times of the application, as happened in \cite{Niwa:Web-presentation-powerpoint} or \cite{Inoue:RealTimeQuestionnaire}, the usability can be assessed quantitatively through usability tests, measuring error rates and time to complete certain tasks and qualitatively using \emph{loud thinking} and interviews \cite{Reindl:automatisierte-user-interface-evaluierung}. If the expectations lied out in chapter \ref{cha:mechanisms} were actually met, if listeners feel more engaged and whether the perception of phone usage in presentations has changed with unveil can also be estimated using qualitative and quantitative evaluation methods. We conducted a quantitative study and offered the users a free text field to give more faceted feedback. We feel, however, that valid results can only be achieved observing the usage of the presented tool over a longer period of time and involving control groups.

Throughout the process of developing the different libraries, several informal presentations and meetings have been held using unveil, both in academic and business settings. This allowed for several iterations of the design, as well as the gradual introduction of more and more interactive mechanisms. The final study, presented in chapter \ref{cha:results}, was then conducted using the final prototype with all mechanisms enabled.

We were pleased to see that all listeners understood the base interface of the application, both on their phones and on the laptop. We have however not had a single tablet user and all participants used iPhones and MacBooks with Safari or Chrome as browsers. During one of the earlier trials, some users experienced glitches in the navigation of the slides in Safari and could scroll within them, moreover local storage is disabled in Safari's \emph{private mode} (thus making it impossible to persist voting results). On bright side, initial concerns regarding the usage of socket.io have not come true, since the presentations were always held in networks we had full control over (so corporate firewalls did not play a role).

As far as the response time of the application is concerned, with up to 30 concurrent users, no noticable declines in performance have been experienced so far and the real-time features feel instantaneous. During the user study, one participant even programatically sent 10,000 clicks on a reaction to the server. We were pleased to see that despite the high amount of requests, only one participant reported that the application did not feel fast. This was probably due to the use of an older generation iPhone (4s), which did not offer the performance necessary. 
The only interaction which is not instantaneous is the transmission of base64-encoded media content through the WebSocket. Since the delay is between listener and speaker, however, the transmission time is not critical for the application to feel responsive. Another point connected to sharing media is the absence of a native sharing feature for phones, which was an anticipated limitation of creating a web application over a mobile one.

The implemented functionalities were generally received well by the users and all participants of our study seemed to understand the interface and found the usage of the mechanisms intuitive. This, however, might be due to the subjects all being digital marketing professionals and therefore technically versed. One participant wrote: ``Could be a better experience if you add more animations making it [the application] smoother'', which is something we want to investe more time in in the future.

Despite our expectations and previous experience using the tool, in the user study, participants did not use the content sharing mechanism much. Instead, the polling feature received overwhelmingly positive feedback. One participant wrote: ``the possibility of giving your anonymous opinion makes the results more trustworthy'', another one states ``[...] it was somehow comforting to know my opinion was shared by most people [...]'. Two participants note that the voting could consist of only one step instead of two (the choice of the option and the submission).

As far as the sharing mechanism is concerned, in our trials, we experienced a large amount of test-data being sent through the mechansisms, especially in the beginning of the presentation. We therefore advice others to provide one or two empty slides in the beginning of the presentation if the audience is new to the software, so the application and its functionality can be explored. Since some users mixed textual content with links in the early tests, we separated media upload, link sharing and questions (or other text) into distinct buttons. Although we did not receive any negative feeback, the feature was not as accepted as well as the other ones. One participant noted the interface could be styled more, most did not include feedback though. The reasons why this mechanism was not perceived as engaging by more participants will have to be studied further.
We were very pleased to see that some of the people who had used unveil, actively mentioned wanting to have the ability to share links and thoughts in meetings without the technology. Resulting from this feedback, an unexpected way of using the software emerged: Instead of enriching an existing presentation, an empty slide was provided, allowing anyone in a meeting to add their own content in the $2$-dimensional space in a brainstorming-like fashion.

The reaction system was also understood by the participants and they stated being familiar with the used emoji. The feedback though positive, included that the reactions for laughter, boredom and approval were ``fun'', but at times redundant. The other ones, however, were perceived as useful ways of interacting with the presenter.

Another observation we made, is that although listeners generally reacted exceedingly positively to QR-codes (linking to the address of the presentation), it was usually faster for them to type the address into their browsers directly. This usually caused a few minutes of interruption, which should be accounted for.

The biggest weakness of unveil for most users was the inability for the presentation to be permanently altered. Especially in meetings and informational presentations, many listeners asked for a link to the collaboratively created presentation afterwards, some users were forced to reload the website due to cross-browser compatibility issues in earlier trials and lost the current state of the slides, late-comers also did not have the possibility to jump into an already altered presentation. This will make it necessary to create a more intelligent and opinionated back end in the future.

\section{From a Developer's Perspective}
\label{sec:discussion-dev}

Most of the feedback we have collected about unveil over the last months came from listeners. This has two reasons: Firstly, the creation of presentations requires knowledge and experience with front end web development technologies, secondly, we have not started promoting the resulting libraries yet, as we feel the system is not stable and mature enough to be used outside our internal settings. However, external developers have provided us with their feedback regarding the syntax used for defining slide decks and seem to not have had any problems understanding the usage of the libraries. This validates our decision to choose semantically-named tags rather than HTML class names to identify different components. On the downside, we seem to have overestimated the level of knowledge necessary to create own presentations, as a few developers were not entirely familiar with the new ES6 syntax and the process of bundling JavaScript and CSS files, so an easier way of importing all necessary libraries should be provided in an example presentation.
In the long run, this product will only be able to increase its popularity if a visual editor for authoring slides, as well as a system to manage (i.e. host) them will be available. Some users also raised the question of how and if it was possible to import PowerPoint presentations, to be able to use the created mechanisms in connection with already existing software.

\section{Architecture}
\label{sec:discussion-architecture}

Generally, it has to be said that the project is still only a prototype and does not provide the stability necessary for us to feel confident promoting the product. One particular problem we have been experiencing seemingly randomly is the navigation getting stuck in an infinite redirect loop when two or more connected presenters try to navigate at the same time. Moreover we have so far ignored security concerns such as adding password protection to the presenter mode.
Although the chosen architecture allows for a lot of flexibility and freedom for developers and in that sense fulfils the criteria impress.js and reveal.js did not, we have discovered more powerful, streamlined and widespread patterns when working with React over the last half year. Since we had no experience with React prior to the start of the development, best practices oftentimes only became apparent throughout the project and through the work on and with other React applications. To make the libraries yet easier to use and extend and more easily debuggable, future iterations of unveil will likely be based on \emph{Redux} \cite{redux}, with the presentation state globally accessible. This will make it easier to decouple components from each other through the introduction of a centralised state container.