Tutorial Manager

Tutorial Manager [v2.0]

The Tutorial Manager (TM for short) is responsible for the following major functionalities:

1. Facilitating multi-step tutorials for all supported platforms.
2. Providing context-relevant help for users.
3. Showing shorter guides for specific actions. [WIP]
4. Prompting the user with sporadic tutorial-like hints. [WIP]

Introduction to Tutorial Systems

Tutorials are there to guide users through a predefined scenario in order for them to learn or better understand some parts of MirageXR. While different tutorials cover different scenarios, on an abstract level they all follow the same rules. Each tutorial consists of steps, atomic intervals of interaction where the user is asked to perform some action. For each step then, the following occurs:

1. The step is entered, relevant checks are made to insure that the step can be set up. If there are no more steps, the tutorial finishes.
2. If there is a target of interaction (such as a button or augmentation), a UI element is called to highlight the target.
3. If there is a message or instruction text, it is shown. Sometimes this is combined with the highlighting UI element.
4. Exit events are defined for the step. The TM needs to know when to advance to the next step (finish step) or exit the tutorial in case it can no longer progress (exit/close step).
5. When the user completes the intended interaction and an exit event is invoked, the step is finished (or exited). All used UI elements to show the current step are removed from sight.
6. The next step is called. (Go to 1.)

Tutorial Data Model

Tutorials are stored in the MirageXR/Resources folder as JSON objects. They are loaded on demand by the Tutorial Manager. A tutorial mainly consists of its steps, as shown on the UML Class Diagram below:

Aside from its steps, it also has a Name and a defined IntendedPlatform (mobile, hololens,...) for which it is meant. These two are mostly there for readability as these files are directly referenced in code anyway.

The main data for tutorials is in its steps. TutorialStepModel is an abstract class which defines the necessary items for each tutorial step:

• StepType - This field holds the information on the step type through an enum. It is necessary for JSON readability as well as converting to the correct type by the NewtonsoftJSON library.
• DelayInMilliseconds - Defines a delay before the step is set up. Mostly used when waiting for certain UI or world-space items to appear.
• IsValid() - Each step type defines which fields and/or other requirements are necessary for a step to be valid.

One tutorial can have multiple steps of each type, in any order, as long the objects it is targeting are loaded and visible when the steps occur.

World-space Steps

These steps are intended for use in World-space environments, i.e. mostly in conjuction with augmentations and AR UI. They have the following relevant fields:

• FocusObject - This is the ID of the object in focus, so that we can find the (Game)object through GameObject.Find(). This will then be the target of the step's intended interaction.
• ActualTarget - If defined, it should be a child object of the FocusObject. This is used in cases where a sub-element of an object fits better as the primary target. Also, can be used to better navigate the object tree, as sometimes objects are named the same.
• Message - The instruction text to show the user.
• ArrowType - The term "Arrow" is used slightly for legacy reasons, but serves as a good name for the World-space UI element used to highlight the target. The "Type" defines which kind of Arrow should be used. Currently there is only the default arrow; the field is mostly there for future-proofing.
• ArrowPositionOffset and ArrowRotationOffset - Offsets for the position and rotation of the arrow relative to the target. Useful, as the arrow doesn't always show up as you'd expect it; improves readability for users.
• FinishEvent - Defines the event which finishes the step. Interacts with the TutorialManager's event system.
• CloseEvents - Defines events that, when they occur, should close the tutorial. Interacts with the TutorialManager's event system.

UI Steps

These steps work in the context of the MirageXR's mobile user interface. They have the following relevant fields:

• Id - Defined ID of the UI element, defined by a TutorialItem component.
• Message - The instruction text to show the user.
• Position - Currently there are three possible positions, based on verticality in the UI: Top, Middle, or Bottom.
• BtnText - Defines what text is shown on the popup dialog's button. Default is "Cancel".

Event-only Steps

These steps are "non-interactable", i.e. there is no UI element that shows these steps to the users, they only wait for an action to be completed through FinishEvent. This step type is used mostly in conjuction with other step types, but when the finish event for the instruction cannot be detected immediately. For example, when calibrating the user goes through multiple pages and interacts with different elements, some of which have their own instructions already present. We want to detect only when calibration is finished and not these intermediate steps.

Tutorial Handlers

Handlers are the main tools the TutorialManager uses to delegate tutorial processing. Here is a UML Class Diagram showing the relevant structure:

As you can see, only UI and WS steps have handlers; event-only steps are handled directly by the Tutorial Manager.

Processing of tutorials happens in the following manner:

1. StartTutorial(TutorialType) is called. Based on the tutorial type, a tutorial model is loaded from the relevant JSON file. NextStep() is called.
2. We go into the first NextStep(), which checks if there are any more steps and starts the HandleStepWithDelay(TutorialStepModel) coroutine.
3. In the HandleStepWithDelay(TutorialStepModel) method, first a delay occurs, if the step defines so. Afterwards, the Tutorial Manager hands over control of the step to the relevant handler and sets the _expectedEvent and _currentClosingEvents. In the case of event-only steps, only the latter happens.

Handling Worlds-space Steps

As the Tutorial Manager handles events and step advancement, handlers' main job is to highlight interaction targets. In the case of world-space steps, these targets are mostly augmentations and world-space UI elements, which we highlight using Arrows.

Arrows are created by the TutorialArrowFactory, a sigleton class which is tasked with generating arrows according to the given type. The factory pattern is used so that the TutorialHandlerWS can be agnostic of which type of arrow it is dealing with.

When the Show(TutorialStepModelWS) method is called in the WS handler, the handler simply unpacks all of the data and either uses an existing Arrow or uses the arrow factory to create a new arrow. Finally, PointTo(...) is called on the arrow to show it in world-space.

Currently, there is only one, default, implementation of the TutorialArrow abstract class and that is the Tutorial3DArrow. Aside from the mandatory methods it also uses Unity's Update() to follow the target if it moves, while maintaining given offsets.

Handling User Interface Steps

As with world-space steps, the UI step handler also mainly deals with highlighting. It uses the following classes to do so:

1. TutorialHandlerUI - Responsible for unpacking TutorialStepUIs and organising interaction for the other classes. Also responsible for masking/unmasking highlighted targets.
2. TutorialMessageView - Script for the popup that shows messages/instructions to the user.
3. TutorialItem - Script that is attached to UI elements so that the element can be found by the handler.

To highlight items on-screen, the UI step handler uses the UnmaskForUGUI library. It basically makes a hole in the otherwise unclickable background so that the element is visible and interactable. See the library for more details.

The UI step system works in the following way:

1. The TM gives over control to the handler through handler.Show(Queue<TutorialStepModelUI). It also sets the expected finish event to UI_FINISHED_QUEUE, which is fired by the handler when its queue of given steps is exhausted. Note: While we pass on a queue, for full-length tutorials this will most likely be a queue with only a single step. Even though multiple steps could be passed at once, this is done to simplify matters as the TM deals with only one step at a time. Multiple steps in the queue can still be seen with context help and other, shorter tutorials.
2. Show() initialises highligting and calls Next() which starts the first step. This checks if there are any more steps in the queue and calls ShowItem(TutorialStepModelUI) if there are.
3. ShowItem(TutorialStepModelUI) first tries to find the target with FindTutorialItem*(string), based on its ID, if given. It does so by searching through GameObjects that have TutorialItem as a component. When a TutorialItem with a matching ID is found, the search is over. Note: This is why it is important to attach TutorialItems to targets beforehand, as well as giving them unique IDs.
4. When the target is found we unmask it (highlight it) and then proceed to MarkTarget(TutorialItem). Here we add listeners to the target, in order to be able to catch when a user is finished interacting with it. This varies for each type of UI element that the user can interact with.
5. Finally, ShowMessage(TutorialStepModelUI) is called to show the instruction text, if there is one. The TutorialMessageView popup is used for this purpose.