Initial Tool and Pen Behavior Functionality

design/behavior.actions.md

@@ -0,0 +1,102 @@
+# Behavior Actions
+
+Behaviors are high level concepts that are made up of one or more actions.  These actions make up the interactions
+that the user makes with the given behavior.  These actions are broken up in to three states: `started`, `performing` and `stopped`.
+
+- `started` - The state of the initial frame that an action has started..
+- `performing` - The state of all frames after `started` while the action is being performed.
+- `stopped` - The state of the frame after the action has been stopped and all other frames after until started again.
+
+These actions result in events being raised from the runtime to the SDK for app developers code to listen to.  These events are based 
+off of the threee action states above as such:
+
+- `started` - Fired once when the action has first transitioned to started.
+- `performing` - Fired during synchronization updates while the action is being performed.  This is an optional event that is exposed only on behaviors that provide them.
+- `stopped` - Fired once when the action has first transitioned to the stopped state.
+
+**Proposed**: In addition, there is an optional update event that can be registered to the action for getting updates while that
+action is being performed on the client.
+
+## Architecture
+
+### Action State
+``` ts
+export type ActionState
+    = 'started'
+    | 'performing'
+	| 'stopped'
+	;
+```
+
+### Action Handler
+``` ts
+/**
+ * The action handler function type.
+ */
+export type ActionHandler = (user: User, actionData?: any) => void;
+```
+
+### Action API
+``` ts 
+/**
+ * Add a handler for the given action state for when it is triggered.
+ * @param actionState The action state that the handle should be assigned to.
+ * @param handler The handler to call when the action state is triggered.
+ */
+public on(actionState: ActionState, handler: ActionHandler): this;
+
+/**
+ * Gets the current state of the action for the user with the given id.
+ * @param user The user to get the action state for.
+ * @returns The current state of the action for the user.
+ */
+public getState(user: User): ActionState;
+
+/**
+ * Get whether the action is active for the user with the given id.
+ * @param user - The user to get whether the action is active for, or null
+ * if active for any user is desired..
+ * @returns - True if the action is active for the user, false if it is not.  In the case
+ * that no user is given, the value is true if the action is active for any user, and false
+ * if not.
+ */
+public isActive(user?: User): boolean
+
+/**
+ * Add a handler for the performing update call for this action.  Callback called on the
+ * standard actor update cadence while an action is being performed, accompanied by optional
+ * action data.
+ * @param handler The handler to call when the performing action update comes in from the client.
+ * **/
+public onPerformingUpdate((handler: ActionHandler): this;
+```
+
+## Network
+
+The `PerformAction` payload contains the type of state that the action is going through, as well as an
+optional `actionData` object for the action being performed.  There is also a third category for the 
+`PerformAction` payload that is `performingAction` which uses the payload to send up `actionData` if the 
+behavior wished to convey this additional data for while the action is being performed.
+
+### Perform Action Payload
+``` ts
+/**
+ * @hidden
+ * Engine to app. The user is performing an action for a behavior.
+ */
+export type PerformAction = Payload & {
+	type: 'perform-action';
+	userId: Guid;
+	targetId: Guid;
+	behaviorType: BehaviorType;
+	actionName: string;
+	actionState: ActionState & `performing`;
+	actionData?: any;
+};
+```
+
+## Client
+
+The client will be respondible for sending `PerformAction` messages when an action is started or stopped, 
+as well as performing updates on the standard actor update cadence.  Any date that the specific action should
+pass along in the context of the behavior will be passed along in the optional action data object in the payload.

design/high.resolution.transform.updates.md

@@ -0,0 +1,42 @@
+# High Resolution Actor Transform Updates
+
+There are cases that a developer may want a higher resolution of transform updates that occur for an actor.
+To facilitate this and allow for still having a regulated bandwidth, we can add an additional API on the actor
+to allow for a developer to register/de-register for high resolution updates that would give an array of transforms
+containing the full high resolution path that occured for the actor during the 10 hz regulated network update.
+
+## API
+
+ A new actor API will be exposed to allow a developer to register and de-register for high resolution
+ transform updates.  
+
+``` ts
+/**
+ * Add a callback to the actor to receive path updates for while the tool is being used
+ */
+public onHighResTransfromUpdate((path: Transform[]) => void) {
+    // Path callback registration.
+    // Enable high resolution updates on the client.
+}
+
+/**
+ * Remove a callback to the actor to receive path updates for while the tool is being used
+ */
+public offHighResTransfromUpdate((path: Transform[]) => void) {
+    // Path callback de-registration.
+    // Disable high resolution updates on the client.
+}
+
+```
+
+## Network
+
+The actor update structure and payload will now need to be able to accept an array of transforms per update.
+This array would contain one transform in the case of normal resolution transform updates, and more in the
+case that high resolution transform updates were enabled for the actor.
+
+## Client Implementation
+
+The client side changes will require a subscription mechanism on the actor runtime instance to signal that 
+high resolution updates need to be gathered for the actor during a fixed update interval and queuded to be 
+sent up with the normal 10 hz update message for the actor.

design/pen.behavior.md

@@ -0,0 +1,87 @@
+# Pen Behavior
+
+The pen behavior is a method of recording the path by which a user moves a grabbed object that they activate so that various scenearios 
+can be carried out that wish to capture the virtual drawing of a user.  This behavior will carry with it client side visuals
+to ensure low-latency feedback, but will supply the complete curve data needed by the MRE app to acto on once the drawing is done.
+The `PenBehavior` will be an extension of the `ToolBehavior`.
+
+## Architecture
+
+There will be a new `PenBehavior` that extends the `ToolBehavior` that will have the ability to generate a wet ink effect along the use 
+path on the client.
+
+### Behavior
+``` ts
+export interface PenBehaviorInitOptions {
+	drawOriginActorId: Guid;
+}
+
+/**
+ * Pen behavior class containing the target behavior actions.
+ */
+export class PenBehavior extends ToolBehavior {
+	public drawOriginActorId: Guid;
+
+	/** @inheritdoc */
+	public get behaviorType(): BehaviorType { return 'pen'; }
+}	
+```
+
+### Draw Data
+
+``` ts
+export interface DrawDataLike {
+	transform: Transform;
+	// Potentially additional data to come, such as:
+	pressure: number;
+}
+```
+
+## Network
+
+The network messages for this behavior will use the standard behavior payloads for actions started and stopped.  The two actions supported 
+by the Pen behavior are hold (begin/end) and drawing (begin/end).
+
+In addition to the standard `PerformAction` start and stop payload, there will be a third `PerformAction` type for `perfoming` which will
+contain the draw data in the optional action data of the `PerformAction` payload.
+
+## Sync layer considerations
+
+Behaviors already have sync layer filtering to ensure that the action messages are for the client they happen on to the MRE app only.  Grab 
+also has special sync filtering based on the client that the grab is happening and the rest of the clients.  
+
+## Client implementation
+
+The client will require the addition of a new pen behavior for the host app to implement, as well as a draw recording system that will facilitate
+the recording of the transform data over time while a draw is happening and to send that data in chunks during the draw based on a set frequency.
+
+## Example Usage
+
+``` ts
+// Create an actor
+this.penModel = MRE.Actor.CreateFromGltf(this.assets, {
+    // from the glTF at the given URL, with box colliders on each mesh
+    uri: `${this.baseUrl}/pen.glb`,
+    colliderType: 'mesh',
+    // Also apply the following generic actor properties.
+    actor: {
+        name: 'penTool',
+        parentId: root.id,
+        transform: {
+            local: {
+                scale: { x: 0.4, y: 0.4, z: 0.4 },
+                position: { x: 0, y: 1, z: -1 }
+            }
+        }
+    }
+});
+
+const penBehavior = Actor.SetBehavior<PenBehavior>();
+
+penBehavior.onDrawEventReceived((drawData) => {
+	// Hypothetical spline generator tool in node.
+	const path = drawData.map(dd => dd.transform);
+	const spline = SplineGenerator.generateSpline(path);
+	// Generate a mesh from the spline and use it to create a new actor at the point of origin of the draw.
+});
+```

design/set.behavior.actor.md

@@ -0,0 +1,33 @@
+# Set Behavior API
+
+An API is needed on the actor for the developer to be able to set a bahavior for the actor to use to 
+allow fo user interaction with that actor.  The API should ensure that the behavior is tied to the one
+actor it is being set on and should be flexible enough to allow setting params on the the behavior at the
+time of creation or initialization.
+
+## API
+
+### In actor.ts
+``` ts
+/**
+ * Sets the behavior on this actor.
+ * @param behavior The type of behavior to set. Pass null to clear the behavior.
+ * @param initOptions The init options object to pass to the behavior to initialize it.
+ */
+public setBehavior<BehaviorT extends Behavior, InitOptionsT>(
+    behavior: { new(initOptions?: InitOptionsT): BehaviorT }, 
+    initOptions?: InitOptionsT
+): BehaviorT {
+    if (behavior) {
+        const newBehavior = new behavior(initOptions);
+
+        this.internal.behavior = newBehavior;
+        this.context.internal.setBehavior(this.id, this.internal.behavior.behaviorType);
+        return newBehavior;
+    }
+
+    this.internal.behavior = null;
+    this.context.internal.setBehavior(this.id, null);
+    return null;
+}
+```

design/tool.behavior.md

@@ -0,0 +1,81 @@
+#  Tool Behavior
+
+The tool behavior allows the user to assign this behavior to an actor so that the actor can be held and
+used.
+
+## Architecture
+
+Create a new behavior that is an `ToolBehavior` for being able to add to an actor, that when added will
+automatically enable grabbable on that actor.  This behavior will then allow the user to execute a primary 
+action to begin recording the transform changes over time.
+
+``` ts
+/**
+ * Tool behavior class containing the target behavior actions.
+ */
+export class ToolBehavior extends TargetBehavior {
+	private _holding: DiscreteAction = new DiscreteAction();
+	private _using: DiscreteAction = new DiscreteAction();
+
+	/** @inheritdoc */
+	public get behaviorType(): BehaviorType { return 'tool'; }
+
+	public get holding() { return this._holding; }
+	public get using() { return this._using; }
+
+	/**
+	 * Add a holding handler to be called when the given hover state is triggered.
+	 * @param holdingState The holding state to fire the handler on.
+	 * @param handler The handler to call when the holding state is triggered.
+	 * @return This tool behavior.
+	 */
+	public onHolding(holdingState: 'grabbed' | 'dropped', handler: ActionHandler): this {
+		const actionState: ActionState = (holdingState === 'grabbed') ? 'started' : 'stopped';
+		this._holding.on(actionState, handler);
+		return this;
+	}
+
+	/**
+	 * Add a using handler to be called when the given hover state is triggered.
+	 * @param usingState The using state to fire the handler on.
+	 * @param handler The handler to call when the using state is triggered.
+	 * @return This tool behavior.
+	 */
+	public onUsing(usingState: 'started' | 'stopped', handler: ActionHandler): this {
+		const actionState: ActionState = (hoverState === 'started') ? 'started' : 'stopped';
+		this._using.on(actionState, handler);
+		return this;
+	}
+}
+```
+
+## Network
+
+The network messages will use the behavior payloads of SetBehavior and PerformAction payloads the same way all
+behaviors do.  In addition, there is an additional message that would come in the form of the path of the tool
+while it is being used.  This will be sent with a higher fidelity than the normal 10 hz transform updates, and 
+will be in the form of a collection of transform recordings.
+
+The message payload for the using path updates could look like the following:
+
+``` ts
+export type UsingToolPath = Payload & {
+	type: 'using-tool-path';
+	actorId: toolActorId;
+	path: Transform[];
+}
+```
+
+## Sync layer considerations
+
+Synchronization will follow the same procedure as all other behaviors.  Nothing special about this behavior.
+The additional sync concerns will center around the path data being sent to the MRE app from the tool behavior
+when it is being used.  This path data only needs to be sent to the MRE app and not the other clients, as the
+MRE app will be responsible for doing what it wants with the path, or behavior that extends `ToolBehavior` may 
+have its own work it does with this path.
+
+## Client implementation
+
+Client implementation is similar to all other behaviors in that a handler will be created for this behavior, as 
+well as an interface for the client app to implement the actual behavior with.  Additionally, a path recording 
+system will need to be developed to generate the path recording while the tool is in use.

packages/functional-tests/src/tests/physics-bounce-test.ts

@@ -17,11 +17,16 @@ export default class PhysicsBounceTest extends Test {
 	public async run(root: MRE.Actor): Promise<boolean> {
 		this.assets = new MRE.AssetContainer(this.app.context);
 
-		this.materials.push(this.assets.createMaterial('mat1', { color: MRE.Color3.FromHexString('#ff0000').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat2', { color: MRE.Color3.FromHexString('#ff7700').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat3', { color: MRE.Color3.FromHexString('#ffbd00').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat4', { color: MRE.Color3.FromHexString('#fcff00').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat5', { color: MRE.Color3.FromHexString('#abf300').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat1', 
+			{ color: MRE.Color3.FromHexString('#ff0000').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat2', 
+			{ color: MRE.Color3.FromHexString('#ff7700').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat3', 
+			{ color: MRE.Color3.FromHexString('#ffbd00').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat4', 
+			{ color: MRE.Color3.FromHexString('#fcff00').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat5', 
+			{ color: MRE.Color3.FromHexString('#abf300').toColor4() }));
 
 		this.createBouncePlane(root, 2, 1.25);
 

packages/functional-tests/src/tests/physics-friction-test.ts

@@ -17,11 +17,16 @@ export default class PhysicsFrictionTest extends Test {
 	public async run(root: MRE.Actor): Promise<boolean> {
 		this.assets = new MRE.AssetContainer(this.app.context);
 
-		this.materials.push(this.assets.createMaterial('mat1', { color: MRE.Color3.FromHexString('#2b7881').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat2', { color: MRE.Color3.FromHexString('#11948b').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat3', { color: MRE.Color3.FromHexString('#664a72').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat4', { color: MRE.Color3.FromHexString('#89133d').toColor4() }));
-		this.materials.push(this.assets.createMaterial('mat5', { color: MRE.Color3.FromHexString('#c7518e').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat1', 
+			{ color: MRE.Color3.FromHexString('#2b7881').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat2', 
+			{ color: MRE.Color3.FromHexString('#11948b').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat3', 
+			{ color: MRE.Color3.FromHexString('#664a72').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat4', 
+			{ color: MRE.Color3.FromHexString('#89133d').toColor4() }));
+		this.materials.push(this.assets.createMaterial('mat5', 
+			{ color: MRE.Color3.FromHexString('#c7518e').toColor4() }));
 
 		this.createSlopePlane(root, 2, 1.25);