Hid Device Api

HID Device API

An API to allow mapping/scripting of HID devices in a similar manner to MIDI devices, hiding ugly details of HID protocol as much as possible.

Development of this API for Mixxx 1.11 is done by Hile in hidscripts branch. This branch will modify files only in the res/controllers/ directory and the files mostly are new scripts, not conflicting with existing files.

Summary and Rationale

When data is sent from a hid controller multiple controls are aggregated into packets so a common api is needed to allow the actual modified controls to be processed without every mapping needing to parse these packets themselves. Similarly when the state of a hid device needs to be updated (leds or other feedback) one or more updates are aggregated into a packet which contains the new state for all the controls within that packet regardless of whether each has changed or not. The common hid api should take care of this decoding/parsing/encoding.

Ideally the API (and future XML device definition file) should abstract the hid layer in such a way that a mapping written for a hid controller could also be used for a midi controller, possibly with no changes at all for a controller that can be used in both midi and hid modes).

For the HID device mappings for 1.11 each mapping has to implement it's own parsing/encoding of HID data. We would like to get some common code into the Javascript library that would later be ported to the C++ core. The concern is that if we add any functions now that mappers rely on we won't be able to change it later on, so even if the implementation is only in Javascript initially it needs to be well considered.

Components

Current proposed API contains three javascript prototype classes:

HIDController implements an abstract HID controller javascript class, which is expected to take care of all HID packet creation and parsing, including updating the field values in these packets.
HIDPacket is a container used to both receive and parse HID packets from devices and to specify the format for packets sent to the device.
HIDBitVector is initialized from HIDPacket automatically and should not be directly handled by user scripts. It's purpose is to combine bits in same HID packet field to group of independently parsed and updated bits. A new HIDBitVector is automatically created when a control or output is added to same HID packet offset.

Each controller script using the API must at least:

Include hid-common-packet-parser.js in the XML file before the actual script
Implement exactly one HIDController instance (instead of new Controller() in MIDI scripts)
Create and register one HIDPacket Input Packet entry named 'control' to the HIDController instance

Optionally, each HID controller script often needs to:

Define other input packets to receive other than controller status from the device
Define output packets to control LEDs or other items on the device, according to device specific details
Define output packets to change HID device state, for example disable mouse, initialize HID mode or request details from device
Add Scaling Functions to the HID controller values, which can be anything from single bit to 32bit integer numbers
Add Filtering Functions to the HID data. The HID controllers are often too high resolution and may send meaningless minor changes which we want to filter out
Register a Callback Function to the control registered in a HID packet

All these details have been implemented by the HID common packet parser classes.

Defining Controls

Signature for the function to add controls:

HIDPacket.prototype.addControl = function(group,name,offset,pack,bitmask,isEncoder) {};

A control identifies the bits/bytes that relate to a specific element on the controller, whether incoming changes from controller or outgoing data to be sent to the controller.

If a control were to be defined from within a hid_mapping_format xml file it should be possible to bind this control to a custom script function via something like a scriptfunction="myprefix.myfunction" attribute or use the midi xml format method of key optionally representing the function name. This is not yet supported in Mixxx 1.11, all controls must be mapped in javascript for now.

There are four types of controls supported by HIDPacket:

Type

button: a binary on/off toggle from device. While some buttons may have multiple states (bitmask not bit), this is not yet supported and the fiedl must be manually processed from a byte.
fader: a numeric value, size 1-4 bytes, from the device. Usually these control mixxx internal values like knobs, jogs or faders. Unlike MIDI, HID does not specify range of such controls and the values are often far off from mixxx ranges. A scaling function can be automatically applied to move the value to expected range (for example, fader from unsigned short value to -1..1 range of mixxx).
encoder: the physical control is an encoder that sends out continous data like a fader, but resets to zero once reaching it's maximum value (and conversely when moved in reverse). An encoder is processed and added exactly like a fader, only difference is setting of isEncoder variable to true. This will change the field parsing code to set the field 'delta' attribute to -1 or +1 offsets, and wrapping over minimum and maximum values automatically. The minimum and maximum values are deduced from the field size, for example a 'byte' encoder wraps from 255 to 0 and from 0 to 255, giving delta -1 and 1 respectively.
LED/Output a bit or byte to be sent back to the controller, most commonly to control status LEDs. Current implementation does not support sending short or integer size values, and sending of bitmask of bits is possible but not yet supported by the API. HIDController takes care of updating only modified parts of the outgoing packet. Right now API is missing a function to set other types of data except LED, but this can be trivially added when we know what is exactly needed.

Packet Header

On many devices, each HID packet has a prefix in the packet to recognize which packet this is: for example, EKS Otus 'control' packet first two bytes are 0x0 and 0x35 (id and length). The packet header is given as an array when creating new packet. If there is no header, pass empty array to the variable.

Offset

The byte offset of the field within the packet, starting from 0.

Packing

Instead of telling the packet field length, HIDPacket parses the fields based on pack attribute. The packing tells the size and numeric range for each field, and allows us to convert the input number to and from exactly correct value. Valid 'pack' values are b (signed byte), B (unsigned byte), h (signed short), H (unsigned short), i (unsigned int) and I (signed int). A field containing bits still needs to be given valid 'pack' value, to calculate bit vector masks and check boundaries.

Bitmask

Bitmask defines the bit to flip for bit controls in the packet: internally, a HIDBitVector packet field is created when bitmasks are seen. If multiple controls with same offset and packing are defined, they are mapped transparently as fields of same HIDBitVector field, which can be updated or read in one go.

Bitmask can be used when defining both button bit inputs and LED output bits.

In current implementation, fader and encoder controls must have bitmask attribute set to 0: we don't support controls like 'high four bits of this byte'. It is not possible to have both fader/encoder numeric fields and bits in same input packet field.

Group

Used in the same way as for a midi mapping - the default group that this control will be bound to.

For automatic deck assignment to dynamic controllers, special names 'deck', 'deck1' and 'deck2' can be used to assign to even/odd decks, or to currently selected deck. All internal controls and connections are updated automatically, when deck status is changed and such group names are used. You can also register some controls with virtual deck mappings and some fields hard coded to specific deck.

For input and output controls not to mapped anywhere automatically, we suggest using using group name 'hid'. All data from and to these fields must be manually scripted or assigned in custom script functions. One example of such variable could be 'deck chooser' control in EKS Otus, or 'modifier fields.

Name

A name for the control. If the control is mapped without a script function directly to mixxx, this name must be valid control name in mixxx for the group where the control is attacked to.

<del>=== Min Value ===

Lowest value of these bytes when the control is at its minimum setting. 0 is the default.

Max Value

The highest value of these bits/bytes when the control is at its maximum setting. 0xFFFF is the default for two bytes </del>

NOTE Minimum and maximum values are calculated automatically for a field from the 'pack' attribute.

Current control defination format is following:

packet.addControl('[Master]','crossfader',34,'H');
packet.addControl('[Playlist]','SelectTrackKnob',15,'B',undefined,true);

Parsing Incoming Controls

HIDController implements parsing of incoming packet named 'control' automatically. Unfortunately due to namespace issues in current qtscript implementation we use, you still need to implement an incomingData function in your script. The standard wrapper to your script is following:

MyDevice.incomingData = function(data,length) {
  MyDevice.parsePacket(data,length);
}

The default parsePacket function has following side effects:

* Only modified input field values are processed by any callbacks or automated calls to engine * If a field has registered callback function, this function is called without running scaling functions or resolving correct decks. All other processing for fields with a custom callback is ignored. * If field has no callback, it's scaling function is called (if defined), virtual deck mapping to actual deck is performed and the assigned control group and name are attempted to be update directly in mix with scaled value. If the field's group name does not match a known mixxx control group or virtual deck, this is not done and the field input is ignored. * Finally, if a function called processDelta is defined, it is called with all modified fields in the packet. This is done after automated fields and callbacks already have been called, so don't handle same field twice!

Binding Actions to Incoming Controls

An API function should exist to bind a control to either an internal function (like binding a midi control to a mixxx engine action from a midi xml mapping file) or to a script function either by name of reference. Passing by function reference would allow inline small code snippets to be bound to the changes.

It should be possible to bind multiple functions to a single control

It should be possible to define what types of value changes should call the bound function such as "when non zero" (useful for mapping actions to button presses instead of presses and releases), "when zero" or "all"

The predefined group (if any) and name of the changed control should be passed to the function. This would allow mappings such as:

// bind directly to an engine action

addEngineBinding("[Channel1]", "play", PRESS, "play");

// bind to a custom function

addScriptBinding("[Channel1]", "play", PRESS, "myprefix.myfunction");

// bind to an anonymous function

addScriptBinding("[Channel1]", "play", ALL, function(group, name, value) {
    if (value > 0) {
      engine.setValue(group, name, !engine.getValue(group, name));
    }
    else {
        // do something different when the button is released
    }
});

These functions should also be called by the future hid device mapping parser.

Sending Data to the HID Device

Since the API will be keeping a cache of the current values for all controls within all packets it should be possible to create a function that can be used like the engine.sendShortMsg()

The API function could be called in one of two ways:

Control Names

If a named control has been defined via a addControlField() call (from the XML parser or otherwise) for the LED in the same way as an incoming control then only this name and new value would need to be specified such as:

sendNamedMsg("[Channel1]", "play", 1);

The API would map the group and name back to the 'hid packet level' byte offset+bitmask, apply this to the cache of that packet by applying a bitwise AND with the inverse of the controls bitmask and then bitwise OR the controls new value and then send out the complete packet.

Control Definition

Some of the same parameters used to define a control could be passed in explicitly such as offset, bitmask value or offset, array of values and length (which again would result in the full packet being sent out even though the functions are only passed a partial delta):

sendShortMsg(offset, bitmask, value);
sendShortMsg(2, 0x3, 0x1);

partial_packet = [0x1, 0x0, 0x4, 0x6, 0x7, 0x8];
byte_offset = 10;
sendLongMsg(byte_offset, partial_packet, partial_packet.length);