Meaning of Scan node's input_mapping and output_mapping parameters

[alexander-camuto]

Me again 😬.

I've been using Scan of late as it is particularly helpful for processing recursive models like LSTMs. However I've been struggling a bit to understand how to process the outputs of a Scan node.

Essentially the node is itself a (sub)-Graph embedded within a larger graph (the model). But say the Scan sub-graph has N explicitly defined Output nodes, the Scan node itself may have M outlets. Where M =/= N. My suspicion is that the information for going from the N output nodes to the M outlets is contained within the output_mapping field, but I don't have a good grasp as to

• whether this is true
• and if so how this mapping is achieved / encoded
  any help would be appreciated :)

[kali]

#[derive(Clone, new, Hash, Default)]
pub struct OutputMapping<F: Clone> {
    pub scan: Option<(usize, ScanInfo)>,
    pub full_dim_hint: Option<F>,
    pub last_value_slot: Option<usize>,
    pub state: bool,
}

output_amppings is a table that must exactly as many output_mapping as the body model has outputs. It is mapped 1-to-1.

• state: whether or not the output is a state that will be fed back to the body. outputs states and input states must be in the same order.
• scan: this means that the output for each turn of this output must be stacked according to ScanInfo and output to the Scan node outlet in the "usize" position. This is how you get every turn h value for your lstm.
• last_value_slot: copy the value for the last turn to the designated spot
• full_dim_hint: for optimisation, it is helpful to know the full length of the output tensor if available

[kali]

Yes, you are correct, except for the previous/current bit. Last means "output of the last iteration" and "full" means "output of all iterations stacked on the axis of the ScanInfo struct. So 11/0 may be something like [1;C] while 11/1 may be [T,C] where T is your time dimension.

[alexander-camuto]

gotcha thank you very much !

[alexander-camuto]

final questions to clarify.

• How then do the input_mappings relate to the outputs of the Scan node -- is this determined by Scan(ScanInfo { axis: 0, chunk: 1 }) in the snippet above ? and what do each of the different input_mappings types mean.
• Secondly where is information about T held within tract i.e how do we know how many iterations to perform on a forward pass for example.

[kali]

input_mappings are relatively independent from the output mapping. As for the output mappings, they input_mappings array matches the body inputs 1-to-1. They are also matching the Scan node input 1-to-1. (So the scan node and the body must have the same number of inputs, which is not the case from outputs as we've covered).

#[derive(Clone, new, Hash, Debug)]
pub enum InputMapping {
    Full,
    State,
    Scan(ScanInfo),
}

• Full: the body input receives the tensor received by the Scan node on the matching slot, unmodified
• State: the body receives the initializing value from the Scan node matching slot on the first loop, then the tensor outputted by the body for this state
• Scan: this is where the actual input usually goes: the Scan node will iterate over the input on the axis from the ScanInfo (in steps of "chunk" size) and feed this slice of the data at each iteration. This determines the number of loops.

The only point of contact between inputs and outputs is the states tensors: they are fed back into the network in the same order that they were outputed. When doing this match, we are only considering the State tagged mapping: first state-tagged output goes to first State input even if it's not the first input, second state-tagged output foes to second State input, etc.

[alexander-camuto]

amazing thank you