Add Flag in the routing protocol to skip further processing and return

[parthshah86]

We are trying to chain 3 GPU based models - model1 -> model2 -> model3 such that model2 and model3 are only run if the prediction from model1 is above a certain threshold.
To achieve this one suggestion is to extend the routing protocol to allow a new value which means to not route the call but return the current result.
https://seldondev.slack.com/archives/C8Y9A8G0Y/p1599082559154700

[ukclivecox]

Presently -1 means route to all children.
We could at -2 for abort and return current input?

[parthshah86]

@cliveseldon that can work, to make sure I understand. The router will get the output from model1 and check the score to decide whether to return -1 or -2. And if its -1 it will forward the output of model1 to model2

[ukclivecox]

Yes, a value X>=0 is to send to child X, no value or -1 means send output of model1 to all children.
The proposed value -2 would be to return the output of model1.
We should also extend to allow a list of N>=0 to send output of model1 to this subset of children.