Transformations available on 'ValueChoice'

[datapaintings]

I would like to search for multiinput neural networks with tuned hidden size of each mode, f. e.:

    self.hidden_size_1 = nn.ValueChoice([16, 32, 64, 128, 256, 512], label='hidden_size_1')
    self.hidden_size_2 = nn.ValueChoice([16, 32, 64, 128, 256, 512], label='hidden_size_2')
    self.output_layer = nn.Linear(self.hidden_size_1 + self.hidden_size_2, 2)

Currently, due to documentation, transformations on 'ValueChoice' are not supported, so concatenation of modes cannot be performed.

[ultmaster]

I think it is possible but requires quite a lot of implementation effort. So IMHO, we should put that in backlog.

@datapaintings, if you are interested, basically we have to trace the computational graph for value choice, and hold a pseudo value choice as parameter. On mutation, it somehow references to the original value choice and redo the computation.