Insights into passing previous actions as observation

[VineetTambe]

Hey Authors,

I was playing around with some params of CALM and was wondering if you have any insight into passing previous actions as observations to the policy.
Other work similar to CALM claim that passing previous actions and states as observations to the policy reduce vibrations and other higher order behaviors in the policy.

Could it be that the 64D latent representation of the reference motion passed has enough signal which when coupled with the obs history is sufficient for the network to learn the same things it would have with the previous action history.

What are your thoughts on this one?

Thanks!

[tesslerc]

Good question.
I'm not sure and it's certainly worth testing!

The question from an optimization standpoint is why does the model learn jittery behavior in the first place. My guess is that could be connected to representation/solution sampling issues.

My take on this -- If you aren't able to properly sample from the set of possible solutions, your model might collapse to the mean. So instead of sampling one of the smooth solutions, the model collapses to the mean which results in jittery behavior.
Methods such as CALM/ASE/PULSE/MaskedMimic use a latent representation to represent "which solution to focus on".