The following library provides a framework for learning the parameters of a Joint-space Task-directed Dynamical System. The training component of the algorithm is primary written in MATLAB. The parameters of the model can then be exported, and imported to the C++ code to run the learned JT-DS on a real robot.
The JT-DS learning framework can be used to extract the underlying behaviors from a series of demonstrated motions, and to encode those motions in a dynamical system. In particular, the demonstrations are joint-space position trajectories for a manipulator moving towards a target task-space position. These demonstrations are converted to a series of locally-exhibited behaviors "explaining" the joint motions in exhibited in each demonstration.
To run the JT-DS learning code, you'll need to have MATLAB installed. (Any version after 2016a will certainly work, though most older versions should too). The code also relies on YALMIP, a semidefinite optimization framework, and Peter Corke's robotics-toolbox framework though you don't need to worry about these - versions of YALMIP and robotics-toolbox are included in the "include/" folder.
Make sure that include/ and src/ are on your MATLAB path. That's it!
For a brief walkthrough on how to utilize the JTDS-Learning library to train your own model, see quickstart.md.
To see a thoroughly documented example of the system at work on a toy problem, check out demo_footstep.m. To see an implementation of the learning algorithm for the KUKA LWR 4+ robot arm, check out demo_kuka.m (note that this requires you to also have rtk-JTDS installed). To see some nice side-by-side comparisons of different A matrices and their effects on motion, check out demo_varying_A.m.
To run the trained model on a real robot, it is recommended that you use the C++ JT-DS library. The parameter file that you export from the learning process can be fed directly into that library, and used to control your robot using your learned behaviors. For an example of using that library to control a KUKA LWR 4+, see rtk_JT_DS.
When creating your own training demonstrations, it's important to understand the fundamental assumptions of the system. Specifically:
- The behaviors are always learned with respect to the final task-position (it's assumed to be the target) so make sure to end your demonstration where you hope the target to be.
- The JT-DS controller, which by default sets P = 1, is always nondecreasing in task-space distance to the target task position. This means that, e.g. if your goal is in front of you and you want your arm to first move backwards and then forwards to the target, no JT-DS can ever encode this. Instead, break this motion up into two behaviors: one where the target is backwards, and the other (beginning from the backwards-position) where the target is the original forwards target.
- Similar joint positions lead to similar joint velocities, so don't give two demonstrations that have the same final position but behave very differently at a shared earlier joint position.
The largest bottleneck in terms of computation time for JTDS_Solver.m is setting up a YALMIP SDP which includes all the GMM components for every datapoint. The optimization, run on a 3.4GHz i7 PC, takes around 3 minutes for ~1000 points and 3 GMM components, and around 15m for ~10000 points and 5 GMM components.
When training a JT-DS, the most important subtask is that the GMM is learned correctly, as this will guarantee a good segmentation of local behaviors. There are a number of options you can specify in the options struct passed into JTDS_learning.m to guide how the GMM will be trained:
options.GMM_sigma_type- Specifies whether the variance matrix is diagonal or full. Full variance matrices generally lead to better fitted Gaussians.options.maximize_BIC- If true, varies the number of GMM components and selects the best one. This should be used if you're not sure how many sub-behaviors your motion has. **options.max_gaussians- The maximum number of sub-behaviors. The higher this number is, the more likely the GMM will overfit and erroneously encode too many small regions as behaviors. **options.BIC_regularization- The Bayesian Information Criterion score is used to select between GMMs to determine the best-fitted one. By regularizing the BIC to penalize higher numbers of Gaussians, we push the system towards a more compact GMM distribution. The precise regularization equation is available in the option description.options.fixed_num_gaussians- If you already know the exact number of behaviors you believe your system should encode, you can directly pre-specify it. In this case, remember to setoptions.maximize_BICto false!
When using computeFullTrajectory.m, the system may end up being poorly behaved and integrate in an unstable manner. You may want to switch the ODE solver used by MATLAB (between ode45, ode15s, or others) if you find yourself getting NaN values as the output of the integration.
The library is based on the paper "Learning Augmented Joint-Space Task-Oriented Dynamical Systems: A Linear Parameter Varying and Synergetic Control Approach", by Yonadav Shavit, Nadia Figueroa, Seyed Sina Mirrazavi Salehian and Aude Billard. For further assistance with the learning, contact yonadav at mit dot edu. or {Nadia dot Figueroa, Sina dot Mirrazavi} at epfl dot ch We hope you and your robot enjoy learning together!