Will Kearns, Aakash Sur, BHI PhD Students and Trevor Cohen, BHI Faculty
Tuesdays, Autumn Quarter: 11:30 am-12:20 pm, Health Sciences Building, T478
Through this course, students will be introduced to reinforcement learning methods and how to apply them to train health dialog systems to address specific problems in healthcare. We will cover a range of machine learning methods including tree search, tree pruning, Markov decision processes, and Q-learning. We will explore both classical methods and recent advances in the development of dialog system components including natural language understanding, dialog management, and natural language generation. The course structure will be a mixture of lectures and interactive coding sessions culminating in the deployment of a health dialog system.
We welcome questions during the class as others might share the same questions. If you need individual help, please see one of the instructors after class or send a question to the group on slack.
Designing Voice User Interfaces: Principles of Conversational Experiences
Introduction to Conversational Agents and Reinforcement Learning
We will introduce conversational agents operating within a natural language environment, an ideal context for employing reinforcement learning (RL). We will survey the methods of RL and its applications of NLP. Finally, we will cover the software requirements for the class, and ensure students can interactively follow along in coding exercises.
None
Natural Language Understanding
We will explore how we can train agents to understand their conversational environments.
ONENET:Joint Domain, Intent, Slot Prediction for Spoken Language Understanding
Dynamic Integration of Background Knowledge in Neural NLU Systems
Train NLU model: with a Pipeline in Rasa
Tree Search
We will model decisions as trees and learn to efficiently search them using classic algorithms such as breadth-first search and depth-first search. In addition, we will introduce heuristic based searches, including A* search.
Joint A* CCG Parsing and Semantic Role Labelling
Advanced Tree Searches
We will cover how to model two player games as trees, and how the optimal strategy can be recovered from these trees. In addition, we will cover how to prune these trees to limit the total search space using alpha-beta pruning, and heuristic pruning.
None
In Class Exercise with Grundy's Game of Nim
Dialog Management
Health dialog systems for patients and consumers
Train Rasa DM with Interactive Learning
Markov Decision Processes
In this class, we will extend our tree based decision models to graphs with Markov models. We will learn how to calculate the best route through a Markov decision process (MDPs) using the Bellman equations. Finally, we will extend these ideas to conversational agents using partially observable Markov decision processes (POMDPs).
POMDP-Based StatisticalSpoken Dialog Systems:A Review
Training a real-world POMDP-based Dialogue System
None
Q-Learning
Here we will introduce one of the key concepts in RL, Q-learning. This approach overcomes the limitations of MDPs and allows us to conduct on-line or off-line learning without complete information.
TBD
TBD
Deep Q-Networks
Moving past basic tabular Q-learning, we will cover current approaches which revolve around Deep Q-Networks (DQN). We will cover popular examples of DQNs used to master video games, and conversations. Finally, we will cover how to efficiently train these models using experience replay.
Playing Atari with Deep Reinforcement Learning
Agenda-based user simulation for bootstrapping a POMDP dialogue system
A User Simulator for Task-Completion Dialogues
Train dialog policy w/ episodic replay
Train a DQN using an agenda based User Simulator
Advanced Neural Methods for Dialog Systems
We will cover advanced neural architectures for training dialog systems, e.g. A2C and MemNN models.
ConveRT: Efficient and Accurate Conversational Representations from Transformers
The Illustrated Transformer (Transformer Tutorial)
TBD
Ethics and NLG
We will finish off the course with a discussion of ethics in the development of dialog systems using two case studies and then focus the discussion on health dialog systems in particular.
The Design and Implementation of XiaoIce, an Empathetic Social Chatbot
Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day
TBD