Advanced Natural Language Processing (CS11-711)

Solutions of the course assignments.

Course objectives

• Describe fundamental tasks in natural language processing.
• Describe modern approaches to solve these tasks.
• Identify weaknesses and strengths of the proposed approaches.

Assignments

Assignment-01 (Minimal Neural Network Toolkit)

• Implementing the base Tensor class.
• Implementing differentiable operations for the Computational Graph.
• Implementing Momentum Training feature for the optimizer.
• Classification head and training loop provided in the skeleton code.

Assignment-02 (Minimal BERT Implementation)

The main implementations is related to the Transformer architecture.

• Implementing the self-attention head.
• Implementing the normalization layer.
• Implementing the multiheaded attention forward layer.

Final Project

This part is a replication of a scientific paper and will be provided in later work.

Credits

• Carnegie Mellon University (CMU), Language Technologies Institute (LTI)
• Course website
• Instructors
  • Graham Neubig (gneubig@cs.cmu.edu)
  • Robert Frederking (ref@cs.cmu.edu)