Alexander Rush and Yoon Kim, Harvard University
Time: Tues/Thurs 1:002:30pm
Location: Pierce 209
Date  Location  Topic  Materials 

Sep. 1, 1011am (Mark)  Pierce 301  Math Review (Linear Algebra, Calculus, Probabilistic Theory)  
Sep. 4, 56pm (Zhirui)  Pierce 301  Math Review (Linear Algebra, Calculus, Probabilistic Theory)  
Sep. 7, 56pm (Rachit)  Pierce 320  Code Review (Python, Numpy, Matplotlib, PyTorch)  
Sep. 8, 1111:59am (Rachit)  MD 223  Code Review (Python, Numpy, Matplotlib, PyTorch) 
The ideal outcome of this project would be a paper that could be submitted to a toptier natural language or machine learning conference such as ACL, EMNLP, NIPS, ICML, or UAI. There are different ways to approach this project, which are dis cussed in a more comprehensive document that is available on the course website. There are four separate components of the project.
You will upload these materials via Canvas. Please see the syllabus (linked in the course website) for a more thorough description of the final project and policies related to collaboration, etc.
Date  Due  Descriptions 

Feb. 20  Proposal  This is a twopage document that describes the problem you intend to solve, your approach to solving it and the experiments that you intend to run to evaluate your solution. 
March 8  Abstract and Status Report  This is a three to four page document that contains a draft of your final abstract, as well as a brief status report on the progress of your project. 
May 6  Talk Session  You will make a conferencestyle poster about your project, with a class poster session during reading week. SEAS will pay for the cost of producing the poster. You will also submit a PDF file of your poster. 
May 11  Final Report  You will write a report of up to ten pages, in the style of a mainstream CS conference paper. See the papers linked on the course website for examples 
Our syllabus this semester consists of two parts. The first part of the semester will be an accelerated background on applied deep learning for natural language processing with a series of Kaggle competitions. The second part of the semester will consist of student led paper presentations on the topic of deep probabilistic sequence modeling with latent variables.
Date  Area  Topic  Demos  Readings  Assignment (DUE: Fri 5pm of this week) 

Jan. 23  Sequence Classification  BagofWords 


Jan. 25  Convolutions 

Classification (Kaggle)  
Jan. 30  Sequence Modeling  NNLMs 


Feb. 1  RNNs 


Feb. 6  Sequence Transduction  Encoding 


Feb. 8  Attention 

Modeling (Kaggle)  
Feb. 13  Search 


Feb. 15  Variational Inference  Basics 


Feb. 20  VAEs 


Feb. 22  Other Latent Variable Techniques  GANs 

Attention (Kaggle)  
Feb. 27.  REINFORCE 


Mar. 1  Guest Lecture (Structured Training for NMT)  
Mar. 6.  Midterm  
Mar. 8  Latent Variable Models  Sequence VAEs 

VI (Kaggle)  
Mar. 20  Sampling Improvements  
Mar. 22  Structured VAEs 

Final Project Topics  
Mar. 27  Discrete Variables  REINFORCE for text  
Mar. 29  Discrete VAEs  Project Abstract Due  
Apr. 3  Dynamic Networks  Training with Search  
Apr. 5  Neural Module Networks  
Apr. 10  NMT  Recent advances in NMT  
Apr. 12  Unsupervised NMT  
Apr. 17  Topics  Latent Variable Models in Science  
Apr. 19  Latent Variable Models in Vision  
Apr. 24  Interpretation  Rationale Generation 