CS287: Machine Learning for Natural Language

Alexander Rush and Yoon Kim, Harvard University

Time: Tues/Thurs 1:00-2:30pm

Location: Pierce 209

Announcements

  • CS 287 will be capped at 30 students this semester. If you are interested in taking the course, please come to our first lecture and fill out the course application
  • Course Info

    Forum and Announcements
    Section Times
    • Friday TBD
    Syllabus and Collaboration Policy
    Instructor
    • Alexander "Sasha" Rush
      OH: Wed 2:30-4, MD 217
    • Email: Piazza preferred or srush at seas.harvard.edu
    Teaching Assistants
    Office Hours
    • Tuesday 2:30-4pm: MD 217 (Sasha)
    • Wednesday 7-9pm: MD 2nd Floor (Yoon)
    Grading
    • Assignments (20%)
    • Scribing (5%)
    • Presentation (10%)
    • Midterm Exam (15%)
    • Final Project (50%)
    Links

    Time and Location

    • Thursday 5-6pm: Pierce Hall 320
    • Friday 11-11:59am: MD 223

    Date Location Topic Materials
    Sep. 1, 10-11am (Mark) Pierce 301 Math Review (Linear Algebra, Calculus, Probabilistic Theory)
    Sep. 4, 5-6pm (Zhirui) Pierce 301 Math Review (Linear Algebra, Calculus, Probabilistic Theory)
    Sep. 7, 5-6pm (Rachit) Pierce 320 Code Review (Python, Numpy, Matplotlib, PyTorch)
    Sep. 8, 11-11:59am (Rachit) MD 223 Code Review (Python, Numpy, Matplotlib, PyTorch)

    The ideal outcome of this project would be a paper that could be submitted to a top-tier natural language or machine learning conference such as ACL, EMNLP, NIPS, ICML, or UAI. There are different ways to approach this project, which are dis- cussed in a more comprehensive document that is available on the course website. There are four separate components of the project.

    You will upload these materials via Canvas. Please see the syllabus (linked in the course website) for a more thorough description of the final project and policies related to collaboration, etc.


    Important Dates

    Date Due Descriptions
    Feb. 20 Proposal This is a two-page document that describes the problem you intend to solve, your approach to solving it and the experiments that you intend to run to evaluate your solution.
    March 8 Abstract and Status Report This is a three to four page document that contains a draft of your final abstract, as well as a brief status report on the progress of your project.
    May 6 Talk Session You will make a conference-style poster about your project, with a class poster session during reading week. SEAS will pay for the cost of producing the poster. You will also submit a PDF file of your poster.
    May 11 Final Report You will write a report of up to ten pages, in the style of a mainstream CS conference paper. See the papers linked on the course website for examples

    Our syllabus this semester consists of two parts. The first part of the semester will be an accelerated background on applied deep learning for natural language processing with a series of Kaggle competitions. The second part of the semester will consist of student led paper presentations on the topic of deep probabilistic sequence modeling with latent variables.

    Date Area TopicDemos ReadingsAssignment (DUE: Fri 5pm of this week)
    Jan. 23 Sequence Classification Bag-of-Words
    Jan. 25 Convolutions Classification (Kaggle)
    Jan. 30 Sequence Modeling NNLMs
    Feb. 1 RNNs
    Feb. 6 Sequence Transduction Encoding
    Feb. 8 Attention Modeling (Kaggle)
    Feb. 13 Search
    Feb. 15 Variational Inference Basics
    Feb. 20 VAEs
    Feb. 22 Other Latent Variable Techniques GANs Attention (Kaggle)
    Feb. 27. REINFORCE
    Mar. 1 Guest Lecture (Structured Training for NMT)
    Mar. 6. Midterm
    Mar. 8 Latent Variable Models Sequence VAEs VI (Kaggle)
    Mar. 20 Sampling Improvements
    Mar. 22 Structured VAEs Final Project Topics
    Mar. 27 Discrete Variables REINFORCE for text
    Mar. 29 Discrete VAEs Project Abstract Due
    Apr. 3 Dynamic Networks Training with Search
    Apr. 5 Neural Module Networks
    Apr. 10 NMT Recent advances in NMT
    Apr. 12 Unsupervised NMT
    Apr. 17 Topics Latent Variable Models in Science
    Apr. 19 Latent Variable Models in Vision
    Apr. 24 Interpretation Rationale Generation

    Citations