Lectures

You can download the lectures here. We will try to upload lectures prior to their corresponding classes. Future lectures listed below likely have broken links.

• 

  Intro to Deep Learning and Course Logistics
  tl;dr: We will introduce the topic of deep learning, a bit about it's history, and what impact it has had. Then we'll go over the homeworks, exams, and other course logistics.
  [slides]
  

  Readings:

  • “Deep Learning” by Yann LeCun, Yoshua Bengio and Geoffrey Hinton (use BU institutional login to access)
  • Understanding Deep Learning, Chapter 1
• 

  Supervised Learning
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 2
• 

  Shallow Neural Networks
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 3
• 

  Deep Neural Networks
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 4
• 

  Loss Functions
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 5
• 

  Fitting Models
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 6
• 

  Gradients
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 7
• 

  Gradients, Initialization, Measuring Performance
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 8
• 

  Regularization
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 9
  • For Valid Generalization the Size of the Weights is More Important than the Size of the Network
  • Train faster, generalize better: Stability of stochastic gradient descent
• 

  Convolutional Networks
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 10
• 

  Residual Networks
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 11
• 

  Midterm review
  [slides]
  

  Readings:

  • n/a
• 

  Midterm starts
  
  

  Please bring a laptop that you can work on.

• 

  Recurrent Neural Networks
  [slides]
  

  Readings:

  • The Unreasonable Effectiveness of Recurrent Neural Networks
• 

  Transformers
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 12
  • Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)
  • The Illustrated Transformer
• 

  Transformer Details
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 12
  • The Illustrated Transformer
  • Andrej Karpathy lecture notes re: LLM tokenization
• 

  Vision Transformers
  [slides]
  

  Readings:

  • Image GPT
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Scaling Vision Transformers
  • Learning Transferable Visual Models From Natural Language Supervision
  • Masked Autoencoders are Scalable Vision Learners (tutorial)
  • Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
  • Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
• 

  Contrastive Learning
  [slides]
  

  Readings:

  • Signature Verification using a “Siamese” Time Delay Neural Network
  • Learning a similarity metric discriminatively, with application to face verification
  • Learning Transferable Visual Models From Natural Language Supervision
• 

  Explanations
  [slides]
  

  Readings:

  • Many links in the slides
• 

  Adversarial Inputs and Generative Adversarial Networks
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 15
  • Many paper links in the slides
• 

  Unsupervised Learning and Variational Autoencoders
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapters 14+17
• 

  Normalizing Flows
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 16
• 

  Diffusion Models
  [slides]
  

  Readings:

  • Understanding Deep Learning, Chapter 18
  • What are Diffusion Models?
• 

  Neural Fields
  [slides]
  

  Readings:

  • Neural Fields in Visual Computing and Beyond
  • Many links in the slides
• 

  Reinforcement Learning
  
  

  Readings:

  • Understanding Deep Learning, Chapter 19
  • RLHF: Reinforcement Learning from Human Feedback
• 

  Class Choice / Recent Developments
  
  

  Readings:

  • TBD
• 

  Final Project Presentations
  
  

  Students will present their final projects.