Schedule

Readings:

• Understanding Deep Learning, Chapter 1

Readings:

• Understanding Deep Learning, Chapter 2

Readings:

• Understanding Deep Learning, Chapter 5
• Mathematics for Machine Learning, Chapters 6 and 8, especially 8.1.3
• Maximum Likelihood Estimation Examples from an MIT course. This video walks through MLE examples for binomial and normal distributions, but does not pull out equivalent loss functions like we did in lecture.

Readings:

• Understanding Deep Learning, Chapter 3
• Mathematics for Machine Learning, Chapter 7

Readings:

• Understanding Deep Learning, Chapter 3

Readings:

• Shared Compute Cluster
• SCC OnDemand Interface

Readings:

• Understanding Deep Learning, Chapter 4

Readings:

• Understanding Deep Learning, Chapter 6

Readings:

• Understanding Deep Learning, Chapters 7.1 - 7.4

Readings:

• Understanding Deep Learning, Chapter 7.5

Readings:

• Understanding Deep Learning, Chapter 8
• Double Descent Demystified

Readings:

• Understanding Deep Learning, Chapter 9
• Double Descent Demystified
• For Valid Generalization the Size of the Weights is More Important than the Size of the Network
• Train faster, generalize better: Stability of stochastic gradient descent

Readings:

• Understanding Deep Learning, Chapter 10

Readings:

• Understanding Deep Learning, Chapter 11

Readings:

• lecture slides

Readings:

• Understanding Deep Learning, Chapter 12
• Optional: Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)
• Optional: The Illustrated Transformer

Readings:

• Understanding Deep Learning, Chapter 12
• Optional The Illustrated Transformer

Readings:

• See slides for references

Readings:

• See slides for references

Readings:

• Intriguing Properties of Neural Networks
• Robustness and Generalization via Generative Adversarial Training
• Adversarial Examples are Not Bugs, They are Features
• Generative Adversarial Nets
• A Style-Based Generator Architecture for Generative Adversarial Networks
• Understanding Deep Learning, Chapters 15

Readings:

• Understanding Variational Autoencoders
• Understanding Deep Learning, Chapter 14, 17 (optional)

Readings:

• Rocca, Understanding Diffusion Probabilistic Models
• Understanding Deep Learning, Chapter 18

Readings:

• Rocca, Understanding Diffusion Probabilistic Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Understanding Deep Learning, Chapter 18

Readings:

• LoRA: Low-Rank Adaptation of Large Language Models
• Adding Conditional Control to Text-to-Image Diffusion Models Examples:
• ControlNet example notebook

Readings:

• Understanding deep learning (still) requires rethinking generalization (access through BU)
• Leveraging Random Label Memorization for Unsupervised Pre-Training
• What Do Neural Networks Learn When Trained With Random Labels?

Readings:

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• Large Language Models are Zero-Shot Reasoners
• Learning to Reason with LLMs
• OpenAI Harmony Response Format
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
• DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning
• AlphaGeometry: An Olympiad-level AI system for geometry
• How Does A Blind Model See The Earth?
• Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
• V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning

Extra Readings:

• s1: Simple test-time scaling
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• Language Models, World Models, and Human Model-Building
• LLMs and World Models

Readings:

• Understanding Deep Learning, Chapter 13

Readings:

• Understanding Deep Learning, Chapter 19
• Data Center Cooling using Model-Predictive Control
• Human-level control through deep reinforcement learning (access via BU)
• Mastering the game of Go without human knowledge (access via BU)
• Gran Turismo Sophie
• Craig Sherstan Keynote at Computers and Games 2024, re: Gran Turismo Sophy
• Aligning language models to follow instructions
• Training language models to follow instructions with human feedback
• RLHF: Reinforcement Learning from Human Feedback