Schedule

Readings:

• Understanding Deep Learning, Chapter 1

Readings:

• Understanding Deep Learning, Chapter 2

Readings:

• Understanding Deep Learning, Chapter 3

Readings:

• Understanding Deep Learning, Chapter 4

Readings:

• Understanding Deep Learning, Chapter 5

Extra Readings:

• Mathematics for Machine Learning, Chapters 6 and 8, especially 8.1.3
• Maximum Likelihood Estimation Examples from an MIT course. This video walks through MLE examples for binomial and normal distributions, but does not pull out equivalent loss functions like we did in lecture.

Readings:

• Understanding Deep Learning, Chapter 5

Readings:

• Understanding Deep Learning, Chapter 6

Readings:

• Understanding Deep Learning, Chapters 7.1 - 7.4

Readings:

• Understanding Deep Learning, Chapter 7.5

Readings:

• Understanding Deep Learning, Chapter 8
• Double Descent Demystified

Readings:

• Understanding Deep Learning, Chapter 9
• Double Descent Demystified
• For Valid Generalization the Size of the Weights is More Important than the Size of the Network
• Train faster, generalize better: Stability of stochastic gradient descent

Readings:

• Understanding Deep Learning, Chapter 10

Readings:

• Understanding Deep Learning, Chapter 11

Readings:

• lecture slides

Readings:

• Understanding Deep Learning, Chapter 12, sections 12.1 - 12.4
• Optional: Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)
• Optional: The Illustrated Transformer

Readings:

• Understanding Deep Learning, Chapter 12, sections 12.5 - 12.9
• Optional The Illustrated Transformer

Readings:

• See slides for references

Readings:

• See slides for references

Readings:

• Intriguing Properties of Neural Networks
• Robustness and Generalization via Generative Adversarial Training
• Adversarial Examples are Not Bugs, They are Features
• Generative Adversarial Nets
• A Style-Based Generator Architecture for Generative Adversarial Networks
• Understanding Deep Learning, Chapters 15

Readings:

• Understanding Variational Autoencoders
• Understanding Deep Learning, Chapter 14, 17 (optional)

Readings:

• Rocca, Understanding Diffusion Probabilistic Models
• Understanding Deep Learning, Chapter 18

Works Cited:

• 2015: Sohl-Dickstein “Deep Unsupervised Learning using Nonequilibrium Thermodynamics”
• 2020 (DDPM): J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models”
• 2022 (DDIM): J. Song, C. Meng, and S. Ermon, “Denoising Diffusion Implicit Models” – more efficient
• 2022 (Stable Diffusion): R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models”
• 2023 (Diffusion Transformer): W. Peebles and S. Xie, “Scalable Diffusion Models with Transformers”

Readings:

• Rocca, Understanding Diffusion Probabilistic Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Understanding Deep Learning, Chapter 18

Works Cited:

• 2015: Sohl-Dickstein “Deep Unsupervised Learning using Nonequilibrium Thermodynamics”
• 2020 (DDPM): J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion Probabilistic Models”
• 2022 (Stable Diffusion): R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models”
• 2023 (Diffusion Transformer): W. Peebles and S. Xie, “Scalable Diffusion Models with Transformers”

Readings:

• Understanding Deep Learning, Chapter 13

Suggested Readings:

• TBD

Readings:

• TBD