Discussions

You can download the discussions here. We will try to upload discussion material prior to their corresponding sessions. Future discussions listed below likely have broken links.

• 

  Discussion_01 - Environment Setup and an Intro to Pytorch, Tensors, and Tensor Operations
  tl;dr: Setting up SCC, virtual environments, and an intro to Pytorch, Tensors, and Tensor Operations. Will also go over on how to run the dl4ds_tutor on SCC.
  [slides]
  

  Notebook: 00_fundamentals.ipynb
  Suggested Readings:

  • SCC Resources
  • Introduction to Pytorch
  • What’s a Tensor? - Dan Fleisch
  • Pytorch Internals - Edward Z. Yang
• 

  Discussion_02 - Autograd, and Computational Graphs in Pytorch. Intro to Model Building in Pytorch.
  tl;dr: Fundamentals of Autograd and Computational Graphs in Pytorch. Introduction to Defining a Neural Network in Pytorch (Basics)
  
  

  Notebook: 01_autograd.ipynb
  Suggested Readings:

  • Fundamentals of Autograd
  • Overview of PyTorch Autograd Engine
  • Pytorch Internals - Edward Z. Yang
  • Builders’ guide (d2l.ai)
• 

  Discussion_03 - Intro to Model Training in Pytorch. (Image Classification, Text Classifcation)
  tl;dr: Model Training Code Walkthrough. Intro on how to handle Text for Deep Learning Networks.
  
  

  Notebook: 02_intro_nn_training.ipynb

• 

  Discussion_04 - Deep Dive 1: How to read, load, and process data. (Examples on Object Detection, Deep Learning on Tabular Data). Using GPUs on SCC.
  tl;dr: Closer look into how to handle data in Pytorch. With examples on how to create and use custom datasets for Object Detection and Tabular Data. Using and monitoring GPUs on SCC to speed up your models.
  
  

  Github Link: disc4
  Google Drive: disc4

• 

  Discussion_05 - Deep Dive 2: Deep Learning Modules in Pytorch (CNN, RNN/LSTM, Transformer)
  tl;dr: Fine-tuning Pre-trained Models, Text Generation (LSTM), Text Summarization (Custom Transformer).
  
  

  Github Link: disc5

• 

  Discussion_06 - Deep Dive 3: Logging, Model Checkpointing, Tracking Experiments, etc. Hyperparameter Tuning/Search (Optuna). Walkthrough of the VizWiz (Midterm Project) codebase.
  tl;dr: Creating a robust training pipeline. How to log, checkpoint, and track experiments. Hyperparameter Tuning/Search using Optuna. Walkthrough of the VizWiz (Midterm Project) codebase.
  
  

  Github Link: disc6

• 

  Discussion_07 - Midterm Project Lab Session
  tl;dr: Midterm Project Lab Session for the VizWiz Captioning Challenge.
  
  
• 

  Discussion_08 - Huggingface, ViT, CLIP, Kosmos2
  tl;dr: Huggingface Transformers, Vision Transformers (ViT), CLIP, Kosmos2.
  
  

  Github Link: disc8

• 

  Discussion_09 - VAEs, and GANs
  tl;dr: Code examples for VAEs, and GANs.
  
  

  Github Link: disc9