Calendar
Week 1
- Jan 23
-
- Lecture 1Advanced ML Introduction 1
- Saining Xie
Large-scale Machine Learning – Neural Network Architectures: Past, Present and Paths Forward
-
- Lecture 1Advanced ML Introduction 2
- Saining Xie
Large-scale Machine Learning – Training objectives: Supervised learning, Self-Supervised Learning, Generative Models and Beyond
-
Reading materials
- The Illustrated Transformer
- Attention is All You Need (Optional, but recommended!)
- Jan 30
- Early Assignment DueDue at 4:00 PM ET
Week 2
- Jan 30
-
- Lecture 2Machine Learning Fundementals
- Saining Xie
- Empirical Risk Minimization, Constrained ERM, Hypothesis Spaces, Excess Risk Decomposition
- Errors - Approximation and Estimation Errors
- Supervised Learning
- Optimizers: Gradient Descent, Stochastic Gradient Descent
- Loss Functions
-
Reading materials
Week 3
- Feb 6
-
- Lecture 3Introduction to Deep Learning
- Saining Xie
- Neural Networks
- Backpropagation
-
Reading materials
Week 4
- Feb 13
-
- Lecture 4Training Deep Neural Networks
- Saining Xie (Zoom due to winter storm)
- Optimization
- Initialization
- Regularization
- Normalization
Transformer Deep Dive 1(next time!)-
Reading materials
- Mar 05
- Assignment 1 DueDue at 4:00 PM ET
Week 5
- Feb 20
-
- Lecture 5Transformer Deep Dive
- Saining Xie
- Attention Layer
- Sequential Models
- Transformer Architecture
- Vision Transformers
-
Reading materials
-
- *Lecture *5Diffusion Transformers (DiT)
- Saining Xie
- Diffusion models
- SORA
-
Reading materials
Week 6
- Feb 27
-
- Lecture 6AOD in Deep Learning (Recap & Case Studies)
- Saining Xie
- Case studies on recent advances in neural architectures, self-supervised learning objectives, and multimodal data preparation
-
Reading materials
Week 7
- Mar 5
-
- Lecture 7Generative Models
- Saining Xie
- GANs, Variational Autoencoders, VQVAE, VQGAN
-
Reading materials
Week 8
- Mar 12
-
- Lecture 8Generative Models - 2
- Saining Xie
- KL-divergence, VAE, VQVAE, VQGAN, GAN, CycleGAN
-
Reading materials
- Apr 2
- Assignment 2 DueDue at 4:00 PM ET
Week 9
- Mar 26
-
- Lecture 9Visualizing, Understanding Neural Networks & Deep Learning on Videos
- Saining Xie
- PCA, t-SNE, Grad-CAM
-
Reading materials
Week 10
- Apr 2
-
- Lecture 10Temporal Data Processing and Reinforcement Learning
- Saining Xie
- Video Classification, Recurrent ConvNet, Spatio-temporal Self-Attention, ViViT
- MDPs, Q Learning, Policy Gradient methods, Actor-Critic Methods, Deep Q-Learning
-
Reading materials
Week 11
- Apr 9
-
- Lecture 11Reinforcement Learning and Alignment
- Saining Xie
- MDPs, Q Learning, Policy Gradient methods, Actor-Critic Methods, Deep Q-Learning, Instruction Fine-tuning, RLHF
-
Reading materials
- Apr 23
- Assignment 3 DueDue at 4:00 PM ET
Week 12
- Apr 16
-
- Guest Lecture 1Towards Flexible, Scalable, and Knowledgeable Generative Intelligence
- Jiatao Gu
- Bio: Jiatao Gu is a staff research scientist at Apple Machine Learning Research (MLR). Prior to Apple, Jiatao was a senior research scientist at Facebook AI Research (FAIR). He received his Ph.D. from the University of Hong Kong after earning his Bachelor’s degree from Tsinghua University. He is the recipient of the Hong Kong PhD Fellowship. His research stands at the intersection of machine learning, natural language processing, and computer vision, with a special focus on generative modeling.
Week 13
- Apr 23
-
- Guest Lecture 2Introduction to Graph Deep Learning
- Jiaxuan You
- Bio: Jiaxuan You is an incoming Assistant Professor in the Computer Science Department at the University of Illinois Urbana-Champaign. He obtained his CS PhD from Stanford University. His research focuses on empowering AI with graph/relational data and building general AI agents. He has published more than 20 publications in NeurIPS, ICML, ICLR, etc, with more than 10,000 citations. Jiaxuan is the creator of GraphGym and a main contributor to PyG, which are popular open-source libraries for graph ML with 20K+ stars. He has served as a program committee member of NeurIPS, ICML, ICLR, AAAI, KDD, WWW, IJCAI more than 30 times. Jiaxuan led the organization of NeurIPS 2022 and 2023 GLFrontiers Workshops: New Frontiers in Graph Learning, and the ICLR 2024 AGI Workshop: How Far Are We from AGI.
Week 14
- Apr 30
- Final Presentations The final presentation of your projects will be on April 30, in-person.