08408005 Deep Learning: Algorithm and Application

Instructor:

Prof. Yadong MU : myd@pku.edu.cn

Teaching Assistant:

Mr. SUN Zhicheng : sunzc@pku.edu.cn
Mr. LIN Chenguo : chenguolin@stu.pku.edu.cn
Mr. LIU Hanwen : hanwenliu@stu.pku.edu.cn

Location: Room 304, Teaching Building 2, Peking University

Time: Tuesday 18:40am - 21:30pm (weekly)

Office hours: Drop an email or wechat message to the instructor or TAs for appointing a course-related face-to-face QA.

Schedule of Lectures

Date	Topics
Feb 20, 2024	Introduction Course logistics A short tour to deep learning: illustrative applications, brief history etc.
Feb 27, 2024	Machine Learning Basics Machine learning paradigms Supervised learning: linear case Supervised learning: non-linear case Gradient Descent Stochastic Gradient Descent Unsupervised learning
March 5, 2024	Deep Learning Basics Neural layers: convolution, pooling, FC, activation, loss functions Gradient back-propagation Auto-differentiation Regularization
March 12, 2024	Backbone - I Conventional Image Classification AlexNet, VGGNet, GoogleNet, ResNet, DenseNet Squeeze-and-Excitation, GroupConv, Depthwise / Depthwise Separable Convolution, ShuffleNet
March 19, 2024	Backbone - II Classic Object Detection Z-F Net RCNN family: RCNN, Fast RCNN, Faster RCNN YOLO, SSD RetinaNet, FPN etc
March 26, 2024	Backbone - III Fully convolutional networks (FCN) Encoder-Decoder (SegNet), DeepLab Contextual inference with graphical models (CNN-as-RNN) Mask R-CNN Generalization to other pixel-to-pixel tasks Resolution matters: Simple Baseline & HRNet
April 2, 2024	Deep Models for Sequential Data RNN LSTM Transformer MLP-Mixer Mamba
April 9, 2024	Foundation Models
April 16, 2024	Generative Models
April 23, 2024	Reinforcement Learning
April 30, 2024	Holiday - no class
May 7, 2024	Learning on Graphs / Adversarial and Backdoor Learning
May 14, 2024	DL for Science / Optimization
May 21, 2024	Course Project Presentation - I
May 28, 2024	Course Project Presentation - II
June 4, 2024	Course Project Presentation - III

Textbook:

Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, MIT Press, 2016

References:

Christopher Bishop, Pattern Recognition and Machine Learning, Springer, 2006
Shai Ben-David and Shai Shalev-Shwartz, Understanding Machine Learning: From Theory to Algorithms, 2014
Richard Sutton, Reinforcement Learning, 2nd edition
Zhi-hua Zhou, Machine Learning (in Chinese), 2016

Course Work

Final Grade	Grading will be based on homeworks (35%), survey (25%) and a course project (40%). The end-of-term grade is curved. Your overall grade will depend on your performance relative to your classmates.
Survey & Project	The topic of survey paper should be DL-related. A Course Project is mandatory for all students. The only requirement for the topic is being deep learning related.