Deep Learning

Winter term 2025 (~ 50 students per semester).

• Supervised machine learning as an interpolation problem

• Simple network architectures: Fully connected layers, rectified linear units, sigmoids, softmax

• Gradient descent for nested functions: The chain rule and it’s implementation via backpropagation

• Stochastic gradient descent on large data sets, acceleration via momentum and ADAM

• Capacity, overfitting and underfitting of neural networks

• Training, testing, and validation data sets

• Improving generalization: data augmentation, dropout, early stopping

• Working with images: Convolutions and pooling layers. Computing derivatives and adjoint linear operators

• Getting the network to train: Data preprocessing, weight initialization schemes, and batch normalization

• Applications and state-of-the-art architectures for image classification, segmentation, and denoising

• Architecture designs: Encoder-decoder idea, unrolled algorithms, skip connections + residual learning, recurrent neural networks

• Implementations in NumPy and PyTorch: Hands-on practical experience by implementing gradient descent on a fully connected network in NumPy.

• Introduction to the deep learning framework PyTorch for training complex models on GPUs.