Deep Learning Notebooks

A curated collection of hands-on notebooks exploring core machine and deep learning concepts. Each notebook focuses on a specific topic - from linear models and foundational elements like broadcasting and autograd to advanced tasks such as custom layers, transfer learning, sequence modeling with RNNs, and representation learning with autoencoders.

This repository is ideal for students, researchers, and practitioners looking to deepen their understanding of deep learning by building key components from scratch and experimenting with real-world datasets like MNIST and CIFAR-10.

📓 Notebooks Overview

1_basic_classifiers.ipynb — Digit Classification with MNIST

Learn how to classify handwritten digits using the MNIST dataset. This notebook covers data preprocessing and applies basic computational learning techniques for digit recognition, with practical applications in document automation, banking, and postal systems.

2_broadcast_tensors.ipynb — Broadcasting Mechanics

Recreate the behavior of expand_as and broadcast_tensors in PyTorch. This notebook demystifies tensor broadcasting by mimicking these essential utilities.

3_autograd.ipynb — Building Autograd and Multinomial

Implement a simplified version of PyTorch’s autograd system and the multinomial sampling function. Understand how gradients are computed under the hood.

4_deciles_vs_percentiles.ipynb — Tabular Learning and Data Loading

Define a flexible DataLoader class for tabular datasets and use it to train multiple neural network models. Learn how deciles and percentiles affect class labeling in regression-to-classification tasks.

5_splitlinear_dropnorm.ipynb — Custom Layers: SplitLinear and DropNorm

Implement two original PyTorch layers:

• SplitLinear: Splits the input, applies shared transformations, and recombines the output.
• DropNorm: A combined Dropout + Normalization module.
  Evaluate their performance against standard PyTorch layers.

6_conv2d_transfer_learning.ipynb — Conv2D from Scratch and Transfer Learning

This comprehensive notebook is divided into three parts:

1. Implement a custom 2D convolutional layer (with padding and stride).
2. Derive the mathematical formulation for backpropagation through convolutions.
3. Apply transfer learning on CIFAR-10 using a pre-trained ResNet, and compare it with a custom CNN.

7_rnn.ipynb — Sentiment Analysis and Custom RNN Cell

1. Train an RNN for sentence-level sentiment classification (SST-2), using truncated BPTT to address gradient issues.
2. Design a novel RNN cell with a reset gate and train it to overfit a small dataset, validating gradient flow and architecture design.

8_autoencoder.ipynb — Deep Autoencoders and Inversion Accuracy

Build a deep autoencoder architecture with symmetric encoder-decoder layers.
Train it on MNIST and evaluate how well each decoder layer approximates the inverse of its encoder counterpart. Compare dense vs. convolutional autoencoders.

🔧 Technologies Used

• Python 3.9+
• PyTorch
• NumPy / Matplotlib / Pandas
• Jupyter Notebooks

📁 Structure

deep-learning-notebooks/
├── 1\_basic\_classifiers.ipynb
├── 2\_broadcast\_tensors.ipynb
├── 3\_autograd.ipynb
├── 4\_deciles\_vs\_percentiles.ipynb
├── 5\_splitlinear\_dropnorm.ipynb
├── 6\_conv2d\_transfer\_learning.ipynb
├── 7\_rnn.ipynb
└── 8\_autoencoder.ipynb