neurology

Deep Learning Fundamentals

From perceptrons to transformers — the neural network architectures, training techniques, and breakthroughs that power modern AI
Co-Created by Kiran Shirol and Claude
Topics Neural Networks CNNs RNNs & LSTMs GANs Autoencoders Transformers
home Learning Portal play_arrow Start Learning summarize Key Insights dictionary Glossary 12 chapters · 4 sections
Section 1

Building Blocks — Neurons & Training

The fundamental units, how they learn, and the math that makes it work.
1
scatter_plot
From Neurons to Networks
Biological inspiration, perceptrons, activation functions (ReLU, sigmoid, tanh), and the universal approximation theorem.
arrow_forward Learn
2
trending_down
Training Deep Networks
Loss functions, backpropagation, the chain rule, and computational graphs that power gradient-based learning.
arrow_forward Learn
3
speed
Optimizers & Learning Rates
SGD, momentum, RMSProp, Adam, learning rate schedules, warmup, and choosing the right optimizer.
arrow_forward Learn
Section 2

Vision — Convolutional Networks

How neural networks learned to see, from filters to ResNet.
4
grid_view
Convolutional Neural Networks
Convolutions, filters, pooling, stride, padding, and how feature maps capture visual patterns.
arrow_forward Learn
5
view_in_ar
CNN Architectures
LeNet, AlexNet, VGG, GoogLeNet, ResNet, skip connections, and the ImageNet revolution.
arrow_forward Learn
Section 3

Sequences & Generation

Modeling time, language, and learning to create — RNNs, autoencoders, and GANs.
6
replay
Recurrent Neural Networks
Sequence modeling, vanilla RNNs, hidden state, and the vanishing/exploding gradient problem.
arrow_forward Learn
7
memory
LSTMs, GRUs & Sequence Models
Gating mechanisms, bidirectional RNNs, seq2seq, encoder-decoder, and the birth of neural machine translation.
arrow_forward Learn
8
compress
Autoencoders & Representation Learning
Undercomplete and overcomplete autoencoders, variational autoencoders (VAEs), latent spaces, and disentanglement.
arrow_forward Learn
9
auto_fix_high
Generative Adversarial Networks
Generator vs. discriminator, the min-max game, DCGAN, StyleGAN, and training instability.
arrow_forward Learn
Section 4

Mastery — Regularization to Transformers

Practical training tricks and the architecture that changed everything.
10
tune
Regularization & Practical Training
Dropout, batch normalization, layer normalization, data augmentation, early stopping, and training recipes.
arrow_forward Learn
11
center_focus_strong
The Attention Mechanism
Bahdanau attention, self-attention, multi-head attention, and why attention replaced recurrence.
arrow_forward Learn
12
token
The Transformer Architecture
Encoder-decoder, positional encoding, “Attention Is All You Need” (Vaswani et al. 2017), and the bridge to LLMs.
arrow_forward Learn
explore

Explore Related Courses

functions
Mathematics for AI
Linear Algebra, Calculus & Stats
hub
Classic Machine Learning
Regression, Trees & SVMs
token
How LLMs Work
Transformers & Attention
school
AI Fundamentals
Core Concepts & Building Blocks