DOCODIVE
Advanced Free Learning Path

Data Science Advanced: Deep Learning & NLP

Master the frontier — neural networks, CNNs, RNNs, transformers, NLP tasks, deployment, MLOps, explainable AI, and ship a production NLP model. This is where you become an AI engineer.

10–14 weeks 40 lessons 1 capstone Intermediate knowledge required
Start Learning
01

Neural Networks from Scratch

25 min
What you'll learn
  • Understand neurons and layers
  • Learn forward propagation
  • Build a tiny network with NumPy

A neural network is layers of 'neurons' that each compute a weighted sum of inputs, add a bias, and pass it through an activation function. Layers stack: input → hidden → output. Forward propagation is just this computation flowing left to right. Even a tiny NumPy network built by hand teaches you what frameworks like TensorFlow and PyTorch automate — and that understanding makes every later concept click.

neural_net.py
import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

# One neuron: weighted sum + bias + activation
inputs = np.array([0.5, -0.2, 0.1])
weights = np.array([0.4, 0.3, -0.5])
bias = 0.1

z = np.dot(inputs, weights) + bias
output = sigmoid(z)
print(f'Neuron output: {output:.4f}')
Live Preview
Neural Networks from Scratch
1
2
3
h1
h2
out
🔎 Important: A neuron = weighted sum + bias + activation. Nothing more. Everything else is composition.
Try it yourself

Write the sigmoid function and compute sigmoid(0).

1 / (1 + exp(-x)).
import numpy as np
def sigmoid(x): return 1/(1+np.exp(-x))
print(sigmoid(0))  # 0.5
02

Activation Functions

22 min
What you'll learn
  • Compare ReLU, sigmoid, tanh
  • Understand why non-linearity matters
  • Pick the right activation

Without activation functions, stacking layers would just be repeated linear math — no matter how deep, it collapses into one line. Activations add non-linearity, letting networks learn curves and complex patterns. ReLU is the default for hidden layers (fast, no vanishing gradient); sigmoid is for binary output; softmax for multi-class output. Choosing correctly is half the battle.

activations.py
import numpy as np

def relu(x): return np.maximum(0, x)
def sigmoid(x): return 1/(1+np.exp(-x))
def softmax(x):
    e = np.exp(x - np.max(x))
    return e / e.sum()

print('ReLU(-2, 3):', relu(np.array([-2, 3])))
print('Softmax([1,2,3]):', softmax(np.array([1.,2.,3.])))
Live Preview
Activation Functions

Sigmoid vs ReLU

✓ Best Practice: Hidden layers: ReLU. Binary classification output: sigmoid. Multi-class: softmax.
Try it yourself

What does ReLU(-5) return?

max(0, -5).
0
03

Loss Functions

22 min
What you'll learn
  • Measure model error
  • Understand MSE and cross-entropy
  • Link loss to training

A loss function scores how wrong your model's predictions are. Regression uses mean squared error (MSE) — squared distance between predicted and actual. Classification uses cross-entropy — how confident the model is in the correct class. Training means minimizing this loss: lower loss = better predictions. Picking the right loss for your task is essential; the optimizer's only job is to reduce it.

loss.py
import numpy as np

def mse(y_true, y_pred):
    return np.mean((y_true - y_pred) ** 2)

def cross_entropy(y_true, y_pred):
    y_pred = np.clip(y_pred, 1e-9, 1-1e-9)
    return -np.sum(y_true * np.log(y_pred))

print('MSE:', mse(np.array([1,2,3]), np.array([1.2,1.8,3.5])))
print('CE:', cross_entropy(np.array([1,0]), np.array([0.9,0.1])))
Live Preview
Loss Functions
💡 Tip: MSE for numbers, cross-entropy for categories. Using the wrong one makes training fail silently.
Try it yourself

Compute MSE between [1,2] and [1,1].

mean((y_true - y_pred)^2).
np.mean((np.array([1,2]) - np.array([1,1]))**2)  # 0.5
04

Optimizers: Gradient Descent

26 min
What you'll learn
  • Understand gradients
  • Learn gradient descent
  • Choose learning rate

The optimizer adjusts weights to minimize loss — gradient descent is the foundation. It computes the gradient (direction of steepest increase) and steps in the opposite direction. The learning rate controls step size: too big overshoots, too small crawls. Adam is the modern default, combining momentum and adaptive rates so it converges fast without much tuning.

optimizer.py
import numpy as np

def loss(w): return (w - 3) ** 2  # minimum at w=3
def grad(w): return 2 * (w - 3)

w = 0.0
lr = 0.1
for step in range(20):
    w = w - lr * grad(w)
print(f'Learned w: {w:.3f} (true 3.0)')
Live Preview
Optimizers: Gradient Descent
🔎 Important: Learning rate is the most important hyperparameter — start small (0.001) and adjust.
Try it yourself

Run gradient descent on (w-5)^2 for 10 steps starting at 0, lr=0.1.

grad = 2*(w-5).
w=0
for _ in range(10): w = w - 0.1*2*(w-5)
print(w)  # close to 5
05

Backpropagation

28 min
What you'll learn
  • Understand how gradients flow backward
  • Learn the chain rule
  • See why deep nets learn

Backpropagation is how neural networks learn. Forward pass makes a prediction and computes loss; backward pass computes how much each weight contributed to that loss (via the chain rule) and updates them. Gradient flows backward layer by layer, hence the name. This is the engine behind every deep learning model — you rarely implement it manually, but understanding it makes debugging and design intuitive.

backprop.py
# Conceptual backprop: loss -> grad -> weight update
def update(w, grad, lr=0.01):
    return w - lr * grad

w1, w2 = 0.5, -0.3
grad_w1, grad_w2 = 0.12, -0.08  # from backprop
w1 = update(w1, grad_w1)
w2 = update(w2, grad_w2)
print(w1, w2)
Live Preview
Backpropagation
1
2
3
h1
h2
out
🔎 Important: Frameworks compute gradients automatically (autograd) — but the idea is still update = w - lr * grad.
Try it yourself

Update w=1.0 with grad=0.2 and lr=0.05.

w - lr*grad.
1.0 - 0.05*0.2 = 0.99
06

Building a Model with PyTorch

28 min
What you'll learn
  • Define a neural network class
  • Use forward pass
  • Train a simple model

PyTorch makes building models clean: subclass nn.Module, define layers in __init__, and write forward(). Tensors replace NumPy arrays and flow through the network with autograd tracking. A simple model is Linear → ReLU → Linear → softmax. PyTorch is the research and production standard today — learning it is a career multiplier for deep learning.

pytorch.py
import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10, 32)
        self.fc2 = nn.Linear(32, 2)
    def forward(self, x):
        x = torch.relu(self.fc1(x))
        return self.fc2(x)

model = Net()
print(model)
Live Preview
Building a Model with PyTorch
✓ Best Practice: nn.Module + forward() is the whole PyTorch mental model.
Try it yourself

Add a dropout layer between fc1 and fc2.

nn.Dropout(0.5).
class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(10,32)
        self.drop = nn.Dropout(0.5)
        self.fc2 = nn.Linear(32,2)
    def forward(self,x):
        x = torch.relu(self.fc1(x))
        x = self.drop(x)
        return self.fc2(x)
07

Training Loop Anatomy

30 min
What you'll learn
  • Write a complete training loop
  • Track loss and accuracy
  • Understand epochs and batches

Training is a loop: for each epoch, iterate over batches, zero gradients, forward pass, compute loss, backward pass, update weights. An epoch is one full pass over the dataset; batches are chunks fed to the model for memory efficiency. Understanding this loop is THE core skill — every deep learning experiment is a variation of it.

training_loop.py
for epoch in range(epochs):
    for batch in dataloader:
        optimizer.zero_grad()
        outputs = model(batch['x'])
        loss = criterion(outputs, batch['y'])
        loss.backward()
        optimizer.step()
    print(f'Epoch {epoch}, Loss: {loss.item():.4f}')
Live Preview
Training Loop Anatomy
✓ Best Practice: zero_grad → forward → loss → backward → step. Memorize that five-step sequence.
Try it yourself

Why must you call optimizer.zero_grad() each batch?

Gradients accumulate.
Otherwise gradients from previous batches add up, corrupting updates.
08

Overfitting in Deep Learning

24 min
What you'll learn
  • Recognize DL overfitting
  • Use early stopping
  • Add regularization

Deep models can memorize training data instead of generalizing — the classic overfitting sign is training loss dropping while validation loss rises. Fixes: more data, dropout (randomly deactivate neurons), weight decay (L2 regularization), and early stopping (stop when validation stops improving). These are your daily tools for building models that work on unseen data.

overfit.py
model = nn.Sequential(
    nn.Linear(100, 64),
    nn.ReLU(),
    nn.Dropout(0.5),
    nn.Linear(64, 2)
)
optimizer = torch.optim.Adam(model.parameters(), weight_decay=0.01)
Live Preview
Overfitting in Deep Learning
⚠️ Common Mistake: Monitor validation loss, not training loss — the gap between them IS overfitting.
Try it yourself

Add dropout of 0.3 after a ReLU layer.

nn.Dropout(0.3).
nn.Sequential(nn.Linear(10,32), nn.ReLU(), nn.Dropout(0.3))
09

Dropout: The Overfitting Killer

20 min
What you'll learn
  • Understand dropout regularization
  • Apply dropout correctly
  • Know train vs eval mode

Dropout randomly deactivates a fraction of neurons during training, forcing the network to learn redundant, robust representations instead of relying on any single pathway. It's like training an ensemble of smaller networks in one. Critical detail: dropout is ON during training and OFF during inference — model.eval() disables it automatically.

dropout.py
import torch.nn as nn

layer = nn.Dropout(p=0.5)
x = torch.ones(10)
print('Train mode:', layer(x))
layer.eval()
print('Eval mode:', layer(x))
Live Preview
Dropout: The Overfitting Killer
1
2
3
4
5
🔎 Important: Always call model.eval() before inference — forgetting this is the most common DL bug.
Try it yourself

Why does dropout output double values in train mode?

Scaling to preserve expected sum.
Dropout scales survivors by 1/(1-p) to keep the expected sum constant.
10

Batch Normalization

22 min
What you'll learn
  • Understand batch norm
  • Speed up training
  • Apply in PyTorch

Batch normalization normalizes each layer's inputs to mean 0, variance 1 within each batch. This stabilizes training, lets you use higher learning rates, reduces sensitivity to initialization, and acts as mild regularization. It's so effective it's now a default layer in almost every modern architecture.

batchnorm.py
model = nn.Sequential(
    nn.Linear(64, 128),
    nn.BatchNorm1d(128),
    nn.ReLU(),
    nn.Linear(128, 10)
)
Live Preview
Batch Normalization
1
2
3
4
5
✓ Best Practice: BatchNorm goes BEFORE the activation — Linear → BatchNorm → ReLU is the canonical order.
Try it yourself

Add BatchNorm1d(64) before ReLU.

nn.BatchNorm1d(64).
nn.Sequential(nn.Linear(10,64), nn.BatchNorm1d(64), nn.ReLU())
11

Convolutional Neural Networks (CNNs)

30 min
What you'll learn
  • Understand convolutions
  • Learn pooling
  • Build a CNN for images

CNNs are the workhorse of computer vision. Convolution filters slide across an image detecting edges, textures, then parts, then objects — a hierarchy of features. Pooling downsamples, reducing size and adding translation invariance. A classic CNN stacks Conv → ReLU → Pool blocks, then flattens and classifies. This architecture powers everything from face recognition to self-driving cars.

cnn.py
import torch.nn as nn

model = nn.Sequential(
    nn.Conv2d(3, 32, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2),
    nn.Conv2d(32, 64, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2),
    nn.Flatten(),
    nn.Linear(64*8*8, 10)
)
Live Preview
Convolutional Neural Networks (CNNs)
🔎 Important: Conv layers detect features; pooling shrinks spatial size; flatten connects to the classifier.
Try it yourself

Add another Conv2d(64,128) block.

nn.Conv2d(64,128,3,padding=1) + ReLU + MaxPool.
nn.Sequential(..., nn.Conv2d(64,128,3,padding=1), nn.ReLU(), nn.MaxPool2d(2), nn.Flatten(), nn.Linear(...))
12

Recurrent Neural Networks (RNNs)

28 min
What you'll learn
  • Understand sequential data
  • Learn hidden state
  • Process time series

RNNs process sequences — text, audio, time series — by maintaining a hidden state that carries information from previous steps. Each step receives the current input plus the previous hidden state. This recurrence lets them capture temporal patterns CNNs can't. Vanilla RNNs suffer vanishing gradients, which is why LSTM and GRU were invented.

rnn.py
import torch.nn as nn

rnn = nn.RNN(input_size=10, hidden_size=20, batch_first=True)
# x shape: (batch, seq_len, input_size)
output, hidden = rnn(x)
print('Output:', output.shape, 'Hidden:', hidden.shape)
Live Preview
Recurrent Neural Networks (RNNs)
x1
x2
x3
h
🔎 Important: Hidden state is RNN's memory — it's what flows from one time step to the next.
Try it yourself

What does batch_first=True change?

Input dimension order.
It makes input shape (batch, seq_len, features) instead of (seq_len, batch, features).
13

LSTM: Long Short-Term Memory

30 min
What you'll learn
  • Understand LSTM gates
  • Solve vanishing gradients
  • Process long sequences

LSTM fixes RNN's main weakness — forgetting important info over long sequences. It has gates: input (what to remember), forget (what to discard), and output (what to expose). A cell state acts as a conveyor belt carrying information across many steps. This lets LSTMs learn dependencies hundreds of steps apart — essential for language, speech, and any long-range sequence task.

lstm.py
import torch.nn as nn

lstm = nn.LSTM(input_size=10, hidden_size=20, num_layers=2, batch_first=True)
output, (hidden, cell) = lstm(x)
print('Output:', output.shape, 'Cell state:', cell.shape)
Live Preview
LSTM: Long Short-Term Memory
inputgate
forgetgate
outputgate
💡 Tip: Cell state is the long-term memory — it flows with minimal change across the sequence.
Try it yourself

Create a 3-layer LSTM with hidden size 64.

nn.LSTM(input_size, 64, num_layers=3).
nn.LSTM(input_size=10, hidden_size=64, num_layers=3)
14

Attention Mechanism

30 min
What you'll learn
  • Understand attention weights
  • Compute context vectors
  • Improve sequence models

Attention lets a model focus on the most relevant parts of an input sequence — like a human reading a sentence and emphasizing key words. Given a query, attention scores each input token, then takes a weighted average (context vector). This dynamic focusing solved RNN bottlenecks and became the foundation of Transformers, the most important architecture in modern AI.

attention.py
import torch
import torch.nn.functional as F

def attention(query, keys, values):
    scores = torch.matmul(query, keys.transpose(-2, -1))
    weights = F.softmax(scores / (keys.size(-1) ** 0.5), dim=-1)
    context = torch.matmul(weights, values)
    return context, weights

context, attention_weights = attention(q, k, v)
print('Context:', context.shape, 'Attention:', attention_weights.shape)
Live Preview
Attention Mechanism

Attention weights

🔎 Important: Attention = softmax(QK^T / sqrt(d)) × V. This one formula powers Transformers, GPT, BERT.
Try it yourself

Why divide by sqrt(d) in attention?

Stability.
It keeps dot products from exploding and softmax from saturating.
15

Word Embeddings

26 min
What you'll learn
  • Understand word vectors
  • Learn semantic similarity
  • Use pre-trained embeddings

Word embeddings map words to dense vectors such that similar words are close in vector space. 'king' is near 'queen', 'dog' near 'puppy'. This semantic geometry lets models understand meaning, not just spelling. Classic embeddings (Word2Vec, GloVe) are learned from co-occurrence; modern models learn contextual embeddings (each word's vector depends on its sentence).

embedding.py
import torch
import torch.nn as nn

vocab_size = 10000
embedding_dim = 300
embedding = nn.Embedding(vocab_size, embedding_dim)
token_ids = torch.tensor([[1, 5, 9]])
vectors = embedding(token_ids)
print('Embedding shape:', vectors.shape)
Live Preview
Word Embeddings
king
queen
dog
✓ Best Practice: Embeddings turn sparse one-hot words into dense, meaningful vectors — the first layer of every NLP model.
Try it yourself

Create an embedding layer for 5000 words, 128 dimensions.

nn.Embedding(5000, 128).
emb = nn.Embedding(5000, 128)
16

Tokenization

24 min
What you'll learn
  • Split text into tokens
  • Understand subword tokenization
  • Use BPE/WordPiece

Tokenization is the first step in any NLP pipeline: breaking text into units the model understands. Simple whitespace splitting fails on punctuation, contractions, and new words. Modern tokenizers (BPE, WordPiece) split rare words into subwords, so 'unhappiness' becomes 'un' + 'happiness' — handling any word with a small vocabulary. This is what powers BERT, GPT, and every transformer.

tokenize.py
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
tokens = tokenizer.tokenize('I love machine learning!')
ids = tokenizer.encode('I love machine learning!')
print('Tokens:', tokens)
print('IDs:', ids)
Live Preview
Tokenization
[CLS] i love machine learning [SEP]
💡 Tip: IDs 101 and 102 are [CLS] and [SEP] — special tokens BERT adds automatically.
Try it yourself

Tokenize the sentence 'Data science is fun!' with BERT.

tokenizer.tokenize('...').
tokenizer.tokenize('Data science is fun!')
17

Text Classification

28 min
What you'll learn
  • Build a text classifier
  • Use transformer encoders
  • Classify sentiment/topics

Text classification assigns labels to text — spam detection, sentiment analysis, topic tagging. With modern transformers, you load a pre-trained model like BERT, fine-tune it on your labeled examples, and get state-of-the-art accuracy with little data. The pipeline: tokenize → encode → train a classifier head → predict. This is the most common NLP task in production.

text_classify.py
from transformers import pipeline

classifier = pipeline('text-classification')
result = classifier('This product is amazing!')
print(result)
Live Preview
Text Classification
Spam98%
Not Spam2%
✓ Best Practice: HuggingFace pipelines let you use SOTA models in 3 lines — fine-tune later when needed.
Try it yourself

Classify 'I hate waiting in lines' with a pipeline.

classifier('...').
classifier('I hate waiting in lines')  # NEGATIVE
18

Sentiment Analysis

24 min
What you'll learn
  • Understand sentiment polarity
  • Use pre-trained models
  • Handle nuanced language

Sentiment analysis determines emotional tone — positive, negative, neutral. Modern models handle sarcasm, negation ('not bad' = positive), and context far better than keyword matching. It's used for brand monitoring, product feedback, and social listening. Fine-tuning on domain data (e.g., movie reviews vs product reviews) dramatically improves accuracy because sentiment is context-dependent.

sentiment.py
from transformers import pipeline

sentiment = pipeline('sentiment-analysis')
texts = ['I loved the movie!', 'The service was terrible.', 'It was okay.']
results = sentiment(texts)
for r in results:
    print(r['label'], f"{r['score']:.2f}")
Live Preview
Sentiment Analysis
😀

Positive 90%

💡 Tip: Domain matters — a model trained on movies may misread product reviews. Fine-tune on your data.
Try it yourself

Run sentiment on 'not bad at all'.

It's an idiom meaning positive.
Returns POSITIVE — models learned this idiom.
19

Named Entity Recognition (NER)

26 min
What you'll learn
  • Identify entities in text
  • Tag people, places, orgs
  • Use NER for information extraction

NER finds and labels entities — people, organizations, locations, dates — in unstructured text. It's the bridge between raw text and structured data: from 'Apple acquired a startup in Berlin' you extract Organization=Apple, City=Berlin. This powers search, recommendation, and knowledge graphs. Modern models achieve near-human accuracy on standard benchmarks.

ner.py
from transformers import pipeline

ner = pipeline('ner', aggregation_strategy='simple')
text = 'Sufyan works at Google in Lahore.'
for entity in ner(text):
    print(entity['word'], '->', entity['entity_group'])
Live Preview
Named Entity Recognition (NER)
Sufyan: PER Google: ORG Lahore: LOC
🔎 Important: NER turns free text into structured facts — the foundation of information extraction systems.
Try it yourself

Run NER on 'Obama visited Paris in 2010.'

Expect PER and LOC entities.
Obama=PER, Paris=LOC, 2010=DATE
20

The Transformer Architecture

32 min
What you'll learn
  • Understand self-attention
  • Learn encoder-decoder structure
  • See why transformers dominate

Transformers replaced RNNs entirely with pure attention and parallel processing. Self-attention lets each token look at every other token simultaneously, capturing long-range dependencies in one step. The architecture has an encoder (understanding input) and decoder (generating output). BERT uses the encoder; GPT uses the decoder. This is the architecture behind virtually all modern AI.

transformer.py
from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
model = AutoModel.from_pretrained('bert-base-uncased')
inputs = tokenizer('Hello world', return_tensors='pt')
outputs = model(**inputs)
print('Last hidden state:', outputs.last_hidden_state.shape)
Live Preview
The Transformer Architecture
Encoder
Self-Attention ×12
Decoder
🔎 Important: Self-attention processes ALL tokens in parallel — this parallelization is why transformers scale.
Try it yourself

Load a tiny BERT and check its hidden size.

AutoModel.from_pretrained('...').
model.config.hidden_size  # e.g. 768
21

Language Models & Generation

30 min
What you'll learn
  • Understand autoregressive generation
  • Use GPT-style models
  • Generate coherent text

Language models predict the next token given previous ones. GPT is autoregressive: it generates one token at a time, each becoming part of the context for the next. This simple 'predict next word' objective, scaled to billions of parameters and trained on massive text, yields models that can write essays, code, and hold conversations. Sampling strategies (temperature, top-p) control creativity vs coherence.

generation.py
from transformers import pipeline

generator = pipeline('text-generation', model='gpt2')
result = generator('The future of AI is', max_length=30, num_return_sequences=1)
print(result[0]['generated_text'])
Live Preview
Language Models & Generation
The future of AI is bright
and will transform healthcare
💡 Tip: temperature < 1 = focused/deterministic; > 1 = creative/random. Tune it for your use case.
Try it yourself

Generate text starting with 'Once upon a time'.

generator('Once upon a time', ...).
generator('Once upon a time', max_length=30)
22

Text Summarization

26 min
What you'll learn
  • Extract key information
  • Build summarization models
  • Handle long documents

Summarization condenses long text into its essential meaning — extractive (selecting important sentences) or abstractive (generating new paraphrased sentences). Modern transformer models generate abstractive summaries that read naturally. Use cases: news digests, document review, meeting notes. The BART and T5 architectures are standard backbones for this task.

summarize.py
from transformers import pipeline

summarizer = pipeline('summarization')
text = '...long article text...'
summary = summarizer(text, max_length=130, min_length=30)
print(summary[0]['summary_text'])
Live Preview
Text Summarization
Long article text...
Concise summary of key points.
✓ Best Practice: max_length/min_length control summary size — tune them to balance brevity and detail.
Try it yourself

Summarize a paragraph of your choice.

summarizer(your_text, max_length=100).
summarizer(text, max_length=100)
23

Machine Translation

28 min
What you'll learn
  • Understand seq2seq translation
  • Use pre-trained translators
  • Handle multiple languages

Machine translation converts text between languages using encoder-decoder transformers. The encoder understands the source sentence; the decoder generates the target translation token by token. Models like M2M100 and NLLB support hundreds of languages. This same architecture underpins summarization, chatbots, and any input-to-output text task.

translate.py
from transformers import pipeline

translator = pipeline('translation', model='Helsinki-NLP/opus-mt-en-ur')
result = translator('Machine learning is transforming the world.')
print(result[0]['translation_text'])
Live Preview
Machine Translation
Hello world
ہیلو دنیا
🔎 Important: The same seq2seq transformer does translation, summarization, and dialogue — one architecture, many tasks.
Try it yourself

Translate a sentence from English to French.

Use a Helsinki en-fr model.
translator = pipeline('translation', model='Helsinki-NLP/opus-mt-en-fr')
result = translator('Hello world')
24

Question Answering

28 min
What you'll learn
  • Extract answers from context
  • Build a QA system
  • Understand extractive QA

Extractive QA takes a question and a context passage, then finds the exact span of text that answers it. BERT-style models do this by predicting start and end positions of the answer within the context. This powers search engines, virtual assistants, and customer support bots. It's a great first NLP project because it's structured and easy to evaluate.

qa.py
from transformers import pipeline

qa = pipeline('question-answering')
context = 'DocoDive is a free library with programming books.'
result = qa(question='What is DocoDive?', context=context)
print(result['answer'])
Live Preview
Question Answering
Q: What is DocoDive?
A: A free library with programming books.
💡 Tip: Extractive QA returns exact spans — for open-domain QA (no context), you need retrieval + generation.
Try it yourself

Ask a model about a fact in a short passage.

Provide both question and context.
qa(question='...', context='...')
25

Building a Chatbot

32 min
What you'll learn
  • Design conversational flow
  • Use dialogue models
  • Add context and memory

Chatbots combine language understanding and generation. Rule-based bots follow scripted flows; modern LLM-based bots converse naturally with instruction-tuned models. Key challenges: maintaining context across turns, staying on-topic, and avoiding hallucinations. A good chatbot keeps conversation history as input and uses system prompts to define behavior.

chatbot.py
from transformers import pipeline

chatbot = pipeline('text-generation', model='microsoft/DialoGPT-small')
response = chatbot('Hi, how are you?')
print(response[0]['generated_text'])
Live Preview
Building a Chatbot
Hi, how are you?
I'm doing well, thanks!
✓ Best Practice: Conversation history IS the context — feed previous turns back into the model each time.
Try it yourself

Build a tiny chatbot that echoes user input back.

Simple Python input/print loop.
while True:
    u = input('You: ')
    if u.lower()=='quit': break
    print(f'Bot: {u}')
26

Speech Recognition (ASR)

28 min
What you'll learn
  • Understand ASR pipeline
  • Use Whisper
  • Convert speech to text

Automatic speech recognition converts audio to text. Modern models like Whisper process audio spectrograms directly with transformers, handling multiple languages and accents robustly. The pipeline: audio → feature extraction → transformer → text tokens. This powers voice assistants, transcription services, and accessibility tools.

asr.py
from transformers import pipeline

asr = pipeline('automatic-speech-recognition', model='openai/whisper-small')
result = asr('audio.mp3')
print(result['text'])
Live Preview
Speech Recognition (ASR)
💡 Tip: Whisper works on 16kHz audio — resample before inference if your input differs.
Try it yourself

Transcribe a short WAV file with Whisper.

asr('file.wav').
result = asr('audio.wav')
print(result['text'])
27

Recommendation Systems

28 min
What you'll learn
  • Understand collaborative filtering
  • Learn matrix factorization
  • Build a simple recommender

Recommendation systems suggest items users might like. Collaborative filtering uses user-item interaction patterns — 'people who liked X also liked Y'. Matrix factorization learns latent factors for users and items, predicting ratings as their dot product. This is what powers Netflix, Amazon, and Spotify. It sits at the intersection of ML and product design.

recommend.py
import numpy as np

# Latent factor model: rating = dot(user_vec, item_vec)
user_vec = np.array([0.8, 0.2, 0.1])
item_vec = np.array([0.6, 0.5, 0.7])
predicted_rating = np.dot(user_vec, item_vec)
print(f'Predicted rating: {predicted_rating:.2f}')
Live Preview
Recommendation Systems
📚
🎬
🎵
✓ Best Practice: The latent factor dot product is the heart of collaborative filtering — embeddings for users and items.
Try it yourself

Compute dot product of [0.5,0.5] and [1,0].

np.dot(...).
np.dot([0.5,0.5],[1,0])  # 0.5
28

Anomaly Detection with Deep Learning

28 min
What you'll learn
  • Detect rare events
  • Use autoencoders
  • Apply to fraud/outliers

Anomaly detection finds rare, unusual data points. Autoencoders are a deep learning approach: train a network to reconstruct normal data, then flag examples with high reconstruction error as anomalies (they can't be reconstructed well). Use cases: fraud detection, equipment failure, network intrusion. The challenge is highly imbalanced labels — often you train only on normal data.

anomaly.py
import torch.nn as nn

autoencoder = nn.Sequential(
    nn.Linear(30, 16),
    nn.ReLU(),
    nn.Linear(16, 8),
    nn.ReLU(),
    nn.Linear(8, 16),
    nn.ReLU(),
    nn.Linear(16, 30)
)
# Reconstruction error = anomaly score
Live Preview
Anomaly Detection with Deep Learning
⚠️ Common Mistake: Train on NORMAL data only. Anomalies are anything the autoencoder can't reconstruct well.
Try it yourself

What makes a good anomaly score?

High reconstruction error.
Reconstruction error — higher means more anomalous.
29

Generative Adversarial Networks (GANs)

30 min
What you'll learn
  • Understand generator vs discriminator
  • Learn adversarial training
  • Generate synthetic data

GANs pit two networks against each other: a generator creates fake data, a discriminator tries to distinguish real from fake. They improve together — the generator gets better at fooling the discriminator, the discriminator gets better at catching fakes. The result: generators that produce stunningly realistic images, audio, and even text. Training GANs is notoriously tricky and sensitive to hyperparameters.

gan.py
class Generator(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(100, 256),
            nn.ReLU(),
            nn.Linear(256, 784),
            nn.Tanh()
        )
    def forward(self, z): return self.net(z)

gen = Generator()
z = torch.randn(1, 100)
fake = gen(z)
print('Generated shape:', fake.shape)
Live Preview
Generative Adversarial Networks (GANs)
Gen
fake
vs
Disc
real?
🔎 Important: The adversarial game is delicate — alternate training generator and discriminator carefully.
Try it yourself

What does the discriminator output represent?

Probability of real.
The probability that the input is real, not generated.
30

Reinforcement Learning Basics

28 min
What you'll learn
  • Understand agents and environments
  • Learn reward maximization
  • Grasp Q-learning

Reinforcement learning trains agents to make sequences of decisions by maximizing cumulative reward. The agent observes a state, takes an action, receives a reward, and transitions to a new state. Q-learning learns a value function estimating long-term value of actions. This powers game AI (AlphaGo), robotics, and recommendation. It's fundamentally different from supervised learning — there's no correct answer, only good outcomes.

rl.py
import numpy as np

# Q-learning update
q_table = np.zeros((10, 4))  # 10 states, 4 actions
alpha, gamma = 0.1, 0.9
state, action, reward, next_state = 0, 1, 1, 2
q_table[state, action] += alpha * (
    reward + gamma * np.max(q_table[next_state]) - q_table[state, action]
)
print(q_table[state, action])
Live Preview
Reinforcement Learning Basics
State4
Actionleft
Reward+10
🔎 Important: Q-learning is the foundation — reward + discounted future value, update toward it.
Try it yourself

What does gamma control in Q-learning?

Discount factor.
How much future rewards matter vs immediate rewards.
31

Transfer Learning

26 min
What you'll learn
  • Reuse pre-trained models
  • Fine-tune on small data
  • Understand feature extraction

Transfer learning lets you take a model trained on massive data (ImageNet, Wikipedia) and adapt it to your task with a fraction of the data. You either freeze the backbone and just train a new head, or fine-tune all layers with a small learning rate. This is why you can build a world-class image classifier or NLP model with hundreds of examples, not millions.

transfer.py
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained(
    'bert-base-uncased', num_labels=2
)
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
print(model)
Live Preview
Transfer Learning
Pre-trained
Fine-tuned
✓ Best Practice: Start from pre-trained, never from scratch — transfer learning is the 80/20 of modern AI.
Try it yourself

Load a pre-trained BERT with 3 labels.

num_labels=3.
AutoModelForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=3)
32

GPU Training

20 min
What you'll learn
  • Use CUDA in PyTorch
  • Move models to GPU
  • Speed up training

Deep learning is computationally heavy — GPUs parallelize matrix operations, making training 10-100x faster. In PyTorch, moving a model and data to GPU is one line: .to('cuda'). Getting comfortable with GPU training, mixed precision, and memory management separates hobbyist experiments from production-scale ML.

gpu.py
import torch

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)

# In the training loop:
x, y = x.to(device), y.to(device)
print(f'Training on {device}')
Live Preview
GPU Training
GPU: CUDA
10-100x faster
💡 Tip: Check torch.cuda.is_available() first, and move BOTH model and data to the device.
Try it yourself

Write the one-liner to check if CUDA is available.

torch.cuda.is_available().
torch.cuda.is_available()
33

Evaluation Metrics for DL Models

24 min
What you'll learn
  • Use accuracy, precision, recall
  • Understand F1 and AUC
  • Pick metrics per task

Different tasks need different metrics. Classification: accuracy (balanced) or precision/recall/F1 (imbalanced). Regression: MAE, RMSE. Ranking: NDCG. Multi-label: average precision. AUC-ROC measures ranking quality independent of threshold. Knowing which metric matches your business goal is as important as the model itself — optimizing the wrong metric is a common, costly mistake.

metrics.py
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

y_true = [0, 1, 1, 0, 1]
y_pred = [0, 1, 0, 0, 1]
print('Accuracy:', accuracy_score(y_true, y_pred))
print('Precision:', precision_score(y_true, y_pred))
print('Recall:', recall_score(y_true, y_pred))
print('F1:', f1_score(y_true, y_pred))
Live Preview
Evaluation Metrics for DL Models
Accuracy
Precision
Recall
⚠️ Common Mistake: For imbalanced data, accuracy is deceptive — always look at precision/recall/F1.
Try it yourself

Compute F1 for y_true=[0,1,1], y_pred=[1,1,0].

f1_score(...).
f1_score([0,1,1],[1,1,0])  # 0.5
34

Explainable AI (XAI)

26 min
What you'll learn
  • Understand model decisions
  • Use SHAP and LIME
  • Build trust and compliance

Black-box models can be accurate but uninterpretable. Explainable AI methods — SHAP (game-theoretic feature attribution) and LIME (local surrogate models) — explain individual predictions: which features pushed the output up or down. This matters for healthcare, finance, and any regulated domain. It also helps debug models by revealing what they're actually learning (sometimes the wrong thing).

xai.py
import shap

explainer = shap.Explainer(model)
shap_values = explainer(X)
shap.plots.waterfall(shap_values[0])
Live Preview
Explainable AI (XAI)
age
income
city
🔎 Important: If feature importances look wrong (e.g., gender driving decisions), your model likely learned bias.
Try it yourself

Name two XAI techniques.

SHAP, LIME.
SHAP and LIME.
35

Bias & Fairness in ML

26 min
What you'll learn
  • Recognize algorithmic bias
  • Measure fairness
  • Mitigate biased outcomes

Models inherit bias from training data — if historical hiring data favored certain groups, the model learns to do the same. Fairness means different groups receive equal treatment: similar accuracy, similar false positive rates. Measuring and mitigating bias is an ethical and legal requirement. Techniques include re-sampling, re-weighting, and using fairness-aware objectives.

fairness.py
# Fairness check: accuracy per group
group_a_accuracy = accuracy_score(y_a_true, y_a_pred)
group_b_accuracy = accuracy_score(y_b_true, y_b_pred)
print('Group A:', group_a_accuracy, 'Group B:', group_b_accuracy)
print('Gap:', abs(group_a_accuracy - group_b_accuracy))
Live Preview
Bias & Fairness in ML
Group A
Group B
⚠️ Common Mistake: A 15% accuracy gap between groups is a red flag — audit your data and model for bias.
Try it yourself

What's the first step in auditing bias?

Check data.
Examine training data distribution for underrepresentation.
36

Privacy-Preserving ML

24 min
What you'll learn
  • Understand differential privacy
  • Learn federated learning
  • Protect user data

ML models can memorize training data, leaking private information. Privacy-preserving techniques protect users: differential privacy adds calibrated noise so individual data points can't be identified; federated learning trains models across devices without collecting raw data centrally. As regulations like GDPR tighten, privacy-aware ML becomes essential, not optional.

privacy.py
# Differential privacy: add noise to gradient
import numpy as np

def privatize(gradient, epsilon=0.5):
    noise = np.random.laplace(0, 1/epsilon, gradient.shape)
    return gradient + noise
Live Preview
Privacy-Preserving ML
🔒 + Laplace noise (ε=0.5)
🔎 Important: Lower epsilon = more privacy but less accuracy. It's a fundamental trade-off.
Try it yourself

What does epsilon control in differential privacy?

Privacy budget.
The privacy budget — smaller means stronger privacy.
37

Model Deployment

28 min
What you'll learn
  • Serve models via API
  • Containerize with Docker
  • Deploy to cloud

A model in a notebook has no business value — deployment puts it behind an API so applications can use it. The standard stack: wrap the model in a Flask/FastAPI service, containerize with Docker, and deploy to cloud (AWS, GCP, or platforms like HuggingFace Spaces). The model is an asset; deployment is what makes it a product.

deploy.py
from fastapi import FastAPI
import joblib

app = FastAPI()
model = joblib.load('model.joblib')

@app.post('/predict')
def predict(features: list):
    prediction = model.predict([features])
    return {'prediction': prediction.tolist()}
Live Preview
Model Deployment
Model
API
Docker
Cloud
✓ Best Practice: Save the model (joblib) + wrap in an API (FastAPI) + containerize (Docker) = production ML.
Try it yourself

Write a simple FastAPI app with a /health endpoint.

@app.get('/health').
@app.get('/health')
def health(): return {'status':'ok'}
38

MLOps: The Full Lifecycle

30 min
What you'll learn
  • Understand MLOps
  • Track experiments
  • Automate training and monitoring

MLOps applies DevOps practices to machine learning: versioning data, tracking experiments (MLflow, Weights & Biases), automating training pipelines, and monitoring deployed models for drift. Models degrade as data changes — MLOps catches that. It's what separates a one-off model from a reliable, maintainable ML system that keeps working in production.

mlops.py
import mlflow

mlflow.set_experiment('my-experiment')
with mlflow.start_run():
    mlflow.log_param('learning_rate', 0.001)
    mlflow.log_metric('accuracy', 0.93)
    mlflow.sklearn.log_model(model, 'model')
Live Preview
MLOps: The Full Lifecycle
📊 Track experiments
🔄 Automate training
📉 Monitor drift
✓ Best Practice: Track EVERYTHING — parameters, metrics, code version. You'll thank yourself when reproducing results.
Try it yourself

Name one model drift detection technique.

Monitor input distribution.
Compare live input distribution against training distribution.
39

Edge AI: Models on Devices

24 min
What you'll learn
  • Understand edge deployment
  • Compress models with quantization
  • Run on mobile/embedded

Edge AI runs models on devices — phones, cameras, sensors — instead of the cloud. Benefits: privacy (data never leaves device), speed (no network latency), and offline capability. The challenge: models are too big, so you compress them — quantization (8-bit instead of 32-bit), pruning, and distillation — with minimal accuracy loss. This is how face unlock and voice assistants work instantly.

edge.py
import torch

model_fp32 = model
model_int8 = torch.quantization.quantize_dynamic(
    model_fp32, {torch.nn.Linear}, dtype=torch.qint8
)
print('Quantized model ready for edge deployment')
Live Preview
Edge AI: Models on Devices
📱
int8 quantized
💡 Tip: Quantization to int8 shrinks models ~4x with minimal accuracy drop — essential for edge.
Try it yourself

Name one benefit of edge AI.

Privacy.
Data stays on device — better privacy and lower latency.
40

Capstone: Deploy an NLP Model End-to-End

60 min
What you'll learn
  • Apply the full DL + NLP + MLOps workflow
  • Build, fine-tune, deploy a model
  • Ship a production-ready system

Your final capstone: pick an NLP task (sentiment, classification, QA), fine-tune a pre-trained transformer, evaluate it rigorously, track experiments, save and containerize it, then serve it behind an API. This is the complete journey from raw text to deployed AI — exactly what a production ML engineer does. It's portfolio-ready proof you can build real systems.

capstone.py
from transformers import AutoModelForSequenceClassification, Trainer
from fastapi import FastAPI

model = AutoModelForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)
# Fine-tune, evaluate, save
model.save_pretrained('./saved_model')

app = FastAPI()
@app.post('/sentiment')
def predict(text: str):
    # tokenize + predict + return
    return {'sentiment': 'positive', 'confidence': 0.95}
Live Preview
Capstone: Deploy an NLP Model End-to-End
Data
Train
Eval
Deploy
✓ Best Practice: This single project demonstrates everything: data, training, evaluation, deployment. Ship it and you're job-ready.
Try it yourself

Write the FastAPI route that returns the sentiment of input text.

@app.post('/sentiment').
@app.post('/sentiment')
def sent(text: str):
    return {'text': text, 'sentiment': 'POSITIVE'}
You've completed all 40 advanced lessons. You're now an AI engineer.

Practice your skills or return to the Data Science hub.

📱 Scan this QR code with your phone camera to instantly open this page.

Works on iOS, Android, and any modern device. No app installation required.

Account Verified!

Your email has been verified successfully.