Machine Learning Beginner Course

Machine learning flips traditional programming: instead of writing rules, you feed examples and let the algorithm learn patterns. Traditional code says 'if spam keywords, mark spam'; ML learns what spam looks like from thousands of labeled emails. The three types: supervised (labeled data), unsupervised (no labels), and reinforcement (learning from rewards). Every recommendation, fraud alert, and voice assistant you use is ML in production.

# Traditional: rules
if 'free' in email and 'click' in email:
    print('spam')

# ML: learn from examples
model.fit(emails, labels)
model.predict(new_email)

Supervised: spam filter. Unsupervised: customer segmentation.

These three buckets organize all of ML. Supervised: you have input-output pairs (house features → price). Unsupervised: only inputs, find structure (group customers by behavior). Reinforcement: an agent takes actions and learns from rewards (game AI, robotics). Picking the right paradigm is the first design decision in any ML project — it determines your data needs, algorithms, and evaluation.

# Supervised: X -> y
X = [[2, 1], [3, 2]]
y = [4, 5]

# Unsupervised: only X
X = [[2, 1], [3, 2], [8, 9]]

# Reinforcement: state, action, reward
state = 'position'
action = 'move_left'
reward = 10

Supervised (regression).

Successful ML follows a disciplined workflow: define the problem, collect data, clean it, explore it, engineer features, train models, evaluate, and deploy. Most projects fail at step one — an ill-defined problem or no clear success metric. Before writing any model code, you should be able to answer: what am I predicting, what data do I have, and how will I know I've succeeded?

workflow = [
    '1. Define problem',
    '2. Collect data',
    '3. Clean data',
    '4. Explore (EDA)',
    '5. Engineer features',
    '6. Train & tune',
    '7. Evaluate & deploy'
]
for step in workflow:
    print(step)

The problem and success metric.

scikit-learn is the standard ML library because every model shares one API: fit() trains, predict() outputs, transform() preprocesses. Master this triad once and you can use hundreds of algorithms. A typical flow: create the model object, call fit(X, y), then predict(X_new). This consistency is why scikit-learn is the best first ML library.

from sklearn.neighbors import KNeighborsClassifier

model = KNeighborsClassifier(n_neighbors=3)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print(predictions[:5])

from sklearn.neighbors import KNeighborsClassifier
model = KNeighborsClassifier(n_neighbors=5)

KNN is the simplest ML algorithm: to classify a new point, find its k nearest neighbors and take a majority vote. No training needed — it stores all data and computes distances at prediction time. It's intuitive, needs no assumptions, and is great for small datasets. The key choice is k: too small overfits (noisy), too large underfits (blurry boundaries). Distance is usually Euclidean.

from sklearn.neighbors import KNeighborsClassifier

model = KNeighborsClassifier(n_neighbors=5)
model.fit(X_train, y_train)
accuracy = model.score(X_test, y_test)
print(f'Accuracy: {accuracy:.3f}')

KNN uses distances; unscaled features dominate the distance calculation.

Distance is the backbone of KNN, clustering, and similarity search. Euclidean is straight-line distance; Manhattan is grid-like (city blocks); cosine measures angle, ignoring magnitude — great for text. Choosing the right metric depends on your data and what 'similar' means in your domain. For high-dimensional data, cosine often beats Euclidean.

from sklearn.metrics.pairwise import euclidean_distances, cosine_similarity
import numpy as np

a = np.array([[1, 2]])
b = np.array([[4, 6]])
print('Euclidean:', euclidean_distances(a, b)[0][0])
print('Cosine:', cosine_similarity(a, b)[0][0])

Cosine similarity.

Every model's error decomposes into bias (wrong assumptions, underfitting) and variance (sensitivity to data, overfitting). Simple models have high bias, low variance; complex models the opposite. The goal is the sweet spot: enough complexity to capture real patterns, not enough to memorize noise. This single concept explains why regularization, cross-validation, and model selection all matter.

# High bias: model too simple (underfits)
# High variance: model too complex (overfits)
# Sweet spot: right complexity
error = bias**2 + variance + irreducible_noise

Overfitting — high variance.

Naive Bayes applies Bayes' theorem to classification, with a 'naive' assumption: features are independent given the class. Despite being wrong in practice, it works surprisingly well for text — spam filtering, sentiment, topic classification. It's fast, needs little data, and gives probabilities. For text, MultinomialNB is the standard; for continuous features, GaussianNB.

from sklearn.naive_bayes import MultinomialNB

model = MultinomialNB()
model.fit(X_train, y_train)
accuracy = model.score(X_test, y_test)
print(f'Accuracy: {accuracy:.3f}')

MultinomialNB.

SVM finds the line (hyperplane) that best separates classes with the largest margin — the widest 'street' between classes. The kernel trick lets it handle non-linear data by implicitly mapping to higher dimensions. SVM excels on small-to-medium datasets with clear margins, and the RBF kernel is the standard default. It's a workhorse for image, text, and biological classification.

from sklearn.svm import SVC

model = SVC(kernel='rbf', C=1.0)
model.fit(X_train, y_train)
accuracy = model.score(X_test, y_test)
print(f'Accuracy: {accuracy:.3f}')

Maps data to higher dimensions so non-linear boundaries become linear.

Boosting trains trees sequentially, each fixing the mistakes of the previous — turning many weak learners into one strong model. XGBoost and LightGBM are the optimized implementations that dominate Kaggle competitions and production tabular ML. They handle missing values, feature importance, and regularization out of the box. For structured data, gradient boosting is usually the answer.

from xgboost import XGBClassifier

model = XGBClassifier(n_estimators=100, learning_rate=0.1)
model.fit(X_train, y_train)
accuracy = model.score(X_test, y_test)
print(f'Accuracy: {accuracy:.3f}')

How much each tree contributes — smaller = slower but often better.

Ensembles combine multiple models to outperform any single one. Bagging (random forests) trains models in parallel on random subsets, reducing variance. Boosting (XGBoost) trains sequentially, reducing bias. Voting ensembles combine diverse models (SVM + RF + KNN) and take majority vote. The intuition: many models make different errors, and combining them cancels those errors out.

from sklearn.ensemble import VotingClassifier, RandomForestClassifier
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier

ensemble = VotingClassifier([
    ('rf', RandomForestClassifier()),
    ('svm', SVC(probability=True)),
    ('knn', KNeighborsClassifier())
], voting='soft')
ensemble.fit(X_train, y_train)
print(ensemble.score(X_test, y_test))

Variance.

Accuracy hides the full story. The confusion matrix shows all four outcomes; precision is 'of my positives, how many right', recall is 'of real positives, how many caught', F1 balances both. The ROC curve plots true positive rate vs false positive rate across thresholds — AUC (area under curve) summarizes ranking quality. These tools tell you not just 'is it good', but 'where does it fail'.

from sklearn.metrics import classification_report, roc_auc_score

print(classification_report(y_test, y_pred))
print('AUC:', roc_auc_score(y_test, y_prob))

You catch most positives but also many false positives.

More features isn't always better — the curse of dimensionality means too many features make data sparse and models overfit. Feature selection keeps the useful ones: filter methods (correlation), wrapper methods (try subsets), and embedded methods (feature importance from trees). Fewer, better features = faster training, simpler models, and often better accuracy.

from sklearn.feature_selection import SelectKBest, f_classif

selector = SelectKBest(f_classif, k=5)
X_selected = selector.fit_transform(X, y)
print('Original features:', X.shape[1])
print('Selected:', X_selected.shape[1])

Curse of dimensionality — data becomes sparse, models overfit.

PCA finds the directions of maximum variance in data and projects onto a few principal components — compressing 100 features into 2-3 that capture most information. It's used for visualization (plot high-dim data in 2D), speeding up training, removing noise, and as a preprocessing step. The trade-off: you lose interpretability because components are combinations of original features.

from sklearn.decomposition import PCA

pca = PCA(n_components=2)
X_reduced = pca.fit_transform(X)
print('Explained variance:', pca.explained_variance_ratio_)

The first component captures 62% of the data's variance.

Time series data has order — stock prices, sales, temperature. Forecasting means predicting future values. Key components: trend (long-term direction), seasonality (repeating patterns), and noise. Simple methods like moving averages and exponential smoothing work surprisingly well as baselines. The critical rule: never shuffle time data, and always use past to predict future.

import pandas as pd

# Moving average baseline
df['MA7'] = df['sales'].rolling(window=7).mean()
forecast = df['MA7'].iloc[-1]
print(f'Next day forecast: {forecast:.1f}')

Trend, seasonality, and noise.

Recommendation systems suggest items users will like. Content-based: recommend items similar to what the user liked (by features). Collaborative filtering: recommend what similar users liked (by behavior). The 'users who bought X also bought Y' pattern is collaborative filtering. Hybrid systems combine both and power Netflix, Amazon, and Spotify.

# Collaborative filtering idea
# similarity between users -> recommend items from similar users
user_a = {'item1': 5, 'item2': 4, 'item3': 3}
user_b = {'item1': 5, 'item2': 4}
# user_b hasn't rated item3; user_a (similar) rated it 3
recommend = 'item3'

Collaborative filtering.

A/B testing is how you prove a model actually improves things: split users into control (old system) and treatment (new model), then compare the metric (click rate, conversion). It's the bridge between offline accuracy and real-world value. Pitfalls: too-small samples, peeking early, and ignoring statistical significance. A model with higher offline accuracy but no A/B lift is a failure.

# A/B test comparison
control_conv = 0.10  # 10% conversion
variant_conv = 0.12   # 12% with new model
lift = (variant_conv - control_conv) / control_conv
print(f'Lift: {lift*100:.1f}%')

Accuracy may not correlate with the business metric users care about.

A model that can't be explained is a liability. Interpretability tells you WHY a prediction was made — essential for trust, debugging, and compliance (GDPR 'right to explanation'). Tree models expose feature_importances_ directly. SHAP values give per-prediction explanations: which features pushed this specific decision up or down. Interpretable models make ML accountable.

from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier()
model.fit(X, y)
importances = model.feature_importances_
for name, imp in sorted(zip(feature_names, importances), key=lambda x: -x[1]):
    print(f'{name}: {imp:.3f}')

Building trust with users, debugging, or regulatory compliance.

ML models learn from data — and if that data reflects historical bias, the model reproduces it. Fairness means different groups receive equal treatment (similar accuracy, similar false positive rates across demographics). Responsible ML requires auditing data, measuring group-wise metrics, and consciously designing for fairness. It's not just ethics — biased models cause legal, reputational, and financial harm.

# Audit fairness: accuracy per group
for group in ['group_a', 'group_b']:
    acc = accuracy_score(y_true[group], y_pred[group])
    print(f'{group}: {acc:.3f}')

Examine training data for group representation imbalance.

Your capstone: a complete classification project. Load a dataset, clean it, explore with EDA, engineer features, then train and compare KNN, SVM, random forest, and gradient boosting. Evaluate each with cross-validation, pick the best, interpret its decisions, and write up why it won. This is the exact deliverable of a working ML engineer — portfolio-ready proof you can do the job.

from sklearn.ensemble import RandomForestClassifier
from xgboost import XGBClassifier
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score

models = {
    'RF': RandomForestClassifier(),
    'XGB': XGBClassifier(eval_metric='logloss'),
    'SVM': SVC()
}
for name, model in models.items():
    scores = cross_val_score(model, X, y, cv=5)
    print(f'{name}: {scores.mean():.3f} (+/- {scores.std():.3f})')

scores = cross_val_score(RandomForestClassifier(), X, y, cv=5)
print(scores.mean())