Data Science Intermediate: Machine Learning

Machine learning is teaching computers to learn patterns from data instead of being explicitly programmed. In supervised learning, you train on labeled examples — inputs with known answers — to predict new ones. In unsupervised learning, the algorithm finds hidden structure without labels. Every ML system, from spam filters to recommendation engines, follows this core workflow: collect data, prepare it, train a model, evaluate it, then deploy it to make predictions.

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

# The canonical ML workflow
X = features  # input data
y = labels    # what to predict
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
model = LinearRegression()
model.fit(X_train, y_train)
score = model.score(X_test, y_test)
print(f'Accuracy: {score:.2f}')

Supervised: spam detection, house price prediction. Unsupervised: customer segmentation, anomaly detection.

scikit-learn is Python's most popular ML library because everything follows one consistent API: every model has fit() to train, predict() for supervised output, and transform() for preprocessing. Once you learn this pattern, you can switch between dozens of models — linear regression, random forest, SVM — with almost no extra learning. This consistency is why scikit-learn is the best starting point for ML.

from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()          # preprocessing: transform()
X_scaled = scaler.fit_transform(X)

model = RandomForestClassifier()   # supervised: fit/predict
model.fit(X_scaled, y)
predictions = model.predict(X_scaled)
print(predictions[:5])

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier()
model.fit(X, y)

If you train and test on the same data, the model can 'cheat' by memorizing answers — and fail on new data. The fix: hold out a test set the model never sees during training. train_test_split randomly divides your data (typically 80% train, 20% test). You train on the train set, evaluate on the test set, and that score tells you how the model will perform on truly new data.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
print('Train size:', len(X_train))
print('Test size:', len(X_test))

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

Linear regression finds the best straight line through your data — the relationship between input variables and a numeric output. It predicts things like prices, temperatures, and sales numbers. The model learns coefficients (slopes) for each feature and an intercept (where the line starts). Despite being simple, it's interpretable, fast, and often the first model you should try for any numeric prediction task.

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X, y)
print('Coefficients:', model.coef_)
print('Intercept:', model.intercept_)
prediction = model.predict([[5]])
print('Prediction for x=5:', prediction[0])

model = LinearRegression()
model.fit(X, y)
print(model.intercept_)

Features are the inputs your model learns from — and their quality matters more than the algorithm. Feature engineering means creating useful new features (combining, transforming, extracting), scaling numeric values so they're comparable, and encoding categories into numbers. A well-engineered simple model often beats a poorly-featured complex one. This is where domain knowledge meets ML.

from sklearn.preprocessing import StandardScaler, OneHotEncoder

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X_numeric)
print('Scaled mean:', X_scaled.mean(), 'std:', X_scaled.std())

from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X)

ML models need numbers, but real data has categories — 'red', 'blue', 'green'. One-hot encoding converts each category into a binary column (is_red, is_blue, is_green). This prevents the model from wrongly assuming 'green' > 'blue' (which ordinal label encoding would imply). For many categories, target encoding or embeddings may be better, but one-hot is the safe default for small cardinality.

from sklearn.preprocessing import OneHotEncoder

colors = [['red'], ['blue'], ['green'], ['red']]
encoder = OneHotEncoder(sparse_output=False)
encoded = encoder.fit_transform(colors)
print(encoded)

enc = OneHotEncoder(sparse_output=False)
X_encoded = enc.fit_transform([['a'],['b'],['c']])

Despite the name, logistic regression is a classification model — it predicts categories, not numbers. It outputs a probability between 0 and 1 (through the sigmoid function), then applies a threshold (usually 0.5) to decide the class. It's the go-to first model for binary problems like spam/not-spam, churn/no-churn, and yes/no predictions, because it's fast, interpretable, and gives calibrated probabilities.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X, y)
probabilities = model.predict_proba(X)
predictions = model.predict(X)
print('Probabilities:', probabilities[:3])
print('Predictions:', predictions[:3])

probs = model.predict_proba(X)

Accuracy (correct / total) sounds perfect but lies when classes are imbalanced. In fraud detection, 99% accuracy is useless if it misses every fraud case. Precision answers 'of my positives, how many were right?' Recall answers 'of actual positives, how many did I catch?' F1 balances both. The confusion matrix shows all four outcomes: true positives, true negatives, false positives, false negatives.

from sklearn.metrics import classification_report, confusion_matrix

y_true = [0, 1, 1, 0, 1, 0, 1, 1]
y_pred = [0, 1, 0, 0, 1, 1, 1, 1]
print(confusion_matrix(y_true, y_pred))
print(classification_report(y_true, y_pred))

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_true, y_pred)

Underfitting: the model is too simple and misses real patterns. Overfitting: the model is too complex and memorizes noise instead of signal. The telltale sign: great training score, terrible test score. Fixes include more data, simpler models, cross-validation, and regularization — which penalizes complexity so the model stays general. This is the single most important concept for building models that work in the real world.

from sklearn.linear_model import Ridge

# Ridge adds L2 regularization
model = Ridge(alpha=1.0)
model.fit(X_train, y_train)
print('Train score:', model.score(X_train, y_train))
print('Test score:', model.score(X_test, y_test))

Overfitting — the model memorized the training data.

A single train/test split can be lucky (or unlucky). Cross-validation fixes this by splitting data into k folds, training k times — each time using a different fold as the test set — then averaging the scores. The result is a robust, honest estimate of your model's performance. 5-fold or 10-fold is standard. This is how serious ML practitioners report results.

from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier()
scores = cross_val_score(model, X, y, cv=5)
print('Scores:', scores)
print('Mean:', scores.mean(), 'Std:', scores.std())

scores = cross_val_score(model, X, y, cv=10)
print(scores.mean())

Decision trees make predictions by asking a series of yes/no questions — 'is age > 30?', 'is income > 50k?' — until reaching a leaf with an answer. They're intuitive, require little data prep, handle both numeric and categorical features, and are fully interpretable: you can trace exactly why a prediction was made. The downside is they overfit easily, which is why ensembles (next lesson) were invented.

from sklearn.tree import DecisionTreeClassifier

model = DecisionTreeClassifier(max_depth=3)
model.fit(X, y)
importances = model.feature_importances_
print('Feature importances:', importances)

model = DecisionTreeClassifier(max_depth=2)
model.fit(X, y)

A random forest is an ensemble of many decision trees that vote on the final answer. Each tree sees a random subset of data and features, so they make different mistakes — and averaging cancels those mistakes out. The result: higher accuracy and far less overfitting than any single tree. Random forests are the workhorse of tabular ML: strong defaults, minimal tuning, and robust to messy data.

from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
accuracy = model.score(X_test, y_test)
print(f'Accuracy: {accuracy:.3f}')

model = RandomForestClassifier(n_estimators=200)
model.fit(X, y)

K-means is unsupervised — it finds natural groupings in unlabeled data. The algorithm picks k centers, assigns each point to the nearest center, moves centers to the average of their points, and repeats until stable. Use it for customer segmentation, image compression, and anomaly detection. The key question is choosing k, often done with the elbow method or domain knowledge.

from sklearn.cluster import KMeans

model = KMeans(n_clusters=3, random_state=42)
model.fit(X)
labels = model.labels_
centers = model.cluster_centers_
print('Labels:', labels)
print('Centers:', centers)

model = KMeans(n_clusters=5)
model.fit(X)
labels = model.labels_

How many clusters should you pick? The elbow method runs k-means for several k values and plots inertia (how tightly packed the clusters are). Inertia always drops as k increases, but the 'elbow' — where the drop suddenly slows — suggests the natural number of clusters. More clusters than the elbow often splits real groups; fewer lumps them together.

from sklearn.cluster import KMeans

inertias = []
for k in range(1, 8):
    model = KMeans(n_clusters=k, random_state=42)
    model.fit(X)
    inertias.append(model.inertia_)
print(inertias)

The data likely has 4 natural clusters.

Imbalanced data means one class vastly outnumbers another — 99% normal transactions, 1% fraud. A model that always predicts 'normal' gets 99% accuracy but is useless. Fixes: oversample the minority (SMOTE creates synthetic examples), undersample the majority, or use class_weight='balanced' so the model pays more attention to rare classes. Fraud, rare disease, and churn all need this.

from sklearn.ensemble import RandomForestClassifier

# class_weight makes minority class mistakes cost more
model = RandomForestClassifier(class_weight='balanced', random_state=42)
model.fit(X_train, y_train)
print('Minority recall:', model.score(X_test, y_test))

model = LogisticRegression(class_weight='balanced')
model.fit(X, y)

A Pipeline chains preprocessing steps and the model into one object — so scaling, encoding, and training happen together, consistently. This prevents the #1 ML bug: data leakage, where you accidentally fit preprocessing on the test data. With a pipeline, every step is applied correctly to train and test separately, and your whole workflow is one clean, reusable object.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LogisticRegression())
])
pipeline.fit(X_train, y_train)
score = pipeline.score(X_test, y_test)
print(f'Pipeline score: {score:.3f}')

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', DecisionTreeClassifier())
])

Models have dials — hyperparameters like max_depth or n_estimators — and tuning them can meaningfully improve performance. GridSearchCV systematically tries every combination in a grid, evaluates each with cross-validation, and returns the best. It's brute force but reliable. RandomSearchCV samples random combos and is better when the search space is large.

from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier

param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [None, 5, 10]
}
grid = GridSearchCV(RandomForestClassifier(), param_grid, cv=5)
grid.fit(X, y)
print('Best params:', grid.best_params_)
print('Best score:', grid.best_score_)

grid = GridSearchCV(DecisionTreeClassifier(), {'max_depth':[3,5,10]}, cv=5)
grid.fit(X, y)

A model that works but can't be explained is dangerous — especially in healthcare, finance, and hiring. Interpretability means knowing which features drive predictions. Tree models give feature_importances_ directly. For any model, permutation importance shuffles a feature and measures how much the score drops. Explaining models builds trust, catches bias, and satisfies regulatory requirements.

from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier()
model.fit(X, y)
importances = model.feature_importances_
for name, imp in zip(feature_names, importances):
    print(f'{name}: {imp:.3f}')

print(model.feature_importances_)

Training a model is expensive — you don't want to redo it every time. joblib saves a trained model to disk, and you load it later to make predictions instantly. This is how models go to production: train once (offline), save, load (online) when serving predictions. Without this, your ML work stays stuck in a notebook.

import joblib

# Save
model.fit(X, y)
joblib.dump(model, 'model.joblib')

# Load
loaded_model = joblib.load('model.joblib')
prediction = loaded_model.predict([[5]])
print('Prediction:', prediction)

joblib.dump(model, 'model.joblib')
model2 = joblib.load('model.joblib')

Your capstone: a complete ML project from raw data to a saved model. Load a dataset, clean it, engineer features, split train/test, build a pipeline, tune with GridSearchCV, evaluate with cross-validation, interpret feature importances, and save the final model. This is the exact workflow a data scientist does on the job — and it's portfolio-ready.

import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier

df = pd.read_csv('data.csv')
X, y = df.drop('target', axis=1), df['target']

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', RandomForestClassifier())
])
param_grid = {'model__n_estimators': [50, 100], 'model__max_depth': [5, 10]}
grid = GridSearchCV(pipeline, param_grid, cv=5)
grid.fit(X, y)
print('Best:', grid.best_score_)
joblib.dump(grid.best_estimator_, 'final_model.joblib')

pipeline = Pipeline([('scaler', StandardScaler()), ('model', RandomForestClassifier())])