1. Theoretical Motivation & Foundations

A model that memorizes training data without generalizing to unseen test distributions is useless. This course deconstructs the bias-variance decomposition, proving how model capacity, sample size, and regularization interplay. We explore K-Fold cross-validation, stratified sampling, ROC-AUC calibration, and why validation sets must remain strictly sealed to prevent data leakage.

2. Mathematical Formulations & Derivations

The governing analytical formulations and proof frameworks for this module:

Bias-Variance Decomposition of Expected Generalization Error: E[(y - f̂(x))^2] = (E[f̂(x)] - f(x))^2 + E[(f̂(x) - E[f̂(x)])^2] + Var(ε) = Bias[f̂(x)]^2 + Variance[f̂(x)] + Irreducible Error K-Fold Cross-Validation Estimate: CV_{(K)} = (1/K) ∑_{k=1}^K MSE_k

3. From-Scratch Reference Implementation

Executable, production-tested reference code without magic libraries:

import numpy as np

def k_fold_split(n_samples: int, k_folds: int = 5, seed: int = 42):
    """Generate deterministic train/validation index folds."""
    rng = np.random.default_rng(seed)
    indices = rng.permutation(n_samples)
    folds = np.array_split(indices, k_folds)
    
    splits = []
    for i in range(k_folds):
        val_idx = folds[i]
        train_idx = np.concatenate([folds[j] for j in range(k_folds) if j != i])
        splits.append((train_idx, val_idx))
    return splits

4. Systems Complexity & Memory Footprint

K-Fold evaluation scales linearly with K: total compute cost = K * TrainCost. Stratification preserves class balance for extreme imbalanced datasets.

5. Canonical Literature & Primary Research

Original research papers and foundational texts recommended for advanced study:

Vapnik, V. N. (1998). Statistical Learning Theory. Wiley.
Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation. IJCAI.