Machine Learning Algorithms — Explained Simply

A supervised algorithm that models a continuous target as a weighted linear combination of input features plus a bias term: y = w·x + b.

Mean Squared Error (MSE) — the average of squared differences between predicted and actual values.

Squaring penalises large errors more, is differentiable everywhere (easy gradients), and yields a convex loss with a unique closed-form solution.

w = (XᵀX)⁻¹Xᵀy. It solves for weights directly without iteration, but inverting XᵀX is O(n³) and fails if features are collinear.

When the feature count is large (inversion too costly) or the data doesn't fit in memory — gradient descent scales better.

Linearity, independence of errors, homoscedasticity (constant error variance), normally distributed residuals, and little multicollinearity.

When features are highly correlated, making coefficient estimates unstable and hard to interpret. Detect it with the Variance Inflation Factor (VIF).

The proportion of variance in the target explained by the model, from 0 to 1 (can be negative for a bad model).

R² never decreases when you add features; adjusted R² penalises extra features, so it only rises if the new feature genuinely helps.

Ridge adds an L2 penalty (shrinks weights toward zero); Lasso adds an L1 penalty that can drive weights exactly to zero, performing feature selection.

L1 produces sparse solutions — it zeros out unimportant features — whereas L2 only shrinks them.

A simple linear model has high bias, low variance; adding polynomial terms lowers bias but raises variance and overfitting risk.

Add polynomial or interaction features, transform variables (log, sqrt), or switch to a non-linear model.

Not for the closed-form solution, but gradient descent converges faster and regularisation penalties are fairer when features are on the same scale.

Non-constant error variance across the range of predictions. Spot it with a residuals-vs-fitted plot showing a fan/cone shape.

When outliers shouldn't dominate the loss — MAE treats all errors linearly, so it's more robust to outliers.

It's the expected change in the target for a one-unit increase in that feature, holding all others constant.

Not well — it outputs unbounded values and is sensitive to outliers; logistic regression is the right tool for classification.

It means perfect multicollinearity or more features than samples; use regularisation (Ridge) or the pseudo-inverse instead.

Use residual plots, Cook's distance, or leverage scores; then cap, remove, or switch to a robust regression / MAE loss.

A linear classifier that models the probability of a class by passing a linear score through the sigmoid (logistic) function.

It regresses the log-odds (logit) of the outcome linearly on the features; the probability mapping then enables classification.

It squashes any real number into (0, 1), turning the linear score into a probability.

Log-loss (binary cross-entropy), which is convex and penalises confident wrong predictions heavily.

MSE with the sigmoid is non-convex (many local minima) and produces weak gradients; log-loss is convex and trains reliably.

Odds = p/(1-p); log-odds (logit) is its natural log. Logistic regression is linear in the log-odds.

exp(coefficient) is the multiplicative change in odds for a one-unit increase in that feature.

Use softmax (multinomial logistic regression) or one-vs-rest, training one binary classifier per class.

Linear in feature space — a line/plane — unless you add polynomial or interaction features.

Use class weights, resampling (SMOTE/undersampling), adjust the decision threshold, and evaluate with precision/recall or PR-AUC rather than accuracy.

The probability cut-off for the positive class; lowering it raises recall at the cost of precision, and vice versa.

An L1 or L2 penalty on the weights to prevent overfitting; C in scikit-learn is the inverse of regularisation strength.

ROC plots true-positive vs. false-positive rate across thresholds; AUC is the area under it — the probability a random positive outranks a random negative.

On highly imbalanced data, where ROC-AUC can look optimistic and precision-recall better reflects positive-class performance.

It helps gradient-based solvers converge and makes regularisation fair, though it's not strictly required for correctness.

Precision = of predicted positives, how many are correct; recall = of actual positives, how many were caught.

Linearity between features and log-odds, independent observations, little multicollinearity, and (ideally) large sample size.

Weights diverge to infinity (the likelihood keeps improving); regularisation prevents this.

It's fast, interpretable, outputs calibrated probabilities, and is hard to beat on linearly separable or text problems.

The harmonic mean of precision and recall; use it when you need a single metric balancing both, especially under imbalance.

A non-parametric algorithm that predicts a point's label from the majority vote (or average) of its K closest training points.

It does no real training — it just stores the data and defers all computation to prediction time.

By cross-validation; small K is noisy/overfits, large K oversmooths. Odd K avoids ties in binary classification.

Distances are dominated by large-range features; without scaling, a feature in thousands swamps one in fractions.

Euclidean (most common), Manhattan, Minkowski, cosine, or Hamming for categorical data.

In high dimensions all points become roughly equidistant, so 'nearest' loses meaning and accuracy degrades.

O(n·d) per query for brute force (n samples, d features) — slow on large datasets.

Use KD-trees or Ball-trees for low dimensions, or approximate nearest-neighbour libraries (FAISS, Annoy) for high dimensions.

Yes — it averages (or distance-weights) the target values of the K nearest neighbours.

Closer neighbours get larger votes (weight ∝ 1/distance), reducing sensitivity to the exact choice of K.

Encode them (one-hot) and use a suitable metric like Hamming, since Euclidean distance is meaningless on raw categories.

Yes for small K; a single noisy neighbour can flip the prediction. Larger K or distance weighting mitigates it.

Small K = low bias, high variance; large K = high bias, low variance.

Poorly — the majority class dominates votes. Use distance weighting, resampling, or class-balanced voting.

Only storing the data (and optionally building a search index); there are no learned parameters.

Simple, no training, naturally handles multi-class, and adapts to complex decision boundaries.

Slow at prediction, memory-heavy, needs scaling, and suffers in high dimensions.

Impute them first (KNN itself is even used as an imputer), since distance can't be computed with gaps.

The model perfectly fits training data (zero training error) but is highly sensitive to noise — classic overfitting.

Filling a missing value using the average/most-common value among the sample's nearest neighbours.

A probabilistic classifier applying Bayes' theorem with the 'naive' assumption that features are conditionally independent given the class.

P(class|features) = P(features|class)·P(class) / P(features); Naive Bayes picks the class maximising the numerator.

Because it assumes all features are independent given the class — usually false, but it still works surprisingly well.

Adding a small count (e.g. 1) to every feature-class tally so an unseen feature doesn't make the whole probability zero.

If a feature value never appeared with a class in training, its probability is 0, zeroing the entire product — fixed by smoothing.

Multinomial (counts, e.g. word frequencies), Bernoulli (binary presence/absence), and Gaussian (continuous features).

Multinomial NB with word counts or TF-IDF; Bernoulli works when only word presence matters.

Training is a single pass counting frequencies; prediction is just multiplying probabilities — both linear in data size.

Multiplying many small probabilities underflows to zero; summing their logs is numerically stable.

No — it performs well even on small datasets because it estimates few, simple parameters.

That each continuous feature is normally distributed within each class, estimated by its per-class mean and variance.

Generative — it models P(features|class) and the class prior, unlike discriminative logistic regression.

When features are strongly correlated, causing over-confident (poorly calibrated) probability estimates.

Often not — the independence assumption pushes probabilities toward 0 or 1, though the ranking/classification can still be good.

Use the matching likelihood per feature type (Gaussian for continuous, multinomial/Bernoulli for categorical) and combine.

NB (generative) trains faster and needs less data; logistic regression (discriminative) usually wins with more data and correlated features.

Spam filtering, sentiment analysis, document/topic classification — high-dimensional text problems.

The base rate of each class, typically estimated as its frequency in the training set.

Discretise them into bins, or use kernel density estimation for the per-class likelihood.

Classification only needs the correct class to have the highest score, not exact probabilities — so independence errors often cancel out.

A classifier that finds the hyperplane maximising the margin — the distance to the nearest points of each class.

The training points lying on or inside the margin; they alone define the boundary — other points don't matter.

The gap between the decision boundary and the closest points; SVM maximises it for better generalisation.

Computing dot products in a high-dimensional space implicitly, letting SVM build non-linear boundaries without explicit transformation.

Linear, polynomial, RBF (Gaussian), and sigmoid; RBF is the popular default for non-linear data.

The trade-off between a wide margin and misclassifications; large C = low tolerance for errors (risk overfitting), small C = wider, softer margin.

The reach of a single training point; high gamma = tight, wiggly boundaries (overfit), low gamma = smooth, broad boundaries.

Hard-margin allows no misclassification (only for separable data); soft-margin permits some via slack variables, controlled by C.

Per-point penalties that allow some points to violate the margin, enabling SVM to handle non-separable data.

Distance- and dot-product-based kernels are dominated by large-scale features, so standardising is essential.

Yes; use class weights (class_weight='balanced') so the minority class isn't ignored.

It's inherently binary, so it uses one-vs-one or one-vs-rest schemes to combine multiple binary SVMs.

max(0, 1 - y·f(x)) — it penalises points inside the margin or misclassified, and is the loss SVM minimises.

Training is roughly O(n²)–O(n³) in samples; use LinearSVC or SGD-based approximations for large data.

The regression variant fitting a tube of width epsilon around the data, penalising only points outside it.

It expresses the optimisation in terms of dot products, enabling the kernel trick and depending only on support vectors.

For high-dimensional, sparse data like text, where data is often already linearly separable and linear SVM is fast.

Grid or randomised search with cross-validation, often over a log scale.

Pros: strong in high dimensions, memory-efficient (uses support vectors), flexible kernels. Cons: slow on big data, sensitive to scaling and C/gamma, no direct probabilities.

Not natively — it fits a logistic calibration (Platt scaling) on the decision scores via cross-validation.

A model that splits data with a sequence of feature-threshold questions, forming a tree whose leaves give predictions.

By the split that most reduces impurity — measured by Gini impurity or entropy (information gain).

The probability of misclassifying a random sample if labelled by the node's class distribution; 0 means pure.

Entropy measures node disorder; information gain is the entropy reduction from a split — the tree maximises it.

They usually give similar trees; Gini is slightly faster (no logs), entropy can be marginally more balanced.

It sorts values and tests thresholds between them, choosing the cut-off that best reduces impurity.

By minimising variance (or MSE) of the target within the resulting child nodes.

Unconstrained, they grow until every leaf is pure — memorising noise. They're low-bias, high-variance.

Limit max_depth, min_samples_split/leaf, max_leaf_nodes, or post-prune with cost-complexity pruning.

Removing branches that add little predictive value to simplify the tree and improve generalisation.

Pruning that penalises tree size via a parameter alpha, trading training fit against complexity.

No — splits are based on thresholds per feature, so monotonic scaling doesn't change them.

Via surrogate splits or by sending missing samples down a default branch; some implementations learn the best direction.

By the total impurity reduction each feature contributes across all its splits, normalised to sum to 1.

Yes — you can read the path of yes/no rules from root to leaf, which stakeholders value.

The tree picks the locally best split at each node, which may not yield the globally optimal tree.

Yes — successive splits on different features naturally model interactions along a path.

By partitioning categories into subsets; scikit-learn requires encoding, while some libraries handle them natively.

Small changes in the data can produce very different splits and tree structures — motivating ensembles.

When interpretability and an explainable decision path matter more than raw accuracy.

An ensemble of decision trees trained on bootstrap samples with random feature subsets, whose predictions are aggregated by vote or average.

Bootstrap Aggregating — training models on random resamples (with replacement) and averaging them to reduce variance.

It decorrelates the trees so they don't all rely on the same dominant feature, which lowers ensemble variance.

Averaging many decorrelated high-variance trees cancels their individual errors, slashing variance with little added bias.

Each tree is tested on the ~37% of samples it didn't train on; averaging gives a built-in validation estimate without a holdout set.

With bootstrap sampling, the chance a given sample is never picked in n draws tends to 1/e ≈ 0.368.

By mean impurity decrease across trees, or more reliably by permutation importance (drop in accuracy when a feature is shuffled).

n_estimators, max_features per split, max_depth, min_samples_leaf, and bootstrap on/off.

No — more trees only stabilise the estimate (diminishing returns); they don't increase overfitting, just compute.

Forest trains independent trees in parallel (reduces variance); boosting trains trees sequentially on errors (reduces bias) and usually scores higher but tunes harder.

No — like single trees, it's invariant to monotonic feature scaling.

Mildly, with very deep trees on noisy data, but far less than a single tree thanks to averaging.

It's robust to noise; missing values need imputation in scikit-learn, though averaging tolerates some outliers well.

Good accuracy out of the box, little tuning, handles mixed feature types, and gives importances and OOB estimates.

How many features each split may consider; lower values increase tree diversity and reduce correlation.

Yes — each tree predicts a number and the forest averages them.

Usually yes — impurity importance is biased toward high-cardinality features; permutation importance measures real predictive impact.

Trees are independent, so they train across CPU cores simultaneously (n_jobs=-1).

Larger memory/inference cost than one tree, less interpretable, and typically a notch below tuned boosting on accuracy.

Average the class proportions in the leaves across all trees (predict_proba).

An ensemble that adds weak learners (shallow trees) sequentially, each fitting the residual errors of the current model.

Bagging trains independent models in parallel to cut variance; boosting trains models sequentially on prior errors to cut bias.

Each new tree fits the negative gradient of the loss w.r.t. current predictions — generalising boosting to any differentiable loss.

It shrinks each tree's contribution; smaller rates need more trees but generalise better (it trades off with n_estimators).

Each only needs to improve slightly; many small corrections combine into a strong learner without overfitting fast.

Too many trees or too-high learning rate overfit; use early stopping, lower learning rate, subsampling, and regularisation.

Halting training when validation performance stops improving for a set number of rounds, preventing overfitting.

Fitting each tree on a random subsample of rows (and sometimes columns), adding randomness that improves generalisation.

L1/L2 regularisation, second-order (Newton) optimisation, sparsity-aware splits, parallelised tree building, and built-in CV.

LightGBM grows trees leaf-wise (faster, great on large data) and uses histogram binning; XGBoost grows level-wise and is often steadier on small data.

Level-wise expands all nodes at a depth (balanced); leaf-wise expands the highest-loss leaf (deeper, more accurate, more overfit-prone).

Modern implementations learn a default direction at each split for missing data, requiring no imputation.

learning_rate, n_estimators, max_depth, subsample, colsample_bytree, and regularisation (lambda/alpha).

Inversely — halving the learning rate roughly requires doubling the trees for similar fit.

No — it's tree-based and invariant to monotonic feature scaling.

It captures complex feature interactions, handles mixed types, and squeezes out top accuracy with proper tuning.

Any differentiable loss — squared error for regression, log-loss for classification, ranking losses, custom objectives.

Via lambda (L2), alpha (L1), gamma (min loss reduction to split), max_depth, and min_child_weight.

Trees are sequential, but the split-finding within each tree is parallelised across features/cores.

More hyperparameters, sensitive to tuning, slower to train sequentially, and easier to overfit if unchecked.

An unsupervised algorithm that partitions data into K clusters by minimising the distance of points to their cluster centroid.

Initialise K centroids, assign each point to the nearest one, recompute centroids as the mean of their members, and repeat until assignments stabilise.

Within-cluster sum of squares (inertia) — the total squared distance from points to their centroids.

The elbow method (inertia vs. K), silhouette score, or gap statistic; plus domain knowledge.

Plot inertia against K and pick the 'elbow' where adding clusters stops sharply reducing inertia.

A measure from -1 to 1 of how well each point fits its cluster vs. the nearest other cluster; higher is better.

Random starts can converge to poor local optima; k-means++ spreads initial centroids to give better, more consistent results.

Chooses initial centroids probabilistically far apart, speeding convergence and improving final cluster quality.

K-Means uses Euclidean distance, so unscaled large-range features dominate the clustering.

Clusters are roughly spherical, similar in size and density, and well separated.

On elongated, non-convex, varying-density, or differently-sized clusters — DBSCAN or GMM handle those better.

No — it converges to a local optimum, so it's run multiple times (n_init) and the best is kept.

O(n·K·d·i) for n points, K clusters, d dimensions, i iterations — efficient and scalable.

K-Medoids uses actual data points as centres and is more robust to outliers, but is slower than mean-based K-Means.

K-Means is a hard-assignment special case of GMM with equal spherical covariances; GMM gives soft, probabilistic memberships.

They pull centroids toward themselves because the mean is sensitive to extreme values.

A variant updating centroids on small random batches, trading a little accuracy for much faster training on big data.

Not directly (mean is undefined); use K-Modes for categorical or K-Prototypes for mixed data.

Profile each cluster by its centroid feature values and size to give it a business meaning (e.g. 'high-value, low-frequency').

Customer segmentation, image colour quantisation, document grouping, and anomaly pre-processing.

A method that builds a tree (dendrogram) of nested clusters, either merging bottom-up or splitting top-down.

Agglomerative (bottom-up) starts with each point as a cluster and merges; divisive (top-down) starts with one cluster and splits. Agglomerative is far more common.

A tree diagram showing the order and distance of merges; cutting it at a height yields a chosen number of clusters.

Cut the dendrogram where the vertical merge distances are largest (the biggest jump), or use a target count.

The rule for measuring distance between clusters — single, complete, average, or Ward.

Uses the minimum distance between clusters; can chain points into long, straggly clusters.

Uses the maximum distance between clusters; tends to produce compact, equal-diameter clusters.

Uses the mean pairwise distance between clusters — a compromise between single and complete.

Merges the pair that minimises the increase in total within-cluster variance; produces balanced, compact clusters.

No — it builds the full hierarchy, and you decide the cluster count afterward by cutting the dendrogram.

Typically O(n²) memory and O(n²–n³) time, limiting it to small/medium datasets.

Hierarchical needs no K up front, gives a full hierarchy, and finds non-spherical shapes, but scales far worse than K-Means.

Yes — like K-Means it depends on distances, so features should be standardised.

Yes, especially with single linkage, unlike K-Means which assumes spherical clusters.

Sensitive — outliers can form their own branches or distort merges, especially with single linkage.

Yes — given the data, metric, and linkage, the dendrogram is fixed (no random initialisation).

Euclidean, Manhattan, cosine, correlation, etc.; Ward's linkage requires Euclidean.

A measure of how faithfully the dendrogram preserves the original pairwise distances; higher means a better representation.

Gene-expression analysis, taxonomy building, document organisation, and any setting wanting an interpretable hierarchy.

Yes — hierarchical clustering on a sample can suggest K, which then seeds a scalable K-Means run.

A density-based clustering algorithm that groups densely packed points and labels sparse points as noise.

eps (neighbourhood radius) and minPts (minimum points within eps to form a dense region).

A point with at least minPts neighbours within eps — it can grow a cluster.

A point within eps of a core point but with fewer than minPts neighbours itself — it joins a cluster but can't extend it.

A point that is neither core nor reachable from any core point; DBSCAN labels it as an outlier.

It finds arbitrary cluster shapes, doesn't need K specified, and explicitly detects outliers.

No — the number emerges from the data density given eps and minPts.

Plot the sorted distance to each point's k-th nearest neighbour (k-distance graph) and pick eps at the 'knee'.

A common heuristic is minPts ≥ dimensions + 1, often 2×dimensions; larger values give denser, more robust clusters.

It struggles when clusters have very different densities, since a single eps can't fit all.

HDBSCAN, which builds a hierarchy over densities and selects clusters across multiple scales.

Yes — eps is a distance, so unscaled features distort the neighbourhood definition.

O(n log n) with a spatial index (KD/Ball tree), or O(n²) in the worst case.

Mostly — only the cluster assignment of border points reachable from multiple clusters can depend on processing order.

Poorly — distances concentrate (curse of dimensionality), making a meaningful eps hard to choose.

Natively — it leaves low-density points unassigned as noise, which is useful for anomaly detection.

A point is density-reachable if there's a chain of core points connecting it within eps to a starting core point.

If density is uniform it may label everything one cluster or all noise — it needs density contrast to work.

DBSCAN scales better, finds noise, and needs no K; hierarchical gives a full interpretable tree but is O(n²).

Spatial/GPS hotspot detection, anomaly and fraud detection, and clustering data with irregular shapes.

Principal Component Analysis — a linear technique that projects data onto new orthogonal axes capturing maximum variance, reducing dimensionality.

A direction (linear combination of features) along which the data varies most; components are ordered by variance explained.

As the eigenvectors of the data's covariance matrix (or via SVD), with eigenvalues giving the variance each explains.

PCA maximises variance, so unscaled large-variance features would dominate the components regardless of importance.

Keep enough to reach a target cumulative explained variance (e.g. 95%), or use a scree-plot elbow.

The fraction of total variance captured by each component, used to decide how many to retain.

No — they're orthogonal and therefore uncorrelated by construction.

Unsupervised — it ignores labels and only considers feature variance.

A plot of explained variance per component; the 'elbow' suggests how many components to keep.

Visualisation, noise reduction, decorrelating features, and speeding up downstream models by cutting dimensions.

Feature selection keeps a subset of original features; PCA creates new combined features, losing direct interpretability.

PCA is typically computed via Singular Value Decomposition of the centred data, which is more numerically stable than eigendecomposition.

No — it's linear; use Kernel PCA, t-SNE, UMAP, or autoencoders for non-linear manifolds.

PCA applied in a kernel-induced high-dimensional space, enabling non-linear dimensionality reduction.

Yes — reconstructing data from the top components discards low-variance directions that often correspond to noise.

Components describe variance around the mean; without centring the first component would just point toward the mean.

Sometimes — loadings show feature contributions — but combined axes are generally harder to explain than raw features.

No — it can discard label-relevant low-variance signal; it mainly helps with speed, collinearity, and visualisation.

The difference between original data and its reconstruction from kept components — it grows as you drop more components.

PCA is unsupervised, maximising variance; LDA is supervised, maximising class separation.

t-distributed Stochastic Neighbor Embedding — a non-linear technique that maps high-dimensional data to 2D/3D for visualisation, preserving local neighbourhoods.

Visualisation only — its output coordinates aren't meaningful features and shouldn't feed a downstream model.

Roughly the effective number of neighbours each point considers; typical values are 5–50 and it noticeably changes the plot.

t-SNE expands dense regions and contracts sparse ones, so cluster sizes and inter-cluster distances aren't faithful.

Its heavy tails prevent crowding, letting moderately distant points spread out and revealing clearer clusters.

PCA is linear, fast, and deterministic (good for preprocessing); t-SNE is non-linear, slower, stochastic, and purely for visualising local structure.

UMAP is faster, scales better, and preserves more global structure, while t-SNE often shows cleaner local clusters.

No — it depends on random initialisation, so different runs/seeds can produce different layouts.

Naively O(n²); Barnes-Hut t-SNE reduces it to about O(n log n) for larger datasets.

Often yes — reducing to ~50 dimensions with PCA first speeds t-SNE and reduces noise.

No — only local neighbourhoods are reliable; the gaps between far-apart clusters are not meaningful.

It minimises the KL divergence between pairwise-similarity distributions in the high- and low-dimensional spaces.

Not naturally — t-SNE has no simple transform for unseen points; UMAP supports this better.

With a bad perplexity or too few iterations it can fragment uniform data into apparent clusters — always vary parameters.

An initial phase that inflates high-dim similarities so clusters form tight, well-separated groups before fine-tuning.

Two or three, since its purpose is human-viewable visualisation.

For large datasets, when you need speed, the ability to transform new data, or better-preserved global structure.

Visualising word/image embeddings, single-cell genomics, and inspecting neural-network feature spaces.

No — it normalises local density, so you can't read relative cluster density from the plot.

Scaling/standardisation and often a PCA pre-reduction; the metric should suit the data (e.g. cosine for embeddings).

A model-free, off-policy reinforcement-learning algorithm that learns the value of taking an action in a state (the Q-value) to derive an optimal policy.

Q(s, a) is the expected cumulative discounted reward from taking action a in state s and acting optimally thereafter.

Q(s,a) ← Q(s,a) + α[r + γ·maxₐ′Q(s′,a′) − Q(s,a)], moving the estimate toward the observed reward plus discounted best next value.

How much each update shifts the Q-value toward the new estimate; high α learns fast but unstably, low α is slow but smooth.

How much future rewards are valued; near 0 is myopic, near 1 is far-sighted.

Balancing trying new actions to discover reward (explore) against using known good actions (exploit).

With probability ε pick a random action, otherwise the current best; ε is usually decayed over time.

It learns the optimal policy's value (via max over next actions) regardless of the exploratory policy actually generating the data.

It expresses the optimal Q-value recursively: Q*(s,a) = E[r + γ·maxₐ′Q*(s′,a′)].

One that learns directly from experience without modelling the environment's transition or reward dynamics.

It needs an entry per state-action pair, which explodes for large or continuous state spaces — motivating function approximation (DQN).

A mapping from states to actions; in Q-learning it's derived by taking the highest-Q action in each state.

A scalar feedback signal the environment returns after each action, which the agent learns to maximise cumulatively.

Reward is immediate; return is the (discounted) sum of future rewards the agent actually tries to maximise.

SARSA is on-policy (uses the action actually taken next); Q-learning is off-policy (uses the max next action). SARSA is more conservative.

Yes, to the optimal Q-values for finite MDPs given sufficient exploration and a suitably decaying learning rate.

A complete sequence of states, actions, and rewards from a start state to a terminal state.

The RL framework of states, actions, transition probabilities, rewards, and a discount factor, with the Markov (memoryless) property.

That the next state and reward depend only on the current state and action, not the full history.

Start with high ε and decay it over episodes so the agent explores first and exploits its learned policy later.

Q-learning where a neural network approximates the Q-function, enabling RL on large or high-dimensional (e.g. pixel) state spaces.

Tables can't cover huge/continuous state spaces; a network generalises Q-values across similar states.

Storing transitions in a buffer and training on random minibatches, which breaks correlation between consecutive samples and improves stability.

A periodically-updated copy of the Q-network used to compute targets, preventing the moving-target instability of bootstrapping off itself.

Correlated samples plus a target that shifts with the same weights create feedback loops; replay and a target network fix this.

The instability risk when combining function approximation, bootstrapping, and off-policy learning together.

The squared temporal-difference error between predicted Q and the target r + γ·maxₐ′Q_target(s′,a′).

The max operator systematically over-estimates Q-values; Double DQN reduces it by decoupling action selection from evaluation.

A variant that selects the best next action with the online network but evaluates it with the target network, curbing overestimation.

An architecture splitting Q into a state-value stream and an action-advantage stream, improving learning where actions matter little.

Sampling high-error transitions more often so the agent learns faster from its most informative experiences.

It stacks several recent frames (for motion) and processes them with convolutional layers to estimate Q-values per action.

No — it needs a discrete action set for the max; continuous control uses DDPG, TD3, or SAC instead.

It learned to play many Atari games from raw pixels at human level using one architecture — a landmark in deep RL.

Either copied every fixed number of steps (hard update) or blended slowly each step (soft update, Polyak averaging).

To keep memory bounded and favour recent, more relevant experience while still decorrelating samples.

Learning rate, discount γ, replay buffer size, batch size, target-update frequency, and the ε-decay schedule.

DQN generalises across states via a network and needs replay/target tricks; tabular stores exact values and only suits small discrete worlds.

Off-policy — it learns from replayed transitions generated by older policies.

Diverging or oscillating Q-values and collapsing episode reward; remedies include lower learning rate, Double DQN, and reward clipping.

RL algorithms that directly learn a parameterised policy by ascending the gradient of expected reward, rather than learning value functions.

Value-based (e.g. DQN) learn action values then act greedily; policy-based learn the action distribution directly, handling continuous actions and stochastic policies.

A Monte-Carlo policy gradient that scales the gradient of log-probability of actions by the episode's return.

It uses full-episode returns as noisy estimates of action quality, making gradient updates unstable.

A reference value (often the state value) subtracted from the return to reduce gradient variance without adding bias.

A(s,a) = Q(s,a) − V(s): how much better an action is than the state's average — used to focus updates.

A hybrid where an actor learns the policy and a critic learns a value function to provide low-variance feedback (the advantage).

Proximal Policy Optimization — a stable, popular policy-gradient method that clips updates so the policy doesn't change too much per step.

It prevents destructively large policy updates that can collapse performance, keeping each step within a trust region.

Trust Region Policy Optimization — constrains each update by a KL-divergence limit; PPO is a simpler, cheaper approximation of the same idea.

Yes — the policy can output parameters of a continuous distribution (e.g. a Gaussian's mean and variance).

A policy outputting action probabilities; it enables exploration and can be optimal in partially observable or adversarial settings.

Vanilla policy gradients and PPO are on-policy (need fresh data each update); off-policy variants (e.g. SAC) reuse a replay buffer.

Adding the policy's entropy to the objective to encourage exploration and prevent premature convergence to a deterministic policy.

Advantage Actor-Critic methods; A3C runs many parallel workers asynchronously, A2C is its synchronous variant.

To estimate value/advantage so the actor's updates have lower variance than raw returns.

Fine-tuning language models with human feedback typically uses PPO to optimise a reward model of human preferences.

Determining which past actions deserve credit/blame for a delayed reward — central and hard in RL.

On-policy methods discard data after each update and rely on high-variance return estimates, needing many environment interactions.

For continuous or high-dimensional action spaces, when you need a stochastic policy, or for stable training via PPO.

A model of layered interconnected neurons that transforms inputs through weighted sums and non-linear activations to learn complex functions.

A non-linear function applied to each neuron's output; without it, stacked layers collapse into a single linear transform.

ReLU (default for hidden layers), sigmoid, tanh, Leaky ReLU, GELU, and softmax for output probabilities.

It's cheap, avoids saturating gradients for positive inputs, and speeds up training versus sigmoid/tanh.

The algorithm that computes loss gradients w.r.t. every weight via the chain rule, propagating error backward through the layers.

An optimisation that updates weights in the negative gradient direction to minimise the loss.

Gradients shrink toward zero in deep nets (with saturating activations), stalling early-layer learning; ReLU, residual connections, and normalisation help.

Gradients grow uncontrollably, destabilising training; mitigated by gradient clipping and careful initialisation.

It scales weight updates; too high diverges, too low trains slowly — schedules and adaptive optimisers help.

An adaptive optimiser combining momentum and per-parameter learning-rate scaling; a robust default for deep nets.

Randomly zeroing a fraction of neurons during training to prevent co-adaptation and reduce overfitting.

Normalising layer inputs per minibatch to stabilise and speed training and allow higher learning rates.

An epoch is one full pass over the data; a batch is a subset processed at once; an iteration is one weight update on a batch.

It balances the stable gradients of full-batch with the speed and noise (regularising effect) of stochastic updates.

Dropout, L2 weight decay, early stopping, data augmentation, and more training data.

Setting starting weights (e.g. He or Xavier) to keep signal/gradient variance stable and avoid vanishing/exploding gradients.

Cross-entropy for classification, MSE/MAE for regression, with softmax outputs for multi-class.

A network with one sufficiently wide hidden layer can approximate any continuous function — though depth is usually more efficient.

Depth learns hierarchical, compositional features more parameter-efficiently than a single huge layer.

Reusing a model pretrained on a large dataset and fine-tuning it on your smaller task, saving data and compute.

A neural network for grid-like data (images) that uses convolutional filters to learn spatial features hierarchically.

Sliding a small learnable filter across the input and computing dot products to produce a feature map highlighting a pattern.

A pattern (e.g. an edge) can appear anywhere, so reusing the same filter gives translation invariance and far fewer parameters.

The output of applying one filter across the input — a map of where that learned feature is detected.

Downsampling (e.g. max pooling) that shrinks feature maps, adds translation invariance, and cuts computation.

Max pooling keeps the strongest activation (sharp features); average pooling smooths — max is more common in practice.

How many pixels the filter moves each step; a larger stride downsamples the output more.

Adding border pixels (often zeros) so filters can cover edges and control output size ('same' vs. 'valid').

The region of the input that influences a given output unit; it grows with depth, letting deep layers see larger context.

They exploit spatial locality and weight sharing, using far fewer parameters while capturing translation-invariant features.

Early layers detect edges/textures; deeper layers combine them into shapes, parts, and whole objects.

A convolution that mixes channels at each pixel — used to reduce/expand channel depth cheaply (e.g. in Inception).

Fine-tuning a CNN pretrained on ImageNet (e.g. ResNet) for your task, which works well with limited data.

Generating varied training images (flips, crops, rotations, colour jitter) to improve generalisation and reduce overfitting.

They add skip connections so gradients flow through very deep networks, fixing degradation and vanishing gradients.

It stabilises and accelerates training by normalising activations, enabling deeper networks and higher learning rates.

Feature maps are flattened or globally pooled, then passed through dense layers and a softmax over classes.

Averaging each feature map to a single value, replacing large dense layers and reducing overfitting.

LeNet, AlexNet, VGG, GoogLeNet/Inception, ResNet, and EfficientNet.

Object detection (YOLO, Faster R-CNN), segmentation (U-Net), and feature extraction for many vision tasks.

A network for sequences that maintains a hidden state passed from step to step, letting earlier inputs influence later outputs.

Ordered/sequential data — text, speech, time series, and any data with temporal dependencies.

Repeated multiplication during backprop-through-time causes vanishing or exploding gradients, so long-range dependencies are lost.

Long Short-Term Memory — an RNN with a cell state and gates that learn what to keep, forget, and output, preserving long-range information.

Forget gate (what to drop), input gate (what new info to store), and output gate (what to emit from the cell).

A protected memory channel that carries information across many steps with minimal interference, easing gradient flow.

Gated Recurrent Unit — a simpler LSTM variant with reset and update gates and no separate cell state; faster, often comparable.

GRU has fewer parameters and trains faster; LSTM can be more expressive on complex tasks — choice is empirical.

Unrolling the RNN across time steps and applying backpropagation over the whole sequence to update shared weights.

Gradient clipping caps the gradient norm, preventing destabilising updates.

One that processes the sequence forward and backward, giving each step context from both past and future (for non-streaming tasks).

A method letting the model weigh and draw on all input positions when producing each output, instead of a single fixed-size state.

A sequence architecture built entirely on self-attention (no recurrence), enabling parallel training and long-range modelling — the basis of modern LLMs.

They parallelise across the sequence (RNNs are sequential), capture long-range dependencies better, and scale to huge data.

Each token computes a weighted combination of all tokens via query-key-value scores, capturing relationships regardless of distance.

Information added to Transformer inputs to convey token order, since attention itself is order-agnostic.

Training a sequence model by feeding the true previous token (not its own prediction) as the next input, speeding convergence.

Encoder-decoder architectures mapping an input sequence to an output sequence — used for translation and summarisation.

Language modelling, sentiment analysis, machine translation, speech recognition, and time-series forecasting.

RNNs use less memory for short/streaming inputs and constant per-step cost; Transformers parallelise and model long context but scale quadratically with sequence length.