mercurial.utils.overfitting_controls module

Overfitting countermeasures: regularization, early stopping, pruning.

class mercurial.utils.overfitting_controls.EarlyStopping(patience: int = 5, min_delta: float = 0.0001)

Bases: object

Stop training when validation loss stops improving.

Methods

step(loss, params)

Return True if training should continue, False if stop.

reset

reset()

step(loss: float, params: ndarray) → bool

Return True if training should continue, False if stop.

class mercurial.utils.overfitting_controls.L2Regularizer(lambda_reg: float = 0.01)

Bases: object

L2 regularization (ridge) for model parameters.

Methods

gradient(params)	2λ * θ.
penalty(params)	λ * ||θ||₂².

gradient(params: ndarray) → ndarray

2λ * θ.

penalty(params: ndarray) → float

λ * ||θ||₂².

class mercurial.utils.overfitting_controls.LinearModel(input_dim: int, lambda_reg: float = 0.01)

Bases: object

Simple linear model with L2 regularization and early stopping.

Methods

train(X_train, y_train, X_val, y_val[, ...])

Train with early stopping.

gradient
loss
predict

gradient(X: ndarray, y: ndarray) → Tuple[ndarray, float]

loss(X: ndarray, y: ndarray) → float

predict(X: ndarray) → ndarray

train(X_train: ndarray, y_train: ndarray, X_val: ndarray, y_val: ndarray, epochs: int = 100, lr: float = 0.01) → dict

Train with early stopping.

mercurial.utils.overfitting_controls.prune_parameters(params: ndarray, threshold: float = 0.01) → ndarray

Set small-magnitude parameters to zero.

mercurial.utils.overfitting_controls.sparsity_ratio(params: ndarray) → float

Fraction of parameters that are zero.