Stopper#

class liesel.optim.Stopper(epochs, patience, atol=0.0, rtol=0.0, min_epochs=0)[source]#

Bases: object

Handles maximum-epoch and early stopping decisions.

Stopper uses a rolling patience window over the recorded monitoring loss. In the experimental optimizer engine, the argument i passed to the stopper is the number of completed epochs, which is also the next epoch index to be written. Therefore, the recent patience window is loss_history[i - patience : i] once at least patience epochs have been completed.

Parameters:

epochs (int) – The maximum number of optimization epochs.
patience (int) – Length of the rolling early-stopping window. Early stopping can only happen after more than patience epochs have been completed.
atol (float, default: 0.0) – Absolute tolerance for early stopping.
rtol (float, default: 0.0) – Relative tolerance for early stopping. The default of 0.0 disables additional relative tolerance beyond exact non-improvement.
min_epochs (int, default: 0) – Minimum number of completed epochs before early stopping can happen. The default of 0 preserves the existing patience-based behavior. The effective earliest stop is max(min_epochs, patience + 1).

Notes

Early stopping happens when the oldest loss value within the patience window is the best loss value within the patience window. With experimental optimizer indexing, a simplified pseudo-implementation is:

def stop(patience, i, loss_history):
    recent_history = loss_history[i - patience : i]
    oldest_within_patience = recent_history[0]
    best_within_patience = np.min(recent_history)

    return oldest_within_patience <= best_within_patience

Absolute and relative tolerance make it possible to stop even in cases when the oldest loss within patience is not the best. Instead, the algorithm stops, when the absolute or relative difference between the oldest loss within patience and the best loss within patience is so small that it can be neglected. To be clear: If either of the two conditions is met, then early stopping happens. The relative magnitude of the difference is calculated with respect to the best loss within patience. A simplified pseudo-implementation is:

def stop(patience, i, loss_history, atol, rtol):
    recent_history = loss_history[i - patience : i]
    oldest_within_patience = recent_history[0]
    best_within_patience = np.min(recent_history)

    diff = oldest_within_patience - best_within_patience
    rel_diff = diff / np.abs(best_within_patience)

    abs_improvement_is_neglectable = diff <= atol
    rel_improvement_is_neglectable = rel_diff <= rtol

    return abs_improvement_is_neglectable | rel_improvement_is_neglectable

Examples

>>> import jax.numpy as jnp
>>> from liesel.optim import Stopper
>>> stopper = Stopper(epochs=10, patience=3)
>>> loss_history = jnp.array([3.0, 2.0, 1.0, 1.5, 1.6, 1.7])
>>> bool(stopper.stop_early(i=5, loss_history=loss_history))
True
>>> int(stopper.which_best_in_recent_history(i=5, loss_history=loss_history))
2

max_iter is available as a read-only alias for epochs:

>>> stopper.max_iter
10

Methods

`continue_`(i, loss_history)	Whether optimization should continue (inverse of `stop_now()`).
`stop_early`(i, loss_history)	Returns whether the recent patience window should trigger early stopping.
`stop_now`(i, loss_history)	Whether optimization should stop now, including at the maximum epoch.
`which_best_in_recent_history`(i, loss_history)	Identifies the index of the best observation in the recent loss window.

Attributes

`atol`	Absolute tolerance for early stopping.
`max_iter`	Read-only alias for `epochs`.
`min_epochs`	Minimum number of completed epochs before early stopping can happen.
`rtol`	Relative tolerance for early stopping.
`epochs`	The maximum number of optimization epochs.
`patience`	Length of the rolling early-stopping window.

Stopper

Contents

Stopper#