Stopper

Contents

Stopper#

class liesel.goose.Stopper(max_iter, patience, atol=0.001, rtol=0.0)[source]#

Bases: object

Handles (early) stopping for optim_flat().

Parameters:

  • max_iter (int) – The maximum number of optimization steps.

  • patience (int) – Early stopping happens only, if there was no improvement for the number of patience iterations, and there were at least as many iterations as the length of the patience window.

  • atol (float) – The absolute tolerance for early stopping. (default: 0.001)

  • rtol (float) – The relative tolerance for early stopping. The default of 0.0 means that no early stopping happens based on the relative tolerance. (default: 0.0)

Notes

Early stopping happens, when the oldest loss value within the patience window is the best loss value within the patience window. A simplified pseudo-implementation is:

def stop(patience, i, loss_history):
    recent_history = loss_history[-patience:]
    oldest_within_patience = recent_history[0]
    best_within_patience = np.min(recent_history)

    return oldest_within_patience <= best_within_patience

Absolute and relative tolerance make it possible to stop even in cases when the oldest loss within patience is not the best. Instead, the algorithm stops, when the absolute or relative difference between the oldest loss within patience and the best loss within patience is so small that it can be neglected. To be clear: If either of the two conditions is met, then early stopping happens. The relative magnitude of the difference is calculaterd with respect to the best lost within patience. A simplified pseudo-implementation is:

def stop(patience, i, loss_history, atol, rtol):
    recent_history = loss_history[-patience:]
    oldest_within_patience = recent_history[0]
    best_within_patience = np.min(recent_history)

    diff = oldest_within_patience - best_within_patience
    rel_diff = diff / np.abs(best_within_patience)

    abs_improvement_is_neglectable = diff <= atol
    rel_improvement_is_neglectable = rel_diff <= rtol

    return (abs_improvement_is_neglectable | rel_improvement_is_neglectable)

Methods

continue_(i, loss_history)

Whether optimization should continue (inverse of stop_now()).

stop_early(i, loss_history)

stop_now(i, loss_history)

Whether optimization should stop now.

which_best_in_recent_history(i, loss_history)

Identifies the index of the best observation in recent history.

Attributes

atol

rtol

max_iter

patience

Contents