LBFGS#

class liesel.optim.LBFGS(position_keys, optimizer=(<function chain.<locals>.init_fn>, <function chain.<locals>.update_fn>), identifier='', activate_after_epochs=0)[source]#

Bases: Optimizer

Optimizer wrapper using Optax L-BFGS.

LBFGS behaves like Optimizer but uses optax.value_and_grad_from_state() inside step(), which lets Optax reuse value/gradient information stored by the L-BFGS transformation.

L-BFGS requires full-data batches, a deterministic objective, and must be the sole optimizer. Other parameter updates would invalidate its cached objective and curvature history. Use one joint L-BFGS for all selected parameters; unselected parameters stay fixed. The engine rejects mini-batches and mixed optimizer configurations, but cannot detect stochastic objective evaluations. For a deterministic objective, the returned scalar is the value used for the update at the supplied pre-update position.

Parameters:

position_keys (Sequence[str]) – Names of the parameter entries owned by this optimizer.
optimizer (GradientTransformationExtraArgs, default: GradientTransformationExtraArgs(init=<function chain.<locals>.init_fn at 0x7524f0cb6340>, update=<function chain.<locals>.update_fn at 0x7524f0cb63e0>)) – Optax L-BFGS transformation. Defaults to optax.lbfgs().
identifier (str, default: '') – Optional optimizer-state identifier filled by OptimEngine when omitted.
activate_after_epochs (int, default: 0) – Number of completed epochs required before this optimizer participates in batch updates. 0 activates it from the first epoch.

Examples

>>> from liesel.optim import LBFGS
>>> lbfgs = LBFGS(["loc"], identifier="loc_lbfgs")
>>> lbfgs.position_keys
('loc',)
>>> lbfgs.identifier
'loc_lbfgs'
>>> repr(lbfgs)
"LBFGS(('loc',), identifier=loc_lbfgs)"

Methods

step(position, loss, carry)

Runs one L-BFGS optimizer step on position.

Attributes

`identifier`	Optional optimizer-state identifier filled by `OptimEngine` when omitted.
`optimizer`	Optax L-BFGS transformation.
`position_keys`	Names of the parameter entries owned by this optimizer.

LBFGS

Contents

LBFGS#