Reproducibility#

PRNG seeding#

Liesel uses JAX’ splittable Threefry counter-based PRNG, which does not have a global state, so you always need to seed it explicitly, for example when initializing an liesel.goose.EngineBuilder. However, even with the same seed, reproducible results cannot be guaranteed across different systems or even for separate runs of the same program on the same hardware.

GPU non-determinism#

Floating point operations on GPUs are generally not completely deterministic, and even though JAX takes some precautions against this surprising behavior, full reproducibility on GPUs should not be assumed. See also this (unmerged) page in the JAX documentation.

Non-reproducibility across systems#

In our experience, results from Liesel, BlackJAX and JAX may differ across systems, even if the exact same code is run on the CPU. Following the Stan documentation, we expect reproducibility only on the CPU and only if all of the following components are identical:

  • the Liesel version,

  • the Python version,

  • the versions of all libraries Liesel depends on,

  • the operating system version,

  • the computer hardware including CPU, motherboard and memory,

  • the compilers, including versions, flags and libraries, used to build Python and all libraries Liesel depends on,

  • the program, including the seed, initialization and data.

See also#