Skip to content

ENH: reproducible Monte Carlo via per-simulation-index seeding#1054

Draft
thc1006 wants to merge 1 commit into
RocketPy-Team:developfrom
thc1006:enh/reproducible-montecarlo-seeding
Draft

ENH: reproducible Monte Carlo via per-simulation-index seeding#1054
thc1006 wants to merge 1 commit into
RocketPy-Team:developfrom
thc1006:enh/reproducible-montecarlo-seeding

Conversation

@thc1006

@thc1006 thc1006 commented Jul 7, 2026

Copy link
Copy Markdown

Pull request type

  • Code changes (bugfix, features)

Draft / RFC. I'm opening this to make the design in #1053 concrete and get your call on the open questions below before polishing it.

Current behavior

MonteCarlo.simulate() seeds the stochastic models per worker in parallel mode (from a fresh, unseeded np.random.SeedSequence().spawn(n_workers)) and once at construction in serial mode. So the sampled inputs depend on the execution mode and the number of workers, and parallel runs aren't reproducible run to run. This is #1053.

New behavior

Adds a keyword-only random_seed to simulate(). From that root I spawn one child seed per simulation index and reseed the models from child_seeds[i] before simulation i. SeedSequence.spawn is prefix-stable, so index i maps to the same seed regardless of which worker runs it. The sampled inputs come out identical across serial, parallel(2) and parallel(N), and reproducible from the seed.

The design follows Scientific-Python SPEC 7 (a per-call keyword-only seed, normalized to accept an int, a SeedSequence, or a Generator, with None = fresh entropy so existing behavior is unchanged unless you pass a seed) and NumPy's documented parallel idiom (SeedSequence(root).spawn(n) then default_rng(child)).

tests/unit/simulation/test_monte_carlo_determinism.py covers it: serial reproducibility, serial == parallel(2) == parallel(4), and the None-seed path. Flight is stubbed, so it's a slow input-sampling test.

Open questions (why this is a draft)

  1. Name: I used random_seed for continuity with the Stochastic* seed. SPEC 7's direction is rng (SciPy 1.15 renamed seed to rng; sklearn is weighing it in #29315). I can switch to rng if you'd rather be forward-looking.
  2. Where it lives: on simulate() (per run) versus also a stored default on MonteCarlo.__init__ (hybrid). I went with simulate() only, the minimal SPEC-7 form.
  3. Default: None = fresh entropy (backward-compatible, not reproducible). A competition or regression context might want a reproducible default instead.
  4. Decorrelation: I split each index seed three ways so env/rocket/flight get independent streams (today they share one seed). I can drop it to keep the change minimal.
  5. Known gap (from Monte Carlo results aren't reproducible across serial and parallel runs (seeding is per-worker, not per-simulation) #1053): list-valued stochastic attributes are sampled with the stdlib random.choice (an unseeded global), so a model with a multi-element thrust_source isn't fully seed-controlled yet. The test uses a single-thrust_source rocket to sidestep it; a full fix would seed the stdlib random per index too.

Breaking change

  • Yes

The exact numbers a seed produces change (per-index seeding, the env/rocket/flight decorrelation, and the serial index now counting from 0 to match parallel), so stored Monte Carlo baselines such as test_monte_carlo_simulate regenerate. There's no API break for users who don't pin exact samples, and random_seed is opt-in.

One more heads-up: Python 3.14 flipped the Linux multiprocessing default from fork to forkserver, so workers no longer inherit memory and the seed objects crossing the process boundary must be picklable. SeedSequence is, and the test passes under fork.

Closes #1053

MonteCarlo seeded the stochastic models per worker in parallel mode (from a
fresh, unseeded SeedSequence) and once at construction in serial mode, so the
sampled inputs depended on the execution mode and the worker count, and parallel
runs were not reproducible run to run.

Add a keyword-only random_seed to simulate() (SPEC 7 style: accepts an int, a
SeedSequence, or a Generator; None keeps the previous fresh-entropy behavior).
Spawn one child seed per simulation index from that root and reseed the
stochastic models from child_seeds[i] before simulation i. SeedSequence.spawn is
prefix-stable, so index i maps to the same seed regardless of which worker runs
it, making the inputs identical across serial, parallel(2) and parallel(N). Each
index seed is split three ways so the environment, rocket and flight draw from
independent streams rather than sharing one.

The serial index field now counts from 0 to match the parallel path. Both changes
alter the numbers a fixed seed produces, so stored baselines regenerate.

Adds tests/unit/simulation/test_monte_carlo_determinism.py: serial
reproducibility, worker invariance (serial == parallel(2) == parallel(4)), and
the None-seed path.

Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant