Experiment Preferences pirate-ship

Deterministic campaign design with seeded RNG will enable snapshot testing of generated game content

determinismtestinggame-designgame-dev
Hypothesis

Deterministic campaign design with seeded RNG will enable snapshot testing of generated game content

Result: confirmed
Key Findings

Snapshot testing became trivial: generate campaign with seed X, diff against expected output. 73 encoding variants all produce deterministic output given the same seed.

Changelog

DateSummary
2026-04-06Audited: added Changelog, domain tag game-dev, expanded all sections, stamped last_audited
2026-02-15Initial creation

Hypothesis

We bet that replacing Math.random() with a seeded PRNG throughout the pirate-ship campaign generation pipeline would make all generated content reproducible and thus snapshot-testable. The campaign system generates encounters, loot tables, enemy stat blocks, narrative branches, and 73 content encoding variants on every play session. Without a seed, each generation was unique and untestable. With a seed, the same input always produces the same output : so a test can simply compare the generated campaign against a stored golden file.

Method

Replaced all Math.random() calls in the campaign generation pipeline with calls to a seeded Mulberry32 PRNG. The seed is a 32-bit integer passed at campaign initialization time. Any campaign generated with seed X will produce identical results across platforms and Node.js versions because Mulberry32 is a pure arithmetic implementation with no platform-specific behavior.

The 73 encoding variants : which apply obfuscation transforms to campaign content to prevent pattern matching by players : were the highest-risk change. Each variant applies a different mathematical transform (bit rotation, XOR, modular arithmetic) to the underlying content values. Ensuring all 73 variants remained deterministic required auditing each transform’s use of randomness and replacing 47 internal Math.random() calls across the variant implementations.

Snapshot testing infrastructure: a Jest test suite that calls generateCampaign(seed) and compares the output (serialized to JSON) against a committed golden file. Any change to the generation logic will produce a diff, which forces an explicit decision: “is this diff intentional?” If yes, update the golden file. If no, it is a regression.

Results

Confirmed. Snapshot testing is fully operational. The test suite generates campaigns with 10 different seeds and compares each against its golden file. Any regression in the generation logic produces a specific diff that identifies exactly which content changed and at which generation step.

The 73 encoding variants all produce deterministic output given the same seed. The audit of 47 internal Math.random() calls found 3 that could not be trivially replaced (they depended on platform timing for entropy seeding in variant initialization). Those 3 were refactored to receive the seed from the campaign-level PRNG rather than generating their own.

See topics/pirate-ship-deterministic-campaign-design and topics/pirate-ship-73-encoding-variants.

Findings

  1. Seeded PRNG is a prerequisite for snapshot testing any stochastic system. There is no way to regression-test generated content without reproducibility. The snapshot testing framework was ready within an hour of the PRNG swap because it was blocked only by non-determinism, not complexity.

  2. 73 variants required auditing, not just a global replacement. Each variant had its own randomness budget. A global find-and-replace of Math.random() would have worked for simple cases but missed the 3 entropy-seeded variants that required architectural changes.

  3. The seed is not just a test tool : it is a gameplay feature. Players can share seeds to replay the same campaign. Speedrunners use known seeds to practice optimal routes. The determinism that enables testing also enables a class of community features.

  4. Determinism and obfuscation are compatible. A common concern was that deterministic output would make the 73 encoding variants predictable and breakable. In practice, the obfuscation prevents content inference regardless of the generation being reproducible : the seed space is 2^32, which is not exhaustively searchable.

Next Steps

Apply the seeded PRNG pattern to the provider API call timing layer, where non-deterministic delays cause flaky integration tests. A seeded timing simulator would make the provider quarantine circuit-breaker testable without real API calls. See projects/pirate-ship/_index for the current project status.