Add timestamp-paced replay (replay_dbn / replay_records)

CHANGELOG.md

@@ -5,7 +5,7 @@ All notable changes to DatabentoBinaryEncoding.jl are documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.1.6] - 2026-06-24
 
 ### Changed
 
@@ -18,6 +18,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- Timestamp-paced **replay**: `replay_dbn(f, filename; ...)` re-emits a DBN file's
+  records in time order, pacing each `f(record)` callback by the gap between record
+  timestamps to simulate a live feed (backtesting, demos, driving real-time
+  consumers). `replay_records(f, records; ...)` does the same over an already-loaded
+  collection. Zstd-aware and skips unknown record types like `DBNStream`. Pacing is
+  anchored to absolute wall-clock targets so callback execution time is absorbed
+  instead of accumulating as drift. Options: `speed` (time-compression multiplier;
+  `Inf` = no waiting), `timestamp` (`:ts_event` or `:ts_recv`, with fallback),
+  `max_sleep` (cap large gaps; re-anchors after a clamp), and `precise` (busy-wait
+  sub-millisecond gaps, since `Base.sleep` only resolves ~1 ms on Unix / ~15 ms on
+  Windows). `clock`/`sleep_fn` are injectable for deterministic testing.
 - `dbn_to_parquet` now compresses with **ZSTD by default** and accepts a `compression`
   keyword — one of `"zstd"` (default), `"snappy"`, `"gzip"`, or `"uncompressed"`.
 - Symbol resolution helpers that join the human-readable `raw_symbol` back onto
@@ -130,7 +141,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   conversion to JSON/Parquet/CSV; byte-for-byte compatibility with the official
   Rust implementation ([#18], [#19]).
 
-[Unreleased]: https://github.com/tbeason/DatabentoBinaryEncoding.jl/compare/v0.1.3...HEAD
+[0.1.6]: https://github.com/tbeason/DatabentoBinaryEncoding.jl/compare/v0.1.5...v0.1.6
 [0.1.3]: https://github.com/tbeason/DatabentoBinaryEncoding.jl/compare/v0.1.2...v0.1.3
 [0.1.2]: https://github.com/tbeason/DatabentoBinaryEncoding.jl/compare/v0.1.1...v0.1.2
 [0.1.1]: https://github.com/tbeason/DatabentoBinaryEncoding.jl/compare/v0.1.0...v0.1.1

Project.toml

@@ -1,6 +1,6 @@
 name = "DatabentoBinaryEncoding"
 uuid = "90689371-c8cb-40d1-831f-18033db90f74"
-version = "0.1.5"
+version = "0.1.6"
 authors = ["Tyler Beason <tbeas12@gmail.com>"]
 
 [deps]

README.md

@@ -10,6 +10,7 @@ For more details, read the [introduction to DBN](https://databento.com/docs/stan
 
 - ✅ Complete DBN v3 Format Support
 - ✅ Efficient streaming support (read and write)
+- ✅ Timestamp-paced replay (simulate a live feed at any speed)
 - ✅ Zstd file compression support (read and write)
 - ✅ Bidirectional format conversion (DBN ↔ JSON/Parquet/CSV)
 - ✅ Byte-for-byte compatibility with official implementations
@@ -79,6 +80,51 @@ end
 println("Total: $(total[])")
 ```
 
+### Replaying DBN Files
+
+Re-emit a file's records in time order, paced by their timestamps, to simulate a
+live feed (useful for backtesting, demos, or driving consumers that expect
+real-time data):
+
+```julia
+# Replay in real time — each record fires after the real gap to the previous one
+replay_dbn("trades.dbn") do rec
+    println(price_to_float(rec.price))
+end
+
+# Replay 100x faster, paced by receive time, with overnight gaps capped at 1s
+replay_dbn("mbo.dbn.zst"; speed = 100, timestamp = :ts_recv, max_sleep = 1.0) do rec
+    handle(rec)
+end
+
+# Replay an already-loaded collection (e.g. from read_dbn)
+records = read_dbn("trades.dbn")
+replay_records(records; speed = 5) do rec
+    handle(rec)
+end
+```
+
+Key options: `speed` (time-compression multiplier; `Inf` = no waiting),
+`timestamp` (`:ts_event` or `:ts_recv`), and `max_sleep` (cap on any single
+wait, in seconds). Compression and unknown record types are handled exactly as
+in `DBNStream`.
+
+**Timing resolution.** DBN timestamps are nanosecond precision, but pacing
+accuracy is bounded by the sleep function. `Base.sleep` resolves to roughly
+1 ms on Unix and as coarse as the system timer tick (~15 ms) on Windows, so
+records spaced more tightly than that arrive clumped rather than as distinct
+waits. Pacing is anchored to absolute wall-clock targets, so this clumping is
+local — the stream re-synchronizes and timing error does not accumulate across
+the file, and records sharing a timestamp are delivered back-to-back. For
+sub-millisecond fidelity (e.g. dense MBO bursts) pass `precise = true`, which
+busy-waits small gaps at the cost of pinning a CPU core:
+
+```julia
+replay_dbn("mbo.dbn"; precise = true) do rec
+    handle(rec)
+end
+```
+
 ### Writing DBN Files
 
 ```julia

src/DatabentoBinaryEncoding.jl

@@ -99,6 +99,7 @@ include("buffered_io.jl")
 include("decode.jl")
 include("encode.jl")
 include("streaming.jl")
+include("replay.jl")
 include("export.jl")
 include("symbols.jl")
 include("import.jl")
@@ -117,6 +118,7 @@ export foreach_record, foreach_record_with_control, foreach_trade, foreach_mbo,
 export record_type_for_dbn_schema  # Schema -> concrete record type
 export foreach_ohlcv, foreach_ohlcv_1s, foreach_ohlcv_1m, foreach_ohlcv_1h, foreach_ohlcv_1d  # OHLCV streaming
 export foreach_cmbp1, foreach_cbbo1s, foreach_cbbo1m, foreach_tcbbo, foreach_bbo1s, foreach_bbo1m  # Consolidated/BBO streaming
+export replay_dbn, replay_records  # Timestamp-paced replay
 export compress_dbn_file, compress_daily_files
 export Schema, Compression, Encoding, SType, RType, Action, Side, InstrumentClass
 export price_to_float, float_to_price, ts_to_datetime, datetime_to_ts, ts_to_date_time, date_time_to_ts, to_nanoseconds