Stage-A: A2 and A4 workflows, direct photodiode paths, and the generic camera session - #12
Open
muthmann wants to merge 57 commits into
Open
Stage-A: A2 and A4 workflows, direct photodiode paths, and the generic camera session#12muthmann wants to merge 57 commits into
muthmann wants to merge 57 commits into
Conversation
PDA1 wire protocol (fragmentation-tolerant parser with CRC resync), v1 ASCII command grammar, typed serial client with idempotent sequence retries and stream-integrity accounting, bounded background I/O worker, .pdq raw-frame writer, JSON run sidecar, calibrated optical log-contrast estimator (clipping/dark-headroom guarded), and a mock controller for hardware-free tests. Wire-compatible with stage-a-controller include/wire_protocol.h.
Live decimated photodiode waveform, calibrated clipping-guarded optical contrast, stream-integrity status, and gated manual controller actions (connect/config/start/stop/expert drive). Fails closed on the ABI v5 execution context: serial I/O only in the active live-capture worker; commands are host actions, never persistent settings.
Statistical core: phase folding (hardware EXT_TRIGGER fiducials or software clock-skew recovery via Rayleigh-power frequency scan), Rayleigh detection with Bonferroni-charged trials, median-background phase-locked excess, probit a_min fit with profile CI, hot-pixel mask from an unmodulated reference window. Sweep engine bisects the drive code, grids the bracketed transition, and records measured optical contrast per point. PDQ + sidecar + results export per run; fails closed on the ABI v5 execution context.
Feature brief for the stage-a-io/monitor/a1 stack, ADR 005 for the device-ownership boundary (plugins own the Teensy, AugurRs stays generic), feature index entry. Note: the legacy plugins (localization/reconstruction/focus-metrics/ evesmlm) on this branch predate the current augur-rs plugin API and do not compile against it — their refresh is in progress on feature/eve-batch-findings; rebasing that work onto plugin ABI v5 only adds the new FfiPreviewFrame.external_triggers field in one test initializer.
… refresh Updates the evesmlm suite, localization, and reconstruction plugins plus repo docs to the current augur-rs plugin interface (host-view actions, dataset display metadata/relations, TableSchema extensions, CompactEvent accessors).
# Conflicts: # docs/features/README.md
Adds the new FfiPreviewFrame.external_triggers field to the two test initializers in focus-metrics and evesmlm-candidates; all plugins now compile and test against the current augur-rs plugin API (ABI v5).
The mock controller spoke an invented protocol (ARM/RUN/FAULT_CLEAR verbs, capabilities HELLO field, BAD_* error codes, arbitrary CONFIG fields), so tests validated commands the Teensy never accepts. It now mirrors stage-a-controller/src/main.cpp verbatim: verbs, SAFE_IDLE/CONFIGURED/RUNNING state machine, RANGE/STATE/SYNTAX/PROTOCOL/VERB error details, single-entry idempotent reply cache, and unknown-CONFIG-field rejection (the host's feature-detection contract). The reserved v2 waveform fields are only accepted behind an explicit with_waveform_extension() opt-in, which also synthesizes photodiode blocks through a Pockels-like sin² transfer. Host fixes uncovered by the faithful mock: - surface async control notices (watchdog !FAULT) through poll_events even with no request in flight; monitor and A1 now react instead of showing a stale acquiring state - A1 sweep issues STOP before CONFIG between measurement points (CONFIG is illegal while RUNNING) - record the ACKed CONFIG fields in the A1 run sidecar - reject wire frames with an unknown protocol version like the reference parser does - stream the PDQ file CRC incrementally instead of buffering the whole recording in memory - monitor maps the unknown_config_field rejection of drive fields to a clear 'no waveform backend' message
New plugins/stage-a-funcgen crate: manual Pockels-cell drive control (sine/square/sawtooth, frequency, center/amplitude DAC codes) with the resulting optical amplitude always measured from the photodiode as a = ln(Vmax/Vmin) — the commanded amplitude is a phase-modulation depth and maps non-linearly to light, so it is never reported as an optical level. The default 'mock' port runs the waveform-extended mock controller on an in-process thread and streams a synthetic sin²-transfer photodiode response, so the full control loop works with zero hardware. Against real firmware 0.2.0 the reserved drive fields are feature-detected via the unknown_config_field rejection and reported as 'no waveform backend'; actual output stays blocked on the hardware freeze per stage-a-controller/docs/features/waveform-drive.md. Same fail-closed safety model as stage-a-monitor: LiveCapture + effects gating, drive parameters as settings but application as an explicit action, local DAC-range validation before any command, watchdog fault surfacing.
…n and photodiode pair - delete stage-a-monitor, stage-a-funcgen, stage-a-a1 (too complex for the current bench stage; retained in git history) - add stage-a-modulation: capped power slider, CONST/SINE/SQUARE with frequency and min threshold, immediate MOD transfer, board-reported DAC code - add stage-a-photodiode: SMA5/pin18/A4 stream readout on the second CDC port with RAW and EXCITATION (I_exc = I_tot - I_pd) modes and rolling chart - extend the stage-a-io mock to firmware 0.3.0 (MOD verb, capabilities) - ADR 006 (two plugins, one port each), rewritten stage-a brief, doc updates
The host settings UI reads enum values with as_u64() and writes the selected variant index; the modulation/photodiode plugins returned and expected variant name strings, so port and mode radio buttons could never be set. Map indices against the schema's variant list in get_setting/set_setting (names still accepted) and pin the contract with round-trip tests.
auto in stage-a-modulation now probes each attached port with HELLO and picks the command port; auto in stage-a-photodiode listens for PD lines and picks the stream port. Filter macOS port lists to the cu.* callout nodes so each device appears once. Verified against the live Teensy (firmware 0.3.0).
Port entries now read '/dev/cu.usbmodem… (Teensyduino Dual Serial)' so the Teensy is recognisable among the enumerated devices; the parenthesised label is display-only and stripped when the setting is applied. stage-a-io gains available_ports_with_labels() for this.
…ut camera frames The host only calls process_frame() while camera frames flow, so the action-button + effects-gate design never connected on a camera-less bench (verified: the serial port stayed free while the GUI ran). Connect is now a checkbox setting handled in set_setting, all serial I/O lives in a plugin-owned device thread (slider drags coalesce into one pending MOD), status comes from shared state, and replay mode still disconnects defensively. Verified end-to-end against the live Teensy: auto-probe, level 800 -> board code 800, level 0 -> 0.
…elope and moving average Track firmware 0.4.0 (ADR 003 in stage-a-controller): the stream port now carries PDA1 SamplesU16 frames at pd_stream_rate_hz instead of 50 Hz ASCII lines, raising the plot's data rate 400x. - parse with stage-a-io's FrameParser (new dep, default-features off: wire parser only); auto port probe detects sample frames - bounded raw ring (130 s / 4 M samples) keyed by device sample index; rate changes and index jumps restart the segment so index/rate stays a consistent time base across acquisition handovers - chart decimates the window into <= 1000 min/mean/max envelope buckets; short windows render raw samples; window down to 10 ms - moving-average indicator for the low-voltage regime: fixed sample window (default 4) or one full period of a user-set sync frequency (window = rate / f), making the mean phase-independent under modulation; overlay line + numeric readout in status views - status table gains rate, moving avg, and stream-integrity columns (device drops, CRC failures, resync bytes, segment restarts) - stage-a-io: gate the Duration import behind the hardware feature so default-features = false builds are warning-free Verified: cargo fmt, clippy -D warnings (photodiode + stage-a-io), 9 plugin tests + 29 stage-a-io tests green.
…e photodiode plugin Two save modes behind one Data settings section: - monitor cache: the raw ring is now cache_s seconds long (default 20 s, 1-130 s) and a Save-cache-snapshot button dumps it as pd_cache_<utc>.csv (sample_index, t_s on the device clock, raw code, raw volts) plus a JSON sidecar carrying rate, integrity counters, display mode, and reference so derived quantities stay reproducible - recording: a Record toggle tees every incoming SamplesU16 frame verbatim to pd_rec_<utc>.pdq via stage-a-io's PdqWriter from the reader thread; stopping (or disabling the plugin) finalizes the file and writes a sidecar with per-recording integrity deltas and validity; the mock synthesizes identical wire frames so recordings parse the same without hardware - data_dir uses the new Path setting kind; record/save failures surface through status entries like connect errors; status shows a live REC indicator with recorded seconds Verified: cargo fmt, clippy -D warnings, 13 plugin tests green.
…odiode plugin - PD Spectrum window: Hann-windowed radix-2 FFT (no deps) over the newest power-of-two window of raw samples (256-16384; ≈1.2 Hz resolution at 20 kSa/s), one-sided amplitude in volts with max-hold bin decimation so narrow peaks survive the plot budget. Window placement means the FFT only runs while the window is open. Verified by test: a synthesized 1 kHz 0.4 V tone is recovered at the right frequency and amplitude. - time_axis setting: BEFORE NOW (scrolling, x ends at 0) or SEGMENT TIME (absolute device-clock seconds) — frozen plots and cursor measurements read as positions instead of implied motion.
Timed MOD sequences without cross-plugin control: the executor lives inside the plugin that already owns the command port. - protocol file: loops = N plus [[steps]] with duration_s, wave (OFF/CONST/SINE/SQUARE), level, min, frequency_hz — fully validated before the run starts (ranges match the firmware grammar) - executor thread walks the steps on an absolute schedule (no drift accumulation) and feeds the same coalescing pending-command slot the device thread drains, so it never touches the serial port itself; the last step holds after completion (set-and-hold), stop is immediate, and disconnecting aborts the run - protocol_path uses the new Path setting kind; run/stop is a settings-driven toggle that works with no camera; progress (loop, step, summary) shows in the status entries Verified: cargo fmt, clippy -D warnings, 8 plugin tests green incl. an end-to-end run against the mock controller.
…ry cells Prepares the plugin for firmware 0.5.0's 500 kSa/s DMA stream (ADR 004 in stage-a-controller): a full raw-window rescan per repaint stops being viable around that rate. - ingest maintains 64:1 min/max/sum summary cells aligned to deque offsets; eviction drops whole cells so the alignment (and the device-clock index base) survives, at the cost of up to one cell of ring slack - chart buckets and every moving-average window combine cells plus raw edge samples via range_summary — O(range/64) instead of O(range), verified exact against naive scans across cell boundaries and after eviction - ring cap raised to 16 M samples (32 s at 500 kSa/s, 32 MiB of codes); cache_s keeps ruling the duration at lower rates Verified: cargo fmt, clippy -D warnings, 16 plugin tests green.
The photodiode plugin dropped the pre-0.4.0 ASCII PD-line path, so a Teensy running old firmware produced a generic 'no PDA1 sample frames' error that gave no hint at the real cause. The auto-probe now classifies each port (PDA1 frames / legacy ASCII / nothing) and, when it sees the 'PD code=…' ASCII stream, tells the user to flash stage-a-controller 0.4.0+ instead — the actual fix, since the plugin and firmware ship in lockstep. Verified: cargo fmt, clippy -D warnings, 16 tests green.
…ntract Introduces `stage-a-a1` as the measurement owner that drives the modulation and photodiode plugins through a leased device contract, plus the `stage-a-plugin-contract` crate that carries the shared settings/telemetry schema between them. Also lands the supporting work these depend on: - optical waveform generation and inversion in the modulation plugin - Pockels transfer calibration (measured V_null / Vpi) with sweep support - photodiode contrast estimation and monitor-cache snapshots - `.pdq` recording format with SHA-256 integrity sidecars Documented in docs/features/stage-a-a1.md, stage-a-a1-automation.md, stage-a-optical-waveform.md, stage-a-pockels-calibration.md and ADRs 007-011.
…display mode The photodiode sits behind the PBS reject port and measures the complement I_pd = I_tot - I_exc — a property of the bench, not of what the operator chose to plot. `optical_summary` picked its geometry from the display mode, so leaving the chart on RAW published the raw detector contrast as `measured_log_contrast`. A1's amplitude sweep settles on that value against a target `a`: it would never settle, time out at every point, and write a wrong `measured_a` into each sweep sidecar. The geometry is now always the rejected complement; the display mode is presentational. A withheld `a` now reports which gate rejected the window instead of silently showing nothing. Also in the photodiode plugin: - dark level is a measured setting with a capture action, applied to both the detector samples and the I_tot anchor so it cancels out of the complement instead of biasing it; `dark_id` names it honestly - phase-0 marker frames are written into the .pdq, so a recorded run stays phase-attributable offline - `save_cache_snapshot` copies the ring and releases the lock before writing the CSV, instead of blocking the reader across millions of writes - the UI mirror keeps the operator's connect intent rather than clearing it every control tick - a 0-byte read backs off instead of spinning a core - the spectrum max-hold seeds each bucket with its own first bin
…ol step `send_modulation` was silenced for an automation lease and a calibration sweep but not for a running protocol, which queues its steps into the same `pending` slot. Because the host re-applies the whole settings snapshot on every sync, any settings change — from this plugin or another — dropped the operator's armed drive on top of the protocol's queued step, and the board held it until the next step boundary. Also in the modulation plugin: - `mod_freq_mhz` is parsed once as f64 and rounded to millihertz; the second u64 parse returned None as soon as the firmware echoed a decimal, which published frequency_millihz: 0 and cost A1 its fallback modulation period - a leased `SetOpticalDepth` now parks the operator's armed depth and `end_lease` restores it, so the board no longer holds the last sweep point's depth after an A1 amplitude sweep finishes
…emoise the fold `S_p(t)` divided by the whole sensor (`width * height`) while `q_p` on the same screen divided by ROI area minus masked pixels, and the rolling numerator counted events from outside the ROI and from masked pixels. With a small ROI that under-reported `S_p` by the ROI/frame ratio, and the status readout printed both numbers under the same "valid pixels" label. The fold is now built from ROI-filtered events and both quantities divide by `valid_pixel_count()`. `current_fold()` is memoised on a fingerprint of its inputs. It is called from `rolling_dataset`, `latest_rolling` (twice), `current_windows` and `current_response`, each allocating a `Vec<FoldedEvent>` over up to MAX_EVENTS — a single repaint could allocate and discard hundreds of megabytes at bench event rates. Also in the A1 plugin: - the no-EventStore fallback trims by the analysis window instead of growing to MAX_EVENTS and then freezing on a stale buffer while the plots still looked live - a failed pilot-window freeze clears the previously loaded windows, so the sidecar cannot record an earlier pilot's windows as this run's
The photodiode's display mode used to select the optical geometry the published log-contrast was computed in, so a UI toggle changed a scientific quantity that A1's amplitude sweep settles against. ADR 012 records that geometry follows the bench, the measured dark level is applied to both sides of the complement (where it cancels), and a withheld `a` states its reason. Also documents the A1 `N_valid` definition (ROI area minus masked pixels, the same denominator `q_p` uses) and generalises the rule in architecture.md: a published field's meaning must not depend on the publisher's UI state.
…etry Ports the per-frequency exact-event-count depth lock (ADR 013, renumbered from 012 to clear the contrast-geometry ADR) onto the branch that carries the photodiode and modulation fixes, and repairs what that combination exposes. The lock closes `a_cmd ← a_cmd · a₀/a_measured` against the photodiode-measured log contrast. On its own branch that value's geometry followed the photodiode's *display toggle*, so under the default RAW mode it was the reject-port detector contrast rather than the excitation contrast — every locked depth would have been wrong. It is correct only together with the fixed geometry. Then the measurement itself: `a = ln(I_exc,max / I_exc,min)` is peak-to-peak, but the photodiode estimated it over a fixed 16 384-sample window — 0.82 s at 20 kSa/s, less than one cycle for every f < 1.2 Hz. Below one cycle the robust extrema see an arc of the waveform, so `a` comes out low and phase-dependent. That is exactly the sub-hertz plateau reference the A1 protocol normalises |H(f)| against, and the lock *divides* by it: a truncated estimate does not add noise, it drives the commanded depth up trial after trial until it rails at 6.0 or the detector clips. The contrast window is now sized from the phase-0 markers to cover CONTRAST_WINDOW_CYCLES whole cycles, floored at the old fixed window and capped by what the ring retains, and `a` is withheld outright below one cycle. The retained markers cannot measure a period longer than the ring — once the ring holds under a cycle it holds at most one marker — so the interval is remembered as markers go past instead of recovered from what survived eviction. `window_seconds` and `covered_cycles` join the optical summary (additive in V1). In the lock: - Find a₀ refuses up front when the published window is under one cycle at the current frequency, naming the cache length to raise. A1 always knows f, so this also covers an owner whose own markers cannot prove it. - the per-trial dwell is at least one estimator window, so a trial cannot average the depth it just replaced; the deadline grows with it - readings are spaced by half a window instead of per service_revision. Consecutive revisions share nearly their whole window, so three of them said no more than one - the trial value is the median and the spread is a stability gate: readings straddling a₀ abort the lock instead of locking onto a drifting drive - the clip warning threshold sits below the estimator's own refusal, where it can actually fire, instead of above it where it never could - a lock-table save failure is appended to the result instead of being overwritten by it
An A1 event-count block is 7-12 frequencies over two or three decades, each one a Find a₀ and a Record a₀ point, repeated over three blocks. ADR 013 left that manual because the ordering decisions are scientific — but they are also expressible, and the checklist wants them frozen in the session plan anyway. Every gap between the two presses was also a gap in which a modulation settings sync could re-apply the operator's own depth on top of the found one. Start frequency sweep runs the whole ladder on ONE modulation lease: per point it retargets the drive frequency, waits for the phase-0 trigger to confirm the new period, runs the unchanged a₀ lock, and records the unchanged event-count point. Both children gained an inherited-lease mode, so they run on the ladder's lease instead of taking their own — which is the substantive guarantee: the operator's drive settings are locked out from the first frequency to the last, so the amplitude provably cannot move between a lock and the point that replays it. `ModulationCommandV1::SetDriveFrequency` is the frequency counterpart of SetOpticalDepth (additive in V1, same scoping: leased only, re-derived through the owner's own drive_command, refused for a manual DAC or constant drive). The owner parks the operator's armed frequency on the first retarget and restores it with the depth when the lease ends, so a finished ladder does not leave the bench on its last point. Robustness, which is most of the work: - the trigger confirms the frequency, not the firmware ACK. A point starts only once enough phase-0 markers at the *new* period agree with the commanded one - retained markers and events are dropped on every frequency change: the measured period is their mean spacing, so keeping them would confirm the new frequency against a mixture of the old drive and the new - pilot windows are dropped with them. Windows frozen at one period are a phase interval of that period; carrying them across would score a point in the wrong window, silently, because a fold always produces something - the plan is validated before the drive moves. The photodiode estimates `a` over one window for the whole ladder, so its lowest frequency decides whether the ladder is measurable at all — checked at the button press, not at the ninth point two hours in - an unreachable a₀, an unconfirmed frequency or a failed recording skips that point and names it in the summary; the remaining decades are still recorded. A refused frequency carries the owner's own wording into the skip The schedule is data: log spacing (|H(f)| is read per decade), ascending / descending / alternating / seeded-random order, and an optional low-frequency reference interleaved every N points. Every point's sidecar gains a [frequency_sweep] section with the executed position, the order and the seed, so a block is interpretable from its files rather than from a notebook. See ADR 014.
Starting an A1 recording appeared to succeed and then reported a finished run
immediately. What landed was a config sidecar in the chosen output folder, a
truncated camera .raw in the host process's working directory, and no .pdq at
all. Three defects combined:
- Every photodiode-leg failure jumped straight to stop_camera, so the host had
been recording for a few hundred milliseconds and produced a stub RAW that
still carried a complete finalization receipt. Photodiode faults also set
stop_requested, conflating them with an operator stop. The camera now runs its
full duration and closes as a camera-only recording instead.
- The specific cause ('set the data directory first') was overwritten by the
generic 'was incomplete' on the way out. The first cause is now preserved and
named in the closing message.
- The RAW, the PDQ, and the sidecar are written by three owners against three
roots, and the host's relative output path resolved to its working directory.
Once both recorders report finalization their files are closed and hashed, so
A1 now gathers them into <output folder>/<id>/ and records the final paths.
PDQ receipts report a label relative to the photodiode data directory, which
the owner now publishes in its summary so the path can be resolved.
Also pre-flights the photodiode before starting the camera (not reporting, not
connected, no data directory, leased elsewhere), surfaces the blocker in the
status view while idle, and stops A1's own pipeline restart from wiping the
row's pilot windows, background floor, and collected response points.
Cherry-picked from fix/stage-a-a1-recording onto this branch, because the
frequency ladder records every one of its points through this coordinator:
without these fixes an unattended ladder would write a folder of truncated
RAWs and no PDQ at all. Its ADR is renumbered 012 to 015 — the third branch to
have claimed 012 independently.
Refs ADR 015, revises ADR 009 decision 3.
The gather-after-finalization from the previous commit left the photodiode's own Data directory in the critical path: an A1 run still failed when it was unset, and changing it mid-experiment could move files out from under a measurement. `PdqStartSpecV1` gains an additive `root_dir: Option<String>` — an absolute directory the client wants the recording written below, replacing the owner's configured data directory for that run. Every safety rule the owner already had survives below the new root (the path stays relative, `..` and non-normal components refused, parent components must be real directories rather than symlinks, the resolved target must stay below the root), plus the root itself must be absolute. An A1-driven run therefore no longer depends on the photodiode's Data directory at all, and the pre-flight stops checking it. The camera RAW still has to be gathered after finalization: the host resolves plugin recording paths below its own output directory and rejects absolute ones, and that rule lives in the other repository. The gather now also runs over the PDQ, where it is normally a no-op because the file was opened in place — which means a run that dies before finalization still leaves its PDQ in the measurement folder. PDQ receipts report the label the client requested, so A1 resolves it against the root it named, falling back to the owner's published `data_dir` and preferring whichever exists — an owner too old to honour `root_dir` still yields a correct path. Cherry-picked from fix/stage-a-a1-recording; revises ADR 015 decision 4.
# Conflicts: # plugins/stage-a-a1/src/runtime.rs # plugins/stage-a-modulation/src/lib.rs # plugins/stage-a-photodiode/src/lib.rs
# Conflicts: # docs/features/README.md # docs/features/stage-a-optical-waveform.md # plugins/stage-a-modulation/README.md # plugins/stage-a-modulation/src/lib.rs
Reworks the Stage-A bench across all four crates so a whole `q_p(a, f)` survey runs unattended and every recording carries its own provenance. A1 (ADR 020-023, 027, 028) - `a` comes from the photodiode (measured) or the commanded calibrated drive (open loop), and the source is recorded in every artefact - `Find a₀` and the lock table apply only to a measured depth; a commanded depth has nothing to search for - the frequency ladder becomes an outer loop over the amplitude sweep, one `a₀` point or a whole depth sweep per `f` - surveys run from a `.csv` or `.toml` protocol naming `ū`, `f` and `a` per recording; examples ship in `protocols/` and install with the plugin - die temperature, pixel dead time and illumination land in every sidecar, absent rather than `0` when the host cannot report them - host sensor telemetry is compacted column-wise into the measurement folder under the run's own name Photodiode and calibration (ADR 017, 019, 024) - the detector learns `I_tot` from the brightest reading it takes; the four anchor settings are gone and dark cancels in the complement - rail detection is span-relative, and a withheld `a` names its gate across the plugin boundary - the transfer sweep judges residual and hysteresis against its own measured noise, over the window it swept Modulation (ADR 025, 026) - `ū` and `a` clamp into the achievable range instead of reverting, so every mode stays selectable - the applied lobe crosses the UI-mirror/live-worker boundary through a process-global generation Gating fixes (ADR 018): A1 requires only the output folder; the measurement and `I_k` ids are provenance, not preconditions.
Both device owners cap the automation lease TTL they hand out at 60 s — a dead-man switch, and correct — but the clamp is silent: the request comes back `Applied`, so A1 believed it held the drive for a whole survey when it held it for a minute. Renewing once per point was therefore only ever right for points shorter than the cap. The shipped example protocol has a 40 s row, every row also pays the camera and photodiode handshakes, and `acquire_photodiode` asks for `duration_s + 60 s`, so any longer recording outlived its own leases. Past the deadline the modulation owner does what an expired lease must do — STOP, output off — and that single event surfaced as three unrelated-looking faults at once: the owner refusing every later command for want of a lease, the sidecar refusing to write because the photodiode had no fresh optical summary, and a "no trigger signal" line that read exactly like an unplugged EXT_TRIGGER cable but was the drive being off. A1 now heartbeats both leases against `expires_at_unix_ms` from the owner's own snapshot — which both owners already published and A1 never read — renewing once less than 20 s of the granted window is left, no more often than every 2 s. The owners' cap is untouched: raising it to survey length would fix the symptom by deleting the safety property that motivated it. Also fixes `on_discontinuity` asking `recording.is_active() || sweep.is_some()` to decide whether a SourceChanged was self-inflicted. Starting and stopping the host recorder raises it twice per recording, and between two points of a protocol or a ladder neither is true — so the run's own boundary was treated as an idle-time reset and wiped the survey's pilot windows, background floor and response curve mid-run. It now asks `automation_active()`: the same set `request_stop` winds down. Refs ADR 029. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The compacted `<stem>.sensor.json` depends on a companion CSV the host writes only when its own "Record sensor monitoring" switch is on. A1 cannot set that switch and cannot query it, so an absent readout was indistinguishable from a camera with no monitoring block — and the sidecar field's own doc comment said exactly that, which is what made the absence so hard to trace. A finished run that produced no readout now says so in the panel and names the switch, instead of leaving a survey to discover months later that it kept none of its bench conditions. The single-point readings in `[sensor]` ride the context bus and are unaffected either way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Installing a plugin required a Rust toolchain, a sibling augur-rs checkout and a working cargo, which made every measurement PC a development machine. Build all runtime plugins on each pull request and each push to main for macOS arm64/x86_64, Linux x86_64 and Windows x86_64, staged in the exact layout ~/.augur/plugins/ expects, so installing is a copy. main also publishes a rolling plugins-latest release, because workflow artifacts need a login and expire after 90 days while a bench should be able to curl a URL. The workspace depends on the host by path, so the job lays out two sibling checkouts. build-runtime-plugins.sh patches a git source whenever it finds a sibling augur-rs checkout; with path deps that patch matches nothing but still costs a fetch, so the checkout's .git is dropped right after cloning. CI calls the repository's own build and install scripts instead of restating the install layout in YAML — those scripts already own plugin discovery, library naming, A1's protocols folder and the macOS install-name rewrite. Plugins are dlopened into the host process, so pin rust-toolchain.toml to the same 1.95.0 augur-rs pins and read the channel out of that file rather than naming a version in the workflow. Every bundle carries a BUILD-INFO.txt with the augur-rs revision and rustc version behind it, which is what makes an ABI mismatch reported from the bench answerable.
Two defects the first run exposed, both independent of the plugin sources. The Linux job borrowed augur-rs/.github/scripts/install-linux-deps.sh from the host checkout to avoid keeping a second dependency list. That script does not exist on every augur-rs revision the job can be pointed at, so the Linux build failed on the value of augur_rs_ref rather than on anything in this repository. It was also a superset: it installs the X11/Wayland/GL stack for the GUI, which no plugin crate links. Install what the plugins actually need instead — pkg-config and libudev-dev for serialport. setup-rust-toolchain injects RUSTFLAGS="-D warnings" by default. That is right for a lint job and wrong for one that ships artifacts: a dead-code warning in one plugin would have denied the bench a bundle for all of them. Lint gating belongs in its own job. Also record what the run proved about the repository itself: these plugins do not compile against augur-rs main, which lacks the TableSchema, host-view and dataset-descriptor API they use.
augur-rs main still has a two-field TableSchema and no Scatter3dFromTable, HostDatasetDescriptor.relations/display or HostViewRegistry.actions, all of which the plugins in this repository already use. Defaulting AUGUR_RS_REF to main is therefore a guaranteed red build that never hands the bench a bundle. Default to the open host branch that does carry the API instead, and record the coupling in the brief and the ADR. BUILD-INFO.txt already names the exact host ref and SHA behind every library, so this stays visible rather than becoming folklore. Move the default back to main in the same commit that the host API lands there.
The eveSMLM chain expressed its stage dependencies directly: fitting depended on the candidates crate, post-processing on the fitting crate. Plugin crates are cdylibs that each export augur_plugin_vtable, so linking one plugin's rlib into another pulled that symbol in twice. Apple's linker tolerates the duplicate. rust-lld and MSVC's link.exe do not: rust-lld: error: duplicate symbol: augur_plugin_vtable LNK2005: augur_plugin_vtable already defined … fatal error LNK1169 That went unnoticed for as long as the only build machine was a Mac. The first CI run on four platforms found it: macOS produced a complete bundle while Linux and Windows failed to link, which also denied the bench a Windows bundle for the Stage-A plugins, since the build is all-or-nothing. Move everything that crosses a stage boundary into evesmlm-types, a plain library crate that exports no vtable — the wire contract plus the current-localization dataset and registry builders that both fitting and post-processing publish. Plugin-private state stays with its plugin: the candidate tracker's TrackedCluster moves back into the candidates crate. Each plugin still re-exports the names it used to own, so downstream use paths keep compiling. This generalizes what stage-a-plugin-contract already does for the Stage-A owners, and replaces the repo convention that shared types belong in the producing plugin's crate.
Port discovery filtered candidates by the two Unix name patterns (`cu.usbmodem`, `ttyACM`) before probing. Windows names no device — every port is `COMn` — so an attached, correctly driven Teensy was filtered out before any probe could run, and both plugins reported "no USB serial device found (looked for usbmodem/ttyACM)": the two things Windows cannot produce. Move the filter into `stage-a-io::transport::candidate_ports()`, where it is platform-aware: the callout node on macOS, `ttyACM*` on Linux, and every USB-classified port on Windows, falling back to the whole list when the OS classifies nothing. What identifies the device is still the probe (HELLO on the command port, PDA1 frames on the stream port); the filter only keeps probes off unrelated ports. Also open every port with DTR asserted. macOS and Linux do this implicitly, Windows does not, so a sketch gating on `if (Serial)` would stay silent even once the right port was found. The filter existed in four places in two implementations; it is now one function with unit tests covering both platform branches, and the failure message names the ports the OS actually enumerated. Refs ADR 032
Excel's "CSV UTF-8" — the obvious save format on a Windows bench — writes a UTF-8 byte-order mark. Unstripped it becomes part of the first header cell, so `mean_u` stops matching `mean_u` and the protocol is refused for missing a required column that is plainly there. The TOML form fails its parse outright. Neither message points at an invisible character. Strip the BOM once for both readers. CRLF was already handled by `str::lines()`; it now has a test so it stays that way. Refs ADR 027
The contract gains the A2 step-latency acquisition configuration and its `PrepareA2` command, so the modulation owner can put the firmware into mode A2 without A2 reaching past it. The photodiode side gains an explicit detector placement. Until now the rejected-port complement was the only geometry the estimator knew, and it is the one geometry in which a DC dark level cancels. A direct sample of the camera or emission path does not cancel it, so those placements carry a captured lamp-off dark reference and refuse `a` until one exists. The splitter fraction travels as provenance only: a constant fraction cancels out of a log contrast and must not be used to rescale it.
The modulation owner is the only crate that speaks to the command port, so putting the board into A2 mode belongs here rather than in A2 itself. The shipped A1 laboratory protocols are parsed as fixtures and checked against the coupled calibrated-drive limits, so a protocol that asks for a `ū`/`a` pair the drive cannot reach is caught here instead of at the bench.
The detector could only be read in the PBS rejected port, where a DC dark level cancels out of the complement. A direct sample of the camera or emission path has no complement to cancel against, so those placements need a real lamp-off reference and refuse `a` until one is captured. The dark value is activated by an explicit button, not by the numeric field alone: a settings replay from the UI mirror would otherwise be able to overwrite a captured reference with a stale draft. Placement is the scientific geometry and is kept separate from the RAW/EXCITATION chart mode, which is only a display choice. The shipped A1 laboratory protocols are parsed as fixtures and checked against the production ring at the bench sample rate, so a sub-hertz rung that could not retain two whole cycles is caught here.
A1 copied the host's camera snapshot, readback, ROI, mask and bias codes into its own sidecar. The copies were not independent measurements and could disagree with the host TOML sitting next to the same RAW, so the v2 sidecar cross-references that file instead of duplicating it and keeps only the conditions that actually move during a run. What the sidecar could not say before, it now says. `[protocol]` names the schedule that produced the point — name, author version, source file, SHA-256 and row identity — and the source is archived once per content hash into the measurement folder, because a path on the operator's disk is not durable provenance. `[depth]` separates the commanded, measured and analysed `a` instead of overlapping names for the three. `[photodiode]` records the detector geometry. The complement model is only correct in the rejected port; applying it to a direct camera or emission-path sample is a scientific error, so the placement travels with every point rather than being assumed. The eight laboratory protocols now in use ship as fixtures. Refs ADR 039.
A2 measures how long the fluorescence chain takes to answer a step. It owns no hardware: the Teensy belongs to the modulation owner and the detector to the photodiode owner, so A2 drives both over the service plane and never opens a port itself. The protocol file is the aggregate root. Optical configuration, the qualified hardware gates, controller settings and the ordered recording rows are validated together before anything happens, because a bring-up file that is half filled in is useful to read and must not be runnable — an explicit TBD gate refuses preflight rather than silently defaulting. Dark rows and stepped rows are separate acquisition types: a dark row forces the modulation safe and carries no trigger-count gate, so a missing trigger cannot be reported as a dark measurement. The plugin records acquisition provenance and live integrity evidence only. Censored first-event fits stay offline, where the model can be chosen with the data in hand. Refs ADR 038.
A4 walks a protocol of diff_on/diff_off pairs at one fixed optical condition and records a RAW file at each, with enough provenance to read an event rate against a threshold setting months later. Every point is confirmed against the sensor's own bias readback before it records. A point whose absolute codes disagree with the row, or whose confirming reading is missing or older than the change, is skipped: a threshold file that is wrong about which codes were live on the die is wrong in a way nobody can detect afterwards. Refusals and flags are split on purpose. The event filters being off, the codes being confirmed and the file being whole are gates, because without them the number means nothing. Temperature drift, illumination drift and event rate are flags that mark a point and keep it — whether a 2 °C drift invalidated a threshold is a judgement to make later with the file in hand, and a runner that discarded the point would have thrown away the evidence for making it. A limit whose quantity could not be measured is flagged rather than passed. The survey runs on the host's generic camera-configuration session: it opens by having the host confirm the configuration the bench is on, and builds every point by cloning that snapshot and setting exactly two fields, so fo, hpf, refr, the ROI, the mask and the trigger are carried forward unchanged. A test asserts that equality field by field. Refs ADR 035, augur-rs ADR 037.
muthmann
force-pushed
the
agent/stage-a1-camera-configuration
branch
from
August 13, 2026 15:35
e490051 to
1b663e9
Compare
The bench time a survey still needs was announced once, in the transient message, and the first point's own message overwrote it a second later. An operator who looked away had no way to get it back. It belongs on the protocol's own status line, where it stays: the points still to run, settling included, counting the point in flight whole. `total_seconds` becomes `remaining_seconds(0)`, and one shared formatter gives both the opening message and the status line the same units.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings
mainup to date with 56 commits of Stage-A work.mainlast moved atPR #6; everything since then has sat on feature branches or in the working
tree.
What lands
hardware; drives the modulation and photodiode owners over the service plane.
The protocol file is the aggregate root, so a half-filled bring-up file is
readable but not runnable (ADR 038).
against the sensor's own bias readback before it records; refusals and flags
are split on purpose (ADR 035).
complement model is only correct in the rejected port, so the direct paths
need a captured lamp-off dark and fail closed without one.
copying the snapshot, identifies the protocol row that produced each point,
and separates commanded/measured/analysed
a(ADR 039).PrepareA2, the A2 acquisition configuration, and thephotodiode placement/dark-reference types.
A4 migrated off the removed host verb
A4 was written against
apply_biases(augur-rs ADR 036), a verb two fieldswide, added for A4 alone. augur-rs removed it in favour of the generic
camera-configuration session (augur-rs ADR 037), and A4 had not been migrated —
the workspace did not compile.
The narrowness moved from the wire into the plugin. A4 now opens each run with
ApplyCameraConfiguration { Current }, keeps the snapshot the host confirms,and builds every point by cloning it and setting exactly two fields; the
closing restore is
RestoreCameraConfiguration. A test asserts a point'sconfiguration equals the baseline field by field except for
diff_on/diff_off.This is the minimum to make A4 build and stay honest. The plugin is not in use
yet and will be revisited before it is.
Known issue, not fixed here
stage-a-modulationandstage-a-photodiodeeach addedaugur-plugin-stage-a-a1as a dev-dependency for the newprotocol_validation_tests.rsfixtures. This is the plugin→plugin dependencyADR 031 forbids: both crates export
augur_plugin_vtable, so the test binarylinks the symbol twice.
It does not affect the shipped
cdylibs or CI, which builds bundles and doesnot run
cargo test. It passes locally only because Apple's linker toleratesduplicate symbols —
rust-lldandlink.exedo not, socargo test --workspacewould fail on Linux and Windows. Fixing it means moving the A1 CSV parser into
stage-a-plugin-contract, which is a design call worth making separately.Verification
cargo test --workspace— 30 suites, all green (macOS)cargo fmt --all --check— cleancargo clippy -p augur-plugin-stage-a-a4 --all-targets— cleanscripts/build-runtime-plugins.sh --profile releasethenscripts/install-built-plugins.sh --profile release— 11 plugins built andinstalled, A2 and A4 among them
Nothing was run against real hardware.