Skip to content

alfred-mls-nav on the single DimSlam module - #3597

Draft
jeff-hykin wants to merge 21 commits into
jeff/fix/ray_tracing_tffrom
jeff/feat/alfred_dim_slam
Draft

alfred-mls-nav on the single DimSlam module#3597
jeff-hykin wants to merge 21 commits into
jeff/fix/ray_tracing_tffrom
jeff/feat/alfred_dim_slam

Conversation

@jeff-hykin

Copy link
Copy Markdown
Member

Stacked on jeff/feat/dim_slam and jeff/fix/ray_tracing_tf; final branch of the stack.

  • alfred-mls-nav runs the single DimSlam module: cuVSLAM on the D455 IR pair + ESKF over visual, wheel odometry and the D455 IMU, in one process
  • merges the robot-proven alfred work from jeff/feat/alfred-mls-nav: rerun camera layout, D455 mount transform, world_frame="odom" on the voxel map, effector event-loop fix, 2 s replay buffer
  • CuvslamOdometry + OdometryFusion blueprint pair replaced by one DimSlam.blueprint; alfred_cuvslam.py and odometry_fusion.py removed
  • demo-cuvslam-replay moves to dimos/mapping/dim_slam and drives DimSlam

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 55.17241% with 221 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/robot/diy/alfred/replay.py 36.09% 108 Missing ⚠️
dimos/robot/diy/alfred/effector_high_level.py 20.33% 47 Missing ⚠️
dimos/mapping/odometry_hist.py 55.55% 16 Missing ⚠️
dimos/memory2/replay.py 74.13% 7 Missing and 8 partials ⚠️
dimos/mapping/dim_slam/demo_cuvslam_replay.py 74.00% 13 Missing ⚠️
dimos/robot/diy/alfred/blueprints/vis_nav.py 73.80% 11 Missing ⚠️
dimos/visualization/rerun/bridge.py 12.50% 7 Missing ⚠️
dimos/core/coordination/worker_manager_python.py 92.85% 0 Missing and 1 partial ⚠️
...mos/e2e_tests/test_manipulation_planning_groups.py 91.66% 1 Missing ⚠️
dimos/mapping/dim_slam/dim_slam.py 88.88% 1 Missing ⚠️
... and 1 more
@@                     Coverage Diff                     @@
##           jeff/fix/ray_tracing_tf    #3597      +/-   ##
===========================================================
+ Coverage                    71.70%   75.92%   +4.22%     
===========================================================
  Files                         1329     1234      -95     
  Lines                       131490   119495   -11995     
  Branches                     12097    11142     -955     
===========================================================
- Hits                         94287    90731    -3556     
+ Misses                       34097    25676    -8421     
+ Partials                      3106     3088      -18     
Flag Coverage Δ
OS-ubuntu-24.04-arm 70.26% <41.98%> (-0.11%) ⬇️
OS-ubuntu-latest 72.10% <53.34%> (-0.05%) ⬇️
Py-3.10 72.10% <53.34%> (-0.05%) ⬇️
Py-3.11 72.10% <53.34%> (-0.05%) ⬇️
Py-3.12 72.10% <53.34%> (-0.05%) ⬇️
Py-3.13 72.10% <53.34%> (-0.05%) ⬇️
Py-3.14 72.10% <53.34%> (-0.05%) ⬇️
Py-3.14t 72.10% <53.34%> (-0.05%) ⬇️
SelfHosted-Large 29.75% <26.12%> (-0.13%) ⬇️
SelfHosted-Linux 35.80% <18.82%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/mapping/dim_slam/demo_cuvslam_realsense.py 75.00% <100.00%> (ø)
dimos/memory2/test_replay.py 100.00% <100.00%> (ø)
dimos/robot/all_blueprints.py 100.00% <ø> (ø)
...imos/robot/diy/alfred/blueprints/alfred_mls_nav.py 100.00% <100.00%> (ø)
dimos/robot/diy/alfred/blueprints/alfred_replay.py 100.00% <100.00%> (ø)
dimos/core/coordination/worker_manager_python.py 71.18% <92.85%> (+1.86%) ⬆️
...mos/e2e_tests/test_manipulation_planning_groups.py 85.45% <91.66%> (+20.71%) ⬆️
dimos/mapping/dim_slam/dim_slam.py 64.78% <88.88%> (+1.28%) ⬆️
dimos/mapping/ray_tracing/test_transformer.py 8.19% <0.00%> (ø)
dimos/visualization/rerun/bridge.py 40.97% <12.50%> (+0.34%) ⬆️
... and 6 more

... and 160 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jeff-hykin
jeff-hykin force-pushed the jeff/feat/alfred_dim_slam branch from 338aa41 to 2259bc2 Compare August 21, 2026 10:48
@jeff-hykin
jeff-hykin changed the base branch from jeff/feat/dim_slam to jeff/fix/ray_tracing_tf August 21, 2026 10:48
@jeff-hykin
jeff-hykin force-pushed the jeff/fix/ray_tracing_tf branch from c8e8540 to c256a9e Compare August 21, 2026 22:47
The camera pointcloud was a 5 fps stand-in on the lidar stream and needed
colour streaming to assemble at all. DimSlam's depth_cloud follows the depth
frame rate and carries the range gate, so RayTracingVoxelMap consumes it
directly and the camera stops assembling a pointcloud nothing reads.
The camera's python pointcloud assembly tops out near 5 Hz regardless of
pointcloud_fps, and the full-resolution depth_cloud drowned the mapper.
DimSlam now publishes the cloud range-gated at 6 m and 3x3-median
decimated (42 mm pitch at the gate, under the 50 mm voxel), and
RayTracingVoxelMap consumes it in place of the camera pointcloud.
@jeff-hykin
jeff-hykin force-pushed the jeff/feat/alfred_dim_slam branch from b0e9b0b to c75734b Compare August 21, 2026 22:48
The multiprocessing resource tracker exits only once every copy of its
keep-alive fd is closed, and its __del__ waits for that during finalization.
rerun_bindings.spawn leaks inheritable fds into a viewer that outlives the run,
and the forkserver holds a copy of its own; either one hangs stop() forever.
Spawn the viewer through Popen, which closes fds above 2, and terminate the
forkserver outright rather than waiting on an EOF that cannot arrive.
alfred-mls-nav carried the whole vision-only stack inline, so anything that
wanted to run it against something other than the robot had to copy it and
drift. vis_nav holds every sensor-independent module -- DimSlam, the ray
tracing voxel map, MLS planning, Dan's planner and controller -- and
alfred-mls-nav is now just that plus the RealSense and high-level drivers.

DimSlam gains the depth2depth fields the stack passes it, so the decimated
cloud is built from the densified depth image rather than the raw one.
AlfredReplay publishes a recording's cameras, IMU and wheel odometry onto the
same stream names the live drivers use, so the blueprint is the replay source
plus vis_nav verbatim -- a replay exercises the code a real run does.

Both imagers are zipped rather than scheduled apart: under decode load they
drift past the tracker's 1 ms skew limit and the pair is rejected. Recorded tf
edges under odom/map/visual_odom are dropped so the replayed filter owns
base_link instead of it having two parents.

Each stream is paced against wall time on its own thread, so a consumer slower
than the recording stretches the run by an amount nothing can predict; a
done-file marks the point where every stream has drained, and a periodic line
reports the achieved rate and the spread between the leading and trailing
stream, which is what shows up downstream as images older than the tracker's
replay buffer.
…lock

A slow consumer used to pull its stream ever further behind the anchor,
something a live sensor can never do. Frames whose slot has passed are now
dropped (decode deferred so drops cost a db fetch, not an image decode),
paced by a per-subscription thread with chunked sleeps because macOS timer
coalescing multiplies a background process's sleep requests severalfold.
Alfred stereo is paired by stamp instead of arrival order, since
uncoordinated drops would misalign an ordinal zip forever.
The whole-repo flake input hash changes with any commit, invalidating the
src derivation even though the build only consumes two crates from it.
Re-copying those crates through builtins.path gives them content-addressed
store paths, so the binary is reused until the crates themselves change.
The __init__ lock was immediately replaced in main(), the max(hz, 1e-3)
guarded a config value that is never zero, and the poll docstring
repeated what now lives as comments at the negation and twist sites.
jeff-hykin and others added 4 commits August 22, 2026 23:31
portal ships in the misc extra, so a top-level import breaks blueprint
validation tests in environments without it.
dimos.memory.transform/type/store no longer exist in the tree, so these
imports failed at runtime and under mypy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants