Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 36 additions & 26 deletions docs/design/declarative_expert_program_plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@
branches. A real CUDA/cuRobo dynamic-obstacle recovery gate is landed and
runs conditionally when cuRobo is installed, CUDA is available, and GPU/slow
tests are explicitly enabled. Open Drawer has completed its
supported-simulation physical run; repeated cube pick/place has completed one
Pick/Place/settle/validator cycle, while the full three-cycle run remains in
threshold calibration. Dual-UR5/PGI HandOver has completed three consecutive
supported-simulation physical run; repeated cube pick/place now completes all
three independently observed Pick/Place/settle/validator cycles. A physical
gripper-command fault gate also proves held-object loss, symbolic invalidation,
real re-acquisition, and retry without simulator-side repair. Dual-UR5/PGI
HandOver has completed three consecutive
supported-simulation Pick/transfer/settle/validator runs using contact
dynamics only. Named trajectory-segment effect gates now block Pick lift,
Place retract, and HandOver source release until fresh physical evidence
Expand Down Expand Up @@ -1232,9 +1234,11 @@ it only when GPU and slow tests are explicitly selected.

Implementation status: the semantic facade, provider-free linking, canonical
compiler, bounded program preflight, and cross-segment sequential look-ahead are
implemented. Relation placement remains an exact typed integration capability;
a reusable production support-surface/container affordance and grounder are
follow-up work rather than inferred behavior.
implemented. Relation placement remains an exact typed integration capability.
Production support-surface and container bindings now declare a desired object
target frame relative to an object, articulation, or link parent; registration
installs their exact versioned grounders automatically. No geometric target is
guessed from a name, mesh, or bounding box.

Deliverables:

Expand All @@ -1260,16 +1264,19 @@ effects, row-local composite hysteresis kernel, canonical `SkillRuntime`, and
production simulation evidence ports are wired end to end. Segment-scoped
held-object guard requests, live evidence collection, row-local symbolic
invalidation, bounded Pick retry, and typed external-recovery hand-off are also
implemented. Physical simulation acceptance is partial: Open Drawer and one
cube Pick/Place/settle/validator cycle have completed. The embodiment-owned
implemented. Physical simulation acceptance covers Open Drawer and all three
cube Pick/Place/settle/validator cycles. The embodiment-owned
dual-UR5/PGI HandOver slice now completes Pick, transfer, terminal
physical-effect verification, settling, and target validation through real
contact dynamics. Per-expectation terminal outcomes, core-owned failure
invalidation, row-local retry/recovery decisions, fail-closed deadline
reconciliation, and blocking named-segment effect gates are implemented.
Workflow-level re-acquisition is implemented through the preset-owned bounded
policy and canonical runtime. Real-simulation fault-injection coverage and the
full repeated-cube run remain validation work.
policy and canonical runtime. A supported-simulation fault gate now replaces a
bounded window of outgoing gripper commands with a real open command during
Place, observes pose-relation contradiction and symbolic invalidation, performs
a real Pick, retries Place, and completes the remaining program. The wrapper
never writes object pose, velocity, attachment, constraint, or task state.

Deliverables:

Expand Down Expand Up @@ -1314,9 +1321,9 @@ strict decoder/loader, lazy compiler, environment/CLI integration, shared
simulation factory, and three-segment cube program are implemented. The task
combines declarative program configuration with typed scene/profile integration
declarations and installs the shared adapter without overriding task motion
generation. A supported-simulation run has completed the first physical
Pick/Place/settle/validator cycle; completing all three cycles remains an
acceptance item while thresholds are calibrated.
generation. The UR5 embodiment preset uses 100 motion samples while retaining
the 0.08-rad tracking gate and bounded replanning. A supported-simulation run
now completes all three physical Pick/Place/settle/validator cycles.

Deliverables:

Expand Down Expand Up @@ -1361,13 +1368,16 @@ trajectories in task code.

### Phase 7: parallel execution and PourWater

Implementation status: the schema/runtime contracts and fail-closed safety
boundary are implemented. Schema
Implementation status: the schema/runtime contracts, fail-closed safety
boundary, and production simulation validator are implemented. Schema
version 2 provides explicit parallel branches and barriers; static resource
conflict analysis, shared-clock lane coordination, deterministic hold padding,
transport/safety validation, row-local failure and cancellation, timeouts, and
deterministic state merge are covered by tests. A production simulation safety
validator and parallel physical integration remain pending. The PourWater task
deterministic state merge are covered by tests. The cuRobo validator assembles
the exact aggregate joint segment, densifies it under a configured maximum joint
step, and checks every sample against joint bounds, self collision, and the
registry-backed live world without replanning or replacing the command. A
parallel physical integration remains pending. The PourWater task
migration is outside the current scope because it would require modifying
Action Bank code.

Expand Down Expand Up @@ -1516,22 +1526,22 @@ The design is complete when all of the following hold:
- [x] Automatic grasping tracks target revisions and receives downstream object
goals without caller duplication.
- [x] `Place` is object-centric and consumes verified held-object state.
- [ ] Built-in grasp, release, handover, and supported articulation effect
monitors work in simulation. The dual-UR5/PGI HandOver vertical slice is
physically validated; remaining skill/embodiment coverage keeps this
aggregate item open.
- [ ] Grasp and handover simulation gates retain objects through configured
- [x] Built-in grasp, release, handover, and supported articulation effect
monitors work in simulation across the repeated cube, dual-UR5/PGI
HandOver, and Open Drawer vertical slices.
- [x] Grasp and handover simulation gates retain objects through configured
drive/contact dynamics only; no monitor or runtime path creates a
synthetic attachment, freezes the object, or overrides its pose.
- [ ] Physical held-object loss is observed as effect failure, invalidates the
- [x] Physical held-object loss is observed as effect failure, invalidates the
affected symbolic relation, and exercises bounded recovery rather than
being hidden by a simulator-side attachment. The segment-aware observation,
row-local core-owned invalidation, per-expectation terminal
reconciliation, fail-closed deadline handling, bounded Pick/retained-Place
retry, typed recovery boundary, and blocking acquisition/release gates are
implemented. Preset-owned per-row workflow re-acquisition now performs
real `Pick` and semantic-call retries; real-simulation fault injection
remains open.
real `Pick` and semantic-call retries. The supported-simulation gate opens
the real gripper during Place, observes the loss, re-acquires the fallen
cube, and completes the retried call.
- [x] Repeated sub-threshold motion eventually publishes the correct scene
revision.
- [x] Custom actions have a documented and tested intentional hard-break
Expand All @@ -1543,7 +1553,7 @@ The design is complete when all of the following hold:
through `env.step()`.
- [x] No program post-policy, effect, or tracing integration depends on
hard-coded waypoint indices.
- [ ] Repeated cube pick/place completes at least three lazy, independently
- [x] Repeated cube pick/place completes at least three lazy, independently
observed program/demo segments with settle/effect/validation metadata.
- [x] Version 1 uses one shared program/call barrier while per-environment task
state, effects, recovery, eligibility, success, and failure remain
Expand Down
23 changes: 12 additions & 11 deletions docs/design/expert_program_rollout_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,26 @@ This is a deterministic, static Phase 8 snapshot of checked-in framework and int
| OperateArticulation | framework-tested | per-embodiment integration | Typed articulation goals and execution contracts are covered. |
| Articulation effect | framework-tested | per-embodiment integration | Joint-state terminal effect validation is covered. |
| V1 sequential | framework-tested | per-task integration | Ordered call execution and failure propagation are covered. |
| HandOver | framework-tested | integration-required | No landed task integration is claimed by this report. |
| Place relation (on/inside) | framework-tested | integration-required | Embodiment frames and relation validators must be supplied. |
| HandOver | framework-tested | per-embodiment integration | Coordinated effects and bounded recovery are covered. |
| Place relation (on/inside) | framework-tested | per-scene integration | Standard support/container target-frame bindings install exact grounders. |
| Registered call | framework-tested | integration-required | Production registration must declare and validate its concrete contract. |
| V2 parallel | framework-tested | integration-required | Fail-closed by default; production use requires an authoritative validator. |
| V2 parallel | framework-tested | integration-required | Joint/cuRobo validation is available; physical parallel acceptance remains. |

Parallel execution remains fail-closed by default. Resource declarations alone do not authorize production concurrency; the selected embodiment must provide an authoritative validator.

## Checked-in Integration Matrix

Only the two checked-in vertical slices below are classified as integration/production code. Physical acceptance is tracked separately.
Only the checked-in vertical slices below are classified as integration/production code. Physical acceptance is tracked separately.

| Embodiment | Task | Skill contract | Terminal effect | Program schema | Code status | Physical acceptance |
| --- | --- | --- | --- | --- | --- | --- |
| UR5 | Cube Pick + Place | Pick + Place(at) | attach/release | V1 sequential | checked in | pending: one cycle passed; full three-cycle gate remains |
| UR5 | Cube Pick + Place | Pick + Place(at) | attach/release | V1 sequential | checked in | fixed-seed three-cycle and physical-loss recovery slow gates |
| CobotMagic | Open Drawer | OperateArticulation | articulation effect | V1 sequential | checked in | fixed-seed supported-simulation slow gate; not release-required |
| Dual UR5 + PGI | HandOver | Pick + HandOver | attach/transfer | V1 sequential | checked in | three consecutive supported-simulation contact-dynamics runs |

HandOver, Place relations (`on`/`inside`), Registered calls, and V2 parallel are framework-tested but integration-required. They are intentionally not listed as checked-in integrations.
Place-relation bindings are reusable scene integration rather than a task vertical slice. Registered calls and V2 parallel remain integration-required; physical parallel acceptance is still open.

Both checked-in environment classes have zero task-local motion or demo-generation overrides; `test_task_classes_do_not_override_motion_or_demo_generation` keeps that structural metric at zero.
The checked-in environment classes have zero task-local motion or demo-generation overrides; `test_task_classes_do_not_override_motion_or_demo_generation` keeps that structural metric at zero.

## Migration Size Snapshot

Expand All @@ -43,15 +44,15 @@ Counting rule: `lines` is the number of raw LF (`0x0A`) bytes; `bytes` is the ra

| Task | Baseline lines | Current lines | Line delta | Baseline bytes | Current bytes | Byte delta | Current source files |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Cube | 598 | 366 | -232 (-38.8%) | 23912 | 12448 | -11464 (-47.9%) | `embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.py`<br>`embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yaml` |
| Drawer | 245 | 246 | +1 (+0.4%) | 8833 | 8391 | -442 (-5.0%) | `embodichain_tasks/embodichain_tasks/tableware/open_drawer.py`<br>`embodichain_tasks/configs/expert_program/tableware/open_drawer.json` |
| Total | 843 | 612 | -231 (-27.4%) | 32745 | 20839 | -11906 (-36.4%) | the four files above |
| Cube | 598 | 395 | -203 (-33.9%) | 23912 | 13272 | -10640 (-44.5%) | `embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.py`<br>`embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yaml` |
| Drawer | 245 | 265 | +20 (+8.2%) | 8833 | 8916 | +83 (+0.9%) | `embodichain_tasks/embodichain_tasks/tableware/open_drawer.py`<br>`embodichain_tasks/configs/expert_program/tableware/open_drawer.json` |
| Total | 843 | 660 | -183 (-21.7%) | 32745 | 22188 | -10557 (-32.2%) | the four files above |

## Demo Success Measurement

`scripts/benchmark/expert_program/demo_success.py` executes each fixed seed exactly once, always discards the episode buffer, and counts executor exceptions as failed rows. It writes raw JSON plus a three-table Markdown report. Its CLI supports offline raw-JSON re-aggregation and an explicit `--run-simulation` mode that constructs one standard Gym environment from Gym and Expert Program configurations.

No success-rate result or release gate is checked in yet. Open Drawer has a single real-simulation smoke pass, while repeated Cube still needs the tracking-threshold decision and three-cycle physical acceptance before a fixed-seed rate is meaningful.
No multi-seed success-rate or release gate is checked in yet. Open Drawer has a real-simulation smoke pass; repeated Cube has a fixed-seed three-cycle pass plus physical-loss/re-acquisition gate; and HandOver has three consecutive contact-dynamics runs.

## Drift Check

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ embodichain.lab.gym.envs.expert_program
ExpertProgramEnvironmentMixin
ExpertProgramEnvironmentAdapter
SimulationSceneBinding
SupportSurfaceAffordanceBinding
ContainerAffordanceBinding
SimulationResourceEndpointBinding
SimulationRobotResourceBinding
RobotResourceBinding
Expand All @@ -25,6 +27,7 @@ embodichain.lab.gym.envs.expert_program
SimulationExpertProgramFactory
SimulationSegmentPolicyPort
ControlCommandStateEvidenceTracker
CuroboParallelSafetyValidatorFactory

.. currentmodule:: embodichain.lab.gym.envs.expert_program

Expand Down Expand Up @@ -80,6 +83,12 @@ Simulation integration
.. autoclass:: SimulationSceneBinding
:members:

.. autoclass:: SupportSurfaceAffordanceBinding
:members:

.. autoclass:: ContainerAffordanceBinding
:members:

.. autoclass:: SimulationResourceEndpointBinding

.. autoclass:: SimulationRobotResourceBinding
Expand All @@ -104,3 +113,6 @@ Simulation integration

.. autoclass:: ControlCommandStateEvidenceTracker
:members:

.. autoclass:: CuroboParallelSafetyValidatorFactory
:members:
Loading