Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 33 additions & 41 deletions docs/design/declarative_expert_program_plan.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
# Declarative Expert Programs and Unified Semantic Skill Runtime

- Status: implementation in progress; Phase 0 and PR1 complete, PR2A and PR2B
implemented on stacked feature branches
- Baseline: `main@e445133c79c8b32019dab1c844b799b43a1658d6`
- Last updated: 2026-08-10
- Status: implementation in progress; Phase 0 and Phase 1 are complete on
`main`, and Phase 2 is next
- Baseline: `main@f4ffb6608f41ea1eaee3020714412e31549dbda7`
- Last updated: 2026-08-18
- Related issues: [#471](https://github.com/DexForce/EmbodiChain/issues/471),
[#474](https://github.com/DexForce/EmbodiChain/issues/474)
- Related implementation:
[#475](https://github.com/DexForce/EmbodiChain/pull/475)
[#475](https://github.com/DexForce/EmbodiChain/pull/475),
[#517](https://github.com/DexForce/EmbodiChain/pull/517),
[#487](https://github.com/DexForce/EmbodiChain/pull/487)

## 1. Executive summary

Expand Down Expand Up @@ -91,12 +93,11 @@ sessions, or verifiers.

## 4. Baseline on current `main`

This plan is updated against committed `main@e445133c` after PR #475. The
implementation series is stacked from that baseline: PR1 is complete on
`refactor/atomic-actions-phase0`, PR2A is implemented by
`feat/atomic-action-pr2a-scene-registry`, and PR2B is implemented by
`feat/atomic-action-pr2b-robot-skill-profile`. These status statements do not
imply that the stacked changes have landed on `main`.
This plan is updated against committed `main@f4ffb660`. PR #517 simplified the
atomic-action core, and PR #487 landed the complete Phase 1 foundation. The
scene registry and robot skill profile APIs are available, but official task
environments have not adopted them yet; that rollout starts only after the
semantic compiler and runtime exist.

| Capability | Current main | Design consequence |
|---|---|---|
Expand All @@ -107,6 +108,8 @@ imply that the stacked changes have landed on `main`.
| Refined planning architecture (#475) | `MotionGenerator.generate()` is the single planning facade; each `ActionPlan` owns one trajectory and one recovery boundary; named `TrajectorySegment`s are metadata | Do not reintroduce `TrajectoryBuilder`, `MotionPlanningAdapter`, or trajectory-segment recovery. |
| Environment cadence through `BaseEnv.step_dt` (#472) | Available | Expert configuration does not expose a separate control period. |
| Adaptive dynamic-object settling (#470) | Reset/event implementation exists | Extract a reusable monitor; demo post-policies must advance through `env.step()`. |
| Authoritative scene registry (#487) | Foundation available; official environments not migrated | Reuse it from the semantic compiler and opt in task scenes explicitly. |
| Declarative robot skill profiles (#487) | Foundation available; official profiles not yet installed | Bind reusable embodiment profiles through the semantic integration layer. |
| Repeated cube pick/place demo | Manually constructs invocations and transform math | First configuration-only vertical slice. |
| Open Drawer task (#473) | Manually builds approach, grasp, pull, and command trajectories | Evidence that the semantic layer needs articulation/link/affordance references and a reusable articulation skill. |
| Action Bank | Configuration plus task-specific Python node/edge functions | Keep only as a compatibility path while semantic coverage is built. |
Expand All @@ -119,27 +122,16 @@ hard break: the project will not provide a compatibility adapter or deprecation
window for that former extension contract. Custom actions must migrate to
`_plan()` so framework-owned scene binding cannot be bypassed.

The remaining #474 prerequisites on `main` are:
Phase 1 closes the core identity, registry-backed collision integration, and
embodiment-owned capability/profile prerequisites. The remaining #474 work is
adoption and semantic orchestration:

- scene pose, semantics, affordance, and collision registration still have
multiple sources of truth;
- ordinary callers still see a large low-level public surface and must perform
semantic transform and verifier plumbing;
- robot capability declarations, resource selection, semantic commands, and
stable policies do not yet have an embodiment-owned source of truth;
- dynamic-obstacle validation is planner-local; provider collision entity IDs
and planner-declared names are not yet fully cross-validated at integration
construction time;
- `MotionPolicy` still exposes implementation-level tuning that should be
hidden behind semantic presets for ordinary users.

PR1 closes the snapshot-grounding and stable-identity bridge. PR2A closes the
first and third gaps for registry-backed integrations by introducing one
authoritative registration boundary, a registry-derived scene provider, and
construction-time collision-world validation. PR2B closes the robot-profile
gap on its stacked branch with generic resources, deterministic binding,
profile-owned commands, and named policy presets. The semantic facade remains
later-phase work.
- official environments do not yet provide authoritative registry population;
- official robot configurations do not yet install reusable skill profiles;
- named presets are not yet selected by a semantic facade/runtime; and
- effect monitoring and the configuration/demo path remain later-phase work.

One #474 finding has changed since its review branch: the ambiguous
`collision_check` switch has been replaced by `DynamicCollisionMode.OFF`,
Expand Down Expand Up @@ -912,8 +904,8 @@ Landed on `main` through #475:
core check. Complete provider/planner cross-validation is deliberately owned
by the authoritative `SceneRegistry` integration in Phase 1.

Exit criteria are met on `main@e445133c`; this remains the foundation for the
completed PR1 bridge. That bridge adds neither a legacy `plan()` adapter nor a
Exit criteria were met by PR #475 and remain the foundation for the completed
Phase 1 bridge. That bridge adds neither a legacy `plan()` adapter nor a
pre-registry duplicate of the integration-level obstacle validator.

### PR1: core snapshot and identity bridge (complete)
Expand Down Expand Up @@ -956,18 +948,18 @@ cross-source uniqueness or collision validation, a `RobotSkillProfile`, or
semantic presets. It does not require official task environments to migrate;
they remain on the compatibility path until a later opt-in vertical slice.

Exit criteria are met on `refactor/atomic-actions-phase0`: canonical object
grounding never mixes snapshot and live poses; explicit missing IDs fail;
dependency metadata matches the poses actually consumed; stable-identity merges
are deterministic; and existing direct-core callers remain usable only through
the documented deprecated fallbacks.
Exit criteria are met on `main` through PR #487: canonical object grounding
never mixes snapshot and live poses; explicit missing IDs fail; dependency
metadata matches the poses actually consumed; stable-identity merges are
deterministic; and existing direct-core callers remain usable only through the
documented deprecated fallbacks.

### Phase 1: unified integration data

Phase 1 is implemented as two focused follow-up PRs that join before the
semantic facade/compiler work.

#### PR2A: SceneRegistry (implemented on the feature branch)
#### PR2A: SceneRegistry (landed in PR #487)

Deliverables:

Expand Down Expand Up @@ -1006,7 +998,7 @@ dynamic registry/provider/planner subsets are validated before execution, the
collision-world batch mode agrees, and cuRobo uses canonical mapping keys end
to end as logical source IDs (and as physical keys for cuboid/mesh worlds).

#### PR2B: RobotSkillProfile (implemented on the feature branch)
#### PR2B: RobotSkillProfile (landed in PR #487)

Deliverables:

Expand All @@ -1032,9 +1024,9 @@ new endpoint kind still needs one shared adapter and a compatible shared atomic
skill before the current core can execute it; adding tasks that reuse that
capability then remains configuration-only.

PR2B may proceed in parallel with PR2A after the PR1 bridge. Neither follow-up
requires official task migration; the repeated-cube vertical slice opts in only
after the registry, profile, compiler, runtime, and demo bridge are available.
PR2A and PR2B landed together through PR #487. That foundation does not migrate
official tasks; the repeated-cube vertical slice opts in only after the
compiler, runtime, and demo bridge are available.

Combined Phase 1 exit criteria: an object is registered once under an
authoritative ID, aliases cannot introduce ambiguity, dynamic-object
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# ----------------------------------------------------------------------------
# Copyright (c) 2021-2026 DexForce Technology Co., Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------------------------

"""Shared binding-contract declarations for built-in manipulation skills."""

from __future__ import annotations

from collections.abc import Mapping

from ..control import ControlCommand
from ..requirements import (
ActionBindingRoute,
DisjointSlotEndpoints,
GRASP_CAPABILITY,
SkillEndpointRequirement,
SkillResourceSlot,
)


def make_motion_slot(
role: str,
*,
capabilities: frozenset[str],
) -> SkillResourceSlot:
"""Build one current-core manipulator slot."""
return SkillResourceSlot(
slot_id=role,
endpoints=(
SkillEndpointRequirement(
endpoint_id="motion",
capabilities=capabilities,
route=ActionBindingRoute("manipulator", role),
),
),
)


def make_manipulation_slot(
role: str,
*,
motion_capabilities: frozenset[str],
grasp_commands: Mapping[str, type[ControlCommand]],
) -> SkillResourceSlot:
"""Build one disjoint motion-and-grasp participant slot."""
return SkillResourceSlot(
slot_id=role,
endpoints=(
SkillEndpointRequirement(
endpoint_id="motion",
capabilities=motion_capabilities,
route=ActionBindingRoute("manipulator", role),
),
SkillEndpointRequirement(
endpoint_id="grasp",
capabilities=frozenset({GRASP_CAPABILITY}),
required_commands=grasp_commands,
route=ActionBindingRoute("end_effector", role),
),
),
constraints=(DisjointSlotEndpoints(("motion", "grasp")),),
)


__all__ = ["make_manipulation_slot", "make_motion_slot"]
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,9 @@
from ..invocation import ActionOptions, ResolvedActionRequest
from ..plans import ActionPlan, normalize_success_mask
from ..requirements import (
ActionBindingRoute,
DisjointResourceSlots,
DisjointSlotEndpoints,
GRASP_CAPABILITY,
INVERSE_KINEMATICS_CAPABILITY,
SkillBindingContract,
SkillEndpointRequirement,
SkillResourceSlot,
)
from ..state import HeldObjectState, PlanningContext
from ..trajectory_ops import interpolate_joint_trajectory, translate_pose_world
Expand All @@ -57,6 +52,7 @@
repeat_qpos,
resolve_batched_pose,
)
from ._binding_contracts import make_manipulation_slot


@dataclass(frozen=True, slots=True, eq=False)
Expand Down Expand Up @@ -344,25 +340,13 @@ class CoordinatedPickment(
end_effector_roles: ClassVar[tuple[str, ...]] = ("left", "right")
binding_contract: ClassVar[SkillBindingContract] = SkillBindingContract(
slots=tuple(
SkillResourceSlot(
slot_id=role,
endpoints=(
SkillEndpointRequirement(
endpoint_id="motion",
capabilities=frozenset({INVERSE_KINEMATICS_CAPABILITY}),
route=ActionBindingRoute("manipulator", role),
),
SkillEndpointRequirement(
endpoint_id="grasp",
capabilities=frozenset({GRASP_CAPABILITY}),
required_commands={
OPEN_COMMAND: JointPositionCommand,
GRASP_COMMAND: JointPositionCommand,
},
route=ActionBindingRoute("end_effector", role),
),
),
constraints=(DisjointSlotEndpoints(("motion", "grasp")),),
make_manipulation_slot(
role,
motion_capabilities=frozenset({INVERSE_KINEMATICS_CAPABILITY}),
grasp_commands={
OPEN_COMMAND: JointPositionCommand,
GRASP_COMMAND: JointPositionCommand,
},
)
for role in ("left", "right")
),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,9 @@
from ..invocation import ActionOptions, ResolvedActionRequest
from ..plans import ActionPlan, normalize_success_mask
from ..requirements import (
ActionBindingRoute,
CARTESIAN_POSE_CAPABILITY,
DisjointResourceSlots,
DisjointSlotEndpoints,
GRASP_CAPABILITY,
SkillBindingContract,
SkillEndpointRequirement,
SkillResourceSlot,
)
from ..state import HeldObjectState, PlanningContext
from ..trajectory_ops import (
Expand All @@ -54,6 +49,7 @@
resolve_batched_pose,
resolve_object_target,
)
from ._binding_contracts import make_manipulation_slot


@dataclass(frozen=True, slots=True, eq=False)
Expand Down Expand Up @@ -146,42 +142,18 @@ class CoordinatedPlacement(
end_effector_roles: ClassVar[tuple[str, ...]] = ("placing", "support")
binding_contract: ClassVar[SkillBindingContract] = SkillBindingContract(
slots=(
SkillResourceSlot(
slot_id="placing",
endpoints=(
SkillEndpointRequirement(
endpoint_id="motion",
capabilities=frozenset({CARTESIAN_POSE_CAPABILITY}),
route=ActionBindingRoute("manipulator", "placing"),
),
SkillEndpointRequirement(
endpoint_id="grasp",
capabilities=frozenset({GRASP_CAPABILITY}),
required_commands={
OPEN_COMMAND: JointPositionCommand,
GRASP_COMMAND: JointPositionCommand,
},
route=ActionBindingRoute("end_effector", "placing"),
),
),
constraints=(DisjointSlotEndpoints(("motion", "grasp")),),
make_manipulation_slot(
"placing",
motion_capabilities=frozenset({CARTESIAN_POSE_CAPABILITY}),
grasp_commands={
OPEN_COMMAND: JointPositionCommand,
GRASP_COMMAND: JointPositionCommand,
},
),
SkillResourceSlot(
slot_id="support",
endpoints=(
SkillEndpointRequirement(
endpoint_id="motion",
capabilities=frozenset({CARTESIAN_POSE_CAPABILITY}),
route=ActionBindingRoute("manipulator", "support"),
),
SkillEndpointRequirement(
endpoint_id="grasp",
capabilities=frozenset({GRASP_CAPABILITY}),
required_commands={GRASP_COMMAND: JointPositionCommand},
route=ActionBindingRoute("end_effector", "support"),
),
),
constraints=(DisjointSlotEndpoints(("motion", "grasp")),),
make_manipulation_slot(
"support",
motion_capabilities=frozenset({CARTESIAN_POSE_CAPABILITY}),
grasp_commands={GRASP_COMMAND: JointPositionCommand},
),
),
constraints=(DisjointResourceSlots(("placing", "support")),),
Expand Down
Loading
Loading