Skip to content

feat(action-engine): add runtime grounding and action adapters - #536

Draft
skywhite1024 wants to merge 1 commit into
ljd/gen-sim-refactor-04-planning-generationfrom
ljd/gen-sim-refactor-05-runtime-grounding
Draft

feat(action-engine): add runtime grounding and action adapters#536
skywhite1024 wants to merge 1 commit into
ljd/gen-sim-refactor-04-planning-generationfrom
ljd/gen-sim-refactor-05-runtime-grounding

Conversation

@skywhite1024

@skywhite1024 skywhite1024 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Description

Stack

Add live target grounding, frame and predicate resolution, collision-cache preparation, action configuration materialization, and the adapter from SeedGraph nodes to the shared AtomicAction engine.

The adapter is rebased onto main's generic endpoint bindings and explicit trajectory timing. It preserves the original GenSim behavior without reverting the newer AtomicAction runtime contracts.

Refs #531

Type of change

  • New feature (non-breaking change which adds functionality)

Validation

  • Focused capability/linker/runtime-adapter coverage - 45 passed
  • Stack-tip GenSim regression - 706 passed, 9 warnings
  • Stack-tip related AtomicAction coverage - 120 passed
  • Black 26.3.1 - 855 Python files unchanged at the stack tip
  • git diff --check - passed

Checklist

  • Code passes Black 26.3.1.
  • Tests cover the affected behavior.
  • Stack-level architecture documentation is included by layer 7.
  • No new third-party dependency is required.

@skywhite1024 skywhite1024 added agent Features related to agentic system atomic action atomic action related functionality enhancement New feature or request labels Aug 20, 2026
@greptile-apps

greptile-apps Bot commented Aug 20, 2026

Copy link
Copy Markdown

Greptile Summary

This PR introduces runtime grounding, predicate evaluation, robot-frame helpers, atomic-action adapters, and a checksummed V-HACD grasp-collision cache.

  • Adapts grounded actions to the shared typed atomic-action planner and execution contracts.
  • Adds live arrangement, placement, relation, and predicate grounding for batched environments.
  • Adds compatibility layers and tests for atomic actions, planning linkage, fallback behavior, and collision-cache handling.

Confidence Score: 3/5

This PR is not safe to merge until the fallback planner preserves dynamic-obstacle collision safety or refuses fallback for collision-aware requests.

Generated scenes enable dynamic collision for rigid objects, but a failed primary plan can be replaced by an accepted IK trajectory whose collision-world mode is explicitly disabled.

Files Needing Attention: embodichain/gen_sim/action_engine/runtime/actions.py

Security Review

The automatic IK fallback disables dynamic-obstacle collision handling for default auto requests. When the primary collision-aware route fails, the fallback can therefore be accepted and executed without respecting the configured moving-object safety boundary.

Important Files Changed

Filename Overview
embodichain/gen_sim/action_engine/runtime/actions.py Adds the main atomic-action planning adapter; its automatic IK fallback drops configured dynamic-obstacle collision handling.
embodichain/gen_sim/action_engine/runtime/grounding.py Adds batched semantic grounding, arrangement, placement, and candidate-generation logic without an accepted actionable defect.
embodichain/gen_sim/action_engine/runtime/grasp_collision_cache.py Adds private, atomic, metadata- and SHA-256-validated V-HACD cache generation; scanner leads did not establish a changed-code vulnerability.
embodichain/gen_sim/action_engine/runtime/predicates.py Adds runtime predicate evaluation supporting the new grounding and action flow.
embodichain/gen_sim/action_engine/runtime/frames.py Adds batched robot-base and relation-frame calculations used by grounding and action configuration.
embodichain/gen_sim/action_engine/runtime/atomic_compat.py Adds compatibility behavior for exact-target held-object motion.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Semantic action] --> B[Runtime grounding]
  B --> C[AtomicActionAdapter]
  C --> D[Collision-aware motion_gen plan]
  D -->|success| E[Selected trajectory]
  D -->|failure| F[ik_interp fallback]
  F -->|DynamicCollisionMode.OFF| E
  E --> G[Execution and state effects]
Loading

Fix all with Greploop Fix All in Codex Fix All in Claude Code

Prompt To Fix All With AI
### Issue 1
embodichain/gen_sim/action_engine/runtime/actions.py:382-385
**Fallback disables obstacle collisions**

When a generated scene configures rigid objects as dynamic obstacles, the default `auto` safety mode allows a failed collision-aware `motion_gen` request to fall back to `ik_interp` with `DynamicCollisionMode.OFF`; successful fallback rows are then executed without considering those obstacles, allowing the robot or held object to collide with them.

**How this was verified:** The generated-scene configuration enables dynamic collision for rigid objects, while OFF mode bypasses the scene collision world and the adapter accepts successful fallback rows.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "feat(action-engine): add runtime groundi..." | Re-trigger Greptile

Comment on lines +382 to +385
fallback_policy = replace(
invocation.motion_policy,
strategy=str(fallback_strategy),
dynamic_collision_mode=DynamicCollisionMode.OFF,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Fallback disables obstacle collisions

When a generated scene configures rigid objects as dynamic obstacles, the default auto safety mode allows a failed collision-aware motion_gen request to fall back to ik_interp with DynamicCollisionMode.OFF; successful fallback rows are then executed without considering those obstacles, allowing the robot or held object to collide with them.

How this was verified: The generated-scene configuration enables dynamic collision for rigid objects, while OFF mode bypasses the scene collision world and the adapter accepts successful fallback rows.

Prompt To Fix With AI
This is a comment left during a code review.
Path: embodichain/gen_sim/action_engine/runtime/actions.py
Line: 382-385

Comment:
**Fallback disables obstacle collisions**

When a generated scene configures rigid objects as dynamic obstacles, the default `auto` safety mode allows a failed collision-aware `motion_gen` request to fall back to `ik_interp` with `DynamicCollisionMode.OFF`; successful fallback rows are then executed without considering those obstacles, allowing the robot or held object to collide with them.

**How this was verified:** The generated-scene configuration enables dynamic collision for rigid objects, while OFF mode bypasses the scene collision world and the adapter accepts successful fallback rows.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

@skywhite1024
skywhite1024 marked this pull request as draft August 21, 2026 07:30
@skywhite1024
skywhite1024 force-pushed the ljd/gen-sim-refactor-05-runtime-grounding branch from 4159ce5 to b666762 Compare August 21, 2026 09:56
@skywhite1024
skywhite1024 force-pushed the ljd/gen-sim-refactor-05-runtime-grounding branch from b666762 to 83c3bc1 Compare August 21, 2026 10:11
@skywhite1024
skywhite1024 force-pushed the ljd/gen-sim-refactor-05-runtime-grounding branch from 83c3bc1 to 553e3f0 Compare August 21, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent Features related to agentic system atomic action atomic action related functionality enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant