feat(task-engine): add orchestration and end-to-end integration

[skywhite1024]

Description

Stack

• Stack: #539
• Layer: 7/7
• Base: ljd/gen-sim-refactor-06-execution-agent
• Depends on: feat(action-engine): add execution runtime and ActionAgent #537
• Followed by: top of stack

Complete the task-first GenSim vertical slice with Task Engine scene feasibility, cross-engine adapters, parallel scene/action orchestration, deterministic recovery and final inspection, run-directory isolation, unified CLI entry points, package data, benchmarks, and architecture documentation.

This layer also ports the upright-grasp ranking and yaw-equivalent downstream reachability enhancement onto main's current AtomicAction endpoint/timing API. It intentionally fixes stale recording and coordinated-grasp test expectations found while rebasing the original branch.

Supersedes #531.

Type of change

• New feature (non-breaking change which adds functionality)
• Refactor (mainline AtomicAction API compatibility)

Validation

• Focused Task Engine and integration coverage - 157 passed
• Full affected GenSim regression - 706 passed, 9 warnings
• Related AtomicAction regression - 120 passed
• Black 26.3.1 - 855 Python files unchanged
• git diff --check - passed

Checklist

• Code passes Black 26.3.1.
• Architecture documentation is included.
• Unit, integration, and compatibility tests cover the affected behavior.
• No new third-party dependency is required.

[greptile-apps]

Greptile Summary

The PR adds a unified Task Engine orchestration pipeline spanning scene preparation, candidate planning, bundle publication, subprocess execution, and CLI integration.

• Adds versioned orchestration, scene adaptation, feasibility, workflow-state, and artifact contracts.
• Adds isolated run directories, configuration defaults, prepared-bundle execution, and end-to-end tests.
• Extends grasp generation and pickup behavior with diagnostics and fallback handling.

Confidence Score: 4/5

The prepared-bundle CLI should not be merged until its success policy is recalculated for the environment count actually launched.

A caller can override the replica count used by execution while acceptance retains a threshold derived from the configured count, causing deterministic false rejection or false acceptance under the all policy.

Files Needing Attention: embodichain/gen_sim/task_engine/cli.py and embodichain/gen_sim/task_engine/config.py

Important Files Changed

Filename	Overview
embodichain/gen_sim/task_engine/cli.py	Adds the unified command interface and prepared-bundle acceptance logic; environment-count overrides are evaluated against a stale configured threshold.
embodichain/gen_sim/task_engine/workflow.py	Implements the end-to-end workflow, parallel interpretation and scene work, bounded retries, publication, and optional execution.
embodichain/gen_sim/task_engine/orchestration/coordinator.py	Coordinates candidate grounding, feasibility checks, planning fallback, preflight, and transactional bundle publication.
embodichain/gen_sim/task_engine/orchestration/scene_source.py	Adds source resolution, content fingerprinting, dependency hashing, and mutation detection for externally owned scene projects.
embodichain/gen_sim/task_engine/_bundle_runner.py	Adds the private subprocess boundary, bundle validation, integrity verification, and Action Engine preflight.
embodichain/gen_sim/task_engine/config.py	Defines strict workflow, planning, and vectorized execution policies loaded from packaged or caller-supplied YAML.
embodichain/lab/sim/atomic_actions/primitives/pick_up.py	Extends pickup planning with grasp diagnostics and support-plane fallback behavior.

Sequence Diagram

sequenceDiagram
    participant CLI
    participant Workflow as TaskEngineWorkflow
    participant Scene as SceneBackend
    participant Coordinator
    participant Executor as SubprocessActionExecutor
    participant Runner as Bundle Runner
    CLI->>Workflow: prepare / run-all request
    Workflow->>Scene: materialize and inspect scene
    Scene-->>Workflow: scene revision and inspection
    Workflow->>Coordinator: prepare candidates and bundle
    Coordinator-->>Workflow: published executable bundle
    alt run-all
        Workflow->>Executor: execute bundle with num_envs
        Executor->>Runner: launch Action Engine subprocess
        Runner-->>Executor: execution report
        Executor-->>Workflow: environment outcomes
    end
    Workflow-->>CLI: manifest and final status

Loading

Prompt To Fix All With AI

### Issue 1
embodichain/gen_sim/task_engine/cli.py:181
**Success threshold uses stale count**

When `--num-envs` differs from the configured environment count, execution uses the override but `required_successes` remains based on the configured count, causing `success_policy=all` to reject every smaller run or accept a larger run without all launched environments succeeding.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "feat(task-engine): add orchestration and..." | Re-trigger Greptile}

[greptile-apps]

Success threshold uses stale count

When --num-envs differs from the configured environment count, execution uses the override but required_successes remains based on the configured count, causing success_policy=all to reject every smaller run or accept a larger run without all launched environments succeeding.

Prompt To Fix With AI

This is a comment left during a code review.
Path: embodichain/gen_sim/task_engine/cli.py
Line: 181

Comment:
**Success threshold uses stale count**

When `--num-envs` differs from the configured environment count, execution uses the override but `required_successes` remains based on the configured count, causing `success_policy=all` to reject every smaller run or accept a larger run without all launched environments succeeding.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.