diff --git a/.changeset/heroic-owls-land.md b/.changeset/heroic-owls-land.md new file mode 100644 index 0000000..4ac6c01 --- /dev/null +++ b/.changeset/heroic-owls-land.md @@ -0,0 +1,7 @@ +--- +"posecode-parser": patch +"posecode-render": patch +"posecode-mcp": patch +--- + +Strengthen Posecode motion authoring and playback with strict contact validation, grounded multi-contact solving, continuous sparse transitions, and more lifelike canonical movement guidance. diff --git a/README.md b/README.md index 07beac0..acaf96e 100644 --- a/README.md +++ b/README.md @@ -426,7 +426,7 @@ It explains biomechanical problems in natural language and proposes specific rev | Package | Purpose | | --- | --- | | [`posecode-language`](packages/posecode-language) | Language definitions and editor support | -| [`posecode-parser`](packages/posecode-parser) | Converts `.posecode` text into validated, ROM-constrained intermediate representation | +| [`posecode-parser`](packages/posecode-parser) | Converts `.posecode` text into a validated, range-constrained intermediate representation | | [`posecode-render`](packages/posecode-render) | Renders animated figures with Three.js, forward kinematics, and IK | | [`posecode-share`](packages/posecode-share) | Encodes Posecode documents into URL-safe share tokens | | [`posecode-mcp`](packages/posecode-mcp) | Exposes Posecode capabilities to AI agents through MCP | @@ -482,7 +482,7 @@ Posecode currently focuses on: The following are outside the current scope: - two-person or partner motion, -- collision detection, +- comprehensive collision detection and rigid-body dynamics, - detailed object physics, - advanced equipment simulation, - multi-joint finger animation, diff --git a/ROADMAP.md b/ROADMAP.md index b861453..481e65d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,8 +1,9 @@ # Posecode: domains & roadmap -Posecode's vision is broad: **describe any single-person movement as text and render it -safely in 3D.** The *protocol* is general; the *current renderer* is deliberately -scoped (one figure, forward kinematics, ground-locked hands/feet, no props). This +Posecode's vision is broad: **describe single-person movement as text and render it +as inspectable 3D motion.** The *protocol* is general; the *current renderer* is deliberately +scoped to one simplified figure with forward kinematics, constrained contacts, +starter props, and no rigid-body dynamics. This doc maps the domains Posecode can serve, what already works, and what each remaining domain needs, so contributions land where they unlock the most. @@ -10,9 +11,9 @@ domain needs, so contributions land where they unlock the most. | Domain | Example uses | Status | | --- | --- | --- | -| **Physiotherapy / rehab** | Range-of-motion demos, home-exercise programs, post-op protocols, cervical/shoulder mobility | Strong fit today: the ROM safety clamp is a clinical feature. Many moves render now; equipment (bands, balls) is future. | -| **Ergonomics / desk & posture** | Posture resets, seated/standing stretch breaks, "do this every hour" prompts | Works today for standing variants; true *seated* needs a chair prop (below). | -| **Yoga & mobility** | Standing poses (chair, side bend, twist), flows, mobility drills | Standing poses work; floor/inversion/lying poses need lying base poses + a mat. | +| **Physiotherapy / rehab** | Range-of-motion demos, home-exercise programs, post-op protocols, cervical/shoulder mobility | Useful for reviewed visualization: ROM constraints and diagnostics are explicit. Clinical use still requires qualified review; bands and balls are future. | +| **Ergonomics / desk & posture** | Posture resets, seated/standing stretch breaks, "do this every hour" prompts | Standing and floor-seated variants work; chair-seated motion needs a dedicated base pose. | +| **Yoga & mobility** | Standing poses (chair, side bend, twist), flows, mobility drills | Standing and basic supine/prone/long-seated poses work; inversions and detailed mat contacts remain limited. | | **Movement education / anatomy** | Demonstrate joint actions ("what is shoulder abduction?"), biomechanics teaching | Excellent fit: single-joint demos are exactly what the rig does. | | **Fitness / strength** | Body-weight and free-form movement coaching | Core today (squat, curl, raise). Barbell/dumbbell/machine work needs props + grip. | | **Functional / elderly care** | Sit-to-stand, balance, gentle ROM, fall-prevention drills | Partial: sit-to-stand works; reaching/balance need reach-IK + props. | diff --git a/docs/market-research.md b/docs/market-research.md index fe16a9e..02efebd 100644 --- a/docs/market-research.md +++ b/docs/market-research.md @@ -8,18 +8,18 @@ ## 1. The thesis Posecode is **"Mermaid for human movement."** A person describes a movement in -words; an LLM writes a short `.posecode` document; the browser parses it, clamps it -to a safe range of motion, and renders an animated 3D mannequin, producing a +words; an LLM writes a short `.posecode` document; the browser validates it, +constrains joint targets to configured ranges, and renders an animated 3D mannequin, producing a **shareable URL**. The loop is: > **ask an LLM for a movement → it renders → share the link.** This is structurally different from video. A `.posecode` doc is **editable text**: an LLM can generate it, a human can tweak one angle, a clinician can fork it, and -it diffs in version control. Diffusion/video models hallucinate anatomy and can't -be safely constrained; Posecode's ROM clamp is a *correctness* feature, not a filter. -Text is also how LLMs natively "think" about structure, so authoring is reliable -and improves as models improve. +it diffs in version control. Posecode exposes the authored anatomy and constraints +for inspection instead of hiding them in a generated trajectory. Range constraints +are one correctness layer, not a certification of the complete movement. Text also +gives LLMs a structured target that can be parsed, evaluated, and revised. ### Why it can spread @@ -30,8 +30,9 @@ and improves as models improve. - **Agent-native.** The [`posecode-mcp`](../packages/posecode-mcp) server lets Claude/ ChatGPT author, validate, and return a render link *inside the chat*: the movement appears where the user already is. -- **Safe to trust.** Clinical ROM limits mean shared links can't depict unsafe - joint angles: important for the health-adjacent buyers below. +- **Inspectable constraints.** Configured ROM limits prevent impossible authored + joint rotations, while contact residuals and movement checks expose failures. + Health-adjacent uses still require qualified review. ## 2. How we scored the domains @@ -67,14 +68,14 @@ on the roadmap. - **Customer:** physios, chiros, athletic trainers, and their patients running home-exercise programs; post-op ROM protocols. -- **Aha use case:** a clinician types a prescription and hands the patient a link - that *shows* the exercise, with a built-in safe-range guarantee. +- **Aha use case:** a clinician authors or reviews a movement and hands the patient + a link that *shows* it, with explicit joint targets and validation diagnostics. - **Viral loop:** clinician → patient link sharing is high-frequency and trusted; patients forward to family; clinics build reusable libraries. - **Ships:** `heel-raises`, `standing-hamstring-curl`, `hip-abduction`, `good-morning` (back-health hinge), plus existing `neck-rotation`, `shoulder-stretch`. -- **Engine-fit:** ★★★★☆, the ROM clamp is a clinical feature; bands/balls and +- **Engine-fit:** ★★★★☆, range constraints support review; bands/balls and lying poses are future (see roadmap). ### 3c. Desk & workplace wellness diff --git a/packages/posecode-embed/test/compat.test.ts b/packages/posecode-embed/test/compat.test.ts index 073c924..eceda0b 100644 --- a/packages/posecode-embed/test/compat.test.ts +++ b/packages/posecode-embed/test/compat.test.ts @@ -10,7 +10,7 @@ const SPORTS_MOVE = `posecode exercise "Crossover" knees: flex 68 ground-lock: feet step "Cross" 0.22s drive: - pelvis: rotate-out 14 + turn: -14 travel: -0.18 0 ground-lock: feet repeat 2 diff --git a/packages/posecode-eval/src/checks.ts b/packages/posecode-eval/src/checks.ts index fb9fe08..081fced 100644 --- a/packages/posecode-eval/src/checks.ts +++ b/packages/posecode-eval/src/checks.ts @@ -6,12 +6,16 @@ */ import type { PhasePose, ProbeResult } from "./probe.js"; +import { REACH_TOLERANCE } from "posecode-render"; import { balanceOverflow, distanceBetween, feetCenterSkateDistance, + fistFloorAngleDeg, footIsSupported, footSkateDistance, + footWorldSkateDistance, + forwardCoordinate, headPropClearance, heightOf, kneeFlexionDeg, @@ -20,7 +24,9 @@ import { phaseMaxLandmarkSpeed, propPenetrationDepth, segmentTiltDeg, + soleUpAngleDeg, spineCurlDeg, + torsoForwardPitchDeg, torsoPitchDeg, } from "./metrics.js"; @@ -37,11 +43,47 @@ export interface MovementChecks { checks: ((result: ProbeResult) => CheckOutcome)[]; } +/** Maximum positional error for a declared reach/pin/grip contact. */ +export const CONTACT_ERROR_MAX = REACH_TOLERANCE; + /** Find a phase by name; throws a failing outcome path if missing. */ function phase(result: ProbeResult, name: string): PhasePose | null { return result.phases.find((p) => p.name === name) ?? null; } +/** Expanded identities of every body part explicitly supported by the floor. */ +function floorSupportEffectors(pose: PhasePose): Set { + const supports = new Set(); + const add = (effector: string): void => { + const groups: Readonly> = { + feet: ["foot_left", "foot_right"], + hands: ["hand_left", "hand_right"], + fists: ["fist_left", "fist_right"], + knees: ["knee_left", "knee_right"], + forearms: ["elbow_left", "elbow_right"], + }; + for (const item of groups[effector] ?? [effector]) supports.add(item); + }; + pose.groundLock.forEach(add); + pose.reaches.filter((reach) => reach.target === "floor").forEach((reach) => add(reach.effector)); + pose.pins.filter((pin) => pin.anchor === "floor").forEach((pin) => add(pin.effector)); + return supports; +} + +function footSupportKind( + pose: PhasePose, + side: "left" | "right", +): "ground-lock" | "pin" | null { + if (pose.pins.some((pin) => + (pin.effector === "feet" || pin.effector === `foot_${side}`) && pin.anchor === "floor")) { + return "pin"; + } + if (pose.groundLock.includes("feet") || pose.groundLock.includes(`foot_${side}`)) { + return "ground-lock"; + } + return null; +} + /** Build a check on one named phase with a measured value and a predicate. */ export function phaseCheck( id: string, @@ -78,6 +120,13 @@ export function genericChecks(result: ProbeResult): CheckOutcome[] { .map((w) => `${w.joint} ${w.action} ${w.requested}→${w.clamped}`) .join("; ") || "no warnings", }, + { + id: "has-phases", + pass: result.phases.length > 0, + detail: result.phases.length > 0 + ? `${result.phases.length} phase(s)` + : "no movement phases to evaluate", + }, ]; for (const p of result.phases) { // The one universal contact invariant: nothing sinks through the floor. @@ -98,22 +147,42 @@ export function genericChecks(result: ProbeResult): CheckOutcome[] { if (p.floorBound) { out.push({ id: `grounded-not-floating:${p.name}`, - pass: p.meshMinY < 0.02, - detail: `mesh floats ${p.meshMinY.toFixed(3)}m above floor (want < 0.020)`, + pass: p.meshMinY > -0.02 && p.meshMinY < 0.02, + detail: `mesh floor offset ${p.meshMinY.toFixed(3)}m (want -0.020 to +0.020)`, }); } + // A contact declaration is an executable promise, not metadata. Every + // reach/pin/grip must either resolve within tolerance or fail explicitly; + // unsupported targets/solver paths must never disappear into a green score. + p.contactResiduals.forEach((contact, index) => { + const suffix = `${p.name}:${contact.kind}:${contact.effector}:${contact.target}:${index}`; + if (contact.status === "unsupported" || contact.error === null) { + out.push({ + id: `contact-supported:${suffix}`, + pass: false, + detail: contact.reason ?? "contact could not be evaluated", + }); + return; + } + out.push({ + id: `contact-position:${suffix}`, + pass: contact.error <= CONTACT_ERROR_MAX, + detail: `${contact.error.toFixed(3)}m residual (want ≤ ${CONTACT_ERROR_MAX.toFixed(3)}m)`, + }); + }); + const floorHands = new Set(); - for (const r of p.reaches) { - if (r.target !== "floor") continue; - if (r.effector === "hands" || r.effector === "hand_left") floorHands.add("left"); - if (r.effector === "hands" || r.effector === "hand_right") floorHands.add("right"); - } - for (const pin of p.pins) { - if (pin.anchor !== "floor") continue; - if (pin.effector === "hands" || pin.effector === "hand_left") floorHands.add("left"); - if (pin.effector === "hands" || pin.effector === "hand_right") floorHands.add("right"); - } + const floorFists = new Set(); + const collectFloorHand = (effector: string): void => { + if (effector === "hands" || effector === "hand_left") floorHands.add("left"); + if (effector === "hands" || effector === "hand_right") floorHands.add("right"); + if (effector === "fists" || effector === "fist_left") floorFists.add("left"); + if (effector === "fists" || effector === "fist_right") floorFists.add("right"); + }; + p.reaches.filter((reach) => reach.target === "floor").forEach((reach) => collectFloorHand(reach.effector)); + p.pins.filter((pin) => pin.anchor === "floor").forEach((pin) => collectFloorHand(pin.effector)); + p.groundLock.forEach(collectFloorHand); for (const side of floorHands) { const angle = palmFloorAngleDeg(p, side as "left" | "right"); out.push({ @@ -122,6 +191,14 @@ export function genericChecks(result: ProbeResult): CheckOutcome[] { detail: `${angle.toFixed(1)}° from palm-down (want < 55°)`, }); } + for (const side of floorFists) { + const angle = fistFloorAngleDeg(p, side as "left" | "right"); + out.push({ + id: `fist-normal:${p.name}:${side}`, + pass: angle < 55, + detail: `${angle.toFixed(1)}° from knuckles-down (want < 55°)`, + }); + } const overflow = balanceOverflow(p); out.push({ @@ -167,14 +244,30 @@ export function genericChecks(result: ProbeResult): CheckOutcome[] { }); } for (const side of ["left", "right"] as const) { - const explicitlyPinned = (phase: PhasePose) => phase.pins.some((p) => - (p.effector === "feet" || p.effector === `foot_${side}`) && p.anchor === "floor"); - if (!explicitlyPinned(previous) || !explicitlyPinned(current)) continue; - const skate = footSkateDistance(previous, current, side); + // With both feet down, a deliberate stance-width change may move each + // foot symmetrically while the support center stays fixed (checked + // above). A single carried support, however, must not jump when the + // author switches between ground-lock and pin semantics. + if (bothSupported || !footIsSupported(previous, side) || !footIsSupported(current, side)) continue; + const previousKind = footSupportKind(previous, side); + const currentKind = footSupportKind(current, side); + // Releasing a world pin into a ground-locked landing deliberately lets + // the foot travel to its new planted position during this phase (step + // closes, marches, and dance phrases). Endpoint displacement is motion, + // not a boundary snap, so it is covered by speed/contact checks instead. + if (previousKind === "pin" && currentKind === "ground-lock") continue; + // Ground-lock anchors travel/yaw with the figure, so compare in its + // choreography-relative frame. A floor pin is a fixed WORLD anchor; + // subtracting authored travel fabricates skate (forward lunge reported + // 30 cm while the pinned foot was actually motionless). Handoffs to or + // from a pin are continuity checks in world space too. + const skate = previousKind === "ground-lock" && currentKind === "ground-lock" + ? footSkateDistance(previous, current, side) + : footWorldSkateDistance(previous, current, side); out.push({ id: `foot-skate:${current.name}:${side}`, - pass: skate < 0.08, - detail: `${skate.toFixed(3)}m pinned-foot drift (want < 0.08m)`, + pass: skate < 0.03, + detail: `${skate.toFixed(3)}m single-support drift (want < 0.03m)`, }); } const speed = phaseMaxLandmarkSpeed(previous, current); @@ -200,6 +293,151 @@ export function genericChecks(result: ProbeResult): CheckOutcome[] { * a leg-swing (the pre-hinge bug) fails these loudly. */ export const MOVEMENT_CHECKS: MovementChecks[] = [ + { + movement: "glute-bridge", + checks: [ + phaseCheck( + "bridge-pelvis-raised", + "Bridge up", + (p) => heightOf(p, "pelvis"), + (v) => v > 0.5, + "pelvis > 0.50m", + ), + phaseCheck( + "lower-pelvis-on-floor", + "Lower", + (p) => heightOf(p, "pelvis"), + (v) => v < 0.16, + "pelvis < 0.16m", + ), + (result) => { + const up = phase(result, "Bridge up"); + const down = phase(result, "Lower"); + if (!up || !down) return { id: "bridge-has-real-height-change", pass: false, detail: "bridge phase missing" }; + const delta = heightOf(up, "pelvis") - heightOf(down, "pelvis"); + return { + id: "bridge-has-real-height-change", + pass: delta > 0.4, + detail: `${delta.toFixed(3)}m pelvis height change (want > 0.40m)`, + }; + }, + ], + }, + { + // Three-point landing: the front sole, rear knee, and opposite fist must + // form distinct supports while the torso stays above the floor. Contact + // residuals alone once allowed a folded body with a vertical planted foot. + movement: "superhero-landing", + checks: [ + phaseCheck( + "torso-forward-not-collapsed", + "Hold the landing", + torsoForwardPitchDeg, + (v) => v >= 65 && v <= 100, + "65–100° character-forward pitch", + ), + (result) => { + const p = phase(result, "Hold the landing"); + if (!p) return { id: "exact-three-supports", pass: false, detail: "phase \"Hold the landing\" not found" }; + const actual = floorSupportEffectors(p); + const expected = ["foot_right", "knee_left", "fist_left"]; + const pass = actual.size === expected.length && expected.every((item) => actual.has(item)); + return { + id: "exact-three-supports", + pass, + detail: `floor supports: ${[...actual].sort().join(", ") || "none"} (want ${expected.join(", ")})`, + }; + }, + (result) => { + const phases = [ + phase(result, "Drop into the landing"), + phase(result, "Make three-point contact"), + phase(result, "Hold the landing"), + ]; + if (phases.some((item) => !item)) { + return { id: "three-supports-stay-planted", pass: false, detail: "landing phase missing" }; + } + let maxDrift = 0; + for (let i = 1; i < phases.length; i++) { + const a = phases[i - 1]!; + const b = phases[i]!; + for (const boneId of ["ankle_right", "knee_left", "wrist_left"]) { + const pa = a.bones.get(boneId)!; + const pb = b.bones.get(boneId)!; + maxDrift = Math.max(maxDrift, Math.hypot(pb[0] - pa[0], pb[2] - pa[2])); + } + } + return { + id: "three-supports-stay-planted", + pass: maxDrift < 0.03, + detail: `${maxDrift.toFixed(3)}m maximum support drift (want < 0.03m)`, + }; + }, + phaseCheck( + "front-sole-planted", + "Hold the landing", + (p) => soleUpAngleDeg(p, "right"), + (v) => v < 25, + "< 25° from flat", + ), + phaseCheck( + "rear-knee-down", + "Hold the landing", + (p) => heightOf(p, "knee_left"), + (v) => v < 0.12, + "left knee < 0.12m", + ), + phaseCheck( + "fist-down", + "Hold the landing", + (p) => heightOf(p, "wrist_left"), + (v) => v < 0.13, + "left fist < 0.13m", + ), + phaseCheck( + "head-clear-of-floor", + "Hold the landing", + (p) => heightOf(p, "head"), + (v) => v > 0.4, + "head > 0.40m", + ), + phaseCheck( + "front-knee-up", + "Hold the landing", + (p) => heightOf(p, "knee_right"), + (v) => v > 0.3, + "right knee > 0.30m", + ), + phaseCheck( + "supports-separated", + "Hold the landing", + (p) => Math.min( + distanceBetween(p, "ankle_right", "knee_left"), + distanceBetween(p, "ankle_right", "wrist_left"), + distanceBetween(p, "knee_left", "wrist_left"), + ), + (v) => v > 0.2, + "every support pair > 0.20m apart", + ), + phaseCheck( + "supports-front-to-back", + "Hold the landing", + (p) => Math.min( + forwardCoordinate(p, "ankle_right") - forwardCoordinate(p, "knee_left"), + forwardCoordinate(p, "wrist_left") - forwardCoordinate(p, "knee_left"), + ), + (v) => v > 0.12, + "front foot and fist > 0.12m ahead of rear knee", + ), + phaseCheck( + "free-arm-raised", + "Hold the landing", + (p) => heightOf(p, "wrist_right") - heightOf(p, "pelvis"), + (v) => v > 0.12, + "right wrist > 0.12m above pelvis", + ), + ], + }, { // `pelvis: hinge`: torso tips forward over vertical legs (deadlift phases // "Lower"/"Lift"). Regressing the hinge into a leg-swing or a backward diff --git a/packages/posecode-eval/src/cli.ts b/packages/posecode-eval/src/cli.ts index 4526224..986364f 100644 --- a/packages/posecode-eval/src/cli.ts +++ b/packages/posecode-eval/src/cli.ts @@ -8,12 +8,37 @@ import { fileURLToPath } from "node:url"; import { dirname, resolve } from "node:path"; import { loadFixtures } from "./generator.js"; import { renderReport, runEval } from "./report.js"; +import { loadXbotCharacter } from "./xbot.js"; const here = dirname(fileURLToPath(import.meta.url)); const examplesDir = resolve(here, "../../../spec/examples"); +const xbotAsset = new URL("../../../playground/public/models/xbot.glb", import.meta.url); -const report = runEval(loadFixtures(examplesDir)); -console.log(renderReport(report)); +// Both figures ship in production: XBot is the default, while the procedural +// mannequin is the deliberate fallback when the character asset cannot load. +// Limb-length differences materially affect contacts, so neither result may +// stand in for the other at the launch gate. +const fixtures = loadFixtures(examplesDir); +const xbot = await loadXbotCharacter(xbotAsset); +const reports = [ + { + label: "XBot visible skeleton + skinned floor (production default)", + report: runEval(fixtures, { proportions: xbot.proportions, character: xbot }), + }, + { + label: "Procedural mannequin (production fallback)", + report: runEval(fixtures), + }, +] as const; -const { checksPassed, checksTotal } = report.summary; -process.exit(checksPassed === checksTotal ? 0 : 1); +for (const { label, report } of reports) { + console.log(`=== ${label} ===`); + console.log(renderReport(report)); + console.log(""); +} +xbot.dispose(); + +const passed = reports.every(({ report }) => + report.summary.checksPassed === report.summary.checksTotal, +); +process.exitCode = passed ? 0 : 1; diff --git a/packages/posecode-eval/src/index.ts b/packages/posecode-eval/src/index.ts index 5307b3f..2cc99fc 100644 --- a/packages/posecode-eval/src/index.ts +++ b/packages/posecode-eval/src/index.ts @@ -1,7 +1,15 @@ /** posecode-eval: public API. */ export { probeMovement } from "./probe.js"; -export type { ProbeResult, PhasePose, Quat, Vec3 } from "./probe.js"; +export type { + ContactKind, + ContactResidual, + ContactStatus, + ProbeResult, + PhasePose, + Quat, + Vec3, +} from "./probe.js"; export { angleBetweenDeg, balanceOverflow, @@ -10,8 +18,11 @@ export { distanceBetween, feetHeight, feetCenterSkateDistance, + fistFloorAngleDeg, footIsSupported, footSkateDistance, + footWorldSkateDistance, + forwardCoordinate, headPropClearance, heightOf, jointAngleDeg, @@ -20,12 +31,14 @@ export { palmFloorAngleDeg, phaseMaxLandmarkSpeed, segmentTiltDeg, + soleUpAngleDeg, spineCurlDeg, + torsoForwardPitchDeg, torsoPitchDeg, } from "./metrics.js"; -export { genericChecks, phaseCheck, MOVEMENT_CHECKS } from "./checks.js"; +export { CONTACT_ERROR_MAX, genericChecks, phaseCheck, MOVEMENT_CHECKS } from "./checks.js"; export type { CheckOutcome, MovementChecks } from "./checks.js"; export { runEval, renderReport } from "./report.js"; -export type { EvalReport, MovementReport, MovementSource } from "./report.js"; +export type { EvalOptions, EvalReport, MovementReport, MovementSource } from "./report.js"; export { loadFixtures } from "./generator.js"; export type { MovementGenerator } from "./generator.js"; diff --git a/packages/posecode-eval/src/metrics.ts b/packages/posecode-eval/src/metrics.ts index b490332..f6b1ed8 100644 --- a/packages/posecode-eval/src/metrics.ts +++ b/packages/posecode-eval/src/metrics.ts @@ -41,6 +41,20 @@ export function torsoPitchDeg(pose: PhasePose): number { return segmentTiltDeg(pose, "pelvis", "neck"); } +/** + * Signed torso pitch in the character's travel-relative sagittal plane. + * Positive values lean character-forward; backward and sideways collapses do + * not masquerade as a valid forward hinge merely because their unsigned tilt + * is large. + */ +export function torsoForwardPitchDeg(pose: PhasePose): number { + const direction = sub(bone(pose, "neck"), bone(pose, "pelvis")); + const c = Math.cos(-pose.rootYaw); + const s = Math.sin(-pose.rootYaw); + const localForward = direction[0] * s + direction[2] * c; + return Math.atan2(localForward, direction[1]) * RAD2DEG; +} + /** Interior angle at joint b formed by segments b→a and b→c (180 = straight). */ export function jointAngleDeg(pose: PhasePose, a: string, b: string, c: string): number { return angleBetweenDeg(sub(bone(pose, a), bone(pose, b)), sub(bone(pose, c), bone(pose, b))); @@ -88,8 +102,27 @@ function rotateByQuat(v: Vec3, q: Quat): Vec3 { export function palmFloorAngleDeg(pose: PhasePose, side: "left" | "right"): number { const q = pose.boneQuaternions.get(`wrist_${side}`); if (!q) return 180; - // The flattened palm's face normal is mirrored local X on the two wrists. - return angleBetweenDeg(rotateByQuat(side === "left" ? [1, 0, 0] : [-1, 0, 0], q), [0, -1, 0]); + // The procedural palm is shallow on local Z for both sides (the same + // geometry axis used by render's production floor-contact solver). + return angleBetweenDeg(rotateByQuat([0, 0, 1], q), [0, -1, 0]); +} + +/** Angle between the semantic fist's knuckle direction and floor-down. */ +export function fistFloorAngleDeg(pose: PhasePose, side: "left" | "right"): number { + const q = pose.boneQuaternions.get(`wrist_${side}`); + if (!q) return 180; + // Same wrist→knuckle axis used by render's production fist contact solver. + return angleBetweenDeg(rotateByQuat([0, -1, 0], q), [0, -1, 0]); +} + +/** Landmark position along character-forward after undoing authored root yaw. */ +export function forwardCoordinate(pose: PhasePose, id: string): number { + const p = bone(pose, id); + const x = p[0] - pose.rootOffset[0]; + const z = p[2] - pose.rootOffset[2]; + const c = Math.cos(-pose.rootYaw); + const s = Math.sin(-pose.rootYaw); + return x * s + z * c; } /** Angle between the sole's local up axis and world up (0 = foot flat). */ @@ -103,8 +136,7 @@ export function soleUpAngleDeg(pose: PhasePose, side: "left" | "right"): number export function palmBarAngleDeg(pose: PhasePose, side: "left" | "right"): number { const q = pose.boneQuaternions.get(`wrist_${side}`); if (!q) return 180; - const localNormal: Vec3 = side === "left" ? [1, 0, 0] : [-1, 0, 0]; - return angleBetweenDeg(rotateByQuat(localNormal, q), [0, 0, 1]); + return angleBetweenDeg(rotateByQuat([0, 0, 1], q), [0, 0, 1]); } /** Distance from a wrist to its side-specific pull-up-bar grip anchor. */ @@ -137,19 +169,37 @@ function supportBoneIds(pose: PhasePose): string[] { const ids = new Set(); const addGroup = (name: string) => { if (name === "feet") { ids.add("ankle_left"); ids.add("ankle_right"); } + if (name === "foot_left") ids.add("ankle_left"); + if (name === "foot_right") ids.add("ankle_right"); if (name === "hands") { ids.add("wrist_left"); ids.add("wrist_right"); } + if (name === "hand_left") ids.add("wrist_left"); + if (name === "hand_right") ids.add("wrist_right"); + if (name === "fists") { ids.add("wrist_left"); ids.add("wrist_right"); } + if (name === "fist_left") ids.add("wrist_left"); + if (name === "fist_right") ids.add("wrist_right"); if (name === "forearms") { ids.add("elbow_left"); ids.add("elbow_right"); } + if (name === "elbow_left") ids.add("elbow_left"); + if (name === "elbow_right") ids.add("elbow_right"); + if (name === "knees") { ids.add("knee_left"); ids.add("knee_right"); } + if (name === "knee_left") ids.add("knee_left"); + if (name === "knee_right") ids.add("knee_right"); }; pose.groundLock.forEach(addGroup); for (const reach of pose.reaches) { if (reach.target !== "floor") continue; addGroup(reach.effector); - const mapped = reach.effector.replace("hand_", "wrist_").replace("foot_", "ankle_"); + const mapped = reach.effector + .replace("hand_", "wrist_") + .replace("fist_", "wrist_") + .replace("foot_", "ankle_"); if (pose.bones.has(mapped)) ids.add(mapped); } for (const pin of pose.pins) { addGroup(pin.effector); - const mapped = pin.effector.replace("hand_", "wrist_").replace("foot_", "ankle_"); + const mapped = pin.effector + .replace("hand_", "wrist_") + .replace("fist_", "wrist_") + .replace("foot_", "ankle_"); if (pose.bones.has(mapped)) ids.add(mapped); } // Floor poses also distribute load through the torso/pelvis even when the @@ -164,9 +214,16 @@ function supportBoneIds(pose: PhasePose): string[] { /** Horizontal COM distance outside the active support bounding box (0 = inside). */ export function balanceOverflow(pose: PhasePose): number { const supports = supportBoneIds(pose).map((id) => bone(pose, id)); - if (supports.length < 2) return 0; + // No authored support information remains unscored for backward + // compatibility. A single support, however, is a real balance constraint: + // measure outside a foot/hand-sized disc instead of auto-passing it. + if (supports.length === 0) return 0; const com = centerOfMass(pose); const margin = 0.14; + if (supports.length === 1) { + const support = supports[0]!; + return Math.max(0, Math.hypot(com[0] - support[0], com[2] - support[2]) - margin); + } const minX = Math.min(...supports.map((p) => p[0])) - margin; const maxX = Math.max(...supports.map((p) => p[0])) + margin; const minZ = Math.min(...supports.map((p) => p[2])) - margin; @@ -232,7 +289,6 @@ const PART_RADII = { torso: 0.13, head: 0.105, thigh: 0.075, shin: 0.055, arm: 0 /** * Worst body penetration into a solid prop face (metres, ≤0 when clear), or - * -Infinity when the document declares no solid-faced prop. Limbs pinned or * reached to a non-floor anchor are that phase's declared prop support and * don't count (a foot standing ON the box is not "in" the box). */ @@ -305,6 +361,17 @@ export function footSkateDistance(previous: PhasePose, pose: PhasePose, side: "l return Math.hypot(b[0] - a[0], b[1] - a[1]); } +/** World-space X/Z drift for a support whose anchor is fixed in the scene. */ +export function footWorldSkateDistance( + previous: PhasePose, + pose: PhasePose, + side: "left" | "right", +): number { + const a = bone(previous, `ankle_${side}`); + const b = bone(pose, `ankle_${side}`); + return Math.hypot(b[0] - a[0], b[2] - a[2]); +} + /** Drift of the planted foot-pair center, ignoring intentional stance-width changes. */ export function feetCenterSkateDistance(previous: PhasePose, pose: PhasePose): number { const delta = (side: "left" | "right") => { diff --git a/packages/posecode-eval/src/probe.ts b/packages/posecode-eval/src/probe.ts index a98e33f..1eb2a4a 100644 --- a/packages/posecode-eval/src/probe.ts +++ b/packages/posecode-eval/src/probe.ts @@ -1,35 +1,80 @@ /** * Headless kinematic probe. * - * Runs a `.posecode` source through the REAL production pipeline (parser → - * timeline → mannequin FK → root choreography (yaw/travel) → ground-lock → - * floor clamp) without a WebGL context (three.js scene-graph math is pure), - * and returns world-space bone positions at the end of every phase. This is - * the ground truth the invariant checks score against. + * Runs a `.posecode` source through the production authored-motion pipeline + * (parser → timeline → mannequin FK → root choreography → contacts → floor + * clamp) without a WebGL context and returns world-space phase endpoints. + * Presentation-only breathing/blinking, free-arm ambience, camera behavior, + * and optional mocap overlays are deliberately outside this deterministic + * authoring gate; browser tests cover the shipped visible character. * - * Floor, body-landmark, and built-in prop pins are deterministic, so the probe - * resolves those exactly like the viewer. Reach-IK remains the one intentional - * gap; phase orientations still receive semantic palm-contact alignment. + * Contact solving reuses the production CCD/ground/contact primitives and + * mirrors the viewer's reach, pin, and grip ordering. Every declared contact + * is returned with an explicit residual (or an explicit unsupported status), + * so a solver gap can never be mistaken for a successful movement. */ import * as THREE from "three"; -import { parse, type TimingMode, type ParseError, type PinTarget, type ReachTarget, type Warning } from "posecode-parser"; +import { + parse, + type GripTarget, + type TimingMode, + type ParseError, + type PinTarget, + type ReachTarget, + type Warning, +} from "posecode-parser"; import { applyGroundLock, - alignFloorPalms, + alignFloorContacts, + alignGripFrames, buildMannequin, buildProps, buildTimeline, depenetrate, + effectorBoneId, + enforceContactRom, + floorContactHeight, + floorTargetForEffector, + formFists, groundFigure, + isDipBarGrip, levelPlantedFeet, + prepareGripFrames, propContactExemptions, + reachChain, resolvePropContacts, + solveCCD, + solveReachToPoint, + wrapGrip, + type Character, + type Proportions, } from "posecode-render"; export type Vec3 = readonly [x: number, y: number, z: number]; export type Quat = readonly [x: number, y: number, z: number, w: number]; +export type ContactKind = "reach" | "pin" | "grip" | "ground-lock"; +export type ContactStatus = "resolved" | "unsupported"; + +/** A declared contact measured against the final, fully solved phase pose. */ +export interface ContactResidual { + kind: ContactKind; + effector: string; + target: string; + effectorBone: string; + /** Contact activation weight (terminal declarations are 1; blended-out reaches approach 0). */ + weight: number; + status: ContactStatus; + /** Final world-space effector/target positions, when they can be resolved. */ + effectorPosition: Vec3 | null; + targetPosition: Vec3 | null; + /** Euclidean positional error in metres; null means the evaluator cannot solve it. */ + error: number | null; + /** Explicit reason for an unsupported path. Never silently treated as a pass. */ + reason?: string; +} + export interface PhasePose { /** Phase name from the document. */ name: string; @@ -39,6 +84,9 @@ export interface PhasePose { groundLock: readonly string[]; pins: readonly PinTarget[]; reaches: readonly ReachTarget[]; + grips: readonly GripTarget[]; + /** Positional truth for every declared reach/pin/grip in this phase. */ + contactResiduals: readonly ContactResidual[]; rootOffset: Vec3; rootYaw: number; /** @@ -49,8 +97,6 @@ export interface PhasePose { * do for authored travel. */ propPush: Vec3; - /** True when the phase relies on pins/reach-IK the probe cannot solve. */ - usesSceneIk: boolean; /** Whether the phase should rest on the floor (no elevated prop/grip support). */ floorBound: boolean; /** @@ -71,22 +117,62 @@ export interface ProbeResult { warnings: readonly Warning[]; phases: readonly PhasePose[]; propTypes: readonly string[]; + /** Flattened contact residuals for scorecards/consumers that do not walk phases. */ + contactResiduals: readonly ContactResidual[]; } const DEG = Math.PI / 180; const EPS = 1e-4; const WORLD_Y = new THREE.Vector3(0, 1, 0); +type TargetReference = + | { kind: "fixed"; point: THREE.Vector3 } + | { kind: "floor"; point: THREE.Vector3 } + | { kind: "landmark"; boneId: string }; + +interface PendingContact { + kind: ContactKind; + effector: string; + target: string; + effectorBone: string; + weight: number; + targetRef: TargetReference | null; + reason?: string; +} + +function fistSidesOf( + reaches: readonly ReachTarget[], + pins: readonly PinTarget[], + groundLock: readonly string[], +): Set<"left" | "right"> { + const sides = new Set<"left" | "right">(); + const add = (effector: string): void => { + if (effector === "fists" || effector === "fist_left") sides.add("left"); + if (effector === "fists" || effector === "fist_right") sides.add("right"); + }; + for (const reach of reaches) add(reach.effector); + for (const pin of pins) add(pin.effector); + for (const effector of groundLock) add(effector); + return sides; +} + /** Probe a movement: FK + root solving at each phase end, viewer-faithful. */ -export function probeMovement(source: string): ProbeResult { +export function probeMovement( + source: string, + proportions?: Proportions, + character?: Character, +): ProbeResult { const { ir, errors, warnings } = parse(source); if (!ir || errors.length > 0) { - return { ok: false, errors, warnings, phases: [], propTypes: [] }; + return { ok: false, errors, warnings, phases: [], propTypes: [], contactResiduals: [] }; } - const m = buildMannequin(); + const m = buildMannequin(undefined, proportions); const tl = buildTimeline(ir); const propScene = buildProps(ir.props); + const authoredFingers = new Set(tl.bonesUsed.filter((id) => + /^(thumb|index|middle|ring|pinky)_(left|right)$/.test(id), + )); // Mirror Viewer.load(): reset bones, apply the base-pose root, pose at t=0, // then drop the figure onto the floor and remember the grounded base root. @@ -97,6 +183,21 @@ export function probeMovement(source: string): ProbeResult { m.root.rotation.set(rx * DEG, ry * DEG, rz * DEG); tl.sample(0, m.bones); m.root.updateMatrixWorld(true); + formFists( + m, + fistSidesOf( + ir.phases[0]?.reaches ?? [], + ir.phases[0]?.pins ?? [], + ir.phases[0]?.groundLock ?? [], + ), + authoredFingers, + ); + alignFloorContacts( + m, + ir.phases[0]?.reaches ?? [], + ir.phases[0]?.pins ?? [], + ir.phases[0]?.groundLock ?? [], + ); depenetrate(m); groundFigure(m); resolvePropContacts(m, propScene.colliders, propContactExemptions([ @@ -121,13 +222,7 @@ export function probeMovement(source: string): ProbeResult { const segmentStartEffectors: Map[] = []; const tempYawQ = new THREE.Quaternion(); - const getEffectorId = (eff: string) => { - if (eff === "hand_left") return "wrist_left"; - if (eff === "hand_right") return "wrist_right"; - if (eff === "foot_left") return "ankle_left"; - if (eff === "foot_right") return "ankle_right"; - return eff; - }; + const getEffectorId = (eff: string) => effectorBoneId(eff); let prevEffectorsMap: Map | null = null; let prevPins: typeof ir.phases[number]["pins"] = []; @@ -178,12 +273,298 @@ export function probeMovement(source: string): ProbeResult { depenetrate(m); groundFigure(m); + const resolveTarget = ( + target: string, + effectorName: string, + ): { point: THREE.Vector3; ref: TargetReference } | null => { + if (target === "floor") { + const point = floorTargetForEffector(m, effectorName); + if (!point) return null; + return { point, ref: { kind: "floor", point: point.clone() } }; + } + const prop = propScene.anchors.get(target); + if (prop) { + const point = prop.clone(); + return { point, ref: { kind: "fixed", point: point.clone() } }; + } + const landmark = m.bones.get(target); + if (landmark) { + return { + point: landmark.getWorldPosition(new THREE.Vector3()), + ref: { kind: "landmark", boneId: target }, + }; + } + return null; + }; + + const unsupported = ( + kind: ContactKind, + effector: string, + target: string, + effectorBone: string, + reason: string, + weight = 1, + ): PendingContact => ({ kind, effector, target, effectorBone, weight, targetRef: null, reason }); + + const applyPins = (pins: readonly PinTarget[], phaseIndex: number): PendingContact[] => { + const dipBarPins = pins.filter((pin) => isDipBarGrip(pin.anchor)); + prepareGripFrames(m, dipBarPins); + const contacts: PendingContact[] = []; + const solvable: Array<{ contact: PendingContact; effector: THREE.Object3D; point: THREE.Vector3 }> = []; + for (const pin of pins) { + const effectorBone = getEffectorId(pin.effector); + const effector = m.bones.get(effectorBone); + if (!effector) { + contacts.push(unsupported("pin", pin.effector, pin.anchor, effectorBone, `unknown effector bone "${effectorBone}"`)); + continue; + } + let resolved: { point: THREE.Vector3; ref: TargetReference } | null = null; + if (pin.anchor === "floor") { + const start = segmentStartEffectors[phaseIndex]?.get(effectorBone); + if (start) { + const point = start.clone(); + point.y = floorTargetForEffector(m, pin.effector)?.y ?? 0; + resolved = { point, ref: { kind: "floor", point: point.clone() } }; + } + } + resolved ??= resolveTarget(pin.anchor, pin.effector); + if (!resolved) { + contacts.push(unsupported("pin", pin.effector, pin.anchor, effectorBone, `unknown anchor "${pin.anchor}"`)); + continue; + } + const contact: PendingContact = { + kind: "pin", + effector: pin.effector, + target: pin.anchor, + effectorBone, + weight: 1, + targetRef: resolved.ref, + }; + contacts.push(contact); + solvable.push({ contact, effector, point: resolved.point }); + } + if (solvable.length > 0) { + const delta = new THREE.Vector3(); + for (const item of solvable) { + delta.add(item.point.clone().sub(item.effector.getWorldPosition(new THREE.Vector3()))); + } + m.root.position.add(delta.multiplyScalar(1 / solvable.length)); + m.root.updateMatrixWorld(true); + } + alignGripFrames(m, dipBarPins); + return contacts; + }; + + const applyGrips = (grips: readonly GripTarget[]): PendingContact[] => { + prepareGripFrames(m, grips); + const contacts: PendingContact[] = []; + const solvable: Array<{ + contact: PendingContact; + effector: THREE.Object3D; + point: THREE.Vector3; + }> = []; + for (const grip of grips) { + const effectorBone = getEffectorId(grip.effector); + const effector = m.bones.get(effectorBone); + if (!effector) { + contacts.push(unsupported("grip", grip.effector, grip.anchor, effectorBone, `unknown effector bone "${effectorBone}"`)); + continue; + } + const resolved = resolveTarget(grip.anchor, grip.effector) + ?? resolveTarget(grip.anchor.replace(/_(left|right)$/, ""), grip.effector); + if (!resolved) { + contacts.push(unsupported("grip", grip.effector, grip.anchor, effectorBone, `unknown anchor "${grip.anchor}"`)); + continue; + } + const contact: PendingContact = { + kind: "grip", + effector: grip.effector, + target: grip.anchor, + effectorBone, + weight: 1, + targetRef: resolved.ref, + }; + contacts.push(contact); + solvable.push({ contact, effector, point: resolved.point }); + } + if (solvable.length > 0) { + const delta = new THREE.Vector3(); + for (const item of solvable) { + delta.add(item.point.clone().sub(item.effector.getWorldPosition(new THREE.Vector3()))); + } + m.root.position.add(delta.multiplyScalar(1 / solvable.length)); + m.root.updateMatrixWorld(true); + } + for (const item of solvable) { + const { joints, limits } = reachChain(m, item.contact.effector); + if (joints.length === 0) { + item.contact.reason = "production IK unsupported: unsupported effector"; + item.contact.targetRef = null; + continue; + } + if (isDipBarGrip(item.contact.target)) { + // Viewer parity: keep the elbow's axial solution fixed so a dip-bar + // palm cannot flip between or away from the parallel rails. + for (let i = 0; i < joints.length; i++) { + if (!joints[i]!.name.startsWith("elbow_")) continue; + const limit = limits[i]; + if (limit) limits[i] = { ...limit, y: [0, 0] }; + } + } + solveCCD({ joints, limits, effector: item.effector, target: item.point }, 12); + } + alignGripFrames(m, grips); + wrapGrip(m, grips); + m.root.updateMatrixWorld(true); + return contacts; + }; + + const applyReaches = ( + reaches: readonly (ReachTarget & { weight: number })[], + ): PendingContact[] => { + const contacts: PendingContact[] = []; + for (const reach of reaches) { + const effectorBone = getEffectorId(reach.effector); + const effector = m.bones.get(effectorBone); + if (!effector) { + contacts.push(unsupported("reach", reach.effector, reach.target, effectorBone, `unknown effector bone "${effectorBone}"`, reach.weight)); + continue; + } + const resolved = resolveTarget(reach.target, reach.effector); + if (!resolved) { + contacts.push(unsupported("reach", reach.effector, reach.target, effectorBone, `unknown target "${reach.target}"`, reach.weight)); + continue; + } + const solved = solveReachToPoint( + m, + reach.effector, + reach.target, + resolved.point, + reach.weight, + ); + if (solved.distance === null) { + contacts.push(unsupported( + "reach", + reach.effector, + reach.target, + effectorBone, + `production IK unsupported: ${solved.reason ?? "unknown reason"}`, + reach.weight, + )); + continue; + } + contacts.push({ + kind: "reach", + effector: reach.effector, + target: reach.target, + effectorBone, + weight: reach.weight, + targetRef: resolved.ref, + }); + } + return contacts; + }; + + const finalizeContacts = (pending: readonly PendingContact[]): ContactResidual[] => + pending.map((contact) => { + // Contact targets are defined against the calibrated production driver + // surfaces (sole/glute/knuckle offsets differ from raw skinned joint + // origins). The visible character is sampled separately below for actual + // skeleton geometry and exact skinned-mesh floor bounds. + const effectorPoint = m.bones.get(contact.effectorBone) + ?.getWorldPosition(new THREE.Vector3()) ?? null; + let targetPoint: THREE.Vector3 | null = null; + if (contact.targetRef?.kind === "fixed") { + targetPoint = contact.targetRef.point.clone(); + } else if (contact.targetRef?.kind === "landmark") { + targetPoint = m.bones.get(contact.targetRef.boneId) + ?.getWorldPosition(new THREE.Vector3()) ?? null; + } else if (contact.targetRef?.kind === "floor" && effectorPoint) { + const height = floorContactHeight(m, contact.effector); + if (height !== null) { + targetPoint = contact.targetRef.point.clone().setY(effectorPoint.y - height); + } + } + const status: ContactStatus = contact.reason || !effectorPoint || !targetPoint + ? "unsupported" + : "resolved"; + return { + kind: contact.kind, + effector: contact.effector, + target: contact.target, + effectorBone: contact.effectorBone, + weight: contact.weight, + status, + effectorPosition: effectorPoint ? vectorTuple(effectorPoint) : null, + targetPosition: targetPoint ? vectorTuple(targetPoint) : null, + error: status === "resolved" ? effectorPoint!.distanceTo(targetPoint!) : null, + ...(contact.reason ? { reason: contact.reason } : {}), + }; + }); + + const measureGroundLocks = (active: readonly string[]): ContactResidual[] => { + const bones = new Set(); + for (const name of active) { + const expanded = m.effectors[name]; + if (expanded) expanded.forEach((id) => bones.add(id)); + else bones.add(effectorBoneId(name)); + } + return [...bones].map((boneId) => { + const effector = m.bones.get(boneId); + const point = effector?.getWorldPosition(new THREE.Vector3()) ?? null; + const height = floorContactHeight(m, boneId); + const target = point && height !== null + ? point.clone().setY(point.y - height) + : null; + const side = boneId.endsWith("_left") ? "left" : "right"; + const semantic = boneId.startsWith("ankle_") + ? `foot_${side}` + : boneId.startsWith("wrist_") + ? `hand_${side}` + : boneId.startsWith("elbow_") + ? `elbow_${side}` + : boneId; + const status: ContactStatus = point && target && height !== null ? "resolved" : "unsupported"; + return { + kind: "ground-lock" as const, + effector: semantic, + target: "floor", + effectorBone: boneId, + weight: 1, + status, + effectorPosition: point ? vectorTuple(point) : null, + targetPosition: target ? vectorTuple(target) : null, + error: status === "resolved" ? Math.abs(height!) : null, + ...(status === "unsupported" ? { reason: "ground-lock surface could not be evaluated" } : {}), + }; + }); + }; + // Sample the end of each phase, applying the viewer's per-frame root // pipeline: base root → yaw/travel → ground-lock → floor safety clamp. const yawQ = new THREE.Quaternion(); + let previousSolvedEffectors: Map | null = null; + const activeFloorPinTargets = new Map(); const phases: PhasePose[] = tl.segments.map((seg, phaseIndex) => { const authored = ir.phases[phaseIndex]!; const info = tl.sample(seg.end - EPS, m.bones); + const currentFloorPins = new Set( + info.pins + .filter((pin) => pin.anchor === "floor") + .map((pin) => effectorBoneId(pin.effector)), + ); + for (const boneId of [...activeFloorPinTargets.keys()]) { + if (!currentFloorPins.has(boneId)) activeFloorPinTargets.delete(boneId); + } + for (const boneId of currentFloorPins) { + let target = activeFloorPinTargets.get(boneId); + if (!target) { + target = previousSolvedEffectors?.get(boneId)?.clone() + ?? segmentStartEffectors[phaseIndex]?.get(boneId)?.clone(); + if (target) activeFloorPinTargets.set(boneId, target); + } + if (target) segmentStartEffectors[phaseIndex]?.set(boneId, target.clone()); + } m.root.position.copy(baseRootPos); m.root.quaternion.copy(baseRootQuat); if (info.rootYaw !== 0) { @@ -193,6 +574,12 @@ export function probeMovement(source: string): ProbeResult { m.root.position.x += info.rootOffset.x; m.root.position.z += info.rootOffset.z; m.root.updateMatrixWorld(true); + formFists( + m, + fistSidesOf(info.reaches, info.pins, info.groundLock), + authoredFingers, + ); + alignFloorContacts(m, info.reaches, info.pins, info.groundLock); // Self-collision resolution, then contact solving (same order as the viewer). depenetrate(m); // Mirror the viewer's per-frame anchors: captured targets carried along @@ -208,48 +595,13 @@ export function probeMovement(source: string): ProbeResult { anchors.set(id, v); } applyGroundLock(m, info.groundLock, anchors); - // Resolve scene-independent pins. Unknown names here are prop anchors and - // intentionally remain for browser-level coverage. - if (info.pins.length > 0) { - const delta = new THREE.Vector3(); - let pinCount = 0; - for (const pin of info.pins) { - const effectorId = pin.effector === "hand_left" - ? "wrist_left" - : pin.effector === "hand_right" - ? "wrist_right" - : pin.effector === "foot_left" - ? "ankle_left" - : pin.effector === "foot_right" - ? "ankle_right" - : pin.effector; - const effector = m.bones.get(effectorId); - if (!effector) continue; - let target: THREE.Vector3 | null = null; - if (pin.anchor === "floor") { - const startPos = segmentStartEffectors[phaseIndex]?.get(effectorId); - if (startPos) { - target = startPos.clone(); - target.y = 0; - } else { - target = effector.getWorldPosition(new THREE.Vector3()); - target.y = 0; - } - } else if (propScene.anchors.has(pin.anchor)) { - target = propScene.anchors.get(pin.anchor)!.clone(); - } else { - const landmark = m.bones.get(pin.anchor); - if (landmark) target = landmark.getWorldPosition(new THREE.Vector3()); - } - if (!target) continue; - delta.add(target.sub(effector.getWorldPosition(new THREE.Vector3()))); - pinCount++; - } - if (pinCount > 0) { - m.root.position.add(delta.multiplyScalar(1 / pinCount)); - m.root.updateMatrixWorld(true); - } - } + // Same production ordering as Viewer.frame(): whole-body pins, bar grips, + // solid-prop correction, then ROM-constrained per-limb reach IK. + const pendingContacts: PendingContact[] = [ + ...applyPins(info.pins, phaseIndex), + ...applyGrips(info.grips), + ]; + let reachContacts: PendingContact[] = []; // Props are solid (viewer parity): after the root solvers place the body, // push it back out of any prop face it crossed and bend swing legs clear. // Limbs pinned/gripped to a prop anchor are declared support, exempt. @@ -260,10 +612,23 @@ export function probeMovement(source: string): ProbeResult { ...info.reaches.map((r) => ({ effector: r.effector, anchor: r.target })), ])); const propPush: Vec3 = [m.root.position.x - prePush.x, 0, m.root.position.z - prePush.z]; - alignFloorPalms(m, info.reaches, info.pins, info.groundLock); + reachContacts = applyReaches(info.reaches); + alignFloorContacts(m, info.reaches, info.pins, info.groundLock); // Plantigrade correction (viewer parity): flatten planted soles. This lifts // the foot mesh a little, so it must run BEFORE the floor clamp reconciles. levelPlantedFeet(m, info.groundLock); + enforceContactRom(m); + // Production bounded multi-contact refinement: a limb reach runs after + // ground-lock and can alter which mesh point is lowest. Replant the root + // support, then solve the independent limbs once more before floor safety. + if (info.groundLock.length > 0 && info.reaches.length > 0) { + for (let refinement = 0; refinement < 3; refinement++) { + applyGroundLock(m, info.groundLock, anchors); + reachContacts = applyReaches(info.reaches); + alignFloorContacts(m, info.reaches, info.pins, info.groundLock); + enforceContactRom(m); + } + } // Viewer safety net: a ground-locked phase is planted, so clamp both ways // (its lowest point sits exactly on the floor); an unlocked phase may be // airborne, so only rescue parts that dip below y=0. Mirror index.ts. @@ -274,26 +639,67 @@ export function probeMovement(source: string): ProbeResult { m.root.position.y -= box.min.y; m.root.updateMatrixWorld(true); } - const finalBox = new THREE.Box3().setFromObject(m.root); + // Production visible-rig path: retarget the solved driver, then reconcile + // the exact skinned surface with the floor exactly as Viewer.frame does. + if (character) { + character.sync(m); + if (floorBound) character.reconcileFloor(); + } + const finalBox = character + ? character.getBounds() + : new THREE.Box3().setFromObject(m.root); + // At an endpoint, timeline blending retains the previous phase's reach at + // a vanishing weight. It is executed for viewer continuity but is not a + // contact declaration of this phase and must not create a false failure. + const contactResiduals = [ + ...finalizeContacts( + [...pendingContacts, ...reachContacts] + .filter((contact) => contact.kind !== "reach" || contact.weight >= 0.99), + ), + ...measureGroundLocks(info.groundLock), + ]; + previousSolvedEffectors = new Map(); + for (const ids of Object.values(m.effectors)) { + for (const id of ids) { + const node = m.bones.get(id); + if (node) previousSolvedEffectors.set(id, node.getWorldPosition(new THREE.Vector3())); + } + } return { name: seg.name, durationSec: authored.durationSec, easing: authored.easing, groundLock: [...info.groundLock], - pins: [...info.pins], - reaches: [...info.reaches], + pins: [...authored.pins], + reaches: [...authored.reaches], + grips: [...authored.grips], + contactResiduals, rootOffset: [info.rootOffset.x, 0, info.rootOffset.z], rootYaw: info.rootYaw, propPush, - usesSceneIk: info.pins.length > 0 || info.reaches.length > 0 || info.grips.length > 0, floorBound, meshMinY: Number.isFinite(finalBox.min.y) ? finalBox.min.y : 0, - bones: snapshotBones(m.bones), - boneQuaternions: snapshotBoneQuaternions(m.bones), + bones: character + ? snapshotCharacterBones(m.bones.keys(), character) + : snapshotBones(m.bones), + boneQuaternions: character + ? snapshotCharacterQuaternions(m.bones.keys(), character) + : snapshotBoneQuaternions(m.bones), }; }); - return { ok: true, errors, warnings, phases, propTypes: [...ir.props] }; + return { + ok: true, + errors, + warnings, + phases, + propTypes: [...ir.props], + contactResiduals: phases.flatMap((phase) => [...phase.contactResiduals]), + }; +} + +function vectorTuple(v: THREE.Vector3): Vec3 { + return [v.x, v.y, v.z]; } function snapshotBoneQuaternions(bones: Map): Map { @@ -315,3 +721,21 @@ function snapshotBones(bones: Map): Map { } return out; } + +function snapshotCharacterBones(ids: Iterable, character: Character): Map { + const out = new Map(); + for (const id of ids) { + const v = character.getJointWorldPosition(id); + if (v) out.set(id, vectorTuple(v)); + } + return out; +} + +function snapshotCharacterQuaternions(ids: Iterable, character: Character): Map { + const out = new Map(); + for (const id of ids) { + const q = character.getJointDriverQuaternion(id); + if (q) out.set(id, [q.x, q.y, q.z, q.w]); + } + return out; +} diff --git a/packages/posecode-eval/src/report.ts b/packages/posecode-eval/src/report.ts index f15f20e..95647d9 100644 --- a/packages/posecode-eval/src/report.ts +++ b/packages/posecode-eval/src/report.ts @@ -5,6 +5,7 @@ import { probeMovement } from "./probe.js"; import { genericChecks, MOVEMENT_CHECKS, type CheckOutcome } from "./checks.js"; +import type { Character, Proportions } from "posecode-render"; export interface MovementSource { /** Identifier matched against MOVEMENT_CHECKS (e.g. file stem "deadlift"). */ @@ -32,8 +33,18 @@ export interface EvalReport { }; } -export function runEval(sources: readonly MovementSource[]): EvalReport { - const movements = sources.map((s) => evalMovement(s)); +export interface EvalOptions { + /** Driver proportions used by the production character being evaluated. */ + proportions?: Proportions; + /** Optional retargeted visible character sampled after each solved phase. */ + character?: Character; +} + +export function runEval( + sources: readonly MovementSource[], + options: EvalOptions = {}, +): EvalReport { + const movements = sources.map((s) => evalMovement(s, options)); return { movements, summary: { @@ -46,8 +57,11 @@ export function runEval(sources: readonly MovementSource[]): EvalReport { }; } -function evalMovement({ movement, source }: MovementSource): MovementReport { - const result = probeMovement(source); +function evalMovement( + { movement, source }: MovementSource, + options: EvalOptions, +): MovementReport { + const result = probeMovement(source, options.proportions, options.character); const specific = MOVEMENT_CHECKS.find((m) => m.movement === movement); const checks = [ ...genericChecks(result), diff --git a/packages/posecode-eval/src/xbot.ts b/packages/posecode-eval/src/xbot.ts new file mode 100644 index 0000000..cecd90e --- /dev/null +++ b/packages/posecode-eval/src/xbot.ts @@ -0,0 +1,21 @@ +/** Node-only loader for the same XBot proportions used by the web playground. */ +import { readFileSync } from "node:fs"; +import * as THREE from "three"; +import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js"; +import { rigCharacter, type Character, type Proportions } from "posecode-render"; + +export async function loadXbotCharacter(asset: URL): Promise { + const bytes = readFileSync(asset); + const buffer = bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength); + const scene = await new Promise((resolve, reject) => { + new GLTFLoader().parse(buffer, "", (gltf) => resolve(gltf.scene), reject); + }); + return rigCharacter(scene); +} + +export async function loadXbotProportions(asset: URL): Promise { + const character = await loadXbotCharacter(asset); + const proportions = character.proportions; + character.dispose(); + return proportions; +} diff --git a/packages/posecode-eval/test/eval.test.ts b/packages/posecode-eval/test/eval.test.ts index 99f8d94..d786fa0 100644 --- a/packages/posecode-eval/test/eval.test.ts +++ b/packages/posecode-eval/test/eval.test.ts @@ -1,6 +1,7 @@ import { describe, it, expect } from "vitest"; import { fileURLToPath } from "node:url"; import { dirname, resolve } from "node:path"; +import type { Proportions } from "posecode-render"; import { loadFixtures, probeMovement, @@ -9,6 +10,7 @@ import { kneeFlexionDeg, balanceOverflow, footSkateDistance, + footWorldSkateDistance, headPropClearance, palmFloorAngleDeg, phaseMaxLandmarkSpeed, @@ -36,6 +38,75 @@ describe("probe", () => { expect(r.phases).toHaveLength(1); expect(r.phases[0]!.bones.size).toBeGreaterThanOrEqual(17); }); + + it("fails an unreachable declared hand-to-floor contact instead of silently passing", () => { + const source = [ + 'posecode posture "Unreachable floor reach"', + " rig humanoid", + " pose start = standing", + ' step "Reach" 1s settle:', + " reach: hand_left floor", + " ground-lock: feet", + ].join("\n"); + const result = runEval([{ movement: "unreachable-floor-reach", source }]); + const contact = probeMovement(source).phases[0]!.contactResiduals[0]!; + + expect(contact.status).toBe("resolved"); + expect(contact.error).toBeGreaterThan(0.03); + expect(result.movements[0]!.checks).toContainEqual(expect.objectContaining({ + id: expect.stringContaining("contact-position:Reach:reach:hand_left:floor"), + pass: false, + })); + }); + + it("forms a semantic fist before measuring its production floor target", () => { + const source = [ + 'posecode posture "Fist contact"', + " rig humanoid", + " pose start = standing", + ' step "Plant" 1s settle:', + " pelvis: hinge 110", + " reach: fist_left floor", + " ground-lock: foot_right", + ].join("\n"); + const phase = probeMovement(source).phases[0]!; + const finger = phase.boneQuaternions.get("index_left")!; + const contact = phase.contactResiduals[0]!; + + expect(finger).not.toEqual([0, 0, 0, 1]); + expect(contact.effector).toBe("fist_left"); + expect(contact.effectorBone).toBe("wrist_left"); + expect(contact.targetPosition).not.toBeNull(); + }); + + it("uses supplied character proportions throughout runEval", () => { + const source = [ + 'posecode posture "Proportion probe"', + " rig humanoid", + " pose start = standing", + ' step "Reach" 1s settle:', + " reach: hand_left knee_left", + " ground-lock: feet", + ].join("\n"); + const longLeftArm: Proportions = { + offsets: { + elbow_left: [0, -0.47, 0], + wrist_left: [0, -0.47, 0], + }, + }; + const contactPasses = (proportions?: Proportions): boolean => { + const report = runEval( + [{ movement: "proportion-probe", source }], + proportions ? { proportions } : {}, + ); + return report.movements[0]!.checks.find((check) => + check.id.startsWith("contact-position:"), + )!.pass; + }; + + expect(contactPasses()).toBe(false); + expect(contactPasses(longLeftArm)).toBe(true); + }); }); describe("metrics", () => { @@ -64,8 +135,10 @@ describe("metrics", () => { const fixtures = loadFixtures(examplesDir); const movement = (name: string) => probeMovement(fixtures.find((f) => f.movement === name)!.source); - const legRaise = movement("supine-leg-raise").phases[0]!; - expect(palmFloorAngleDeg(legRaise, "left")).toBeLessThan(5); + const palmDownQuaternions = new Map(standing.boneQuaternions); + const halfTurn = Math.SQRT1_2; + palmDownQuaternions.set("wrist_left", [halfTurn, 0, 0, halfTurn]); + expect(palmFloorAngleDeg({ ...standing, boneQuaternions: palmDownQuaternions }, "left")).toBeLessThan(0.01); const deadlift = movement("deadlift"); expect(footSkateDistance(deadlift.phases[0]!, deadlift.phases[1]!, "left")).toBeLessThan(0.2); @@ -83,16 +156,134 @@ describe("metrics", () => { const heights = deadBug.phases.map((p) => p.bones.get(id)![1]); expect(Math.max(...heights) - Math.min(...heights)).toBeLessThan(0.01); } + + const lunge = movement("forward-lunge"); + // The back-foot pin is fixed in the scene. Removing authored travel from + // its measurement fabricates 30 cm of skate even though world drift is 0. + expect(footWorldSkateDistance(lunge.phases[0]!, lunge.phases[1]!, "left")).toBeLessThan(1e-4); + expect(footSkateDistance(lunge.phases[0]!, lunge.phases[1]!, "left")).toBeGreaterThan(0.25); + }); + + it("does not auto-pass a single support point", () => { + const bones = new Map(standing.bones); + for (const [id, point] of bones) { + if (id === "ankle_left") continue; + bones.set(id, [point[0] + 1, point[1], point[2]]); + } + const offBalance = { ...standing, groundLock: ["foot_left"], bones }; + + expect(balanceOverflow(offBalance)).toBeGreaterThan(0.5); + }); + + it("carries a solved ground-lock support into a newly introduced floor pin", () => { + const source = [ + 'posecode exercise "Support handoff"', + " rig humanoid", + " pose start = standing", + ' step "Lower" 1s settle:', + " hip_left: flex 80", + " knee_left: flex 95", + " ground-lock: foot_left", + ' step "Transfer" 0.5s settle:', + " pin: foot_left floor", + ].join("\n"); + const result = probeMovement(source); + expect(result.errors).toEqual([]); + expect(footWorldSkateDistance(result.phases[0]!, result.phases[1]!, "left")).toBeLessThan(0.03); }); }); describe("fixture scorecard", () => { - it("every canonical example passes every invariant", () => { - const report = runEval(loadFixtures(examplesDir)); - const failures = report.movements - .flatMap((m) => m.checks.filter((c) => !c.pass).map((c) => `${m.movement}/${c.id}: ${c.detail}`)); - expect(failures).toEqual([]); + it("scores every declared canonical contact and keeps the fallback catalog green", () => { + const fixtures = loadFixtures(examplesDir); + const report = runEval(fixtures); + const declaredContacts = fixtures.reduce( + (count, fixture) => count + probeMovement(fixture.source).contactResiduals.length, + 0, + ); + const contactChecks = report.movements.flatMap((movement) => + movement.checks.filter((check) => check.id.startsWith("contact-")), + ); + + expect(contactChecks).toHaveLength(declaredContacts); + expect(contactChecks.every((check) => check.pass)).toBe(true); expect(report.summary.parseFailures).toBe(0); expect(report.summary.clampWarnings).toBe(0); + expect(report.summary.checksPassed).toBe(report.summary.checksTotal); + }); + + it("does not give the supplied bad superhero landing a perfect score", () => { + const source = `posecode posture "Superhero Three-Point Landing" + rig humanoid + pose start = standing + + step "Drop into the landing" 0.7s drive: + pelvis: hinge 35 + hip_left: flex 95 + hip_right: flex 65 + knee_left: flex 125 + knee_right: flex 90 + ankle_left: dorsiflex 12 + ankle_right: dorsiflex 15 + chest: rotate-out 18 + shoulder_left: flex 55 + shoulder_right: flex 145 + elbow_left: flex 20 + elbow_right: flex 95 + fingers: flex 65 + ground-lock: feet + cue "Drop low with the right foot forward and the left knee close to the floor" + + step "Plant the fist" 0.35s settle: + pelvis: hinge 50 + hip_left: flex 110 + hip_right: flex 80 + knee_left: flex 138 + knee_right: flex 105 + ankle_left: dorsiflex 10 + ankle_right: dorsiflex 15 + shoulder_left: flex 70 + shoulder_right: flex 150 + shoulder_right: abduct 35 + elbow_left: flex 10 + elbow_right: flex 105 + wrist_left: hold neutral 0 + wrist_right: flex 20 + fingers_left: flex 70 + fingers_right: flex 55 + reach: hand_left floor + ground-lock: feet + cue "Plant the left fist beside the body while the right arm arcs overhead" + + step "Hero pose" 1.5s linear: + neck: extend 8 + head: rotate-out 12 + chest: extend 10 + chest: rotate-out 22 + shoulder_left: flex 70 + shoulder_right: flex 155 + shoulder_right: abduct 40 + elbow_left: flex 8 + elbow_right: flex 110 + wrist_right: flex 25 + fingers_left: flex 75 + fingers_right: flex 60 + reach: hand_left floor + ground-lock: feet + cue "Hold the low three-point stance with the chest open and the raised arm curved overhead" + + repeat 1`; + const supplied = runEval([{ movement: "superhero-landing", source }]).movements[0]!; + expect(supplied.passed).toBeLessThan(supplied.total); + + // Strict language validation may reject the supplied trailing `0`. Remove + // only that syntax error and verify the kinematic/contact score still fails: + // a parse failure must not be the evaluator's sole line of defence. + const syntacticallyValid = source.replace("wrist_left: hold neutral 0", "wrist_left: hold neutral"); + const movement = runEval([{ movement: "superhero-landing", source: syntacticallyValid }]).movements[0]!; + const failed = movement.checks.filter((check) => !check.pass); + expect(movement.parseOk).toBe(true); + expect(movement.passed).toBeLessThan(movement.total); + expect(failed.some((check) => check.id.startsWith("contact-position:"))).toBe(true); }); }); diff --git a/packages/posecode-language/src/completion.ts b/packages/posecode-language/src/completion.ts index 283e661..a59f463 100644 --- a/packages/posecode-language/src/completion.ts +++ b/packages/posecode-language/src/completion.ts @@ -10,9 +10,12 @@ import { POSES, EFFECTORS, REACH_EFFECTORS, + PIN_EFFECTORS, + GRIP_EFFECTORS, MODES, JOINT_NAMES, ACTION_NAMES, + actionsForJoint, TOP_KEYWORDS, CHILD_KEYWORDS, KEYWORD_DOCS, @@ -39,6 +42,8 @@ type Context = | "easing" | "effector" | "reach-effector" + | "pin-effector" + | "grip-effector" | "action" | "joint" | "top" @@ -48,15 +53,17 @@ function contextFor(prefix: string, line: number): Context { if (line === 0) { return /^\s*posecode\s+[\w-]*$/.test(prefix) ? "kind" : "none"; } - if (/^\s*pose\s+start\s*=\s*[\w-]*$/.test(prefix)) return "pose"; - if (/^\s*step\s+"[^"]*"\s+[0-9.]+s\s+[\w-]*$/.test(prefix)) return "easing"; - if (/^\s*ground-lock\s*:\s*[\w,\s-]*$/.test(prefix)) return "effector"; - if (/^\s*(reach|pin|grip)\s*:\s*[\w-]*$/.test(prefix)) return "reach-effector"; - if (/^\s*[\w-]+\s*:\s*[\w-]*$/.test(prefix)) return "action"; + const indent = prefix.length - prefix.trimStart().length; + if (indent === 2 && /^\s*pose\s+start\s*=\s*[\w-]*$/.test(prefix)) return "pose"; + if (indent === 2 && /^\s*step\s+"[^"]*"\s+[0-9.]+s\s+[\w-]*$/.test(prefix)) return "easing"; + if (indent >= 4 && /^\s*ground-lock\s*:\s*[\w,\s-]*$/.test(prefix)) return "effector"; + if (indent >= 4 && /^\s*reach\s*:\s*[\w-]*$/.test(prefix)) return "reach-effector"; + if (indent >= 4 && /^\s*pin\s*:\s*[\w-]*$/.test(prefix)) return "pin-effector"; + if (indent >= 4 && /^\s*grip\s*:\s*[\w-]*$/.test(prefix)) return "grip-effector"; + if (indent >= 4 && /^\s*[\w-]+\s*:\s*[\w-]*$/.test(prefix)) return "action"; if (/^\s*[\w-]*$/.test(prefix)) { - const indent = prefix.length - prefix.trimStart().length; - return indent >= 2 ? "joint" : "top"; + return indent >= 4 ? "joint" : indent >= 2 ? "top" : "none"; } return "none"; } @@ -85,8 +92,16 @@ export function getCompletions( return EFFECTORS.map((e) => item(e, "effector")); case "reach-effector": return REACH_EFFECTORS.map((e) => item(e, "effector")); + case "pin-effector": + return PIN_EFFECTORS.map((e) => item(e, "effector")); + case "grip-effector": + return GRIP_EFFECTORS.map((e) => item(e, "effector")); case "action": - return [...ACTION_NAMES, "hold"].map((a) => item(a, "action")); + { + const joint = /^\s*([\w-]+)\s*:/.exec(prefix)?.[1]; + const actions = joint ? actionsForJoint(joint) : ACTION_NAMES; + return [...actions, "hold"].map((a) => item(a, "action")); + } case "joint": return [ ...JOINT_NAMES.map((j) => item(j, "joint")), diff --git a/packages/posecode-language/src/diagnostics.ts b/packages/posecode-language/src/diagnostics.ts index 4be6042..b8c8a85 100644 --- a/packages/posecode-language/src/diagnostics.ts +++ b/packages/posecode-language/src/diagnostics.ts @@ -33,7 +33,7 @@ export function getDiagnostics(text: string): Diagnostic[] { diagnostics.push({ line: w.line, severity: "warning", - message: `${joint} ${w.action} ${w.requested}° exceeds range of motion, clamped to ${w.clamped}° (safe ${w.limit.min}–${w.limit.max}°)`, + message: `${joint} ${w.action} ${w.requested}° exceeds the configured range, clamped to ${w.clamped}° (${w.limit.min}–${w.limit.max}°)`, }); } @@ -51,6 +51,18 @@ export function getDiagnostics(text: string): Diagnostic[] { message: `"${tok}" is deprecated; use "${LEGACY_MODE_ALIASES[tok]}"`, }); } + + const axial = /^\s*(spine|chest|neck|head)\s*:\s*(rotate-in|rotate-out)\b/.exec( + lineText, + ); + if (axial) { + const replacement = axial[2] === "rotate-in" ? "twist-left" : "twist-right"; + diagnostics.push({ + line: idx + 1, + severity: "hint", + message: `"${axial[2]}" is ambiguous on ${axial[1]}; use "${replacement}"`, + }); + } }); return diagnostics; diff --git a/packages/posecode-language/src/hover.ts b/packages/posecode-language/src/hover.ts index 3f175ff..bbeffaa 100644 --- a/packages/posecode-language/src/hover.ts +++ b/packages/posecode-language/src/hover.ts @@ -1,5 +1,5 @@ /** - * Hover info. The biggest win is showing a joint+action's safe range of motion + * Hover info. The biggest win is showing a joint+action's configured range of motion * right where it's authored, so the "why was my angle clamped?" answer is one * hover away. Falls back to short docs for keywords, kinds, poses, and easings. */ @@ -58,7 +58,7 @@ export function getHover( const rom = romFor(bone, token); if (rom) { return md( - `**${boneType(bone)} · ${token}**: safe range **${rom.min}–${rom.max}°**. Angles beyond this are hard-clamped.`, + `**${boneType(bone)} · ${token}**: configured range **${rom.min}–${rom.max}°**. Angles beyond this are clamped with a diagnostic.`, ); } } diff --git a/packages/posecode-language/src/vocab.ts b/packages/posecode-language/src/vocab.ts index febb4c5..49b350a 100644 --- a/packages/posecode-language/src/vocab.ts +++ b/packages/posecode-language/src/vocab.ts @@ -10,24 +10,39 @@ import { EASINGS, MODES, LEGACY_MODE_ALIASES, - EFFECTOR_NAMES, + REACH_EFFECTOR_NAMES, + PIN_EFFECTOR_NAMES, + GRIP_EFFECTOR_NAMES, GROUND_LOCK_EFFECTOR_NAMES, + MOVEMENT_KINDS, + START_POSE_NAMES, + PROP_TYPES, + actionsForJoint, } from "posecode-parser"; -export { JOINT_NAMES, ACTION_NAMES, EASINGS, MODES, LEGACY_MODE_ALIASES }; +export { + JOINT_NAMES, + ACTION_NAMES, + EASINGS, + MODES, + LEGACY_MODE_ALIASES, + actionsForJoint, +}; /** Movement kinds in the header (`posecode "..."`). */ -export const KINDS = ["exercise", "stretch", "posture"]; +export const KINDS: string[] = [...MOVEMENT_KINDS]; /** Recognised start poses (`pose start = ...`). */ -export const POSES = ["neutral", "standing", "plank", "supine", "prone", "seated"]; +export const POSES: string[] = [...START_POSE_NAMES]; /** Floor contacts that can be ground-locked. */ export const EFFECTORS = [...GROUND_LOCK_EFFECTOR_NAMES]; -/** Reach/pin effectors (groups + per-side aliases), sourced from the parser. */ -export const REACH_EFFECTORS = EFFECTOR_NAMES; -export const PROPS = ["chair", "wall", "bar", "box", "dip-bars"]; +/** Capability-specific constraint effectors, sourced from the parser. */ +export const REACH_EFFECTORS = [...REACH_EFFECTOR_NAMES]; +export const PIN_EFFECTORS = [...PIN_EFFECTOR_NAMES]; +export const GRIP_EFFECTORS = [...GRIP_EFFECTOR_NAMES]; +export const PROPS: string[] = [...PROP_TYPES]; /** Top-level directives (excluding the `posecode` header keyword). */ export const TOP_KEYWORDS = ["rig", "prop", "pose", "clip", "step", "repeat"]; @@ -39,7 +54,7 @@ export const CHILD_KEYWORDS = ["ground-lock", "reach", "pin", "grip", "turn", "t export const KEYWORD_DOCS: Record = { posecode: 'Document header: `posecode ""`.', rig: "Selects the rig (currently `humanoid`).", - prop: "Adds a scene object: `prop chair | wall | bar | box | dip-bars`. Supplies reach/pin anchors.", + prop: "Adds a scene object: `prop chair | wall | bar | box | dip-bars`. Supplies declared reach, pin, and grip anchors.", pose: "Sets the starting pose: `pose start = standing | neutral | plank | supine | prone | seated`.", start: "Used in `pose start = `.", clip: 'Optional mocap clip: `clip "walk"`. A renderer with a matching retargeted animation plays it crossfaded over the procedural pose; others ignore it.', @@ -50,13 +65,13 @@ export const KEYWORD_DOCS: Record = { snap: "Timing mode: fast, near-immediate arrival with an accent.", linear: "Timing mode: constant velocity; intentionally mechanical.", repeat: "How many times the movement loops.", - "ground-lock": "Keeps grouped, per-side, or axial contacts on the floor: `feet`, `foot_left` (also `left foot`), `hands`, `forearms`, or `back` for supine work. Planted feet auto-level unless intentionally plantarflexed.", + "ground-lock": "Keeps declared floor supports planted for this phase: grouped, per-side, or axial contacts such as `feet`, `foot_left` (also written `left foot`), `hands`, `forearms`, and `back` for supine work. Repeat the declaration in every phase that needs it; planted feet auto-level unless intentionally plantarflexed.", reach: - "Drives an effector to a target via ROM-constrained IK: `reach: hand_left ankle_left`, `reach: hands floor`.", - pin: "Moves the body so an effector sits on an anchor: `pin: hands bar` (hang, pull up, step up, dip).", + "Drives a limb effector toward a validated target via constrained IK: `reach: hand_left ankle_left`, `reach: fist_left floor`, `reach: knee_left floor`.", + pin: "Moves the body so one primary effector sits on a validated anchor. Use `grip` for a two-hand bar or rails contact.", grip: "Holds a bar/rail: `grip: hands bar`. Each hand gets its own two-point anchor (bar_left/bar_right), the arm bends via IK onto it, and the fingers wrap the bar. Use for pull-up, dead-hang, hanging knee raise.", turn: "Turns the figure to face a new direction: `turn: 360` (degrees, yaw about vertical). Absolute, carried across phases. Standing poses only.", travel: "Moves the figure across the floor: `travel: 0.4 0` (world x z metres from the start spot). Absolute, carried across phases. Standing poses only.", cue: "A short coaching cue shown while this phase plays.", - hold: "Keep the joint at its neutral / rest angle.", + hold: "Reset every rotation channel on this joint to its neutral / rest angle: `: hold neutral`.", }; diff --git a/packages/posecode-language/test/language.test.ts b/packages/posecode-language/test/language.test.ts index b3fc2ee..3f732f2 100644 --- a/packages/posecode-language/test/language.test.ts +++ b/packages/posecode-language/test/language.test.ts @@ -34,7 +34,13 @@ describe("getDiagnostics", () => { }); it("returns nothing for a clean document", () => { - const clean = 'posecode posture "P"\n rig humanoid\n pose start = standing'; + const clean = [ + 'posecode posture "P"', + " rig humanoid", + " pose start = standing", + ' step "Hold" 1s linear:', + " pelvis: hold neutral", + ].join("\n"); expect(getDiagnostics(clean)).toEqual([]); }); }); @@ -57,10 +63,28 @@ describe("getCompletions", () => { expect(l).toContain("cue"); }); + it("suggests document directives, not joints, at the two-space top level", () => { + const l = onLine(" ", 2); + expect(l).toEqual(expect.arrayContaining(["step", "repeat", "pose"])); + expect(l).not.toContain("knees"); + expect(l).not.toContain("cue"); + }); + it("suggests actions after `: `", () => { - expect(onLine(" knees: ", 11)).toEqual( - expect.arrayContaining(["flex", "extend"]), + const kneeActions = onLine(" knees: ", 11); + expect(kneeActions).toEqual(expect.arrayContaining(["flex", "extend", "hold"])); + expect(kneeActions).not.toContain("abduct"); + expect(kneeActions).not.toContain("rotate-in"); + }); + + it("offers joint-specific wrist deviation and explicit axial twist", () => { + expect(onLine(" wrist_left: ", 16)).toEqual( + expect.arrayContaining(["flex", "extend", "abduct", "adduct"]), ); + const chestActions = onLine(" chest: ", 11); + expect(chestActions).toEqual(expect.arrayContaining(["twist-left", "twist-right"])); + expect(chestActions).not.toContain("rotate-in"); + expect(chestActions).not.toContain("rotate-out"); }); it("suggests timing modes inside a step header", () => { @@ -89,9 +113,41 @@ describe("getCompletions", () => { it("suggests reach effectors (groups + sides) after `reach: ` and `pin: `", () => { expect(onLine(" reach: ", 11)).toEqual( - expect.arrayContaining(["hands", "hand_left", "foot_right"]), + expect.arrayContaining(["hands", "fists", "knees", "hand_left", "fist_right", "foot_right"]), ); expect(onLine(" pin: ", 9)).toEqual(expect.arrayContaining(["feet"])); + expect(onLine(" reach: ", 11)).not.toContain("pelvis"); + expect(onLine(" pin: ", 9)).toContain("pelvis"); + expect(onLine(" grip: ", 10)).toEqual( + expect.arrayContaining(["hands", "hand_left", "hand_right"]), + ); + expect(onLine(" grip: ", 10)).not.toContain("feet"); + }); +}); + +describe("strict movement diagnostics", () => { + it("reports unsupported joint/action combinations as errors", () => { + const doc = [ + 'posecode posture "Bad knee"', + " rig humanoid", + ' step "Pose" 1s linear:', + " knee_left: abduct 20", + ].join("\n"); + expect(getDiagnostics(doc)).toContainEqual( + expect.objectContaining({ line: 4, severity: "error", message: expect.stringMatching(/not supported/i) }), + ); + }); + + it("nudges legacy axial rotation toward explicit direction", () => { + const doc = [ + 'posecode posture "Twist"', + " rig humanoid", + ' step "Pose" 1s linear:', + " chest: rotate-out 20", + ].join("\n"); + expect(getDiagnostics(doc)).toContainEqual( + expect.objectContaining({ line: 4, severity: "hint", message: expect.stringContaining("twist-right") }), + ); }); }); diff --git a/packages/posecode-mcp/README.md b/packages/posecode-mcp/README.md index e5a6bb5..7fd0400 100644 --- a/packages/posecode-mcp/README.md +++ b/packages/posecode-mcp/README.md @@ -1,9 +1,9 @@ # posecode-mcp A [Model Context Protocol](https://modelcontextprotocol.io) server for **Posecode**. -It gives any MCP-capable agent (Claude Desktop, Cursor, …) a native way to *show -movement*: learn the `.posecode` language, validate a movement against healthy -range-of-motion limits, and get a link that animates it as a 3D figure. +It lets a model in an MCP client learn the `.posecode` language, validate a +movement against configured range-of-motion limits, and return a link that +animates the movement as a 3D figure. This closes the loop the playground left open: no copy-pasting a system prompt or shuttling text between a chat window and the editor. @@ -12,14 +12,14 @@ or shuttling text between a chat window and the editor. | Tool | What it does | | --- | --- | -| `posecode_authoring_guide` | Returns the Posecode authoring guide (grammar, joints, actions, example) so the model can write valid `.posecode`. | -| `validate_posecode` | Parses a `.posecode` document and returns errors plus any **range-of-motion safety clamps** (the angles that were out of healthy range). | +| `posecode_authoring_guide` | Returns the Posecode authoring guide (grammar, joints, actions, example) so a capable model can draft raw `.posecode` for validation. | +| `validate_posecode` | Parses a `.posecode` document and returns errors plus any **range-of-motion clamps** (angles outside the configured rig bounds). | | `render_posecode` | Validates, then returns a **permalink** that renders the movement in the Posecode playground. Hand it to the user to watch. | `validate_posecode` / `render_posecode` flag invalid documents as MCP error results so the model knows to fix and retry. -## Run +## Use from an MCP client Run the latest published server directly from npm: @@ -27,13 +27,7 @@ Run the latest published server directly from npm: npx -y posecode-mcp@latest ``` -For local repository development: - -```bash -npm start -w posecode-mcp # tsx src/stdio.ts -``` - -### Add to Claude Desktop / Cursor +For an MCP client that accepts JSON server configuration: ```json { @@ -50,9 +44,20 @@ npm start -w posecode-mcp # tsx src/stdio.ts `POSECODE_BASE_URL` is optional: it sets the playground that render permalinks point at (defaults to the hosted playground). +## Local development + +From the Posecode monorepo, run the TypeScript source with the workspace script: + +```bash +npm start -w posecode-mcp +``` + +That development command uses `tsx src/stdio.ts`; MCP consumers do not need a +repository checkout, an absolute source path, or a separate `tsx` install. + ## How it fits `render_posecode` builds its links with [`posecode-share`](../posecode-share), the same permalink primitive the playground uses, and validates with -[`posecode-parser`](../posecode-parser). The server holds no rendering itself; 3D math -runs client-side when the link is opened. +[`posecode-parser`](../posecode-parser). The server does not render the movement +itself; 3D math runs in the user's browser when the link is opened. diff --git a/packages/posecode-mcp/src/analyze.ts b/packages/posecode-mcp/src/analyze.ts index 8ff3fe8..b7036c3 100644 --- a/packages/posecode-mcp/src/analyze.ts +++ b/packages/posecode-mcp/src/analyze.ts @@ -2,7 +2,7 @@ * Pure, transport-agnostic core of the Posecode MCP server. * * Turns a `.posecode` document into the structured results the MCP tools - * return: a validation summary (errors + ROM-safety clamps) and a render + * return: a validation summary (errors + configured ROM clamps) and a render * result that adds a playground permalink. No MCP or I/O here: just parse → * shape, so it is trivially unit-testable. */ @@ -31,7 +31,7 @@ export interface ValidationSummary { phases?: PhaseSummary[]; /** Fatal parse/validation errors (empty when ok). */ errors: { line: number; message: string }[]; - /** Angles that exceeded a healthy range of motion and were clamped. */ + /** Angles that exceeded the configured rig range and were clamped. */ romWarnings: Warning[]; } diff --git a/packages/posecode-mcp/src/guide.ts b/packages/posecode-mcp/src/guide.ts index f230f03..cc2fcb7 100644 --- a/packages/posecode-mcp/src/guide.ts +++ b/packages/posecode-mcp/src/guide.ts @@ -5,7 +5,7 @@ * development; falls back to a compact inline grammar if neither is reachable. */ -import { existsSync, readFileSync } from "node:fs"; +import { readFileSync } from "node:fs"; import { fileURLToPath } from "node:url"; import { dirname, resolve } from "node:path"; @@ -13,44 +13,58 @@ let cached: string | null = null; export function authoringGuide(): string { if (cached !== null) return cached; - let guide: string; - try { - const here = dirname(fileURLToPath(import.meta.url)); - const packagedGuide = resolve(here, "llm-authoring.md"); - const repositoryGuide = resolve(here, "../../../spec/llm-authoring.md"); - guide = readFileSync( - existsSync(packagedGuide) ? packagedGuide : repositoryGuide, - "utf8", - ); - } catch { - guide = FALLBACK_GUIDE; + const here = dirname(fileURLToPath(import.meta.url)); + // Published builds ship a generated copy beside the bundle. Repository + // development reads the canonical spec directly. Trying both fixes the old + // package behavior where every installed MCP server silently fell back to a + // stale, much smaller grammar because `../../../spec` was not published. + const candidates = [ + resolve(here, "llm-authoring.md"), + resolve(here, "../../../spec/llm-authoring.md"), + ]; + let guide = ""; + for (const path of candidates) { + try { + guide = readFileSync(path, "utf8"); + break; + } catch { + // Try the next distribution layout. + } } + if (!guide) guide = FALLBACK_GUIDE; cached = guide; return guide; } const FALLBACK_GUIDE = `# Authoring Posecode -Output ONLY a \`.posecode\` document in a code block, no prose. +When the request is representable, output ONLY the raw \`.posecode\` document, +with no Markdown fence or prose. If it needs free flight, multiple people, +arbitrary equipment, exact sign language, or detailed facial/scapular motion, +say that Posecode cannot yet represent the missing capability. ## Grammar \`\`\` posecode "" # kind = exercise | stretch | posture rig humanoid - pose start = # neutral | standing | plank + prop # optional: chair | wall | bar | box | dip-bars + pose start = # neutral | standing | plank | supine | prone | seated step "" : # mode = flow | settle | drive | snap | linear : - ground-lock: # feet/hands/forearms/back, or foot_left / left foot + ground-lock: # repeat feet/hands/forearms/back or side-specific supports + reach: # supported hand/fist/elbow/knee/foot toward a validated target + pin: # move the body around one primary contact + grip: hands # declared bar / rails, two independent hand contacts cue "" repeat \`\`\` Joints: neck head spine chest pelvis, and (singular or plural) shoulders elbows wrists hips knees ankles. Actions (degrees are absolute targets): flex/extend, -abduct/adduct, rotate-in/rotate-out, dorsiflex/plantarflex, hold neutral, and -hinge (hips only, closed-chain hip flexion: torso tips over planted feet with -a neutral spine; use for deadlift / forward fold instead of hips: flex). -Stay within healthy range of motion; the renderer hard-clamps anything beyond. -Use ground-lock: feet when standing, hands and feet in a high plank, forearms -and feet in a forearm plank, and back for supine floor work such as a dead bug. -Do not invent other contact names.`; +abduct/adduct, rotate-in/rotate-out (shoulder/hip), twist-left/twist-right +(axial joints), dorsiflex/plantarflex, hold neutral, and hinge (pelvis only). +Use only joint/action pairs and declared prop anchors accepted by the validator. +Author the gross pose before reach; a parsed reach is not proof of contact. +Keep cues, sides, and declared contacts consistent through every phase. Floor +contacts are a closed vocabulary: use feet when standing, hands and feet in a +high plank, forearms and feet in a forearm plank, and back for supine work.`; diff --git a/packages/posecode-mcp/src/server.ts b/packages/posecode-mcp/src/server.ts index 14d6805..349c922 100644 --- a/packages/posecode-mcp/src/server.ts +++ b/packages/posecode-mcp/src/server.ts @@ -3,7 +3,7 @@ * * Three tools turn "an LLM knows biomechanics" into "an LLM can show movement": * - posecode_authoring_guide: learn the .posecode language inline - * - validate_posecode: parse + surface range-of-motion safety clamps + * - validate_posecode: parse + surface configured range-of-motion clamps * - render_posecode: get a playground link that animates the movement * * The server is decoupled from any transport so it can be driven over stdio in @@ -49,7 +49,7 @@ export function createPosecodeServer(opts: PosecodeServerOptions = {}): McpServe { title: "Validate a Posecode movement", description: - "Parse a .posecode document and return any errors plus range-of-motion (ROM) safety clamps. Use this to check a movement before showing it to a user.", + "Parse a .posecode document and return any errors plus configured range-of-motion (ROM) clamps. Use this to check the source before showing it to a user; a clean result is not a clinical safety assessment.", inputSchema: sourceSchema, }, async ({ source }) => { diff --git a/packages/posecode-parser/README.md b/packages/posecode-parser/README.md index ce7be16..c5ed8cd 100644 --- a/packages/posecode-parser/README.md +++ b/packages/posecode-parser/README.md @@ -56,9 +56,10 @@ aliases `ease-in`, `ease-out`, and `ease-in-out` remain accepted so existing movement files keep working. `parse()` never throws: malformed or out-of-range documents come back as -structured `errors`/`warnings` instead. Every joint angle in `ir` is hard-clamped -to a healthy range of motion, so a hallucinated `knee: flex 200` renders at its -safe ceiling with a warning, never an anatomically impossible joint. +structured `errors`/`warnings` instead. Every joint angle in `ir` is clamped to +Posecode's configured range-of-motion bounds, so a hallucinated +`knee: flex 200` renders at the configured 144° ceiling with a warning. These +rig constraints are not a clinical safety assessment of the full movement. ## License diff --git a/packages/posecode-parser/src/clamp.ts b/packages/posecode-parser/src/clamp.ts index a4984cb..fcb318f 100644 --- a/packages/posecode-parser/src/clamp.ts +++ b/packages/posecode-parser/src/clamp.ts @@ -3,11 +3,12 @@ * * This is the biomechanics-aware stage: it expands symmetric joint groups, * maps semantic actions to rotation axes (with left/right mirroring), and - * clamps every angle into its safe Range of Motion (§5.1). All inputs are + * clamps every angle into the configured rig bounds (§5.1). All inputs are * treated as immutable: a fresh IR is built and returned. */ import type { + Axis, EulerDeg, GripTarget, JointTarget, @@ -19,8 +20,9 @@ import type { Warning, } from "./types.js"; import { POSECODE_VERSION } from "./types.js"; -import type { AstDoc, AstStep } from "./parser.js"; +import type { AstDoc, AstJointTarget, AstStep } from "./parser.js"; import { + BONES, actionAxis, boneType, expandEffector, @@ -29,7 +31,8 @@ import { isGroundLockEffector, isLeft, } from "./joints.js"; -import { clampAngle, romFor } from "./rom.js"; +import { actionsForJoint, clampAngle, romFor } from "./rom.js"; +import { anchorsForProps, propForAnchor } from "./protocol.js"; export interface ResolveResult { ir: PosecodeIR; @@ -38,14 +41,30 @@ export interface ResolveResult { } const ZERO: EulerDeg = { x: 0, y: 0, z: 0 }; +const AXES: Axis[] = ["x", "y", "z"]; +const BODY_TARGETS = new Set(BONES); + +interface AuthoredEuler { + euler: EulerDeg; + axes: Set; +} export function resolve(ast: AstDoc): ResolveResult { const warnings: Warning[] = []; const errors: ParseError[] = []; + const declaredAnchors = anchorsForProps(ast.props); + const semanticState = initialSemanticState(ast.startPose); + const phases: Phase[] = []; - const phases: Phase[] = ast.steps.map((step) => - resolveStep(step, warnings, errors), - ); + // Resolve in document order so cross-joint mechanics can be checked against + // the pose carried from preceding phases. This matters for a hinge authored + // after a deep hip flexion (and vice versa), not only when both share a step. + for (const step of ast.steps) { + const phase = resolveStep(step, declaredAnchors, warnings, errors); + enforceHipHingeLimit(step, phase, semanticState, warnings); + applyPhaseToState(semanticState, phase); + phases.push(phase); + } const ir: PosecodeIR = { version: POSECODE_VERSION, @@ -64,11 +83,41 @@ export function resolve(ast: AstDoc): ResolveResult { function resolveStep( step: AstStep, + declaredAnchors: ReadonlySet, warnings: Warning[], errors: ParseError[], ): Phase { + // `ground-lock`, `pin`, and `grip` each solve the floating root. Combining + // those solver families in one phase makes the later pass invalidate the + // earlier contact (for example, a knee pin sliding a ground-locked foot). + // Express multi-point support with one root anchor plus per-limb `reach`s. + if (step.groundLock.length > 0 && step.pins.length > 0) { + errors.push({ + line: step.pins[0]!.line, + message: "`pin` cannot be combined with `ground-lock` in one step; use one primary pin plus `reach` for additional contacts", + }); + } + if (step.groundLock.length > 0 && step.grips.length > 0) { + errors.push({ + line: step.grips[0]!.line, + message: "`grip` cannot be combined with `ground-lock` in one step; use one root-support family per step", + }); + } + if (step.pins.length > 0 && step.grips.length > 0) { + errors.push({ + line: step.grips[0]!.line, + message: "`grip` cannot be combined with `pin` in one step; use the grip as the primary support plus `reach` for additional contacts", + }); + } + if (step.pins.length > 1) { + errors.push({ + line: step.pins[1]!.line, + message: "a step may declare only one primary `pin`; use a grouped effector or per-limb `reach` constraints for additional contacts", + }); + } + // Accumulate per-bone Euler so multiple action lines on the same joint merge. - const byBone = new Map(); + const byBone = new Map(); for (const target of step.targets) { const bones = expandJoint(target.joint); @@ -79,7 +128,11 @@ function resolveStep( // `hold ` keeps the joint at neutral. if (target.action === "hold") { - for (const bone of bones) ensure(byBone, bone); + for (const bone of bones) { + const authored = ensure(byBone, bone); + authored.euler = { ...ZERO }; + for (const axis of AXES) authored.axes.add(axis); + } continue; } @@ -96,6 +149,18 @@ function resolveStep( continue; } + const unsupported = bones.filter((bone) => romFor(bone, target.action) === null); + if (unsupported.length > 0) { + const allowed = actionsForJoint(target.joint); + errors.push({ + line: target.line, + message: + `action "${target.action}" is not supported for ${target.joint}; ` + + `expected one of ${[...allowed, "hold"].join(", ")}`, + }); + continue; + } + for (const bone of bones) { const clamped = clampAngle(bone, target.action, target.degrees); if (clamped !== target.degrees) { @@ -116,14 +181,16 @@ function resolveStep( : 1; const mirror = isLeft(bone) && aa.axis !== "x" ? -1 : 1; const sign = aa.sign * flexFlip * mirror; - const euler = ensure(byBone, bone); - euler[aa.axis] = sign * clamped; + const authored = ensure(byBone, bone); + authored.euler[aa.axis] = sign * clamped; + authored.axes.add(aa.axis); } } - const targets: JointTarget[] = [...byBone.entries()].map(([boneId, euler]) => ({ + const targets: JointTarget[] = [...byBone.entries()].map(([boneId, authored]) => ({ boneId, - euler, + euler: authored.euler, + axes: AXES.filter((axis) => authored.axes.has(axis)), })); const groundLock: string[] = []; @@ -148,6 +215,23 @@ function resolveStep( errors.push({ line: r.line, message: `unknown reach effector: "${r.effector}"` }); continue; } + if (sides.includes("pelvis")) { + errors.push({ + line: r.line, + message: 'reach effector "pelvis" is unsupported; use `pin: pelvis `', + }); + continue; + } + if (!validateReachTarget(r.target, r.line, declaredAnchors, errors)) { + continue; + } + if (sides.some((effector) => contactBoneId(effector) === r.target)) { + errors.push({ + line: r.line, + message: `reach effector "${r.effector}" cannot target its own joint "${r.target}"`, + }); + continue; + } for (const effector of sides) reaches.push({ effector, target: r.target }); } const pins: PinTarget[] = []; @@ -157,6 +241,9 @@ function resolveStep( errors.push({ line: p.line, message: `unknown pin effector: "${p.effector}"` }); continue; } + if (!validatePinAnchor(p.anchor, p.line, declaredAnchors, errors)) { + continue; + } for (const effector of sides) pins.push({ effector, anchor: p.anchor }); } @@ -170,6 +257,31 @@ function resolveStep( errors.push({ line: g.line, message: `unknown grip effector: "${g.effector}"` }); continue; } + if (sides.some((effector) => !effector.startsWith("hand_"))) { + errors.push({ + line: g.line, + message: `grip effector "${g.effector}" must resolve to a hand`, + }); + continue; + } + if (!validateGripAnchor(g.anchor, g.line, declaredAnchors, errors)) { + continue; + } + const writtenSide = /_(left|right)$/.exec(g.anchor)?.[1]; + if (writtenSide && sides.length > 1) { + errors.push({ + line: g.line, + message: `grouped grip effector "${g.effector}" requires a bare anchor such as "${g.anchor.replace(/_(left|right)$/, "")}"`, + }); + continue; + } + if (writtenSide && sides[0] && !sides[0].endsWith(`_${writtenSide}`)) { + errors.push({ + line: g.line, + message: `grip anchor "${g.anchor}" does not match effector "${sides[0]}"`, + }); + continue; + } for (const effector of sides) { grips.push({ effector, anchor: sideAnchor(g.anchor, effector) }); } @@ -199,6 +311,213 @@ function resolveStep( }; } +/** Reach accepts the floor, body landmarks, and declared prop anchors. */ +function validateReachTarget( + value: string, + line: number, + declaredAnchors: ReadonlySet, + errors: ParseError[], +): boolean { + if (value === "floor" || BODY_TARGETS.has(value) || declaredAnchors.has(value)) { + return true; + } + const prop = propForAnchor(value); + errors.push({ + line, + message: prop + ? `reach target "${value}" requires declared prop "${prop}"` + : `unknown reach target: "${value}"`, + }); + return false; +} + +/** Pins are world/root anchors. A body landmark moves with the root and cannot anchor it. */ +function validatePinAnchor( + value: string, + line: number, + declaredAnchors: ReadonlySet, + errors: ParseError[], +): boolean { + if (value === "floor" || declaredAnchors.has(value)) return true; + if (BODY_TARGETS.has(value)) { + errors.push({ + line, + message: `pin anchor "${value}" is body-relative and moves with the root; use floor or a declared prop anchor`, + }); + return false; + } + const prop = propForAnchor(value); + errors.push({ + line, + message: prop + ? `pin anchor "${value}" requires declared prop "${prop}"` + : `unknown pin anchor: "${value}"`, + }); + return false; +} + +/** Grips only have contact frames on the built-in overhead bar and dip rails. */ +function validateGripAnchor( + value: string, + line: number, + declaredAnchors: ReadonlySet, + errors: ParseError[], +): boolean { + const prop = propForAnchor(value); + if (prop !== "bar" && prop !== "dip-bars") { + errors.push({ + line, + message: `grip anchor "${value}" must be an overhead-bar or dip-rail anchor`, + }); + return false; + } + if (!declaredAnchors.has(value)) { + errors.push({ line, message: `grip anchor "${value}" requires declared prop "${prop}"` }); + return false; + } + return true; +} + +function contactBoneId(effector: string): string { + return effector + .replace(/^hand_/, "wrist_") + .replace(/^fist_/, "wrist_") + .replace(/^foot_/, "ankle_"); +} + +/** + * Keep the renderer's coupled local hip angle inside the flexion ceiling. + * Runtime local hip X is `semantic hip X - pelvis hinge X`; independently-safe + * authored values can therefore add to an impossible angle (110° + 50°). + */ +function enforceHipHingeLimit( + step: AstStep, + phase: Phase, + previous: ReadonlyMap, + warnings: Warning[], +): void { + const candidate = cloneState(previous); + applyPhaseToState(candidate, phase); + + let pelvisX = candidate.get("pelvis")?.x ?? 0; + const hips = ["hip_left", "hip_right"] as const; + const pelvisSource = sourceForAxis(step, "pelvis", "x"); + let pelvisWarning: Warning | null = null; + const clampPelvisTo = (safeValue: number): void => { + if (!pelvisSource || pelvisX <= safeValue) return; + const requestedPelvisX = pelvisX; + const clamped = Math.max(0, safeValue); + setPhaseAxis(phase, "pelvis", "x", clamped); + ensureState(candidate, "pelvis").x = clamped; + pelvisX = clamped; + if (pelvisWarning) { + pelvisWarning.clamped = clamped; + pelvisWarning.limit = { min: 0, max: clamped }; + return; + } + pelvisWarning = { + line: pelvisSource.line, + phase: step.name, + joint: "pelvis", + action: pelvisSource.action, + requested: pelvisSource.degrees ?? requestedPelvisX, + clamped, + limit: { min: 0, max: clamped }, + }; + warnings.push(pelvisWarning); + }; + + // When a newly-authored hinge meets a hip carried from an earlier phase, + // the hinge must yield first. Doing this before clamping same-step hips + // avoids needless left/right asymmetry in mixed carried/authored poses. + const carriedHipLimits = hips + .filter((hip) => sourceForAxis(step, hip, "x") === null) + .map((hip) => (candidate.get(hip)?.x ?? 0) + COMBINED_HIP_FLEX_MAX); + if (carriedHipLimits.length > 0) { + clampPelvisTo(Math.min(120, ...carriedHipLimits)); + } + + for (const hip of hips) { + const hipX = candidate.get(hip)?.x ?? 0; + if (hipX - pelvisX >= -COMBINED_HIP_FLEX_MAX) continue; + + const source = sourceForAxis(step, hip, "x"); + if (!source) continue; + const safeHipX = pelvisX - COMBINED_HIP_FLEX_MAX; + setPhaseAxis(phase, hip, "x", safeHipX); + ensureState(candidate, hip).x = safeHipX; + const maxDegrees = Math.max(0, COMBINED_HIP_FLEX_MAX - pelvisX); + warnings.push({ + line: source.line, + phase: step.name, + joint: hip, + action: source.action, + requested: source.degrees ?? Math.abs(hipX), + clamped: maxDegrees, + limit: { min: 0, max: maxDegrees }, + }); + } + + // If the hips were carried from a previous phase, the newly-authored hinge + // is the channel that must yield. Use the most restrictive side. + const safePelvisX = Math.min( + 120, + ...hips.map((hip) => (candidate.get(hip)?.x ?? 0) + COMBINED_HIP_FLEX_MAX), + ); + clampPelvisTo(safePelvisX); +} + +const COMBINED_HIP_FLEX_MAX = 135; + +function sourceForAxis( + step: AstStep, + bone: string, + axis: Axis, +): AstJointTarget | null { + for (let i = step.targets.length - 1; i >= 0; i--) { + const target = step.targets[i]!; + if (!expandJoint(target.joint).includes(bone)) continue; + if (target.action === "hold" || actionAxis(target.action)?.axis === axis) return target; + } + return null; +} + +function setPhaseAxis(phase: Phase, bone: string, axis: Axis, value: number): void { + const target = phase.targets.find((entry) => entry.boneId === bone); + if (!target) return; + target.euler[axis] = value; + if (target.axes && !target.axes.includes(axis)) target.axes.push(axis); +} + +function initialSemanticState(startPose: string | undefined): Map { + const state = new Map(); + if (startPose === "seated") { + state.set("hip_left", { x: -90, y: 0, z: 0 }); + state.set("hip_right", { x: -90, y: 0, z: 0 }); + } + return state; +} + +function applyPhaseToState(state: Map, phase: Phase): void { + for (const target of phase.targets) { + const current = ensureState(state, target.boneId); + for (const axis of target.axes ?? AXES) current[axis] = target.euler[axis]; + } +} + +function cloneState(state: ReadonlyMap): Map { + return new Map([...state].map(([bone, euler]) => [bone, { ...euler }])); +} + +function ensureState(map: Map, bone: string): EulerDeg { + let euler = map.get(bone); + if (!euler) { + euler = { ...ZERO }; + map.set(bone, euler); + } + return euler; +} + /** Max travel offset from the load spot, metres, in any single axis. */ const TRAVEL_MAX = 3; @@ -219,11 +538,11 @@ function sideAnchor(anchor: string, effector: string): string { return anchor; } -function ensure(map: Map, bone: string): EulerDeg { - let euler = map.get(bone); - if (!euler) { - euler = { ...ZERO }; - map.set(bone, euler); +function ensure(map: Map, bone: string): AuthoredEuler { + let authored = map.get(bone); + if (!authored) { + authored = { euler: { ...ZERO }, axes: new Set() }; + map.set(bone, authored); } - return euler; + return authored; } diff --git a/packages/posecode-parser/src/index.ts b/packages/posecode-parser/src/index.ts index 7c16daf..85bf857 100644 --- a/packages/posecode-parser/src/index.ts +++ b/packages/posecode-parser/src/index.ts @@ -53,6 +53,9 @@ export { JOINT_NAMES, ACTION_NAMES, EFFECTOR_NAMES, + REACH_EFFECTOR_NAMES, + PIN_EFFECTOR_NAMES, + GRIP_EFFECTOR_NAMES, GROUND_LOCK_EFFECTOR_NAMES, expandJoint, expandEffector, @@ -60,5 +63,31 @@ export { actionAxis, boneType, } from "./joints.js"; -export { romFor, clampAngle, eulerRomFor, type RomLimit, type EulerRom } from "./rom.js"; +export { + romFor, + clampAngle, + eulerRomFor, + isActionAllowed, + actionsForJoint, + isLegacyAxialAction, + type RomLimit, + type EulerRom, +} from "./rom.js"; export { EASINGS, MODES, LEGACY_MODE_ALIASES, normalizeMode } from "./schema.js"; +export { + MOVEMENT_KINDS, + RIG_NAMES, + START_POSE_NAMES, + PROP_TYPES, + PROP_ANCHORS, + isMovementKind, + isRigName, + isStartPoseName, + isPropType, + propForAnchor, + anchorsForProps, + type MovementKind, + type RigName, + type StartPoseName, + type PropType, +} from "./protocol.js"; diff --git a/packages/posecode-parser/src/joints.ts b/packages/posecode-parser/src/joints.ts index cb0fe66..74b1101 100644 --- a/packages/posecode-parser/src/joints.ts +++ b/packages/posecode-parser/src/joints.ts @@ -101,8 +101,14 @@ export function expandJoint(name: string): string[] { const EFFECTOR_SIDES = [ "hand_left", "hand_right", + // Closed-fist floor contacts use the wrist chain for positioning, but retain + // their semantic contact surface for the renderer's orientation solver. + "fist_left", + "fist_right", "elbow_left", "elbow_right", + "knee_left", + "knee_right", "foot_left", "foot_right", // Axial support point for floor-based poses such as cobra. Unlike a reach, @@ -113,13 +119,20 @@ const EFFECTOR_SIDES = [ /** Symmetric effector groups → the per-side effectors they expand to. */ const EFFECTOR_GROUPS: Record = { hands: ["hand_left", "hand_right"], + fists: ["fist_left", "fist_right"], forearms: ["elbow_left", "elbow_right"], + knees: ["knee_left", "knee_right"], feet: ["foot_left", "foot_right"], }; /** Every effector name `reach:` / `pin:` accept: groups + per-side aliases. */ export const EFFECTOR_NAMES = [...Object.keys(EFFECTOR_GROUPS), ...EFFECTOR_SIDES]; +/** Capability-specific contact vocabularies exposed to validators/editors. */ +export const REACH_EFFECTOR_NAMES = EFFECTOR_NAMES.filter((name) => name !== "pelvis"); +export const PIN_EFFECTOR_NAMES = [...EFFECTOR_NAMES]; +export const GRIP_EFFECTOR_NAMES = ["hands", "hand_left", "hand_right"] as const; + /** * Contacts accepted by `ground-lock:`. Ground locking has historically * supported the symmetric hand/forearm/foot groups; per-side aliases let a @@ -177,6 +190,12 @@ const ACTIONS: Record = { adduct: { axis: "z", sign: 1 }, "rotate-in": { axis: "y", sign: 1 }, "rotate-out": { axis: "y", sign: -1 }, + // Axial rotation must use world-independent anatomical directions. Internal + // and external rotation make sense for paired ball-and-socket joints, but + // are ambiguous on the spine/cervical chain. Facing +Z, +Y turns the torso + // toward the person's left (+X), and -Y turns it right (-X). + "twist-left": { axis: "y", sign: 1 }, + "twist-right": { axis: "y", sign: -1 }, supinate: { axis: "y", sign: 1 }, pronate: { axis: "y", sign: -1 }, // The foot points FORWARD (+Z): lifting the toes toward the shin diff --git a/packages/posecode-parser/src/parser.ts b/packages/posecode-parser/src/parser.ts index 9a65872..8778c3d 100644 --- a/packages/posecode-parser/src/parser.ts +++ b/packages/posecode-parser/src/parser.ts @@ -10,8 +10,19 @@ import { tokenize, TokenizeError, type Line, type Token } from "./tokenizer.js"; import type { ParseError } from "./types.js"; import { normalizeMode, MODES } from "./schema.js"; import { GROUND_LOCK_EFFECTOR_NAMES } from "./joints.js"; +import { + MOVEMENT_KINDS, + PROP_TYPES, + RIG_NAMES, + START_POSE_NAMES, + isMovementKind, + isPropType, + isRigName, + isStartPoseName, +} from "./protocol.js"; const GROUND_LOCK_EFFECTORS = new Set(GROUND_LOCK_EFFECTOR_NAMES); +const TOP_LEVEL_HEADS = new Set(["rig", "prop", "clip", "pose", "repeat", "step"]); export interface AstJointTarget { joint: string; @@ -91,7 +102,13 @@ export function parseToAst(source: string): ParseAstResult { // Header: `posecode ""` const header = lines[0]!; const ht = header.tokens; - if (word(ht[0]) !== "posecode" || ht[1]?.type !== "word" || ht[2]?.type !== "str") { + if ( + header.indent !== 0 || + ht.length !== 3 || + word(ht[0]) !== "posecode" || + ht[1]?.type !== "word" || + ht[2]?.type !== "str" + ) { return { ast: null, errors: [ @@ -103,6 +120,16 @@ export function parseToAst(source: string): ParseAstResult { }; } + if (!isMovementKind(ht[1].value)) { + return { + ast: null, + errors: [{ + line: header.line, + message: `unknown movement kind "${ht[1].value}"; expected one of ${MOVEMENT_KINDS.join(", ")}`, + }], + }; + } + const doc: AstDoc = { kind: ht[1].value, name: ht[2].value, @@ -114,6 +141,9 @@ export function parseToAst(source: string): ParseAstResult { let current: AstStep | null = null; let invalidStepIndent: number | null = null; + let topLevelIndent: number | null = null; + let currentStepIndent: number | null = null; + let currentChildIndent: number | null = null; for (let i = 1; i < lines.length; i++) { const ln = lines[i]!; @@ -127,40 +157,107 @@ export function parseToAst(source: string): ParseAstResult { const head = word(ln.tokens[0]); const t = ln.tokens; + // Indentation is syntax, not presentation. All document declarations share + // one indentation level beneath the header, and all children of a step + // share one deeper level. Without this gate an unindented joint silently + // attached to the preceding step, while an indented `repeat` or `rig` + // mutated the whole document from inside that step. + if (head && TOP_LEVEL_HEADS.has(head)) { + if (ln.indent <= header.indent) { + errors.push({ line: ln.line, message: `top-level \`${head}\` must be indented beneath the posecode header` }); + current = null; + currentStepIndent = null; + currentChildIndent = null; + if (head === "step") invalidStepIndent = ln.indent; + continue; + } + if (topLevelIndent === null) topLevelIndent = ln.indent; + if (ln.indent !== topLevelIndent) { + errors.push({ line: ln.line, message: `top-level \`${head}\` must use the document indentation level (${topLevelIndent} spaces)` }); + current = null; + currentStepIndent = null; + currentChildIndent = null; + if (head === "step") invalidStepIndent = ln.indent; + continue; + } + if (head !== "step") { + current = null; + currentStepIndent = null; + currentChildIndent = null; + } + } else { + if (!current || currentStepIndent === null) { + const err = parseStepChild(ln, null); + if (err) errors.push(err); + continue; + } + if (ln.indent <= currentStepIndent) { + errors.push({ line: ln.line, message: "step children must be indented beneath their `step` header" }); + continue; + } + if (currentChildIndent === null) currentChildIndent = ln.indent; + if (ln.indent !== currentChildIndent) { + errors.push({ line: ln.line, message: `step children must use one indentation level (${currentChildIndent} spaces)` }); + continue; + } + } + switch (head) { case "rig": { const r = word(t[1]); - if (!r) errors.push({ line: ln.line, message: "rig requires a name" }); + if (t.length !== 2 || !r) { + errors.push({ line: ln.line, message: "expected `rig humanoid`" }); + } else if (!isRigName(r)) { + errors.push({ + line: ln.line, + message: `unknown rig "${r}"; expected one of ${RIG_NAMES.join(", ")}`, + }); + } else doc.rig = r; break; } case "prop": { - // `prop `: a scene object (chair | wall | bar), repeatable. + // `prop `: a built-in scene object, repeatable. const p = word(t[1]); - if (!p) errors.push({ line: ln.line, message: "prop requires a type" }); + if (t.length !== 2 || !p) { + errors.push({ line: ln.line, message: "expected `prop `" }); + } else if (!isPropType(p)) { + errors.push({ + line: ln.line, + message: `unknown prop "${p}"; expected one of ${PROP_TYPES.join(", ")}`, + }); + } else doc.props.push(p); break; } case "clip": { // `clip ""`: an optional mocap clip the renderer may play // (retargeted) instead of / blended with the procedural phases. - if (t[1]?.type === "str") doc.clip = t[1].value; + if (t.length === 2 && t[1]?.type === "str") doc.clip = t[1].value; else errors.push({ line: ln.line, message: 'expected `clip ""`' }); break; } case "pose": { // `pose start = ` - const name = t.length > 0 ? t[t.length - 1] : undefined; - if (word(t[1]) === "start" && name && name.type === "word") { - doc.startPose = name.value; + const name = word(t[3]); + if (t.length === 4 && word(t[1]) === "start" && t[2]?.type === "eq" && name) { + if (!isStartPoseName(name)) { + errors.push({ + line: ln.line, + message: `unknown start pose "${name}"; expected one of ${START_POSE_NAMES.join(", ")}`, + }); + break; + } + doc.startPose = name; } else { errors.push({ line: ln.line, message: "expected `pose start = `" }); } break; } case "repeat": { - if (t[1]?.type === "num") doc.repeat = Math.max(1, Math.round(Number(t[1].value))); - else errors.push({ line: ln.line, message: "repeat requires a count" }); + const count = t[1]?.type === "num" ? Number(t[1].value) : NaN; + if (t.length === 2 && Number.isInteger(count) && count >= 1) doc.repeat = count; + else errors.push({ line: ln.line, message: "repeat requires a positive integer count" }); break; } case "step": { @@ -176,7 +273,8 @@ export function parseToAst(source: string): ParseAstResult { dur?.type !== "dur" || !easingTok || resolved.mode === null || - colon?.type !== "colon" + colon?.type !== "colon" || + t.length !== 5 ) { errors.push({ line: ln.line, @@ -186,6 +284,8 @@ export function parseToAst(source: string): ParseAstResult { : 'expected `step "" :`', }); current = null; + currentStepIndent = null; + currentChildIndent = null; invalidStepIndent = ln.indent; break; } @@ -201,6 +301,8 @@ export function parseToAst(source: string): ParseAstResult { line: ln.line, }; doc.steps.push(current); + currentStepIndent = ln.indent; + currentChildIndent = null; break; } default: { @@ -224,7 +326,9 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { if (head === "cue") { if (!current) return { line: ln.line, message: "`cue` outside of a step" }; - if (t[1]?.type !== "str") return { line: ln.line, message: "`cue` requires a quoted string" }; + if (t.length !== 2 || t[1]?.type !== "str") { + return { line: ln.line, message: "`cue` requires exactly one quoted string" }; + } current.cue = t[1].value; return null; } @@ -234,13 +338,16 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { if (t[1]?.type !== "colon") { return { line: ln.line, message: "expected `ground-lock: `" }; } + if (t.slice(2).some((tok) => tok.type !== "word" && tok.type !== "comma")) { + return { line: ln.line, message: "expected `ground-lock: `" }; + } const words = t.slice(2).filter((tok) => tok.type === "word").map((tok) => tok.value); const effectors: string[] = []; for (let i = 0; i < words.length; i++) { const value = words[i]!; if (value === "and") continue; const next = words[i + 1]; - if ((value === "left" || value === "right") && next) { + if (value === "left" || value === "right") { const base = next === "hand" ? "hand" : next === "foot" ? "foot" : next === "forearm" || next === "elbow" ? "elbow" : null; if (base) { @@ -248,6 +355,7 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { i++; continue; } + return { line: ln.line, message: `unknown ground-lock effector: "${value}${next ? ` ${next}` : ""}"` }; } if (!GROUND_LOCK_EFFECTORS.has(value)) { return { line: ln.line, message: `unknown ground-lock effector: "${value}"` }; @@ -267,7 +375,7 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { if (!current) return { line: ln.line, message: "`reach` outside of a step" }; const effector = t[2]?.type === "word" ? t[2].value : null; const target = t[3]?.type === "word" ? t[3].value : null; - if (t[1]?.type !== "colon" || !effector || !target) { + if (t.length !== 4 || t[1]?.type !== "colon" || !effector || !target) { return { line: ln.line, message: "expected `reach: `" }; } current.reaches.push({ effector, target, line: ln.line }); @@ -279,7 +387,7 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { if (!current) return { line: ln.line, message: "`pin` outside of a step" }; const effector = t[2]?.type === "word" ? t[2].value : null; const anchor = t[3]?.type === "word" ? t[3].value : null; - if (t[1]?.type !== "colon" || !effector || !anchor) { + if (t.length !== 4 || t[1]?.type !== "colon" || !effector || !anchor) { return { line: ln.line, message: "expected `pin: `" }; } current.pins.push({ effector, anchor, line: ln.line }); @@ -293,7 +401,7 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { if (!current) return { line: ln.line, message: "`grip` outside of a step" }; const effector = t[2]?.type === "word" ? t[2].value : null; const anchor = t[3]?.type === "word" ? t[3].value : null; - if (t[1]?.type !== "colon" || !effector || !anchor) { + if (t.length !== 4 || t[1]?.type !== "colon" || !effector || !anchor) { return { line: ln.line, message: "expected `grip: `" }; } current.grips.push({ effector, anchor, line: ln.line }); @@ -304,7 +412,7 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { // `turn: `: the figure's facing (root yaw about world Y) at the // end of this phase. Absolute, accumulated forward like a joint target. if (!current) return { line: ln.line, message: "`turn` outside of a step" }; - if (t[1]?.type !== "colon" || t[2]?.type !== "num") { + if (t.length !== 3 || t[1]?.type !== "colon" || t[2]?.type !== "num") { return { line: ln.line, message: "expected `turn: `" }; } current.turn = Number(t[2].value); @@ -315,25 +423,44 @@ function parseStepChild(ln: Line, current: AstStep | null): ParseError | null { // `travel: `: the figure's ground position (world X/Z metres) at the // end of this phase. Absolute offset from the load spot, accumulated forward. if (!current) return { line: ln.line, message: "`travel` outside of a step" }; - if (t[1]?.type !== "colon" || t[2]?.type !== "num" || t[3]?.type !== "num") { + if ( + t.length !== 4 || + t[1]?.type !== "colon" || + t[2]?.type !== "num" || + t[3]?.type !== "num" + ) { return { line: ln.line, message: "expected `travel: `" }; } current.travel = { x: Number(t[2].value), z: Number(t[3].value) }; return null; } - // Joint target: `: []` + // Joint target: `: ` or `: hold neutral`. if (!current) { return { line: ln.line, message: "joint target outside of a step" }; } if (head === null || t[1]?.type !== "colon" || t[2]?.type !== "word") { - return { line: ln.line, message: "expected `: []`" }; + return { line: ln.line, message: "expected `: `" }; + } + const action = t[2].value; + if (action === "hold") { + if (t.length !== 4 || word(t[3]) !== "neutral") { + return { + line: ln.line, + message: "expected `: hold neutral` (no trailing angle)", + }; + } + current.targets.push({ joint: head, action, degrees: null, line: ln.line }); + return null; } const degTok = t[3]; + if (t.length !== 4 || degTok?.type !== "num") { + return { line: ln.line, message: "expected `: `" }; + } current.targets.push({ joint: head, - action: t[2].value, - degrees: degTok && degTok.type === "num" ? Number(degTok.value) : null, + action, + degrees: Number(degTok.value), line: ln.line, }); return null; diff --git a/packages/posecode-parser/src/protocol.ts b/packages/posecode-parser/src/protocol.ts new file mode 100644 index 0000000..e2a3fdf --- /dev/null +++ b/packages/posecode-parser/src/protocol.ts @@ -0,0 +1,63 @@ +/** Closed vocabulary for document-level Posecode protocol declarations. */ + +export const MOVEMENT_KINDS = ["exercise", "stretch", "posture"] as const; +export type MovementKind = (typeof MOVEMENT_KINDS)[number]; + +export const RIG_NAMES = ["humanoid"] as const; +export type RigName = (typeof RIG_NAMES)[number]; + +export const START_POSE_NAMES = [ + "neutral", + "standing", + "plank", + "supine", + "prone", + "seated", +] as const; +export type StartPoseName = (typeof START_POSE_NAMES)[number]; + +export const PROP_TYPES = ["chair", "wall", "bar", "box", "dip-bars"] as const; +export type PropType = (typeof PROP_TYPES)[number]; + +/** World-anchor names supplied by each built-in prop. Sided grip aliases are included. */ +export const PROP_ANCHORS: Readonly> = { + chair: ["seat"], + wall: ["wall"], + bar: ["bar", "bar_left", "bar_right"], + box: ["box"], + "dip-bars": ["bars", "bars_left", "bars_right"], +}; + +export function isMovementKind(value: string): value is MovementKind { + return (MOVEMENT_KINDS as readonly string[]).includes(value); +} + +export function isRigName(value: string): value is RigName { + return (RIG_NAMES as readonly string[]).includes(value); +} + +export function isStartPoseName(value: string): value is StartPoseName { + return (START_POSE_NAMES as readonly string[]).includes(value); +} + +export function isPropType(value: string): value is PropType { + return (PROP_TYPES as readonly string[]).includes(value); +} + +/** The prop type that owns an anchor, or null when the word is not a prop anchor. */ +export function propForAnchor(anchor: string): PropType | null { + for (const type of PROP_TYPES) { + if (PROP_ANCHORS[type].includes(anchor)) return type; + } + return null; +} + +/** All anchor names made available by the props declared in one document. */ +export function anchorsForProps(props: readonly string[]): Set { + const anchors = new Set(); + for (const value of props) { + if (!isPropType(value)) continue; + for (const anchor of PROP_ANCHORS[value]) anchors.add(anchor); + } + return anchors; +} diff --git a/packages/posecode-parser/src/rom.ts b/packages/posecode-parser/src/rom.ts index 73bdfcd..3b20864 100644 --- a/packages/posecode-parser/src/rom.ts +++ b/packages/posecode-parser/src/rom.ts @@ -1,10 +1,11 @@ /** - * Range-of-Motion (ROM) hard limits per joint type and action, in degrees. + * Configured Range-of-Motion (ROM) limits per joint type and action, in degrees. * - * Values follow the maximum healthy limits in the project research - * (CDC / clinical normative data), §5.1 Tables 1 & 2. The clamp pass treats + * Values are implementation bounds informed by the project's cited references; + * they are not a clinical assessment or a guarantee that a complete motion is + * safe for a particular person. The clamp pass treats * `max` as a HARD ceiling: e.g. a requested knee flexion of 200° is clamped to - * 144°, so the renderer can never produce an anatomically impossible joint. + * 144°, preventing that authored channel from exceeding its configured bound. * * `min` is the floor of the achievable angle for that action direction (0 for * most; small positive ceilings on extension capture hyperextension limits). @@ -37,6 +38,9 @@ const ROM: Record = { wrist: { flex: { min: 0, max: 80 }, extend: { min: 0, max: 70 }, + // Radial/ulnar deviation expressed in the DSL's midline-relative terms. + abduct: { min: 0, max: 20 }, + adduct: { min: 0, max: 30 }, }, // --- Lower extremity (research Table 2) --- hip: { @@ -70,12 +74,16 @@ const ROM: Record = { adduct: { min: 0, max: 35 }, "rotate-in": { min: 0, max: 45 }, "rotate-out": { min: 0, max: 45 }, + "twist-left": { min: 0, max: 45 }, + "twist-right": { min: 0, max: 45 }, }, chest: { flex: { min: 0, max: 30 }, extend: { min: 0, max: 20 }, "rotate-in": { min: 0, max: 35 }, "rotate-out": { min: 0, max: 35 }, + "twist-left": { min: 0, max: 35 }, + "twist-right": { min: 0, max: 35 }, }, neck: { flex: { min: 0, max: 50 }, @@ -84,6 +92,21 @@ const ROM: Record = { adduct: { min: 0, max: 45 }, "rotate-in": { min: 0, max: 80 }, "rotate-out": { min: 0, max: 80 }, + "twist-left": { min: 0, max: 80 }, + "twist-right": { min: 0, max: 80 }, + }, + // The head control is the upper-cervical share of motion, not a second full + // neck. Conservative limits prevent neck+head targets from silently doubling + // the complete cervical range. + head: { + flex: { min: 0, max: 25 }, + extend: { min: 0, max: 25 }, + abduct: { min: 0, max: 20 }, + adduct: { min: 0, max: 20 }, + "rotate-in": { min: 0, max: 40 }, + "rotate-out": { min: 0, max: 40 }, + "twist-left": { min: 0, max: 40 }, + "twist-right": { min: 0, max: 40 }, }, // --- Hand / fingers (single-DOF curl per finger) --- index: { flex: { min: 0, max: 100 }, extend: { min: 0, max: 20 } }, @@ -98,7 +121,14 @@ const ROM: Record = { }, }; -import { actionAxis, boneType, flexionSign, isLeft } from "./joints.js"; +import { + ACTION_NAMES, + actionAxis, + boneType, + expandJoint, + flexionSign, + isLeft, +} from "./joints.js"; import type { Axis } from "./types.js"; /** Look up the ROM limit for a bone + action, or null if undefined. */ @@ -108,6 +138,35 @@ export function romFor(boneId: string, action: string): RomLimit | null { return limits[action] ?? null; } +/** True when an anatomical action is defined for this concrete bone. */ +export function isActionAllowed(boneId: string, action: string): boolean { + return romFor(boneId, action) !== null; +} + +/** + * Actions valid for every bone represented by a DSL joint/group name. The + * intersection makes completions for heterogeneous groups such as `fingers` + * safe: thumb-only abduction is not offered for all fingers at once. + */ +export function actionsForJoint(joint: string): string[] { + const bones = expandJoint(joint); + if (bones.length === 0) return []; + const axial = bones.every((bone) => AXIAL_TYPES.has(boneType(bone))); + return ACTION_NAMES.filter( + (action) => + (!axial || (action !== "rotate-in" && action !== "rotate-out")) && + bones.every((bone) => isActionAllowed(bone, action)), + ); +} + +const AXIAL_TYPES = new Set(["spine", "chest", "neck", "head"]); + +/** True for the accepted-but-deprecated ambiguous axial rotation spelling. */ +export function isLegacyAxialAction(boneId: string, action: string): boolean { + return AXIAL_TYPES.has(boneType(boneId)) && + (action === "rotate-in" || action === "rotate-out"); +} + /** Signed per-axis rotation range, degrees, in a bone's LOCAL Euler frame. */ export type EulerRom = Record; @@ -117,8 +176,8 @@ export type EulerRom = Record; * (flexion-sign per joint, Y/Z mirrored on left-side bones). Each axis range is * the union of every action that rotates it; axes with no ROM entry stay * `{min: 0, max: 0}`, locking them (a knee is a pure hinge). This is what lets - * the IK solver honour the same hard limits as authored angles: any solved - * joint rotation clamped into this box is inside the healthy ROM. + * the IK solver honour the same configured limits as authored angles: any + * solved joint rotation clamped into this box is inside that ROM box. * * Returns null for bones without ROM data (e.g. `head`). */ diff --git a/packages/posecode-parser/src/schema.ts b/packages/posecode-parser/src/schema.ts index bb33ba5..f9cef04 100644 --- a/packages/posecode-parser/src/schema.ts +++ b/packages/posecode-parser/src/schema.ts @@ -10,6 +10,12 @@ import { z } from "zod"; import type { ParseError, TimingMode } from "./types.js"; import type { AstDoc } from "./parser.js"; +import { + MOVEMENT_KINDS, + PROP_TYPES, + RIG_NAMES, + START_POSE_NAMES, +} from "./protocol.js"; export const MODES = ["flow", "settle", "drive", "snap", "linear"] as const; @@ -71,13 +77,13 @@ const stepSchema = z.object({ }); const docSchema = z.object({ - kind: z.string().min(1), + kind: z.enum(MOVEMENT_KINDS), name: z.string().min(1), - rig: z.string().min(1), - startPose: z.string().optional(), - props: z.array(z.string()), + rig: z.enum(RIG_NAMES), + startPose: z.enum(START_POSE_NAMES).optional(), + props: z.array(z.enum(PROP_TYPES)), repeat: z.number().int().positive(), - steps: z.array(stepSchema), + steps: z.array(stepSchema).min(1, "a Posecode document requires at least one step"), }); /** diff --git a/packages/posecode-parser/src/types.ts b/packages/posecode-parser/src/types.ts index cb229e0..df10e34 100644 --- a/packages/posecode-parser/src/types.ts +++ b/packages/posecode-parser/src/types.ts @@ -27,6 +27,12 @@ export interface EulerDeg { export interface JointTarget { boneId: string; euler: EulerDeg; + /** + * Euler channels explicitly authored by this target. Renderers merge only + * these channels into the carried pose; omitted channels retain their prior + * value. Absent on legacy/manually-built IR, where all channels are applied. + */ + axes?: Axis[]; } /** diff --git a/packages/posecode-parser/test/rom.test.ts b/packages/posecode-parser/test/rom.test.ts index 6970143..55286df 100644 --- a/packages/posecode-parser/test/rom.test.ts +++ b/packages/posecode-parser/test/rom.test.ts @@ -77,7 +77,10 @@ describe("euler ROM boxes (eulerRomFor)", () => { expect(eulerRomFor("pelvis")!.x).toEqual({ min: 0, max: 120 }); }); - it("returns null for a bone without ROM data", () => { - expect(eulerRomFor("head")).toBeNull(); + it("bounds the upper-cervical head share independently from the neck", () => { + const box = eulerRomFor("head")!; + expect(box.x).toEqual({ min: -25, max: 25 }); + expect(box.y).toEqual({ min: -40, max: 40 }); + expect(box.z).toEqual({ min: -20, max: 20 }); }); }); diff --git a/packages/posecode-parser/test/strict-validation.test.ts b/packages/posecode-parser/test/strict-validation.test.ts new file mode 100644 index 0000000..b3b8060 --- /dev/null +++ b/packages/posecode-parser/test/strict-validation.test.ts @@ -0,0 +1,268 @@ +import { describe, expect, it } from "vitest"; +import { parse } from "../src/index.js"; + +function doc(...body: string[]): string { + return [ + 'posecode posture "Test"', + " rig humanoid", + ' step "Pose" 1s linear:', + ...body.map((line) => ` ${line}`), + ].join("\n"); +} + +describe("closed protocol vocabulary and arity", () => { + it.each([ + ['posecode dance "X"\n rig humanoid', /movement kind/i], + ['posecode posture "X"\n rig robot', /unknown rig/i], + ['posecode posture "X"\n pose start = crouching', /unknown start pose/i], + ['posecode posture "X"\n prop sword', /unknown prop/i], + ])("rejects declarations outside the protocol: %s", (source, message) => { + const result = parse(source); + expect(result.ir).toBeNull(); + expect(result.errors.some((error) => message.test(error.message))).toBe(true); + }); + + it("accepts exactly `hold neutral` and marks all channels explicit", () => { + const result = parse(doc("wrist_left: hold neutral")); + expect(result.errors).toEqual([]); + expect(result.ir?.phases[0]?.targets[0]?.axes).toEqual(["x", "y", "z"]); + }); + + it("rejects the legacy trailing zero instead of silently ignoring it", () => { + const result = parse(doc("wrist_left: hold neutral 0")); + expect(result.ir).toBeNull(); + expect(result.errors[0]?.message).toContain("no trailing angle"); + }); + + it("rejects surplus tokens on top-level and phase directives", () => { + expect(parse('posecode posture "X" extra').ir).toBeNull(); + expect(parse('posecode posture "X"\n rig humanoid extra').ir).toBeNull(); + expect(parse(doc("cue \"one\" extra")).ir).toBeNull(); + }); + + it("requires at least one movement step", () => { + const result = parse('posecode posture "Empty"\n rig humanoid'); + expect(result.ir).toBeNull(); + expect(result.errors[0]?.message).toMatch(/at least one step/i); + }); +}); + +describe("indentation grammar", () => { + it("rejects an unindented joint instead of attaching it to the preceding step", () => { + const result = parse([ + 'posecode posture "Indent"', + " rig humanoid", + ' step "Pose" 1s linear:', + "hip_left: flex 30", + ].join("\n")); + expect(result.ir).toBeNull(); + expect(result.errors[0]?.message).toMatch(/step children must be indented/i); + }); + + it("rejects a top-level directive nested inside a step", () => { + const result = parse([ + 'posecode posture "Indent"', + " rig humanoid", + ' step "Pose" 1s linear:', + " hips: flex 30", + " repeat 8", + ].join("\n")); + expect(result.ir).toBeNull(); + expect(result.errors[0]?.message).toMatch(/document indentation level/i); + }); + + it("rejects inconsistent child indentation", () => { + const result = parse([ + 'posecode posture "Indent"', + " rig humanoid", + ' step "Pose" 1s linear:', + " hips: flex 30", + " knees: flex 40", + ].join("\n")); + expect(result.ir).toBeNull(); + expect(result.errors[0]?.message).toMatch(/one indentation level/i); + }); +}); + +describe("joint/action compatibility", () => { + it("errors instead of applying an unsupported action without a ROM", () => { + const result = parse(doc("knee_left: abduct 20")); + expect(result.ir).toBeNull(); + expect(result.errors).toEqual([ + expect.objectContaining({ line: 4, message: expect.stringMatching(/not supported.*knee_left/i) }), + ]); + }); + + it("enforces conservative upper-cervical head ROM", () => { + const result = parse(doc("head: flex 50")); + expect(result.errors).toEqual([]); + expect(result.warnings[0]).toEqual(expect.objectContaining({ clamped: 25 })); + expect(result.ir?.phases[0]?.targets[0]?.euler.x).toBe(25); + }); + + it("supports bounded wrist radial and ulnar deviation", () => { + const result = parse(doc("wrist_left: abduct 30", "wrist_right: adduct 25")); + expect(result.errors).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toEqual(expect.objectContaining({ clamped: 20 })); + }); + + it("supports explicit left/right axial twist while accepting the legacy alias", () => { + const result = parse(doc("chest: twist-left 20", "neck: rotate-out 15")); + expect(result.errors).toEqual([]); + expect(result.ir?.phases[0]?.targets.find((t) => t.boneId === "chest")?.euler.y).toBe(20); + expect(result.ir?.phases[0]?.targets.find((t) => t.boneId === "neck")?.euler.y).toBe(-15); + }); +}); + +describe("contact namespace", () => { + it("accepts floor, body landmarks, declared props, fists, and knees", () => { + const source = [ + 'posecode posture "Landing"', + " rig humanoid", + " prop bar", + ' step "Land" 1s settle:', + " reach: fists floor", + " pin: knees floor", + " reach: hand_left knee_left", + " reach: hand_right bar", + ].join("\n"); + const result = parse(source); + expect(result.errors).toEqual([]); + expect(result.ir?.phases[0]?.reaches).toEqual(expect.arrayContaining([ + { effector: "fist_left", target: "floor" }, + { effector: "fist_right", target: "floor" }, + ])); + expect(result.ir?.phases[0]?.pins).toEqual([ + { effector: "knee_left", anchor: "floor" }, + { effector: "knee_right", anchor: "floor" }, + ]); + }); + + it("requires prop anchors to be declared", () => { + const result = parse(doc("reach: hand_left bar")); + expect(result.ir).toBeNull(); + expect(result.errors[0]?.message).toMatch(/requires declared prop "bar"/i); + }); + + it("keeps pelvis as a pin-only effector", () => { + const result = parse(doc("reach: pelvis floor")); + expect(result.ir).toBeNull(); + expect(result.errors[0]).toEqual(expect.objectContaining({ + line: 4, + message: expect.stringMatching(/pelvis.*unsupported.*pin/i), + })); + expect(parse(doc("pin: pelvis floor")).errors).toEqual([]); + }); + + it("rejects unknown anchors and non-hand grips", () => { + expect(parse(doc("pin: foot_left moon")).errors[0]?.message).toMatch(/unknown pin anchor/i); + expect(parse(doc("grip: fists floor")).errors[0]?.message).toMatch(/must resolve to a hand/i); + }); + + it("keeps reach, pin, and grip target namespaces semantically distinct", () => { + expect(parse(doc("pin: hand_left knee_left")).errors[0]?.message).toMatch(/body-relative/i); + expect(parse(doc("grip: hands floor")).errors[0]?.message).toMatch(/bar.*dip-rail/i); + expect(parse(doc("grip: hands knee_left")).errors[0]?.message).toMatch(/bar.*dip-rail/i); + }); + + it("rejects collapsed or crossed sided grip anchors", () => { + const grouped = [ + 'posecode exercise "Grip"', + " rig humanoid", + " prop bar", + ' step "Hold" 1s settle:', + " grip: hands bar_left", + ].join("\n"); + expect(parse(grouped).errors[0]?.message).toMatch(/grouped grip.*bare anchor/i); + const crossed = grouped.replace("hands bar_left", "hand_right bar_left"); + expect(parse(crossed).errors[0]?.message).toMatch(/does not match/i); + }); + + it("rejects competing whole-root contact solvers in one step", () => { + const result = parse(doc("pin: knee_left floor", "ground-lock: foot_right")); + expect(result.ir).toBeNull(); + expect(result.errors[0]?.message).toMatch(/pin.*cannot be combined.*ground-lock/i); + }); + + it("rejects multiple raw pins and self-referential reaches", () => { + expect(parse(doc("pin: pelvis floor", "pin: feet floor")).errors[0]?.message) + .toMatch(/only one primary.*pin/i); + expect(parse(doc("reach: hand_left wrist_left")).errors[0]?.message) + .toMatch(/cannot target its own joint/i); + expect(parse(doc("reach: knee_left knee_left")).errors[0]?.message) + .toMatch(/cannot target its own joint/i); + }); +}); + +describe("sparse channels and coupled hip mechanics", () => { + it("records only the axis authored by each phase", () => { + const source = [ + 'posecode exercise "Curl"', + " rig humanoid", + ' step "Curl" 1s flow:', + " elbow_left: flex 70", + ' step "Turn palm" 1s settle:', + " elbow_left: pronate 40", + ].join("\n"); + const result = parse(source); + expect(result.errors).toEqual([]); + expect(result.ir?.phases[0]?.targets[0]?.axes).toEqual(["x"]); + expect(result.ir?.phases[1]?.targets[0]?.axes).toEqual(["y"]); + }); + + it("clamps same-phase hinge plus hip flexion to 135 degrees local", () => { + const result = parse(doc("pelvis: hinge 50", "hips: flex 110")); + expect(result.errors).toEqual([]); + const hips = result.ir?.phases[0]?.targets.filter((target) => target.boneId.startsWith("hip_")); + expect(hips?.map((target) => target.euler.x)).toEqual([-85, -85]); + expect(result.warnings.filter((warning) => warning.joint.startsWith("hip_"))).toHaveLength(2); + }); + + it("clamps a later hinge against carried hip flexion", () => { + const source = [ + 'posecode posture "Fold"', + " rig humanoid", + ' step "Lift knee" 1s flow:', + " hips: flex 110", + ' step "Hinge" 1s settle:', + " pelvis: hinge 50", + ].join("\n"); + const result = parse(source); + expect(result.errors).toEqual([]); + expect(result.ir?.phases[1]?.targets[0]?.euler.x).toBe(25); + expect(result.warnings).toContainEqual(expect.objectContaining({ joint: "pelvis", clamped: 25 })); + }); + + it("accounts for hip flexion in the seated base pose", () => { + const source = [ + 'posecode posture "Seated fold"', + " rig humanoid", + " pose start = seated", + ' step "Fold" 1s settle:', + " pelvis: hinge 60", + ].join("\n"); + const result = parse(source); + expect(result.errors).toEqual([]); + expect(result.ir?.phases[0]?.targets[0]?.euler.x).toBe(45); + }); + + it("reduces a new hinge before touching a same-step hip when the opposite hip is carried", () => { + const source = [ + 'posecode posture "Asymmetric fold"', + " rig humanoid", + ' step "Carry left" 1s flow:', + " hip_left: flex 110", + ' step "Add hinge and right" 1s settle:', + " pelvis: hinge 50", + " hip_right: flex 110", + ].join("\n"); + const result = parse(source); + expect(result.errors).toEqual([]); + const phase = result.ir?.phases[1]; + expect(phase?.targets.find((target) => target.boneId === "pelvis")?.euler.x).toBe(25); + expect(phase?.targets.find((target) => target.boneId === "hip_right")?.euler.x).toBe(-110); + expect(result.warnings).toContainEqual(expect.objectContaining({ joint: "pelvis", clamped: 25 })); + expect(result.warnings.some((warning) => warning.joint === "hip_right")).toBe(false); + }); +}); diff --git a/packages/posecode-render/src/character.ts b/packages/posecode-render/src/character.ts index 6879683..9b07780 100644 --- a/packages/posecode-render/src/character.ts +++ b/packages/posecode-render/src/character.ts @@ -75,6 +75,10 @@ export interface Character { correctContacts(driver: Mannequin, boneIds: readonly string[]): void; /** Precise CPU-skinned world bounds for diagnostics/export validation. */ getBounds(): THREE.Box3; + /** World position of one mapped character joint, addressed by driver id. */ + getJointWorldPosition(driverId: string): THREE.Vector3 | null; + /** Character orientation expressed back in the driver's semantic frame. */ + getJointDriverQuaternion(driverId: string): THREE.Quaternion | null; /** Fast sampled visible-surface correction; returns the applied Y delta. */ reconcileFloor(floorY?: number): number; /** @@ -447,6 +451,24 @@ export function rigCharacter(charScene: THREE.Object3D): Character { return foundSkin ? box : new THREE.Box3().setFromObject(group); } + function getJointWorldPosition(driverId: string): THREE.Vector3 | null { + const mappedBone = mappedById.get(driverId); + if (!mappedBone) return null; + return mappedBone.node.getWorldPosition(new THREE.Vector3()); + } + + function getJointDriverQuaternion(driverId: string): THREE.Quaternion | null { + const mappedBone = mappedById.get(driverId); + if (!mappedBone) return null; + // sync writes characterWorld = driverWorld * calibratedRestWorld. + // Removing the rest frame yields an orientation whose local axes match the + // driver metrics (sole/palm/knuckle normals) while still sampling the + // actual retargeted character skeleton. + return mappedBone.node + .getWorldQuaternion(new THREE.Quaternion()) + .multiply(mappedBone.restWorld.clone().invert()); + } + // A dense uniform sample plus every rest-pose axis extremum. Xbot's 28k // vertices reduce to ~3.6k skin transforms per frame while retaining sole, // back, head, hand, and limb surface coverage under arbitrary articulation. @@ -497,6 +519,8 @@ export function rigCharacter(charScene: THREE.Object3D): Character { sync, correctContacts, getBounds, + getJointWorldPosition, + getJointDriverQuaternion, reconcileFloor, skinnedMesh, drivenNodes, diff --git a/packages/posecode-render/src/contacts.ts b/packages/posecode-render/src/contacts.ts index 0a1fdee..8735939 100644 --- a/packages/posecode-render/src/contacts.ts +++ b/packages/posecode-render/src/contacts.ts @@ -2,57 +2,234 @@ import * as THREE from "three"; import { eulerRomFor, type PinTarget, type ReachTarget, type GripTarget } from "posecode-parser"; import type { Mannequin } from "./mannequin.js"; +import { effectorBoneId } from "./reach.js"; const DOWN = new THREE.Vector3(0, -1, 0); const DEG = Math.PI / 180; +const CONTACT_EULER = new THREE.Euler(); + +export type HandSide = "left" | "right"; +export type HandContactKind = "palm" | "fist"; /** - * Rotate contacting wrists so the palm face normal points into the floor. - * Applies to hands pressed to the floor via `reach`/`pin: hands floor` AND via - * `ground-lock: hands` (a high plank / push-up / mountain-climber): those bear - * weight flat on the ground and must be palm-down, not left in their raw FK - * (pronated) orientation. + * Actual outward normal of the flattened procedural palm geometry. The palm + * ellipsoid is shallow on local Z for both hands (see mannequin.addPalm). */ -export function alignFloorPalms( +export const PALM_LOCAL_NORMAL = [0, 0, 1] as const; +/** Knuckle-bearing direction from wrist to the curled finger bases. */ +export const FIST_LOCAL_NORMAL = [0, -1, 0] as const; + +function strictClampToRom(node: THREE.Object3D, boneId: string): boolean { + const rom = eulerRomFor(boneId); + if (!rom) return false; + CONTACT_EULER.setFromQuaternion(node.quaternion, "XYZ"); + const x = THREE.MathUtils.clamp(CONTACT_EULER.x, rom.x.min * DEG, rom.x.max * DEG); + const y = THREE.MathUtils.clamp(CONTACT_EULER.y, rom.y.min * DEG, rom.y.max * DEG); + const z = THREE.MathUtils.clamp(CONTACT_EULER.z, rom.z.min * DEG, rom.z.max * DEG); + if ( + Math.abs(x - CONTACT_EULER.x) < 1e-10 && + Math.abs(y - CONTACT_EULER.y) < 1e-10 && + Math.abs(z - CONTACT_EULER.z) < 1e-10 + ) return false; + CONTACT_EULER.set(x, y, z, "XYZ"); + node.quaternion.setFromEuler(CONTACT_EULER); + return true; +} + +/** + * Final safety assertion for renderer-authored terminal corrections. It is + * intentionally limited to wrists/ankles: timeline hip counter-rotation for a + * pelvis hinge may legitimately sit outside a raw isolated hip box, whereas + * contact wrist and ankle locals have no such coupled exception. All three + * ankle axes are clamped, including the normally locked Y/Z axes. + */ +export function enforceContactRom(m: Mannequin): void { + let changed = false; + for (const side of ["left", "right"] as const) { + for (const joint of [`wrist_${side}`, `ankle_${side}`]) { + const node = m.bones.get(joint); + if (node) changed = strictClampToRom(node, joint) || changed; + } + } + if (changed) m.root.updateMatrixWorld(true); +} + +function alignWristNormal( m: Mannequin, + side: HandSide, + normal: readonly [number, number, number], + weight: number, +): boolean { + const wristId = `wrist_${side}`; + const wrist = m.bones.get(wristId); + if (!wrist?.parent) return false; + const world = wrist.getWorldQuaternion(new THREE.Quaternion()); + const current = new THREE.Vector3(...normal).applyQuaternion(world).normalize(); + const correction = new THREE.Quaternion().setFromUnitVectors(current, DOWN); + const safeWeight = THREE.MathUtils.clamp(weight, 0, 1); + if (safeWeight < 1) { + correction.slerp(new THREE.Quaternion(), 1 - safeWeight); + } + const desiredWorld = correction.multiply(world); + const parentWorld = wrist.parent.getWorldQuaternion(new THREE.Quaternion()); + wrist.quaternion.copy(parentWorld.invert().multiply(desiredWorld)); + // Contact correction is never allowed to buy contact with a broken wrist. + strictClampToRom(wrist, wristId); + return true; +} + +function collectFloorHandContacts( reaches: readonly (ReachTarget & { weight?: number })[], pins: readonly PinTarget[], - groundLock: readonly string[] = [], -): void { - const sides = new Map<"left" | "right", number>(); - const collect = (effector: string, target: string, weight = 1) => { + groundLock: readonly string[], +): Map { + const contacts = new Map(); + const collect = (effector: string, target: string, weight = 1): void => { if (target !== "floor") return; - if (effector === "hands" || effector === "hand_left") { - sides.set("left", Math.max(sides.get("left") ?? 0, weight)); - } - if (effector === "hands" || effector === "hand_right") { - sides.set("right", Math.max(sides.get("right") ?? 0, weight)); - } + const fistMatch = /^fist_(left|right)$/.exec(effector); + const palmMatch = /^(?:hand|wrist)_(left|right)$/.exec(effector); + const isFist = effector === "fists" || Boolean(fistMatch); + const isPalm = effector === "hands" || Boolean(palmMatch); + if (!isFist && !isPalm) return; + const kind: HandContactKind = isFist ? "fist" : "palm"; + const add = (side: HandSide): void => { + const previous = contacts.get(side); + // An explicit fist wins over a simultaneous generic hand contact. + if (!previous || kind === "fist" || previous.kind !== "fist") { + contacts.set(side, { + kind, + weight: Math.max(previous?.weight ?? 0, weight), + }); + } + }; + if (effector === "hands" || effector === "fists" || fistMatch?.[1] === "left" || palmMatch?.[1] === "left") add("left"); + if (effector === "hands" || effector === "fists" || fistMatch?.[1] === "right" || palmMatch?.[1] === "right") add("right"); }; reaches.forEach((r) => collect(r.effector, r.target, r.weight)); pins.forEach((p) => collect(p.effector, p.anchor)); - if (groundLock.includes("hands") || groundLock.includes("hand_left")) { - sides.set("left", 1); + groundLock.forEach((effector) => collect(effector, "floor")); + return contacts; +} + +/** + * Orient floor contacts by their real geometry: a palm presents its flattened + * +Z face, while a fist presents the wrist→knuckle (-Y) direction. The two are + * intentionally distinct, and both corrections are strict wrist-ROM-clamped. + */ +export function alignFloorContacts( + m: Mannequin, + reaches: readonly (ReachTarget & { weight?: number })[], + pins: readonly PinTarget[], + groundLock: readonly string[] = [], +): void { + const contacts = collectFloorHandContacts(reaches, pins, groundLock); + let changed = false; + for (const [side, contact] of contacts) { + const normal = contact.kind === "fist" ? FIST_LOCAL_NORMAL : PALM_LOCAL_NORMAL; + changed = alignWristNormal(m, side, normal, contact.weight) || changed; + } + if (changed) m.root.updateMatrixWorld(true); +} + +/** Backwards-compatible name retained for eval/embedders; now handles fists too. */ +export function alignFloorPalms( + m: Mannequin, + reaches: readonly (ReachTarget & { weight?: number })[], + pins: readonly PinTarget[], + groundLock: readonly string[] = [], +): void { + alignFloorContacts(m, reaches, pins, groundLock); +} + +/** + * Bone-origin target whose corresponding visible contact surface rests on y=0. + * Joint effectors (knee/elbow) use their local contact radius instead of the + * whole descendant subtree—otherwise a knee target incorrectly measures the + * shin/foot and folds the knee upward while still claiming floor contact. + */ +export function floorTargetForEffector( + m: Mannequin, + effectorName: string, +): THREE.Vector3 | null { + const effector = m.bones.get(effectorBoneId(effectorName)); + if (!effector) return null; + const p = effector.getWorldPosition(new THREE.Vector3()); + if (effectorName === "pelvis") { + p.y = pelvisFloorDrop(m); + return p; + } + if (effectorName.startsWith("knee_")) { + // Measure only the two rounded caps meeting at the knee. A subtree would + // wrongly include the whole shin/foot, while a fixed proxy radius misses + // the vertical extent as the bent limb changes orientation. + const side = effectorName.endsWith("_left") ? "left" : "right"; + p.y = jointSurfaceDrop( + m, + `knee_${side}`, + m.contactSurfaces[`knee_${side}`], + Math.max(m.collision.shin, m.collision.thigh), + ); + return p; } - if (groundLock.includes("hands") || groundLock.includes("hand_right")) { - sides.set("right", 1); + if (effectorName.startsWith("elbow_")) { + p.y = m.collision.arm; + return p; } + const box = new THREE.Box3().setFromObject(effector); + p.y = Number.isFinite(box.min.y) ? Math.max(0, p.y - box.min.y) : 0; + return p; +} - for (const [side, weight] of sides) { - const wrist = m.bones.get(`wrist_${side}`); - if (!wrist?.parent) continue; - const world = wrist.getWorldQuaternion(new THREE.Quaternion()); - const localNormal = side === "left" - ? new THREE.Vector3(1, 0, 0) - : new THREE.Vector3(-1, 0, 0); - const current = localNormal.applyQuaternion(world).normalize(); - const correction = new THREE.Quaternion().setFromUnitVectors(current, DOWN); - if (weight < 1) correction.slerp(new THREE.Quaternion(), 1 - weight); - const desiredWorld = correction.multiply(world); - const parentWorld = wrist.parent.getWorldQuaternion(new THREE.Quaternion()); - wrist.quaternion.copy(parentWorld.invert().multiply(desiredWorld)); +/** Signed height of an effector's actual contact surface above the floor. */ +export function floorContactHeight(m: Mannequin, effectorName: string): number | null { + const effector = m.bones.get(effectorBoneId(effectorName)); + if (!effector) return null; + const originY = effector.getWorldPosition(new THREE.Vector3()).y; + if (effectorName.startsWith("elbow_")) return originY - m.collision.arm; + if (effectorName.startsWith("knee_")) { + const side = effectorName.endsWith("_left") ? "left" : "right"; + return originY - jointSurfaceDrop( + m, + `knee_${side}`, + m.contactSurfaces[`knee_${side}`], + Math.max(m.collision.shin, m.collision.thigh), + ); } - if (sides.size > 0) m.root.updateMatrixWorld(true); + if (effectorName === "pelvis") return originY - pelvisFloorDrop(m); + const minY = new THREE.Box3().setFromObject(effector).min.y; + return Number.isFinite(minY) ? minY : null; +} + +function jointSurfaceDrop( + m: Mannequin, + boneId: string, + surfaces: readonly THREE.Object3D[], + fallback: number, +): number { + const bone = m.bones.get(boneId); + if (!bone) return fallback; + const originY = bone.getWorldPosition(new THREE.Vector3()).y; + let minY = Infinity; + for (const surface of surfaces) { + const y = new THREE.Box3().setFromObject(surface).min.y; + if (Number.isFinite(y)) minY = Math.min(minY, y); + } + return Number.isFinite(minY) ? Math.max(0.001, originY - minY) : fallback; +} + +/** Current trunk-surface drop below the pelvis origin (translation invariant). */ +function pelvisFloorDrop(m: Mannequin): number { + const pelvis = m.bones.get("pelvis"); + if (!pelvis) return Math.max(0.08, m.collision.torso * 0.85); + const originY = pelvis.getWorldPosition(new THREE.Vector3()).y; + let minY = Infinity; + for (const surface of m.contactSurfaces.pelvis) { + const y = new THREE.Box3().setFromObject(surface).min.y; + if (Number.isFinite(y)) minY = Math.min(minY, y); + } + return Number.isFinite(minY) + ? Math.max(0.08, originY - minY) + : Math.max(0.08, m.collision.torso * 0.85); } const SOLE_LOCAL = new THREE.Vector3(0, -1, 0); @@ -85,8 +262,12 @@ export function levelPlantedFeet(m: Mannequin, activeGroundLock: readonly string // deliberate relevé / calf-raise / demi-plié — leave it on its toes. TMP_EULER.setFromQuaternion(ankle.quaternion, "XYZ"); const authoredX = TMP_EULER.x; - const authoredZ = TMP_EULER.z; - if (authoredX > PLANTARFLEX_SKIP) continue; + if (authoredX > PLANTARFLEX_SKIP) { + // Preserve deliberate tiptoe pitch, but still assert the ankle's locked + // axial/frontal axes and configured plantarflexion ceiling. + changed = strictClampToRom(ankle, `ankle_${side}`) || changed; + continue; + } // Planted-ness weight from the foot mesh bottom height: fully level when the // sole is on the floor, fading out as a swing foot lifts past PLANT_FADE. const box = new THREE.Box3().setFromObject(ankle); @@ -101,36 +282,118 @@ export function levelPlantedFeet(m: Mannequin, activeGroundLock: readonly string const desiredWorld = correction.multiply(world); const parentWorld = ankle.parent.getWorldQuaternion(new THREE.Quaternion()); const local = parentWorld.invert().multiply(desiredWorld); - // Clamp the corrected ankle to its ROM, widened to admit the authored angle - // so leveling can never push the joint past a healthy range. - const rom = eulerRomFor(`ankle_${side}`); - if (rom) { - TMP_EULER.setFromQuaternion(local, "XYZ"); - const cx = THREE.MathUtils.clamp( - TMP_EULER.x, - Math.min(rom.x.min * DEG, authoredX), - Math.max(rom.x.max * DEG, authoredX), - ); - const cz = THREE.MathUtils.clamp( - TMP_EULER.z, - Math.min(rom.z.min * DEG, authoredZ), - Math.max(rom.z.max * DEG, authoredZ), - ); - TMP_EULER.set(cx, TMP_EULER.y, cz, "XYZ"); - local.setFromEuler(TMP_EULER); - } ankle.quaternion.copy(local); + // Clamp X/Y/Z strictly. In particular, ankle Y/Z are locked axes; leaving + // either component from the world-space correction created twisted soles. + strictClampToRom(ankle, `ankle_${side}`); changed = true; } if (changed) m.root.updateMatrixWorld(true); } /** Finger curl (radians about the knuckle X axis) that wraps a gripping hand. */ -export const FINGER_CURL = 1.35; +export const FINGER_CURL = -1.35; /** Thumb opposition curl (radians) toward the fingers. */ -export const THUMB_CURL = 0.9; +export const THUMB_CURL = -0.9; +/** Sideways thumb opposition kept inside the thumb's adduction ROM. */ +export const THUMB_OPPOSE = 0.5; +/** Closed-fist finger curl, still inside the 100deg finger-flexion limit. */ +export const FIST_CURL = -1.5; +export const FIST_THUMB_CURL = -1.15; +export const FIST_THUMB_OPPOSE = 0.45; const FINGERS = ["index", "middle", "ring", "pinky"] as const; +/** True for the parallel-rail anchors belonging to the dip-bars prop. */ +export function isDipBarGrip(anchor: string): boolean { + return anchor === "bars" || anchor.startsWith("bars_"); +} + +/** + * Establish a deterministic dip-bar forearm frame before body translation/IK. + * Removing inherited elbow axial twist makes the palm face available for the + * downward support contact instead of leaving each hand turned toward a thigh. + */ +export function prepareGripFrames( + m: Mannequin, + grips: readonly { effector: string; anchor: string }[], +): void { + let changed = false; + for (const grip of grips) { + if (!isDipBarGrip(grip.anchor)) continue; + const match = /_(left|right)$/.exec(grip.effector); + if (!match) continue; + const side = match[1] as HandSide; + const elbowId = `elbow_${side}`; + const elbow = m.bones.get(elbowId); + if (!elbow) continue; + CONTACT_EULER.setFromQuaternion(elbow.quaternion, "XYZ"); + CONTACT_EULER.y = 0; + elbow.quaternion.setFromEuler(CONTACT_EULER); + strictClampToRom(elbow, elbowId); + changed = true; + } + if (changed) m.root.updateMatrixWorld(true); +} + +/** + * Finish the stable dip-bar frame after IK: the flattened palm presses down on + * the rail while wrist flexion/extension remains strictly ROM-safe. + */ +export function alignGripFrames( + m: Mannequin, + grips: readonly { effector: string; anchor: string }[], +): void { + let changed = false; + for (const grip of grips) { + if (!isDipBarGrip(grip.anchor)) continue; + const match = /_(left|right)$/.exec(grip.effector); + if (!match) continue; + changed = alignWristNormal( + m, + match[1] as HandSide, + PALM_LOCAL_NORMAL, + 1, + ) || changed; + } + if (changed) m.root.updateMatrixWorld(true); +} + +/** + * Close semantic fist effectors without overwriting explicitly authored digit + * bones. Wrist/contact corrections never touch these locals, so the curl is + * preserved while the knuckles are oriented onto a target. + */ +export function formFists( + m: Mannequin, + sides: ReadonlySet, + authoredFingers: ReadonlySet = new Set(), +): void { + let changed = false; + for (const side of sides) { + for (const finger of FINGERS) { + const id = `${finger}_${side}`; + if (authoredFingers.has(id)) continue; + const bone = m.bones.get(id); + if (!bone) continue; + bone.rotation.set(FIST_CURL, 0, 0); + changed = true; + } + const thumbId = `thumb_${side}`; + if (!authoredFingers.has(thumbId)) { + const thumb = m.bones.get(thumbId); + if (thumb) { + thumb.rotation.set( + FIST_THUMB_CURL, + 0, + side === "left" ? -FIST_THUMB_OPPOSE : FIST_THUMB_OPPOSE, + ); + changed = true; + } + } + } + if (changed) m.root.updateMatrixWorld(true); +} + /** * Curl the fingers of each gripping hand around the bar. Grips are per-side * after resolution (`hand_left` / `hand_right`), so the side comes straight off @@ -152,7 +415,7 @@ export function wrapGrip(m: Mannequin, grips: readonly GripTarget[]): void { const thumb = m.bones.get(`thumb_${side}`); if (thumb) { // Thumb wraps from the opposite side: curl plus a sideways opposition. - thumb.rotation.set(THUMB_CURL, 0, side === "left" ? -THUMB_CURL : THUMB_CURL); + thumb.rotation.set(THUMB_CURL, 0, side === "left" ? -THUMB_OPPOSE : THUMB_OPPOSE); changed = true; } } @@ -164,16 +427,18 @@ export function wrapGrip(m: Mannequin, grips: readonly GripTarget[]): void { * relaxed hand is not flat: the fingers settle into a soft inward hook (~30°), * which reads as a natural cupped hand instead of a stiff splayed palm. */ -export const REST_CURL = 0.55; -/** Slight finger adduction (radians) drawing splayed digits toward the middle - * finger, so a relaxed hand closes softly rather than fanning like jazz-hands. */ -export const REST_ADDUCT = 0.12; +export const REST_CURL = -0.55; +/** Gentle thumb opposition for a relaxed hand, within adduction ROM. */ +export const REST_THUMB_OPPOSE = 0.4; +/** Finger bones are intentionally single-DOF; their authored rest offsets + * provide natural spacing without inventing an out-of-ROM lateral rotation. */ +export const REST_ADDUCT = 0; /** * Near-flat curl (radians) for a hand pressed onto the floor (plank, push-up, * cobra). The palm lies flat with the fingers extended forward; the resting * inward hook would instead claw the fingertips into the ground. */ -export const FLOOR_CURL = 0.06; +export const FLOOR_CURL = -0.06; /** * Give idle hands a natural relaxed shape instead of a flat splayed palm. @@ -219,7 +484,7 @@ export function relaxHands( if (thumb) { // Planted: thumb lies alongside the flat palm. Free: opposes softly. const thumbCurl = planted ? FLOOR_CURL : REST_CURL * 0.6; - const thumbOppose = planted ? 0 : REST_CURL; + const thumbOppose = planted ? 0 : REST_THUMB_OPPOSE; thumb.rotation.set(thumbCurl, 0, side === "left" ? -thumbOppose : thumbOppose); changed = true; } @@ -238,16 +503,16 @@ const HIP_EULER = new THREE.Euler(); * (right leg forward ↔ left arm forward). Adds a swing to each free shoulder * proportional to the OPPOSITE hip's sagittal (local X) angle, so any move that * animates the hips (walk, march, box-step) gets natural arm swing for free. - * Skips shoulders the document authors and any gripping side. + * Skips shoulders the document authors and any contact-constrained hand side. */ export function swingArms( m: Mannequin, authoredShoulders: ReadonlySet, - gripSides: ReadonlySet<"left" | "right">, + protectedSides: ReadonlySet<"left" | "right">, ): void { let changed = false; for (const side of ["left", "right"] as const) { - if (gripSides.has(side)) continue; + if (protectedSides.has(side)) continue; const shoulderId = `shoulder_${side}`; if (authoredShoulders.has(shoulderId)) continue; const shoulder = m.bones.get(shoulderId); diff --git a/packages/posecode-render/src/depenetrate.ts b/packages/posecode-render/src/depenetrate.ts index b4cea52..f6ef964 100644 --- a/packages/posecode-render/src/depenetrate.ts +++ b/packages/posecode-render/src/depenetrate.ts @@ -1,12 +1,13 @@ /** - * Self-collision resolution: stop limbs from passing through the body. + * Bounded self-penetration correction for selected limb/body samples. * * Authored poses are pure per-joint rotations, so nothing prevents a biceps * curl from dragging the forearm through the thighs, or a cross-body reach * from sweeping the hand through the chest. This pass approximates the body * with capsules (torso, head, thighs, shins), samples points along each * forearm/hand and each lower leg, and when a sample sits inside an obstacle - * it rotates the limb's proximal joint (shoulder / hip) just enough to clear. + * it rotates the limb's proximal joint (shoulder / hip) toward clearance. It + * is not a comprehensive collision detector or physics solver. * * Principles: * - **Minimal**: corrections only remove actual overlap, so intentional @@ -14,8 +15,8 @@ * inside it. A pose with no overlap is untouched. * - **Deterministic**: corrections are a pure function of the pose, so looping * animations stay smooth (no frame-to-frame jitter). - * - **Safe**: each adjusted joint is clamped back into its healthy ROM - * (widened to admit the authored angle), the same guarantee reach-IK gives. + * - **Bounded**: each adjusted joint is clamped back into its configured ROM + * (widened to admit the authored angle), matching reach-IK's constraint. * * Runs on the driver skeleton right after FK sampling, before ground-lock, in * both the viewer's frame loop and its load-time anchor capture, so ground diff --git a/packages/posecode-render/src/groundlock.ts b/packages/posecode-render/src/groundlock.ts index 0bea147..e9c0829 100644 --- a/packages/posecode-render/src/groundlock.ts +++ b/packages/posecode-render/src/groundlock.ts @@ -33,6 +33,7 @@ import * as THREE from "three"; import type { Mannequin } from "./mannequin.js"; +import { floorTargetForEffector } from "./contacts.js"; const ROOT_X = new THREE.Vector3(1, 0, 0); @@ -121,11 +122,24 @@ export function applyGroundLock( // Newton iterations: rotate about the toes until the authored upper-body // support reaches the floor (palms for high plank, elbows for forearm // plank). The final bbox correction accounts for the support mesh radius. + const upperSurfaceError = (): number => { + let sum = 0; + let count = 0; + for (const id of upperSupports) { + const node = m.bones.get(id); + if (!node) continue; + const semantic = id.replace("wrist_", "hand_"); + const targetY = floorTargetForEffector(m, semantic)?.y ?? 0; + sum += node.getWorldPosition(new THREE.Vector3()).y - targetY; + count++; + } + return count > 0 ? sum / count : 0; + }; for (let i = 0; i < 8; i++) { - const y0 = avgWorld(m, upperSupports).y; + const y0 = upperSurfaceError(); if (Math.abs(y0) < 0.004) break; rotateRootAboutPivot(m, pivot, 0.01); - const y1 = avgWorld(m, upperSupports).y; + const y1 = upperSurfaceError(); rotateRootAboutPivot(m, pivot, -0.01); const deriv = (y1 - y0) / 0.01; if (Math.abs(deriv) < 1e-4) break; diff --git a/packages/posecode-render/src/ik.ts b/packages/posecode-render/src/ik.ts index 1317dc0..531593f 100644 --- a/packages/posecode-render/src/ik.ts +++ b/packages/posecode-render/src/ik.ts @@ -7,8 +7,9 @@ * world target while the rest of the body moves. Chains may carry per-joint * Euler angle limits (the joint's Range of Motion expressed as a local-frame * box, see posecode-parser's `eulerRomFor`); each iteration clamps the joint back - * inside its box, so a solved pose can never exceed the healthy ROM any more - * than an authored angle can. + * inside its box, so a solved joint cannot exceed the configured ROM any more + * than an authored joint can. This is a mechanical constraint, not a safety + * or clinical assessment of the complete movement. */ import * as THREE from "three"; diff --git a/packages/posecode-render/src/index.ts b/packages/posecode-render/src/index.ts index 64ba663..8aa747e 100644 --- a/packages/posecode-render/src/index.ts +++ b/packages/posecode-render/src/index.ts @@ -12,7 +12,6 @@ import * as THREE from "three"; import { OrbitControls } from "three/examples/jsm/controls/OrbitControls.js"; import { RoomEnvironment } from "three/examples/jsm/environments/RoomEnvironment.js"; -import { eulerRomFor } from "posecode-parser"; import type { PosecodeIR, ReachTarget, PinTarget, GripTarget } from "posecode-parser"; import { buildMannequin, type Mannequin } from "./mannequin.js"; import { applyGroundLock as applyGroundLockTo, groundFigure as groundFigureOf } from "./groundlock.js"; @@ -22,7 +21,6 @@ import { type PhaseSegment, type WeightedReachTarget, } from "./timeline.js"; -import { solveCCD, type JointLimits } from "./ik.js"; import { buildProps, type PropScene } from "./props.js"; import { loadCharacter, type Character } from "./character.js"; import { @@ -34,7 +32,30 @@ import { } from "./clips.js"; import { depenetrate } from "./depenetrate.js"; import { resolvePropContacts, propContactExemptions } from "./propcontact.js"; -import { alignFloorPalms, levelPlantedFeet, wrapGrip, relaxHands, swingArms, aimHead } from "./contacts.js"; +import { + alignFloorContacts, + alignGripFrames, + enforceContactRom, + floorContactHeight, + floorTargetForEffector, + formFists, + isDipBarGrip, + levelPlantedFeet, + prepareGripFrames, + wrapGrip, + relaxHands, + swingArms, + aimHead, +} from "./contacts.js"; +import { + REACH_TOLERANCE, + effectorBoneId, + missingReachTarget, + reachChain, + solveReachToPoint, + type ReachResidual, +} from "./reach.js"; +import { solveCCD } from "./ik.js"; const DEG = Math.PI / 180; @@ -65,6 +86,8 @@ export interface Viewer { /** True while a retargeted mocap clip is driving (or fading over) the pose. */ get clipActive(): boolean; getTimeline(): TimelineInfo | null; + /** Diagnostics for every active reach, including missing/unreachable targets. */ + getReachResiduals(): readonly ReachResidual[]; /** Precise visible world bounds; intended for audits and deterministic export. */ getVisibleBounds(): THREE.Box3; getMannequin(): any; @@ -312,6 +335,14 @@ export function createViewer( // The last loaded document, kept so the viewer can re-solve base pose and // ground anchors when the character (with its own proportions) arrives. let lastIR: PosecodeIR | null = null; + // Rebuilt on every solved frame. Missing target names and unreachable + // effectors remain visible here instead of disappearing behind `continue`. + let reachResiduals: ReachResidual[] = []; + type ReachResidualTarget = + | { kind: "fixed"; point: THREE.Vector3 } + | { kind: "floor"; point: THREE.Vector3 } + | { kind: "landmark"; boneId: string }; + let reachResidualTargets: Array = []; let groundTargets = new Map(); const segmentStartEffectors: Map[] = []; // World-space anchor points contributed by scene props (chair seat, bar grip, @@ -332,6 +363,11 @@ export function createViewer( let loopCb: () => void = () => {}; let lastPhaseName = ""; let activeSegIndex = 0; + // `load()` briefly solves each real phase endpoint to seed any floor pin + // introduced by the following phase from the *fully solved* prior pose. + // Keep those internal solves out of callbacks, character/mocap state, and + // the canvas; they are deterministic anchor preparation, not visible frames. + let precomputingAnchors = false; // Camera easing targets. const desiredTarget = new THREE.Vector3(0, 0.9, 0); @@ -398,79 +434,12 @@ export function createViewer( return frameAnchorMap; } - // Friendly DSL effector aliases → the distal bone whose world position is - // driven to the reach target. - const EFFECTOR_BONE: Record = { - hand_left: "wrist_left", - hand_right: "wrist_right", - foot_left: "ankle_left", - foot_right: "ankle_right", - }; - - /** - * The rotatable joint chain (proximal → distal) that moves an effector, with - * each joint's ROM expressed as local Euler limits for the constrained solve. - * A limit box is widened to include the joint's CURRENT (authored FK) angle: - * the timeline pose is already ROM-clamped in author terms, but rig mechanics - * such as the hip-hinge counter-rotation can place a bone outside its raw box - * on purpose: IK must never fight the authored pose, only be prevented from - * pushing beyond it. - */ - function reachChain(effectorBone: string): { - joints: THREE.Object3D[]; - limits: (JointLimits | null)[]; - } { - const side = effectorBone.endsWith("_left") ? "left" : "right"; - const ids = effectorBone.startsWith("wrist") - ? [`shoulder_${side}`, `elbow_${side}`] - : effectorBone.startsWith("elbow") - ? [`shoulder_${side}`] - : effectorBone.startsWith("ankle") - ? [`hip_${side}`, `knee_${side}`] - : []; - const joints: THREE.Object3D[] = []; - const limits: (JointLimits | null)[] = []; - for (const id of ids) { - const node = mannequin.bones.get(id); - if (!node) continue; - joints.push(node); - limits.push(jointLimitsFor(id, node)); - } - return { joints, limits }; - } - - const REACH_EULER = new THREE.Euler(); - - /** A bone's ROM as radian Euler limits, widened to admit its current pose. */ - function jointLimitsFor(boneId: string, node: THREE.Object3D): JointLimits | null { - const rom = eulerRomFor(boneId); - if (!rom) return null; - REACH_EULER.setFromQuaternion(node.quaternion, "XYZ"); - return { - x: widen(rom.x.min * DEG, rom.x.max * DEG, REACH_EULER.x), - y: widen(rom.y.min * DEG, rom.y.max * DEG, REACH_EULER.y), - z: widen(rom.z.min * DEG, rom.z.max * DEG, REACH_EULER.z), - }; - } - - function widen(min: number, max: number, current: number): [number, number] { - return [Math.min(min, current), Math.max(max, current)]; - } - /** Resolve a reach target name to a world point: floor / prop anchor / landmark. */ function resolveReachTarget( target: string, - effector: THREE.Object3D, + effectorName: string, ): THREE.Vector3 | null { - if (target === "floor") { - // Rest the effector's MESH on the floor, not its bone origin: the hand - // mesh extends below the wrist bone, so a bone target of y=0 would sink - // the palm and force the floor safety clamp to lift the whole body. - const p = effector.getWorldPosition(new THREE.Vector3()); - const box = new THREE.Box3().setFromObject(effector); - p.y = Number.isFinite(box.min.y) ? Math.max(0, p.y - box.min.y) : 0; - return p; - } + if (target === "floor") return floorTargetForEffector(mannequin, effectorName); const anchor = propAnchors.get(target); if (anchor) return anchor.clone(); const bone = mannequin.bones.get(target); @@ -487,24 +456,63 @@ export function createViewer( * joints keep their authored FK pose. */ function applyReaches(reaches: WeightedReachTarget[]): void { + reachResiduals = []; + reachResidualTargets = []; for (const r of reaches) { - const effectorBone = EFFECTOR_BONE[r.effector] ?? r.effector; + const effectorBone = effectorBoneId(r.effector); const effector = mannequin.bones.get(effectorBone); - if (!effector) continue; - const target = resolveReachTarget(r.target, effector); - if (!target) continue; - const { joints, limits } = reachChain(effectorBone); - if (joints.length === 0) continue; - const before = joints.map((joint) => joint.quaternion.clone()); - solveCCD({ joints, limits, effector, target }, 12); - for (let i = 0; i < joints.length; i++) { - const solved = joints[i]!.quaternion.clone(); - joints[i]!.quaternion.slerpQuaternions(before[i]!, solved, r.weight); + if (!effector) { + reachResiduals.push( + solveReachToPoint(mannequin, r.effector, r.target, new THREE.Vector3(), r.weight), + ); + reachResidualTargets.push(null); + continue; } - mannequin.root.updateMatrixWorld(true); + const target = resolveReachTarget(r.target, r.effector); + if (!target) { + reachResiduals.push(missingReachTarget(r.effector, r.target, r.weight)); + reachResidualTargets.push(null); + continue; + } + reachResiduals.push( + solveReachToPoint(mannequin, r.effector, r.target, target, r.weight), + ); + reachResidualTargets.push( + r.target === "floor" + ? { kind: "floor", point: target.clone() } + : mannequin.bones.has(r.target) + ? { kind: "landmark", boneId: r.target } + : { kind: "fixed", point: target.clone() }, + ); } } + /** Re-measure after later contacts/root grounding so diagnostics are final. */ + function refreshReachResiduals(): void { + reachResiduals = reachResiduals.map((residual, index) => { + const targetRef = reachResidualTargets[index]; + if (!targetRef || residual.distance === null) return residual; + const effector = mannequin.bones.get(effectorBoneId(residual.effector)); + if (!effector) return residual; + // Body landmarks and contact surfaces can move during later root/contact + // reconciliation. Re-read their final geometry while preserving a floor + // contact's solved world-space X/Z anchor. + const point = effector.getWorldPosition(new THREE.Vector3()); + let target: THREE.Vector3 | undefined; + if (targetRef.kind === "fixed") { + target = targetRef.point; + } else if (targetRef.kind === "landmark") { + target = mannequin.bones.get(targetRef.boneId)?.getWorldPosition(new THREE.Vector3()); + } else { + const height = floorContactHeight(mannequin, residual.effector); + if (height !== null) target = targetRef.point.clone().setY(point.y - height); + } + if (!target) return residual; + const distance = point.distanceTo(target); + return { ...residual, distance, reached: distance <= REACH_TOLERANCE }; + }); + } + /** * Contact pins: translate the WHOLE figure so each pinned effector sits on its * anchor. Where ground-lock keeps a planted foot on the floor, a pin keeps a @@ -515,10 +523,12 @@ export function createViewer( */ function applyPins(pins: PinTarget[]): void { if (pins.length === 0) return; + const dipBarPins = pins.filter((pin) => isDipBarGrip(pin.anchor)); + prepareGripFrames(mannequin, dipBarPins); const delta = new THREE.Vector3(); let n = 0; for (const p of pins) { - const effectorBone = EFFECTOR_BONE[p.effector] ?? p.effector; + const effectorBone = effectorBoneId(p.effector); const effector = mannequin.bones.get(effectorBone); if (!effector) continue; let anchor: THREE.Vector3 | null = null; @@ -526,14 +536,12 @@ export function createViewer( const startPos = segmentStartEffectors[activeSegIndex]?.get(effectorBone); if (startPos) { anchor = startPos.clone(); - const isFoot = effectorBone.startsWith("ankle") || effectorBone.startsWith("foot"); - const drop = isFoot ? (character?.proportions.soleDrop ?? 0.042) : 0; - anchor.y = drop; + anchor.y = floorTargetForEffector(mannequin, p.effector)?.y ?? 0; } else { - anchor = resolveReachTarget(p.anchor, effector); + anchor = resolveReachTarget(p.anchor, p.effector); } } else { - anchor = resolveReachTarget(p.anchor, effector); + anchor = resolveReachTarget(p.anchor, p.effector); } if (!anchor) continue; delta.add(anchor.sub(effector.getWorldPosition(new THREE.Vector3()))); @@ -543,6 +551,7 @@ export function createViewer( mannequin.root.position.add(delta.multiplyScalar(1 / n)); mannequin.root.updateMatrixWorld(true); } + alignGripFrames(mannequin, dipBarPins); } /** @@ -555,17 +564,20 @@ export function createViewer( */ function applyGrips(grips: GripTarget[]): void { if (grips.length === 0) return; - const resolveGrip = (anchor: string, effector: THREE.Object3D): THREE.Vector3 | null => - resolveReachTarget(anchor, effector) ?? - resolveReachTarget(anchor.replace(/_(left|right)$/, ""), effector); + // Dip support begins from a deterministic untwisted forearm frame; doing + // this before translation/IK means the later wrist target remains exact. + prepareGripFrames(mannequin, grips); + const resolveGrip = (anchor: string, effectorName: string): THREE.Vector3 | null => + resolveReachTarget(anchor, effectorName) ?? + resolveReachTarget(anchor.replace(/_(left|right)$/, ""), effectorName); // 1. Body translate (the vertical pull). const delta = new THREE.Vector3(); let n = 0; for (const g of grips) { - const effectorBone = EFFECTOR_BONE[g.effector] ?? g.effector; + const effectorBone = effectorBoneId(g.effector); const effector = mannequin.bones.get(effectorBone); if (!effector) continue; - const target = resolveGrip(g.anchor, effector); + const target = resolveGrip(g.anchor, g.effector); if (!target) continue; delta.add(target.clone().sub(effector.getWorldPosition(new THREE.Vector3()))); n++; @@ -576,17 +588,26 @@ export function createViewer( } // 2. Per-hand arm IK onto each grip point (ROM-clamped via reachChain). for (const g of grips) { - const effectorBone = EFFECTOR_BONE[g.effector] ?? g.effector; + const effectorBone = effectorBoneId(g.effector); const effector = mannequin.bones.get(effectorBone); if (!effector) continue; - const target = resolveGrip(g.anchor, effector); + const target = resolveGrip(g.anchor, g.effector); if (!target) continue; - const { joints, limits } = reachChain(effectorBone); + const { joints, limits } = reachChain(mannequin, g.effector); if (joints.length === 0) continue; + if (isDipBarGrip(g.anchor)) { + // Keep forearm twist stable while CCD positions the hand. Otherwise + // equally-valid axial solutions flip palms between/away from the rails. + for (let i = 0; i < joints.length; i++) { + if (!joints[i]!.name.startsWith("elbow_")) continue; + const limit = limits[i]; + if (limit) limits[i] = { ...limit, y: [0, 0] }; + } + } solveCCD({ joints, limits, effector, target }, 12); } - // 3. Finger wrap. Wrist orientation remains inherited until the semantic - // contact-frame solver can account for this character's calibrated axes. + // 3. Stable contact frame followed by an anatomically signed finger wrap. + alignGripFrames(mannequin, grips); wrapGrip(mannequin, grips); } @@ -599,12 +620,12 @@ export function createViewer( if (authoredHead) return; const pts: THREE.Vector3[] = []; const collect = (effectorName: string, anchorName: string): void => { - const bone = EFFECTOR_BONE[effectorName] ?? effectorName; + const bone = effectorBoneId(effectorName); const eff = mannequin.bones.get(bone); if (!eff) return; const t = - resolveReachTarget(anchorName, eff) ?? - resolveReachTarget(anchorName.replace(/_(left|right)$/, ""), eff); + resolveReachTarget(anchorName, effectorName) ?? + resolveReachTarget(anchorName.replace(/_(left|right)$/, ""), effectorName); if (t) pts.push(t); }; for (const g of info.grips) collect(g.effector, g.anchor); @@ -671,7 +692,7 @@ export function createViewer( } // Life layer rides on wall-clock time (not timeline time) so the figure // keeps breathing and blinking while paused or scrubbing. - applyLife(performance.now() / 1000); + if (!precomputingAnchors) applyLife(performance.now() / 1000); // Recompute root contact from the grounded base each frame (no drift). mannequin.root.position.copy(baseRootPos); mannequin.root.quaternion.copy(baseRootQuat); @@ -687,6 +708,32 @@ export function createViewer( mannequin.root.position.x += info.rootOffset.x; mannequin.root.position.z += info.rootOffset.z; mannequin.root.updateMatrixWorld(true); + // A semantic fist is geometry as well as an endpoint name. Close any + // unauthored digits before floor target/bounds resolution; authored curl + // remains untouched and survives the later wrist contact correction. + const fistSides = fistSidesOf(info.reaches, info.pins, info.groundLock); + const activeGripSides = gripSidesOf(info.grips); + const constrainedHandSides = contactHandSidesOf( + info.reaches, + info.pins, + info.grips, + info.groundLock, + ); + const palmFloorSides = floorHandSidesOf(info.reaches, info.pins, info.groundLock); + formFists(mannequin, fistSides, authoredFingers); + // Finger surface shape is part of a floor target too. Flatten palms (and + // relax free hands) before measuring their subtree bounds; fist/grip + // sides are protected from this pass. + relaxHands( + mannequin, + unionHandSides(activeGripSides, fistSides), + authoredFingers, + palmFloorSides, + ); + // Establish the intended palm/knuckle surface BEFORE any floor target is + // measured. The post-IK pass below restores the same frame after parent + // joints move, keeping contact-surface height consistent at both ends. + alignFloorContacts(mannequin, info.reaches, info.pins, info.groundLock); // Self-collision: nudge limbs out of the body BEFORE contact solving so // ground-lock and pins see the corrected pose (same order as load()). depenetrate(mannequin); @@ -710,22 +757,35 @@ export function createViewer( // bug. Ground-lock and pins have already fixed the root placement that // floor/landmark targets resolve against. applyReaches(info.reaches); - alignFloorPalms(mannequin, info.reaches, info.pins, info.groundLock); + alignFloorContacts(mannequin, info.reaches, info.pins, info.groundLock); // Plantigrade correction: keep planted soles flat to the floor so grounded // lower-body poses (squat, lunge, deadlift) don't balance on the toes. // Runs before the floor clamp so the leveled sole is what rests on y=0. levelPlantedFeet(mannequin, info.groundLock); // L4.2 aliveness: contralateral arm swing during locomotion (free arms only). - swingArms(mannequin, authoredShoulders, gripSidesOf(info.grips)); - // L4.1 aliveness: relax idle hands into a natural curl (grips still wrap). - relaxHands( - mannequin, - gripSidesOf(info.grips), - authoredFingers, - floorHandSidesOf(info.reaches, info.pins, info.groundLock), - ); + swingArms(mannequin, authoredShoulders, constrainedHandSides); // L4.3 aliveness: turn the head toward the active contact (bar / floor reach). applyLookAt(info); + // Final renderer-authored contact mutations stay inside strict terminal + // ROM (notably wrist and every ankle axis, including locked ankle Y/Z). + enforceContactRom(mannequin); + // A floor reach can change a limb after floating-root ground-lock has + // placed the existing supports. Reconcile once more in constraint order: + // replant the root support, then re-solve the independent limbs. Without + // this bounded refinement, the global floor safety clamp could rescue a + // reached knee/hand by lifting the declared planted foot several cm. + if (info.groundLock.length > 0 && info.reaches.length > 0) { + for (let refinement = 0; refinement < 3; refinement++) { + applyGroundLockTo( + mannequin, + info.groundLock, + frameAnchors(info.rootYaw, info.rootOffset), + ); + applyReaches(info.reaches); + alignFloorContacts(mannequin, info.reaches, info.pins, info.groundLock); + enforceContactRom(mannequin); + } + } // Safety net: reconcile the fully-solved pose with the floor. // // A ground-locked phase asserts its effectors (feet, and for a plank the @@ -751,11 +811,13 @@ export function createViewer( mannequin.root.position.y -= box.min.y; mannequin.root.updateMatrixWorld(true); } - if (info.phaseName !== lastPhaseName) { + refreshReachResiduals(); + if (!precomputingAnchors && info.phaseName !== lastPhaseName) { lastPhaseName = info.phaseName; phaseCb({ phaseName: info.phaseName, ...(info.cue ? { cue: info.cue } : {}) }); } } + if (precomputingAnchors) return; // Mirror the fully-solved driver pose onto the skinned character. character?.sync(mannequin); // Mocap layer: ease the crossfade weight, then blend the retargeted clip @@ -822,6 +884,29 @@ export function createViewer( timeline = buildTimeline(ir); time = 0; lastPhaseName = ""; + reachResiduals = []; + reachResidualTargets = []; + authoredFingers = new Set(timeline.bonesUsed.filter(isFingerId)); + authoredShoulders = new Set(timeline.bonesUsed.filter((id) => id.startsWith("shoulder_"))); + authoredHead = timeline.bonesUsed.some((id) => id === "head" || id === "neck"); + const initialPhase = ir.phases[0]; + const initialFistSides = fistSidesOf( + initialPhase?.reaches ?? [], + initialPhase?.pins ?? [], + initialPhase?.groundLock ?? [], + ); + const initialGripSides = gripSidesOf(initialPhase?.grips ?? []); + const initialConstrainedHandSides = contactHandSidesOf( + initialPhase?.reaches ?? [], + initialPhase?.pins ?? [], + initialPhase?.grips ?? [], + initialPhase?.groundLock ?? [], + ); + const initialPalmFloorSides = floorHandSidesOf( + initialPhase?.reaches ?? [], + initialPhase?.pins ?? [], + initialPhase?.groundLock ?? [], + ); // Scene props: tear down any previous set, build the declared ones, and // expose their anchors to reach-IK. if (propScene) { @@ -841,23 +926,28 @@ export function createViewer( applyBaseRoot(); timeline.sample(0, mannequin.bones); mannequin.root.updateMatrixWorld(true); + formFists(mannequin, initialFistSides, authoredFingers); + relaxHands( + mannequin, + unionHandSides(initialGripSides, initialFistSides), + authoredFingers, + initialPalmFloorSides, + ); + alignFloorContacts( + mannequin, + initialPhase?.reaches ?? [], + initialPhase?.pins ?? [], + initialPhase?.groundLock ?? [], + ); depenetrate(mannequin); groundFigureOf(mannequin); if (propScene) { resolvePropContacts(mannequin, propScene.colliders, contactExemptionsOf(ir.phases[0] ?? {})); } levelPlantedFeet(mannequin, ir.phases[0]?.groundLock ?? []); - authoredFingers = new Set(timeline.bonesUsed.filter(isFingerId)); - authoredShoulders = new Set(timeline.bonesUsed.filter((id) => id.startsWith("shoulder_"))); - authoredHead = timeline.bonesUsed.some((id) => id === "head" || id === "neck"); - swingArms(mannequin, authoredShoulders, gripSidesOf(ir.phases[0]?.grips ?? [])); - relaxHands( - mannequin, - gripSidesOf(ir.phases[0]?.grips ?? []), - authoredFingers, - floorHandSidesOf(ir.phases[0]?.reaches ?? [], ir.phases[0]?.pins ?? [], ir.phases[0]?.groundLock ?? []), - ); + swingArms(mannequin, authoredShoulders, initialConstrainedHandSides); applyLookAt({ grips: ir.phases[0]?.grips ?? [], reaches: ir.phases[0]?.reaches ?? [] }); + enforceContactRom(mannequin); captureGroundTargets(); baseRootPos.copy(mannequin.root.position); baseRootQuat.copy(mannequin.root.quaternion); @@ -872,8 +962,8 @@ export function createViewer( for (const bone of mannequin.bones.values()) bone.quaternion.identity(); const info = timeline.sample(seg.start, mannequin.bones); - const wasPinned = (id: string) => prevPins.some(p => (EFFECTOR_BONE[p.effector] ?? p.effector) === id && p.anchor === "floor"); - const isPinned = (id: string) => info.pins.some(p => (EFFECTOR_BONE[p.effector] ?? p.effector) === id && p.anchor === "floor"); + const wasPinned = (id: string) => prevPins.some(p => effectorBoneId(p.effector) === id && p.anchor === "floor"); + const isPinned = (id: string) => info.pins.some(p => effectorBoneId(p.effector) === id && p.anchor === "floor"); mannequin.root.position.copy(baseRootPos); mannequin.root.quaternion.copy(baseRootQuat); @@ -904,16 +994,76 @@ export function createViewer( prevPins = info.pins; } + // The raw FK pass above provides deterministic first-phase anchors, + // but a pin introduced later must inherit the contact position from + // the preceding phase *after* ground-lock, reach IK, ROM enforcement, + // and floor reconciliation. Otherwise a ground-lock→pin handoff snaps + // the whole body to the next phase's unsolved FK location (the original + // superhero knee planted abruptly; bridge lowers jumped by ~60 cm). + let previousSolvedEffectors: Map | null = null; + const activeFloorPinTargets = new Map(); + precomputingAnchors = true; + try { + for (let i = 0; i < timeline.segments.length; i++) { + const seg = timeline.segments[i]!; + const currentPins = new Set( + (ir.phases[i]?.pins ?? []) + .filter((pin) => pin.anchor === "floor") + .map((pin) => effectorBoneId(pin.effector)), + ); + for (const id of [...activeFloorPinTargets.keys()]) { + if (!currentPins.has(id)) activeFloorPinTargets.delete(id); + } + for (const id of currentPins) { + let anchor = activeFloorPinTargets.get(id); + if (!anchor) { + anchor = previousSolvedEffectors?.get(id)?.clone() + ?? segmentStartEffectors[i]?.get(id)?.clone(); + if (anchor) activeFloorPinTargets.set(id, anchor); + } + if (anchor) segmentStartEffectors[i]?.set(id, anchor.clone()); + } + + // Stay inside the segment because sample(duration) wraps to zero. + time = Math.max(seg.start, seg.end - 1e-5); + frame(); + previousSolvedEffectors = new Map(); + for (const ids of Object.values(mannequin.effectors)) { + for (const id of ids) { + const node = mannequin.bones.get(id); + if (node) previousSolvedEffectors.set(id, node.getWorldPosition(new THREE.Vector3())); + } + } + } + } finally { + precomputingAnchors = false; + time = 0; + } + // Restore initial pose for (const bone of mannequin.bones.values()) bone.quaternion.identity(); applyBaseRoot(); timeline.sample(0, mannequin.bones); + formFists(mannequin, initialFistSides, authoredFingers); + relaxHands( + mannequin, + unionHandSides(initialGripSides, initialFistSides), + authoredFingers, + initialPalmFloorSides, + ); + alignFloorContacts( + mannequin, + initialPhase?.reaches ?? [], + initialPhase?.pins ?? [], + initialPhase?.groundLock ?? [], + ); mannequin.root.position.copy(baseRootPos); mannequin.root.quaternion.copy(baseRootQuat); mannequin.root.updateMatrixWorld(true); depenetrate(mannequin); groundFigureOf(mannequin); levelPlantedFeet(mannequin, ir.phases[0]?.groundLock ?? []); + enforceContactRom(mannequin); } requestClip(ir); @@ -964,6 +1114,9 @@ export function createViewer( segments: timeline.segments, }; }, + getReachResiduals() { + return reachResiduals.map((residual) => ({ ...residual })); + }, getVisibleBounds() { return character?.getBounds() ?? new THREE.Box3().setFromObject(mannequin.root); }, @@ -1052,6 +1205,10 @@ function contactBoneIds(info: { if (effector === "feet" || effector === "foot_right") ids.add("ankle_right"); if (effector === "hands" || effector === "hand_left") ids.add("wrist_left"); if (effector === "hands" || effector === "hand_right") ids.add("wrist_right"); + if (effector === "fists" || effector === "fist_left") ids.add("wrist_left"); + if (effector === "fists" || effector === "fist_right") ids.add("wrist_right"); + if (effector === "knees" || effector === "knee_left") ids.add("knee_left"); + if (effector === "knees" || effector === "knee_right") ids.add("knee_right"); if (effector === "forearms") { ids.add("elbow_left"); ids.add("elbow_right"); @@ -1096,6 +1253,50 @@ function gripSidesOf(grips: readonly { effector: string }[]): Set<"left" | "righ return sides; } +/** Every hand side whose arm participates in an active contact constraint. */ +function contactHandSidesOf( + reaches: readonly { effector: string }[], + pins: readonly { effector: string }[], + grips: readonly { effector: string }[], + groundLock: readonly string[] = [], +): Set<"left" | "right"> { + const sides = gripSidesOf(grips); + const add = (effector: string): void => { + if (/^(?:hand|fist|elbow)_left$/.test(effector) + || effector === "hands" || effector === "fists" || effector === "forearms") sides.add("left"); + if (/^(?:hand|fist|elbow)_right$/.test(effector) + || effector === "hands" || effector === "fists" || effector === "forearms") sides.add("right"); + }; + reaches.forEach((contact) => add(contact.effector)); + pins.forEach((contact) => add(contact.effector)); + groundLock.forEach(add); + return sides; +} + +/** Hand sides whose semantic endpoint is a closed fist (at any target). */ +function fistSidesOf( + reaches: readonly { effector: string }[], + pins: readonly { effector: string }[], + groundLock: readonly string[] = [], +): Set<"left" | "right"> { + const sides = new Set<"left" | "right">(); + const add = (effector: string): void => { + if (effector === "fists" || effector === "fist_left") sides.add("left"); + if (effector === "fists" || effector === "fist_right") sides.add("right"); + }; + reaches.forEach((reach) => add(reach.effector)); + pins.forEach((pin) => add(pin.effector)); + groundLock.forEach(add); + return sides; +} + +function unionHandSides( + a: ReadonlySet<"left" | "right">, + b: ReadonlySet<"left" | "right">, +): Set<"left" | "right"> { + return new Set([...a, ...b]); +} + /** * Hand sides pressed onto the floor this phase — via `reach`/`pin: hands floor` * OR `ground-lock: hands` (a high plank / push-up / mountain-climber, where the @@ -1147,6 +1348,16 @@ export { applyGroundLock, groundFigure } from "./groundlock.js"; export type { Mannequin, Proportions, CollisionRadii } from "./mannequin.js"; export { buildTimeline } from "./timeline.js"; export { solveCCD, type IkChain, type JointLimits } from "./ik.js"; +export { + EFFECTOR_BONE, + REACH_TOLERANCE, + effectorBoneId, + reachChain, + solveReachToPoint, + missingReachTarget, + type ReachResidual, + type ReachResidualReason, +} from "./reach.js"; export { buildProps, type PropScene, type FaceCollider, type BlockedPart } from "./props.js"; export { resolvePropContacts, propContactExemptions, type PropContactExemptions } from "./propcontact.js"; export { loadCharacter, rigCharacter, type Character } from "./character.js"; @@ -1158,5 +1369,19 @@ export { type ClipSource, } from "./clips.js"; export { depenetrate } from "./depenetrate.js"; -export { alignFloorPalms, levelPlantedFeet } from "./contacts.js"; +export { + PALM_LOCAL_NORMAL, + FIST_LOCAL_NORMAL, + alignFloorContacts, + alignFloorPalms, + prepareGripFrames, + alignGripFrames, + isDipBarGrip, + floorTargetForEffector, + floorContactHeight, + enforceContactRom, + formFists, + levelPlantedFeet, + wrapGrip, +} from "./contacts.js"; export type { PhaseSegment } from "./timeline.js"; diff --git a/packages/posecode-render/src/mannequin.ts b/packages/posecode-render/src/mannequin.ts index e9e086d..dcaaf5c 100644 --- a/packages/posecode-render/src/mannequin.ts +++ b/packages/posecode-render/src/mannequin.ts @@ -24,6 +24,14 @@ export interface Mannequin { effectors: Record; /** Body-part radii for the self-collision pass (metres). */ collision: CollisionRadii; + /** Visible meshes that form semantic local support surfaces. */ + contactSurfaces: { + /** Trunk/mat surface used by `pin: pelvis floor`. */ + pelvis: THREE.Object3D[]; + /** Rounded thigh/shin caps that meet at each supporting knee. */ + knee_left: THREE.Object3D[]; + knee_right: THREE.Object3D[]; + }; } /** Capsule/sphere radii approximating the visible body for self-collision. */ @@ -185,6 +193,7 @@ export function buildMannequin(material?: THREE.Material, proportions?: Proporti root.name = "posecode-mannequin"; const bones = new Map(); + const jointCaps = new Map(); for (const spec of SKELETON) { const bone = new THREE.Object3D(); @@ -201,9 +210,25 @@ export function buildMannequin(material?: THREE.Material, proportions?: Proporti // Draw the segment from the parent joint to this joint, on the parent. if (spec.parent && spec.radius) { const mat = segmentMaterial(spec.id, mats); - const seg = isFinger(spec.id) - ? makeSegment(offset.length(), spec.radius, mats.skin) - : makeTaperedSegment(offset.length(), spec.radius, spec.radiusEnd ?? spec.radius, mat); + let seg: THREE.Object3D; + if (isFinger(spec.id)) { + seg = makeSegment(offset.length(), spec.radius, mats.skin); + } else { + const tapered = makeTaperedSegment( + offset.length(), + spec.radius, + spec.radiusEnd ?? spec.radius, + mat, + ); + seg = tapered.group; + const addCap = (joint: string, cap: THREE.Object3D): void => { + const caps = jointCaps.get(joint) ?? []; + caps.push(cap); + jointCaps.set(joint, caps); + }; + addCap(spec.parent, tapered.proximalCap); + addCap(spec.id, tapered.distalCap); + } orientSegment(seg, offset); bones.get(spec.parent)!.add(seg); } @@ -227,6 +252,23 @@ export function buildMannequin(material?: THREE.Material, proportions?: Proporti addShoe(bones.get("ankle_left")!, mats, proportions?.soleDrop); addShoe(bones.get("ankle_right")!, mats, proportions?.soleDrop); + // A pelvis floor pin represents the trunk's mat contact, whose thickness + // changes with orientation (supine/prone) and spinal pose. A fixed proxy + // radius cannot serve both a flat prone lower and a supine bridge. Record + // the actual trunk meshes so contact code can measure that surface without + // accidentally including descendant arms or legs in the pelvis subtree. + const boneIdByNode = new Map( + [...bones].map(([id, node]) => [node, id]), + ); + const trunkIds = new Set(["pelvis", "spine", "chest", "neck", "head"]); + const pelvisSurface: THREE.Object3D[] = []; + root.traverse((obj) => { + if (!(obj as THREE.Mesh).isMesh) return; + let owner = obj.parent; + while (owner && !boneIdByNode.has(owner)) owner = owner.parent; + if (owner && trunkIds.has(boneIdByNode.get(owner)!)) pelvisSurface.push(obj); + }); + return { root, bones, @@ -234,9 +276,15 @@ export function buildMannequin(material?: THREE.Material, proportions?: Proporti hands: ["wrist_left", "wrist_right"], hand_left: ["wrist_left"], hand_right: ["wrist_right"], + fists: ["wrist_left", "wrist_right"], + fist_left: ["wrist_left"], + fist_right: ["wrist_right"], forearms: ["elbow_left", "elbow_right"], elbow_left: ["elbow_left"], elbow_right: ["elbow_right"], + knees: ["knee_left", "knee_right"], + knee_left: ["knee_left"], + knee_right: ["knee_right"], feet: ["ankle_left", "ankle_right"], foot_left: ["ankle_left"], foot_right: ["ankle_right"], @@ -244,6 +292,11 @@ export function buildMannequin(material?: THREE.Material, proportions?: Proporti back: ["pelvis", "spine", "chest"], }, collision: proportions?.collision ?? DEFAULT_COLLISION, + contactSurfaces: { + pelvis: pelvisSurface, + knee_left: jointCaps.get("knee_left") ?? [], + knee_right: jointCaps.get("knee_right") ?? [], + }, }; } @@ -265,7 +318,11 @@ function makeTaperedSegment( rProx: number, rDist: number, mat: THREE.Material, -): THREE.Object3D { +): { + group: THREE.Group; + proximalCap: THREE.Mesh; + distalCap: THREE.Mesh; +} { const group = new THREE.Group(); // +Y end maps to the CHILD joint after orientSegment (dir = offset). const shaft = new THREE.Mesh(new THREE.CylinderGeometry(rDist, rProx, length, 22, 1), mat); @@ -276,7 +333,7 @@ function makeTaperedSegment( const capProx = new THREE.Mesh(new THREE.SphereGeometry(rProx, 22, 16), mat); capProx.position.y = -length / 2; group.add(capProx); - return group; + return { group, proximalCap: capProx, distalCap: capDist }; } /** Position/orient a +Y segment so it spans from the parent joint to `offset`. */ @@ -372,7 +429,11 @@ function addHead(head: THREE.Object3D, mats: FigureMaterials, headLength?: numbe /** A flattened palm instead of a ball: hands read as hands, not maracas. */ function addPalm(wrist: THREE.Object3D, mat: THREE.Material): void { - addEllipsoid(wrist, 0.045, [0.85, 1.05, 0.5], [0, -0.02, 0.004], mat); + // The ellipsoid's thinnest axis is local Z, so +Z is the procedural palm's + // outward face normal on BOTH sides. Contact orientation must use this mesh + // convention (not an anatomical left/right guess such as +/-X). + const palm = addEllipsoid(wrist, 0.045, [0.85, 1.05, 0.5], [0, -0.02, 0.004], mat); + palm.name = `palm_${wrist.name.replace("wrist_", "")}`; } /** diff --git a/packages/posecode-render/src/propcontact.ts b/packages/posecode-render/src/propcontact.ts index 1f7624f..2f8420a 100644 --- a/packages/posecode-render/src/propcontact.ts +++ b/packages/posecode-render/src/propcontact.ts @@ -1,13 +1,13 @@ /** - * Prop contact solving: props are solid, not just visual. + * Bounded prop-contact correction for selected blocking surfaces. * * Authored poses are pure joint rotations relative to the root, and the root * solvers (ground-lock, pins) only know about the floor and named anchors, so * nothing stopped a wall-sit's pelvis from hinging straight through the wall * or a sit-to-stand's torso from sinking into the chair's backrest. This pass * samples the body as capsules (torso, head, thighs, shins, forearms) against - * each prop's declared solid faces (`FaceCollider`, see props.ts) and removes - * any overlap: + * selected declared faces (`FaceCollider`, see props.ts) and reduces sampled + * overlap. This is deliberately not comprehensive collision detection: * * - **Body-resolved faces** (wall, backrest, seat edge) translate the WHOLE * figure along the face normal — the physical resolution of leaning into a @@ -21,7 +21,7 @@ * * Same principles as the self-collision pass (depenetrate.ts): minimal (a * pose with no overlap is untouched, contact settles ON the surface), - * deterministic (pure function of the pose), and ROM-safe. + * deterministic (pure function of the pose), and ROM-bounded. * * Runs after ground-lock / pins / grips (it must see the final root * placement) and before reach-IK (reached hands must not be dragged off @@ -127,8 +127,20 @@ export function propContactExemptions( for (const c of contacts) { if (c.anchor === "floor") continue; for (const side of ["left", "right"] as const) { - if (c.effector === "feet" || c.effector === `foot_${side}` || c.effector === `ankle_${side}`) legs.add(side); - if (c.effector === "hands" || c.effector === `hand_${side}` || c.effector === `wrist_${side}`) arms.add(side); + if ( + c.effector === "feet" || + c.effector === "knees" || + c.effector === `foot_${side}` || + c.effector === `ankle_${side}` || + c.effector === `knee_${side}` + ) legs.add(side); + if ( + c.effector === "hands" || + c.effector === "fists" || + c.effector === `hand_${side}` || + c.effector === `fist_${side}` || + c.effector === `wrist_${side}` + ) arms.add(side); } } return { legs, arms }; diff --git a/packages/posecode-render/src/props.ts b/packages/posecode-render/src/props.ts index 4fe981e..43434c6 100644 --- a/packages/posecode-render/src/props.ts +++ b/packages/posecode-render/src/props.ts @@ -147,7 +147,9 @@ export function buildProps(types: string[], material?: THREE.Material): PropScen // the floor. The single `bars` grip anchor sits at the midpoint between // the rails at grip height: pins translate the BODY so the average hand // position meets the anchor, which leaves each authored hand over its - // own rail. + // own rail. Side-specific anchors are required because grouped grips are + // resolved to `bars_left` / `bars_right`; falling back to the centre + // collapses both hands onto one point and twists the shoulders together. const railH = 1.1; const halfSpan = 0.22; for (const x of [-halfSpan, halfSpan]) { @@ -168,6 +170,8 @@ export function buildProps(types: string[], material?: THREE.Material): PropScen } } anchors.set("bars", new THREE.Vector3(0, railH, 0)); + anchors.set("bars_left", new THREE.Vector3(halfSpan, railH, 0)); + anchors.set("bars_right", new THREE.Vector3(-halfSpan, railH, 0)); } else if (type === "box") { // A low step/plateau placed IN FRONT of the figure (+Z): the lead foot // steps forward and up onto it. Top surface at ~0.30 m; `box` anchor sits diff --git a/packages/posecode-render/src/reach.ts b/packages/posecode-render/src/reach.ts new file mode 100644 index 0000000..761fb34 --- /dev/null +++ b/packages/posecode-render/src/reach.ts @@ -0,0 +1,170 @@ +/** Reach-effector resolution and ROM-constrained IK shared by the viewer/tests. */ +import * as THREE from "three"; +import { eulerRomFor } from "posecode-parser"; +import type { Mannequin } from "./mannequin.js"; +import { solveCCD, type JointLimits } from "./ik.js"; + +const DEG = Math.PI / 180; +const REACH_EULER = new THREE.Euler(); + +/** Friendly/canonical contact effectors mapped to their driven rig bone. */ +export const EFFECTOR_BONE: Readonly> = { + hand_left: "wrist_left", + hand_right: "wrist_right", + fist_left: "wrist_left", + fist_right: "wrist_right", + foot_left: "ankle_left", + foot_right: "ankle_right", + knee_left: "knee_left", + knee_right: "knee_right", +}; + +export type ReachResidualReason = + | "missing-effector" + | "missing-target" + | "unsupported-effector"; + +/** + * Runtime outcome for one active reach. `distance` is the actual post-blend + * world-space effector error in metres. A null distance means the target or + * effector could not be resolved; these cases are deliberately retained + * instead of being silently discarded by the renderer. + */ +export interface ReachResidual { + effector: string; + target: string; + weight: number; + distance: number | null; + reached: boolean; + reason?: ReachResidualReason; +} + +/** Viewer/testing tolerance for considering a terminal contact reached. */ +export const REACH_TOLERANCE = 0.03; + +/** Resolve a canonical effector name to the bone that occupies its endpoint. */ +export function effectorBoneId(effector: string): string { + return EFFECTOR_BONE[effector] ?? effector; +} + +/** + * The proximal-to-distal chain allowed to move a terminal effector. + * Knee contact rotates the hip only: the knee itself is the endpoint, not a + * joint that may rotate itself toward its own target. Fists share the arm + * chain with palms while retaining distinct contact geometry in contacts.ts. + */ +export function reachChain( + m: Mannequin, + effector: string, +): { joints: THREE.Object3D[]; limits: (JointLimits | null)[] } { + const boneId = effectorBoneId(effector); + const side = boneId.endsWith("_left") ? "left" : "right"; + const ids = boneId.startsWith("wrist") + ? [`shoulder_${side}`, `elbow_${side}`] + : boneId.startsWith("elbow") + ? [`shoulder_${side}`] + : boneId.startsWith("ankle") + ? [`hip_${side}`, `knee_${side}`] + : boneId.startsWith("knee") + ? [`hip_${side}`] + : []; + const joints: THREE.Object3D[] = []; + const limits: (JointLimits | null)[] = []; + for (const id of ids) { + const node = m.bones.get(id); + if (!node) continue; + joints.push(node); + limits.push(widenedJointLimits(id, node)); + } + return { joints, limits }; +} + +/** A bone's ROM in radians, widened only enough to preserve authored FK. */ +export function widenedJointLimits( + boneId: string, + node: THREE.Object3D, +): JointLimits | null { + const rom = eulerRomFor(boneId); + if (!rom) return null; + REACH_EULER.setFromQuaternion(node.quaternion, "XYZ"); + return { + x: widen(rom.x.min * DEG, rom.x.max * DEG, REACH_EULER.x), + y: widen(rom.y.min * DEG, rom.y.max * DEG, REACH_EULER.y), + z: widen(rom.z.min * DEG, rom.z.max * DEG, REACH_EULER.z), + }; +} + +function widen(min: number, max: number, current: number): [number, number] { + return [Math.min(min, current), Math.max(max, current)]; +} + +/** + * Solve one named effector to an already-resolved world point and report the + * residual after reach-weight blending. This low-level entry point lets tests + * and non-WebGL consumers exercise the exact viewer solve. + */ +export function solveReachToPoint( + m: Mannequin, + effectorName: string, + targetName: string, + target: THREE.Vector3, + weight = 1, +): ReachResidual { + const safeWeight = THREE.MathUtils.clamp(weight, 0, 1); + const effector = m.bones.get(effectorBoneId(effectorName)); + if (!effector) { + return { + effector: effectorName, + target: targetName, + weight: safeWeight, + distance: null, + reached: false, + reason: "missing-effector", + }; + } + const { joints, limits } = reachChain(m, effectorName); + if (joints.length === 0) { + return { + effector: effectorName, + target: targetName, + weight: safeWeight, + distance: null, + reached: false, + reason: "unsupported-effector", + }; + } + + const before = joints.map((joint) => joint.quaternion.clone()); + solveCCD({ joints, limits, effector, target }, 12); + for (let i = 0; i < joints.length; i++) { + const solved = joints[i]!.quaternion.clone(); + joints[i]!.quaternion.slerpQuaternions(before[i]!, solved, safeWeight); + } + m.root.updateMatrixWorld(true); + const distance = effector + .getWorldPosition(new THREE.Vector3()) + .distanceTo(target); + return { + effector: effectorName, + target: targetName, + weight: safeWeight, + distance, + reached: distance <= REACH_TOLERANCE, + }; +} + +/** Construct an explicit diagnostic for a target name that did not resolve. */ +export function missingReachTarget( + effector: string, + target: string, + weight = 1, +): ReachResidual { + return { + effector, + target, + weight: THREE.MathUtils.clamp(weight, 0, 1), + distance: null, + reached: false, + reason: "missing-target", + }; +} diff --git a/packages/posecode-render/src/timeline.ts b/packages/posecode-render/src/timeline.ts index fd3cad2..ed66850 100644 --- a/packages/posecode-render/src/timeline.ts +++ b/packages/posecode-render/src/timeline.ts @@ -8,12 +8,21 @@ */ import * as THREE from "three"; -import type { PosecodeIR, ReachTarget, PinTarget, GripTarget, TimingMode } from "posecode-parser"; +import type { + Axis, + PosecodeIR, + ReachTarget, + PinTarget, + GripTarget, + TimingMode, +} from "posecode-parser"; import { poseFor, type PoseSpec } from "./poses.js"; const DEG = Math.PI / 180; type EulerDegTuple = [number, number, number]; +const AXIS_INDEX: Record = { x: 0, y: 1, z: 2 }; +const ALL_AXES: Axis[] = ["x", "y", "z"]; interface Keyframe { time: number; @@ -228,7 +237,11 @@ export function buildTimeline(ir: PosecodeIR): BuiltTimeline { let t = 0; for (const phase of ir.phases) { for (const target of phase.targets) { - curr.set(target.boneId, [target.euler.x, target.euler.y, target.euler.z]); + const next = [...(curr.get(target.boneId) ?? [0, 0, 0])] as EulerDegTuple; + for (const axis of target.axes ?? ALL_AXES) { + next[AXIS_INDEX[axis]] = target.euler[axis]; + } + curr.set(target.boneId, next); } if (phase.turnDeg !== undefined) currYaw = phase.turnDeg; if (phase.travel) currPos = { x: phase.travel.x, z: phase.travel.z }; @@ -404,7 +417,10 @@ function snapshot(curr: Map): Map if (pelvisX !== 0) { for (const hip of ["hip_left", "hip_right"]) { const [hx, hy, hz] = curr.get(hip) ?? [0, 0, 0]; - out.set(hip, [hx - pelvisX, hy, hz]); + // Parser-produced IR already enforces this coupled limit. Clamp here too + // because PosecodeIR is public and hosts may construct legacy/manual IR + // without running the parser first. + out.set(hip, [THREE.MathUtils.clamp(hx - pelvisX, -135, 20), hy, hz]); } } return out; diff --git a/packages/posecode-render/test/contacts.test.ts b/packages/posecode-render/test/contacts.test.ts index cf968fd..67192fe 100644 --- a/packages/posecode-render/test/contacts.test.ts +++ b/packages/posecode-render/test/contacts.test.ts @@ -1,7 +1,20 @@ import { describe, it, expect } from "vitest"; import * as THREE from "three"; import { buildMannequin } from "../src/mannequin.js"; -import { levelPlantedFeet, relaxHands, swingArms, aimHead } from "../src/contacts.js"; +import { + FIST_LOCAL_NORMAL, + PALM_LOCAL_NORMAL, + alignFloorContacts, + alignGripFrames, + enforceContactRom, + floorTargetForEffector, + formFists, + levelPlantedFeet, + prepareGripFrames, + relaxHands, + swingArms, + aimHead, +} from "../src/contacts.js"; import { groundFigure } from "../src/groundlock.js"; const DEG = Math.PI / 180; @@ -65,20 +78,122 @@ describe("levelPlantedFeet", () => { }); }); +describe("contact geometry", () => { + const down = new THREE.Vector3(0, -1, 0); + + it("uses the flattened palm mesh's +Z face on both sides and stays in wrist ROM", () => { + const m = buildMannequin(); + for (const side of ["left", "right"] as const) { + const palm = m.bones.get(`wrist_${side}`)!.getObjectByName(`palm_${side}`) as THREE.Mesh; + palm.geometry.computeBoundingBox(); + const size = palm.geometry.boundingBox!.getSize(new THREE.Vector3()).multiply(palm.scale); + // Ground the axis convention in the actual mesh: Z is its shallow face. + expect(size.z).toBeLessThan(size.x); + expect(size.z).toBeLessThan(size.y); + // Twenty degrees of safe shoulder extension lets the wrist's 70deg + // extension ceiling reach a fully palm-down 90deg world frame. + m.bones.get(`shoulder_${side}`)!.rotation.x = 20 * DEG; + } + m.root.updateMatrixWorld(true); + + alignFloorContacts(m, [], [{ effector: "hands", anchor: "floor" }], []); + for (const side of ["left", "right"] as const) { + const wrist = m.bones.get(`wrist_${side}`)!; + const normal = new THREE.Vector3(...PALM_LOCAL_NORMAL) + .applyQuaternion(wrist.getWorldQuaternion(new THREE.Quaternion())) + .normalize(); + expect(normal.dot(down)).toBeGreaterThan(0.995); + const local = new THREE.Euler().setFromQuaternion(wrist.quaternion, "XYZ"); + expect(local.x).toBeLessThanOrEqual(70 * DEG + 1e-6); + expect(Math.abs(local.y)).toBeLessThan(1e-6); + expect(Math.abs(local.z)).toBeLessThan(1e-6); + } + }); + + it("plants a fist on its knuckles, distinctly from a palm, without losing curl", () => { + const m = buildMannequin(); + m.bones.get("wrist_left")!.rotation.x = -30 * DEG; + formFists(m, new Set(["left"])); + const curlBefore = m.bones.get("index_left")!.quaternion.clone(); + alignFloorContacts(m, [{ effector: "fist_left", target: "floor", weight: 1 }], [], []); + + const wristWorld = m.bones.get("wrist_left")!.getWorldQuaternion(new THREE.Quaternion()); + const knuckles = new THREE.Vector3(...FIST_LOCAL_NORMAL).applyQuaternion(wristWorld).normalize(); + const palm = new THREE.Vector3(...PALM_LOCAL_NORMAL).applyQuaternion(wristWorld).normalize(); + expect(knuckles.dot(down)).toBeGreaterThan(0.995); + expect(Math.abs(palm.dot(down))).toBeLessThan(0.1); + expect(m.bones.get("index_left")!.quaternion.angleTo(curlBefore)).toBeLessThan(1e-8); + expect(m.bones.get("index_left")!.rotation.x).toBeLessThan(-1); + }); + + it("targets the knee surface, not the bottom of its shin/foot subtree", () => { + const m = buildMannequin(); + m.root.updateMatrixWorld(true); + const wristBefore = m.bones.get("wrist_left")!.quaternion.clone(); + const target = floorTargetForEffector(m, "knee_left"); + expect(target).not.toBeNull(); + expect(m.contactSurfaces.knee_left).toHaveLength(2); + expect(target!.y).toBeGreaterThan(0.05); + expect(target!.y).toBeLessThan(0.07); + alignFloorContacts(m, [{ effector: "knee_left", target: "floor", weight: 1 }], [], []); + expect(m.bones.get("wrist_left")!.quaternion.angleTo(wristBefore)).toBeLessThan(1e-8); + }); + + it("targets the local pelvis surface instead of its whole body subtree", () => { + const m = buildMannequin(); + m.root.updateMatrixWorld(true); + const target = floorTargetForEffector(m, "pelvis"); + expect(target).not.toBeNull(); + expect(m.contactSurfaces.pelvis.length).toBeGreaterThan(0); + expect(target!.y).toBeGreaterThan(0.08); + expect(target!.y).toBeLessThan(0.16); + }); + + it("strictly clears forbidden ankle Y/Z and clamps terminal joints", () => { + const m = buildMannequin(); + m.bones.get("ankle_left")!.rotation.set(0.2, 0.5, -0.4); + m.bones.get("wrist_right")!.rotation.set(0.2, 0.3, -0.8); + enforceContactRom(m); + const ankle = new THREE.Euler().setFromQuaternion(m.bones.get("ankle_left")!.quaternion, "XYZ"); + const wrist = new THREE.Euler().setFromQuaternion(m.bones.get("wrist_right")!.quaternion, "XYZ"); + expect(Math.abs(ankle.y)).toBeLessThan(1e-7); + expect(Math.abs(ankle.z)).toBeLessThan(1e-7); + expect(ankle.x).toBeLessThanOrEqual(50 * DEG + 1e-6); + expect(wrist.x).toBeLessThanOrEqual(70 * DEG + 1e-6); + expect(Math.abs(wrist.y)).toBeLessThan(1e-7); + expect(wrist.z).toBeGreaterThanOrEqual(-20 * DEG - 1e-6); + expect(wrist.z).toBeLessThanOrEqual(30 * DEG + 1e-6); + }); + + it("builds a stable palm-down dip-bar frame", () => { + const m = buildMannequin(); + m.bones.get("shoulder_left")!.rotation.x = 20 * DEG; + m.bones.get("elbow_left")!.rotation.y = 80 * DEG; + const grips = [{ effector: "hand_left", anchor: "bars_left" }]; + prepareGripFrames(m, grips); + expect(Math.abs(m.bones.get("elbow_left")!.rotation.y)).toBeLessThan(1e-7); + alignGripFrames(m, grips); + const normal = new THREE.Vector3(...PALM_LOCAL_NORMAL).applyQuaternion( + m.bones.get("wrist_left")!.getWorldQuaternion(new THREE.Quaternion()), + ); + expect(normal.dot(down)).toBeGreaterThan(0.995); + }); +}); + describe("relaxHands (L4.1)", () => { it("curls the fingers of an idle, un-authored hand into a natural rest", () => { const m = buildMannequin(); expect(m.bones.get("index_left")!.rotation.x).toBeCloseTo(0, 5); // flat at rest relaxHands(m, new Set(), new Set()); - expect(m.bones.get("index_left")!.rotation.x).toBeGreaterThan(0.1); - expect(m.bones.get("middle_right")!.rotation.x).toBeGreaterThan(0.1); + expect(m.bones.get("index_left")!.rotation.x).toBeLessThan(-0.1); + expect(m.bones.get("middle_right")!.rotation.x).toBeLessThan(-0.1); }); it("leaves a gripping hand for wrapGrip (skips grip sides)", () => { const m = buildMannequin(); relaxHands(m, new Set(["left"]), new Set()); expect(m.bones.get("index_left")!.rotation.x).toBeCloseTo(0, 5); // untouched - expect(m.bones.get("index_right")!.rotation.x).toBeGreaterThan(0.1); // right relaxed + expect(m.bones.get("index_right")!.rotation.x).toBeLessThan(-0.1); // right relaxed }); it("does not override an explicitly authored finger", () => { @@ -93,11 +208,11 @@ describe("relaxHands (L4.1)", () => { // A free hand takes the soft inward hook... relaxHands(m, new Set(), new Set(), new Set()); const freeCurl = m.bones.get("index_left")!.rotation.x; - expect(freeCurl).toBeGreaterThan(0.3); + expect(freeCurl).toBeLessThan(-0.3); // ...but a hand pressed to the floor (plank/push-up) lies extended. relaxHands(m, new Set(), new Set(), new Set(["left"])); expect(m.bones.get("index_left")!.rotation.x).toBeLessThan(0.1); // flat - expect(m.bones.get("index_right")!.rotation.x).toBeGreaterThan(0.3); // right still hooked + expect(m.bones.get("index_right")!.rotation.x).toBeLessThan(-0.3); // right still hooked }); }); diff --git a/packages/posecode-render/test/propcontact.test.ts b/packages/posecode-render/test/propcontact.test.ts index c6531e3..fdd983f 100644 --- a/packages/posecode-render/test/propcontact.test.ts +++ b/packages/posecode-render/test/propcontact.test.ts @@ -91,11 +91,11 @@ describe("solid prop contact", () => { it("maps effector spellings and floor anchors correctly in propContactExemptions", () => { const e = propContactExemptions([ - { effector: "feet", anchor: "box" }, - { effector: "hand_left", anchor: "bar" }, + { effector: "knee_left", anchor: "seat" }, + { effector: "fist_right", anchor: "wall" }, { effector: "foot_right", anchor: "floor" }, // floor: not a prop contact ]); - expect([...e.legs].sort()).toEqual(["left", "right"]); - expect([...e.arms]).toEqual(["left"]); + expect([...e.legs]).toEqual(["left"]); + expect([...e.arms]).toEqual(["right"]); }); }); diff --git a/packages/posecode-render/test/render.test.ts b/packages/posecode-render/test/render.test.ts index a2b219f..aad7e5f 100644 --- a/packages/posecode-render/test/render.test.ts +++ b/packages/posecode-render/test/render.test.ts @@ -7,6 +7,7 @@ import { poseFor } from "../src/poses.js"; import { buildProps } from "../src/props.js"; import { applyGroundLock, groundFigure } from "../src/groundlock.js"; import { levelPlantedFeet, wrapGrip } from "../src/contacts.js"; +import { effectorBoneId, missingReachTarget, solveReachToPoint } from "../src/reach.js"; import { parse, eulerRomFor } from "posecode-parser"; const DEG = Math.PI / 180; @@ -36,6 +37,11 @@ describe("mannequin", () => { expect(m.effectors.feet).toEqual(["ankle_left", "ankle_right"]); expect(m.effectors.foot_right).toEqual(["ankle_right"]); expect(m.effectors.back).toEqual(["pelvis", "spine", "chest"]); + expect(m.effectors.fist_left).toEqual(["wrist_left"]); + expect(m.effectors.fists).toEqual(["wrist_left", "wrist_right"]); + expect(m.effectors.knees).toEqual(["knee_left", "knee_right"]); + expect(effectorBoneId("fist_right")).toBe("wrist_right"); + expect(effectorBoneId("knee_left")).toBe("knee_left"); }); }); @@ -368,6 +374,45 @@ describe("reach-IK", () => { expect(after).toBeLessThan(0.06); }); + + it("solves a canonical knee effector through the hip and reports residual", () => { + const m = buildMannequin(); + m.root.updateMatrixWorld(true); + const hip = m.bones.get("hip_left")!; + const knee = m.bones.get("knee_left")!; + const hipPos = hip.getWorldPosition(new THREE.Vector3()); + const radius = knee.getWorldPosition(new THREE.Vector3()).sub(hipPos); + const target = hipPos.clone().add(radius.applyAxisAngle(new THREE.Vector3(1, 0, 0), -0.2)); + const kneeBefore = knee.quaternion.clone(); + + const residual = solveReachToPoint(m, "knee_left", "world", target); + + expect(residual.reason).toBeUndefined(); + expect(residual.distance).not.toBeNull(); + expect(residual.distance!).toBeLessThan(0.04); + expect(residual.reached).toBe(true); + // The endpoint knee does not rotate itself; its hip is the reach chain. + expect(knee.quaternion.angleTo(kneeBefore)).toBeLessThan(1e-8); + expect(hip.quaternion.angleTo(new THREE.Quaternion())).toBeGreaterThan(0.05); + }); + + it("retains an unreachable fist residual and an unresolved target diagnostic", () => { + const m = buildMannequin(); + m.root.updateMatrixWorld(true); + const far = new THREE.Vector3(5, 5, 5); + const residual = solveReachToPoint(m, "fist_right", "far", far); + expect(residual.distance).not.toBeNull(); + expect(residual.distance!).toBeGreaterThan(5); + expect(residual.reached).toBe(false); + + expect(missingReachTarget("fist_right", "missing_anchor")).toMatchObject({ + effector: "fist_right", + target: "missing_anchor", + distance: null, + reached: false, + reason: "missing-target", + }); + }); }); describe("hand rig", () => { @@ -568,13 +613,11 @@ describe("contact pins", () => { ' step "Hang" 1.5s ease-in-out:', " shoulders: flex 175", " elbows: flex 5", - " pin: hand_left bar", - " pin: hand_right bar", + " pin: hands bar", ' step "Pull up" 1.2s ease-out:', " shoulders: flex 150", " elbows: flex 130", - " pin: hand_left bar", - " pin: hand_right bar", + " pin: hands bar", " repeat 2", ].join("\n"); @@ -675,7 +718,7 @@ describe("ROM-constrained reach-IK", () => { const { joints, limits, effector } = armChain(m); const [shoulder, elbow] = joints; // High behind the back (figure faces +Z): reaching it would demand - // shoulder extension far past the 60° healthy ceiling. + // shoulder extension far past the configured 60° ceiling. const target = shoulder! .getWorldPosition(new THREE.Vector3()) .add(new THREE.Vector3(0, 0.3, -0.5)); @@ -709,7 +752,7 @@ describe("ROM-constrained reach-IK", () => { free.joints[0]!.quaternion, "XYZ", ); - expect(eFree.x).toBeGreaterThan(60 * DEG + 0.05); // past healthy extension + expect(eFree.x).toBeGreaterThan(60 * DEG + 0.05); // past configured extension }); it("keeps a knee hinge-only: no lateral splay while a foot reaches", () => { @@ -813,6 +856,10 @@ describe("dip bars prop", () => { expect(grip.y).toBeGreaterThan(0.9); // high enough that feet clear the floor expect(grip.y).toBeLessThan(1.6); // but well below the pull-up bar expect(group.children.length).toBeGreaterThan(0); + expect(anchors.get("bars_left")!.x).toBeGreaterThan(0); + expect(anchors.get("bars_right")!.x).toBeLessThan(0); + expect(anchors.get("bars_left")!.y).toBeCloseTo(grip.y, 6); + expect(anchors.get("bars_left")!.distanceTo(anchors.get("bars_right")!)).toBeGreaterThan(0.4); }); // Same root-translation logic as the viewer's applyPins (see contact pins). @@ -1011,9 +1058,11 @@ describe("bar grip (L3.2)", () => { const restIndex = m.bones.get("index_left")!.rotation.x; const restThumb = m.bones.get("thumb_left")!.rotation.x; wrapGrip(m, [{ effector: "hand_left", anchor: "bar_left" }]); - expect(m.bones.get("index_left")!.rotation.x).toBeGreaterThan(restIndex + 0.5); - expect(m.bones.get("middle_left")!.rotation.x).toBeGreaterThan(0.5); - expect(m.bones.get("thumb_left")!.rotation.x).toBeGreaterThan(restThumb + 0.3); + // Finger flexion is local -X in the rig/parser ROM convention. The former + // positive renderer curl was actually extension (digits bent off the palm). + expect(m.bones.get("index_left")!.rotation.x).toBeLessThan(restIndex - 0.5); + expect(m.bones.get("middle_left")!.rotation.x).toBeLessThan(-0.5); + expect(m.bones.get("thumb_left")!.rotation.x).toBeLessThan(restThumb - 0.3); // the un-gripped right hand is untouched expect(m.bones.get("index_right")!.rotation.x).toBeCloseTo(0, 5); }); diff --git a/packages/posecode-render/test/timeline-sparse.test.ts b/packages/posecode-render/test/timeline-sparse.test.ts new file mode 100644 index 0000000..587b244 --- /dev/null +++ b/packages/posecode-render/test/timeline-sparse.test.ts @@ -0,0 +1,75 @@ +import { describe, expect, it } from "vitest"; +import * as THREE from "three"; +import { parse, type PosecodeIR } from "posecode-parser"; +import { buildMannequin } from "../src/mannequin.js"; +import { buildTimeline } from "../src/timeline.js"; + +const DEG = 180 / Math.PI; + +function eulerDegrees(node: THREE.Object3D): THREE.Euler { + return new THREE.Euler().setFromQuaternion(node.quaternion, "XYZ"); +} + +describe("sparse timeline targets", () => { + it("preserves an unauthored standing-pose forearm rotation", () => { + const result = parse([ + 'posecode posture "Curl"', + " rig humanoid", + " pose start = standing", + ' step "Curl" 1s settle:', + " elbow_left: flex 30", + ].join("\n")); + const timeline = buildTimeline(result.ir!); + const mannequin = buildMannequin(); + timeline.sample(1, mannequin.bones); + const euler = eulerDegrees(mannequin.bones.get("elbow_left")!); + expect(euler.x * DEG).toBeCloseTo(-30, 4); + expect(euler.y * DEG).toBeCloseTo(80, 4); + }); + + it("carries flexion when a later phase authors only axial rotation", () => { + const result = parse([ + 'posecode exercise "Curl and turn"', + " rig humanoid", + ' step "Curl" 1s flow:', + " elbow_right: flex 60", + ' step "Turn" 1s settle:', + " elbow_right: pronate 35", + ].join("\n")); + const timeline = buildTimeline(result.ir!); + const mannequin = buildMannequin(); + timeline.sample(2, mannequin.bones); + const euler = eulerDegrees(mannequin.bones.get("elbow_right")!); + expect(euler.x * DEG).toBeCloseTo(-60, 4); + expect(euler.y * DEG).toBeCloseTo(-35, 4); + }); + + it("defensively clamps coupled hips in manually constructed legacy IR", () => { + const ir: PosecodeIR = { + version: "0.2", + kind: "posture", + name: "Legacy fold", + rig: "humanoid", + props: [], + repeat: 1, + phases: [{ + name: "Fold", + durationSec: 1, + easing: "linear", + targets: [ + { boneId: "pelvis", euler: { x: 50, y: 0, z: 0 } }, + { boneId: "hip_left", euler: { x: -110, y: 0, z: 0 } }, + ], + groundLock: [], + reaches: [], + pins: [], + grips: [], + }], + }; + const timeline = buildTimeline(ir); + const mannequin = buildMannequin(); + timeline.sample(1, mannequin.bones); + const hip = eulerDegrees(mannequin.bones.get("hip_left")!); + expect(hip.x * DEG).toBeCloseTo(-135, 4); + }); +}); diff --git a/playground/index.html b/playground/index.html index 9d58b63..72a879a 100644 --- a/playground/index.html +++ b/playground/index.html @@ -8,7 +8,7 @@ @@ -31,7 +31,7 @@ @@ -75,9 +75,9 @@ "url": "https://www.posecode.org/", "applicationCategory": "DeveloperApplication", "operatingSystem": "Any (web browser)", - "description": "Posecode is a tiny language LLMs can write to describe human movement: exercises, stretches, physiotherapy, yoga, and dance. The browser renders it as an animated 3D figure, with every joint clamped to a safe range of motion.", + "description": "Posecode is a small language capable LLMs can use to describe single-person movement. The browser validates a closed vocabulary, constrains joint targets to configured ranges, and renders an inspectable animated 3D figure.", "offers": { "@type": "Offer", "price": "0", "priceCurrency": "USD" }, - "license": "https://opensource.org/license/mit", + "license": "https://www.apache.org/licenses/LICENSE-2.0", "sameAs": ["https://github.com/posecode-dev/posecode"], "keywords": "text to motion, LLM, kinematic motion, 3D animation, exercise, physiotherapy, open source" }, @@ -90,7 +90,7 @@ "name": "What is Posecode?", "acceptedAnswer": { "@type": "Answer", - "text": "Posecode is an open-source text-to-motion language. You describe a human movement in plain text (phases, joint angles, timing) and the browser renders it as an animated 3D figure. It is designed so LLMs like ChatGPT, Claude, and Gemini can write it reliably." + "text": "Posecode is an open-source text-to-motion language. You describe single-person movement as phases, joint targets, timing, and contacts; the browser validates the document and renders it as an animated 3D figure." } }, { @@ -98,15 +98,15 @@ "name": "How do I generate an exercise animation with an LLM?", "acceptedAnswer": { "@type": "Answer", - "text": "Copy the Posecode LLM prompt from posecode.org, paste it into any chat model, and ask for a movement such as a deadlift or a hamstring stretch. Paste the model's .posecode reply into the playground and it animates instantly." + "text": "Copy the authoring guide into a capable chat model and ask for a representable movement such as a hip hinge or hamstring stretch. For a representable request, the model should return raw .posecode text. Paste it into the playground: valid documents animate, while parse errors, range clamps, and unresolved reach targets appear as diagnostics." } }, { "@type": "Question", - "name": "Is the movement anatomically safe?", + "name": "Does Posecode guarantee a movement is safe?", "acceptedAnswer": { "@type": "Answer", - "text": "Every joint angle is hard-clamped to a clinical range of motion, so even a hallucinated angle cannot produce an impossible pose. Range-of-motion values are general literature data, not medical advice." + "text": "No. Posecode constrains authored and IK-generated joint targets to configured reference ranges and reports unresolved reach targets. A simplified rig cannot certify that a whole movement is safe, accurate, or appropriate for a person. It is not medical advice." } }, { @@ -114,7 +114,7 @@ "name": "Is Posecode free to use?", "acceptedAnswer": { "@type": "Answer", - "text": "Yes. Posecode is Apache-2.0-licensed and open source: the parser, 3D renderer, share codec, MCP server, and VS Code extension are all free, and rendering runs client-side with no GPU required." + "text": "Yes. Posecode is Apache-2.0-licensed and open source: the parser, 3D renderer, share codec, MCP server, and VS Code extension are all free, and rendering runs locally in a WebGL-capable browser without server-side motion generation." } } ] @@ -151,7 +151,7 @@

Write the move.
Watch it happen.

- Turn readable instructions into precise 3D movement. Build exercises, + Turn readable instructions into structured, inspectable 3D motion. Build exercises, stretches, dance phrases, and rehab routines, then edit every beat live in the browser.

@@ -161,30 +161,32 @@

Copy LLM prompt -

- 65+ moves included  ·  60 fps  ·  open source  ·  runs on-device +

+ 48 launch-ready moves  ·  25 experimental previews  ·  validated text  ·  open source

-
+
-
Out
+
Drop into the landing
- -
@@ -195,23 +197,23 @@

Semantic, not matrices

Models write phases and joint angles, like knees: flex 95, - instead of 3D transforms. It's the biomechanics an LLM already knows, given a syntax. + instead of 3D transforms. The closed syntax makes the intended anatomy inspectable.

-

Anatomically safe

+

Range-aware & testable

- Every angle is hard-clamped to a clinical range of motion. A hallucinated - “knee flex 200°” can't produce an impossible joint: it's pinned to 144°. + Joint targets are constrained to configured reference ranges, while reach + residuals expose a fist, foot, or knee that remains short of its target.

Cheap & client-side

- Generation is a fraction of a cent of text. Rendering is forward kinematics in - Three.js: 60fps on a phone, no diffusion model, no GPU farm. + Generation is compact text. Rendering is forward kinematics and constrained IK + in Three.js on the user's device, with no server-side diffusion pipeline.

@@ -223,7 +225,7 @@

Cheap & client-side

One language, many practices

A taste of the library, from physiotherapy and posture to yoga, dance, and - sign language. Every movement opens in the playground, editable and + martial arts. Every movement opens in the playground, editable and shareable.

@@ -239,18 +241,18 @@

From a sentence to a moving figure

  1. 1 -

    An LLM writes .posecode

    -

    Paste the Posecode prompt into any model and ask for a movement. It replies with a small, readable document.

    +

    A capable model writes .posecode

    +

    Give the Posecode authoring guide to a capable model and ask for a representable movement. For a supported request, its raw .posecode output remains editable and reviewable.

  2. 2

    The parser validates & clamps

    -

    A pure-TypeScript parser turns the text into a typed IR and pins every angle to its safe range of motion.

    +

    A TypeScript parser enforces the closed vocabulary, joint/action compatibility, contacts, and configured range limits.

  3. 3

    Three.js renders it

    -

    Forward kinematics plus ground-lock IK animate a low-poly figure live in the browser, at 60fps.

    +

    Forward kinematics plus constrained contact IK animate a low-poly figure live in the browser.

@@ -268,8 +270,8 @@

Playground →

MCP server

-

Let any agent validate a movement and get back a render link, natively.

- npx tsx posecode-mcp +

Let a model in an MCP client validate a movement and get back a render link.

+ npx -y posecode-mcp

VS Code extension

@@ -292,27 +294,28 @@

Common questions

What is Posecode?
- Posecode is an open-source text-to-motion language. You describe a human - movement in plain text (phases, joint angles, timing) and the browser renders - it as an animated 3D figure. It is designed so LLMs like ChatGPT, Claude, and - Gemini can write it reliably. + Posecode is an open-source text-to-motion language. You describe single-person + movement as phases, joint targets, timing, and contacts; the browser validates + the document and renders it as an animated 3D figure.
How do I generate an exercise animation with an LLM?
- Copy the LLM prompt above, paste it into any chat model, and ask for a - movement such as “write a deadlift” or “a hamstring stretch”. - Paste the model's .posecode reply into the playground and it - animates instantly. + Copy the authoring guide into a capable chat model and ask for a representable + movement such as a hip hinge or hamstring stretch. For a representable request, + the model should return raw .posecode text. Paste it into the + playground: valid documents animate, while parse errors, range clamps, and + unresolved reach targets appear as diagnostics.
-
Is the movement anatomically safe?
+
Does Posecode guarantee a movement is safe?
- Every joint angle is hard-clamped to a clinical range of motion, so even a - hallucinated angle cannot produce an impossible pose. Range-of-motion values - are general literature data, not medical advice. + No. Posecode constrains authored and IK-generated joint targets to configured + reference ranges and reports unresolved reach targets. A simplified rig cannot + certify that a whole movement is safe, accurate, or appropriate for a person. + It is not medical advice.
@@ -320,7 +323,7 @@

Common questions

Yes. Posecode is Apache-2.0-licensed and open source: the parser, 3D renderer, share codec, MCP server, and VS Code extension are all free, and rendering runs - client-side with no GPU required. + locally in a WebGL-capable browser without server-side motion generation.
diff --git a/playground/package.json b/playground/package.json index 52036b3..e320846 100644 --- a/playground/package.json +++ b/playground/package.json @@ -6,6 +6,7 @@ "type": "module", "scripts": { "dev": "vite", + "prebuild": "node ../scripts/generate-content-pages.mjs", "build": "vite build", "preview": "vite preview" }, diff --git a/playground/play.html b/playground/play.html index ce48ac3..398d4e8 100644 --- a/playground/play.html +++ b/playground/play.html @@ -17,14 +17,14 @@ /> @@ -75,6 +75,7 @@ > Movement + @@ -203,7 +204,7 @@

How to use Posecode

1
Copy the prompt. Click Copy LLM prompt (top right). It teaches - any model (ChatGPT, Claude, Gemini) to write the Posecode language. + a capable model how to write the Posecode language.
  • @@ -211,15 +212,16 @@

    How to use Posecode

    Ask an LLM for a movement. Paste the prompt, then ask for what you want: “write a deadlift”, “a hamstring stretch”, “cat-cow”. - It replies with a .posecode block. + For a representable request, it should reply with only the raw + .posecode document.
  • 3
    - Paste it here. Drop the .posecode text into the editor on the - left and watch it animate. Out-of-range angles are clamped automatically and - flagged below the editor. + Paste it here. Drop the raw .posecode text into the editor on + the left. Valid documents animate; parse errors, range clamps, and unresolved + reach targets appear below the editor.
  • @@ -248,6 +250,7 @@

    Movement library

    aria-label="Search movements" autocomplete="off" /> +
    diff --git a/playground/public/llm-guide.html b/playground/public/llm-guide.html index 57ccf2a..7c637d9 100644 --- a/playground/public/llm-guide.html +++ b/playground/public/llm-guide.html @@ -6,11 +6,11 @@ How to Teach an LLM to Write Posecode: The Authoring Guide - + - + @@ -122,7 +122,9 @@

    Authoring .posecode with an LLM

    Paste the prompt below into ChatGPT, Claude, or any capable model. Then ask for a movement ("write a squat", "show a hamstring stretch") and paste the reply into the Posecode playground.


    -

    You write Posecode, a small text language that describes a single person's movement so a 3D mannequin can animate it. Output ONLY a .posecode document in a code block, no prose.

    +

    You write Posecode, a small text language that describes one person's movement so a 3D figure can animate it.

    +

    When the request is representable, output ONLY the raw .posecode document: no Markdown fence and no prose. Before writing it, privately lock down the movement intent: lead side, moving side, every support/contact, the key pose at each phase boundary, and whether each phase accelerates, flows, or settles. Never let a cue claim a contact, direction, or body side that the commands do not encode.

    +

    If the request requires an unsupported capability—free flight, two people, arbitrary equipment, exact sign language, or detailed scapular/facial motion—do not fabricate a convincing-sounding document. Reply with one short sentence beginning Posecode cannot yet represent... and name the missing capability.

    Grammar

    posecode <kind> "<Name>"          # kind = exercise | stretch | posture
       rig humanoid
    @@ -130,8 +132,10 @@ 

    Grammar

    pose start = <pose> # neutral | standing | plank | supine | prone | seated step "<Phase name>" <Ns> <mode>: # mode = flow | settle | drive | snap | linear <joint>: <action> <degrees> - reach: <effector> <target> # optional: drive a hand/foot to a target via IK - ground-lock: <contacts> # groups/back, foot_left, or natural left foot + reach: <effector> <target> # limb IK to a landmark, floor, or declared prop anchor + pin: <effector> <anchor> # move the body while one primary contact stays fixed + grip: hands <anchor> # two-hand grip on a declared bar / rails + ground-lock: <contacts> # planted supports; groups/back or side-specific names turn: <degrees> # optional: face this yaw by phase end (standing only) travel: <x> <z> # optional: move to this x z (metres) by phase end cue "<short coaching cue>" @@ -139,23 +143,14 @@

    Grammar

    Joints

    neck head spine chest pelvis and (singular or plural) shoulders elbows wrists hips knees ankles. Plural names move both sides symmetrically; use elbow_left etc. for one side. Fingers: fingers (or fingers_left / fingers_right), and individually thumb_* index_* middle_* ring_* pinky_*.

    Actions (degrees are absolute targets)

    -
    • flex / extend: bend / straighten (sagittal)
    • abduct / adduct: away from / toward midline (frontal)
    • rotate-in / rotate-out: internal / external rotation
    • supinate / pronate: forearm turn (palm up / down)
    • dorsiflex / plantarflex: ankle up / down
    • hinge: hip hinge (on pelvis only): tip the torso forward over the
    -

    hips with a flat back, legs staying planted. Use this, not spinal flex, for a deadlift, bent-over row, good-morning, or a bow.

    -
    • hold neutral: keep the joint at rest
    +
    • flex / extend: bend / straighten (sagittal)
    • abduct / adduct: away from / toward midline (frontal)
    • rotate-in / rotate-out: internal / external rotation of a shoulder or hip
    • twist-left / twist-right: turn an axial joint (spine, chest, neck, or head) toward the named side
    • supinate / pronate: forearm turn (palm up / down)
    • dorsiflex / plantarflex: ankle up / down
    • hinge: hip hinge (on pelvis only): tip the torso forward over the hips with a flat back, legs staying planted. Use this, not spinal flex, for a deadlift, bent-over row, good-morning, or a bow.
    • hold neutral: set every channel on that joint to its rest value (no angle)
    +

    Use only anatomically compatible pairs. Examples: knees take flex/extend, ankles take dorsiflex/plantarflex, elbows take flex/extend/pronate/supinate, wrists take flex/extend/abduct/adduct, and hinge belongs only to the pelvis. The validator rejects globally-known actions on the wrong joint; never use the absence of a warning as permission to invent a pairing.

    Rules

    -
    1. Break the movement into 2–5 concurrent phases; each step is one phase.
    2. Set the joints that actually move in that phase; unset joints hold their
    -

    previous value.

    -
    1. Stay within healthy range of motion (e.g. knee flex ≤ 144°, elbow flex ≤
    -

    154°, shoulder flex ≤ 180°). The renderer hard-clamps anything beyond it.

    -
    1. Add a one-line cue per phase. Ground contacts are a closed vocabulary:
    -

    use ground-lock: feet when standing, ground-lock: hands, feet in a high plank, ground-lock: forearms, feet in a forearm plank, and ground-lock: back for supine floor work such as a dead bug. Do not invent other contact names.

    -
    1. repeat the rep count.
    +
    1. Break the movement into 2–6 concurrent phases; each step is the time taken to arrive at one key pose. A duration is not a dwell.
    2. Set only the channels that change. Unset channels hold their previous value; hold neutral deliberately resets the whole joint.
    3. Stay within Posecode's configured range-of-motion bounds (for example, knee flex ≤ 144°, elbow flex ≤ 154°, shoulder flex ≤ 180°). These bounds constrain the rig; they do not certify a movement as safe. Do not stack a pelvis hinge and hip flexion past the hip's combined limit.
    4. Choose one lead/trail convention and keep it across joint targets, contacts, phase names, and cues. Mirror all four together when switching sides.
    5. Declare every load-bearing contact in every phase where it remains active. Contacts do not inherit. Use side-specific support for lunges, kneeling, and single-leg work; ground-lock: feet is for two genuinely planted feet. Ground contacts are a closed vocabulary: use ground-lock: hands, feet in a high plank, ground-lock: forearms, feet in a forearm plank, and ground-lock: back for supine floor work. Do not invent contact names.
    6. Use ground-lock only for floor supports already meant to stay planted. Use reach to move a limb to a target, pin when a single contact must move the whole body, and grip for a bar or rails. Never combine ground-lock, pin, or grip root solvers in one step; use one primary support plus per-limb reach constraints for the remaining contacts.
    7. Author the gross body position before adding reach. A hand cannot reach a floor that the torso/legs leave outside the arm's reachable workspace.
    8. Derive each cue from the actual commands. Remove phrases such as “foot forward,” “knee down,” “fist planted,” or “arm overhead” unless that exact side and constraint are encoded.
    9. Use linear only for an unchanged dwell or intentionally mechanical motion. A moving phase that arrives at a landing or hold should normally settle.
    10. Add an explicit unchanged step for a visible hold, repeating its active contacts. Then author a controlled recovery when the movement should loop.
    11. Declare a prop before using its anchor; never invent target names.
    12. Set repeat to the requested repetition count.
    +

    Before returning the document, privately run this final check:

    +
    • Every word comes from the closed vocabulary, and every action is compatible with its joint.
    • Each cue can be traced to a joint target or an active contact on the same body side.
    • Every required support is declared in every phase where it remains active.
    • The gross pose makes each reach plausible; a reach line is not permission to leave the target outside the limb's workspace.
    • The final hold and recovery are explicit, warning-free, and do not rely on an undeclared prop or unsupported physics.

    Timing modes

    -
    • flow: carry momentum through an interior pose; use for continuous dance,
    -

    locomotion, and multi-part sports motion.

    -
    • settle: decelerate into a real rest; use at a squat bottom, landing, hold,
    -

    or final pose.

    -
    • drive: accelerate from rest; use for a jump, push, lift, or recoil.
    • snap: arrive quickly and stop sharply; use for a strike or release.
    • linear: constant timing; best for deliberate holds or mechanical motion.
    +
    • flow: carry momentum through an interior pose; use for continuous dance, locomotion, and multi-part sports motion.
    • settle: decelerate into a real rest; use at a squat bottom, landing, hold, or final pose.
    • drive: accelerate from rest; use for a jump, push, lift, or recoil.
    • snap: arrive quickly and stop sharply; use for a strike or release.
    • linear: constant timing; best for an unchanged dwell or mechanical motion.

    The older names ease-in, ease-out, and ease-in-out still parse for compatibility, but do not author new documents with them.

    Example

    posecode exercise "Body-weight squat"
    @@ -165,69 +160,117 @@ 

    Example

    step "Descend" 1.6s settle: hips: flex 80 knees: flex 95 - ankles: dorsiflex 14 + pelvis: hinge 25 + spine: flex 0 + shoulders: flex 70 + neck: extend 10 ground-lock: feet - cue "Sit the hips back, chest proud, knees track over the toes" + cue "Flex both hips and knees as the pelvis hinges over the planted feet" step "Drive up" 1.2s drive: hips: flex 0 knees: flex 0 - ankles: dorsiflex 0 + pelvis: hinge 0 + spine: flex 0 + shoulders: flex 0 + neck: extend 0 ground-lock: feet - cue "Drive through the heels to stand tall" + cue "Straighten both hips and knees and return the torso and arms to neutral" repeat 8

    Hip-hinge example

    A flat-back hinge bends at the hips, not the spine. Hinge the pelvis and let the arms hang; ground-lock: feet.

    -
    posecode exercise "Deadlift"
    +
    posecode exercise "Body-weight hip hinge"
       rig humanoid
       pose start = standing
     
       step "Lower" 1.8s settle:
    -    pelvis: hinge 95
    +    pelvis: hinge 75
         knees: flex 25
    -    shoulders: flex 90
    +    shoulders: flex 70
         ground-lock: feet
    -    cue "Hips back, flat back: let the arms hang to the bar"
    +    cue "Hips back and back flat as the arms hang below the shoulders"
     
       step "Lift" 1.4s drive:
         pelvis: hinge 0
         knees: flex 0
         shoulders: flex 0
         ground-lock: feet
    -    cue "Drive the hips forward to stand tall"
    +    cue "Drive the hips forward and return to a tall stance"
     
       repeat 8

    Reaching, props, lying poses & hands

    -
    • Reach a target: reach: <effector> <target> drives a hand or foot to a
    -

    world point via IK. Effectors: hand_left hand_right foot_left foot_right, or hands / feet for both sides at once. Targets: a body landmark bone (ankle_left, knee_right…), floor, or a prop anchor (bar, seat, wall). The solve is ROM-constrained: the arm/leg can never exceed the safe joint limits chasing a target, so an out-of-reach target just yields the closest healthy pose. Author the gross pose (e.g. a pelvis: hinge), then let reach finish the hand placement. Example, touch your toes:

    -

    ``posecode step "Fold" 2.5s settle: pelvis: hinge 95 knees: flex 12 reach: hand_left ankle_left reach: hand_right ankle_right ground-lock: feet cue "Hinge and reach toward the ankles" ``

    -
    • Props: prop chair | wall | bar | box | dip-bars (top level). The chair
    -

    sits behind the figure (sit-to-stand, box squat), the wall behind that (wall sit), the bar overhead, the box in front (step-ups), and the dip bars either side at hip-press height (pin: hands bars + elbow flex = triceps dips).

    -
    • Pins: pin: <effector> <anchor> moves the whole BODY so the effector sits
    -

    on the anchor (vs reach, which moves just the limb). Same effectors as reach, including hands / feet. Use it for hanging and climbing: pin: hands bar + flexing the elbows = a pull-up; pin: foot_right box + straightening the leg = a step-up; pin: hands bars (dip bars) + bending the elbows = a triceps dip.

    -
    • Lying / seated: pose start = supine | prone | seated for floor and mat
    -

    work (glute bridge, dead bug, cobra, seated forward fold). In a supine exercise whose torso stays down, add ground-lock: back to each phase.

    -
    • Hands: fingers: flex 80 makes a fist; curl individual fingers for shapes
    -

    (index_right: flex 95). Single-DOF per finger, good for grip and rough gesture, not exact sign language.

    +
    • Reach a target: reach: <effector> <target> drives a supported limb endpoint to a world point via IK. Effectors include side-specific hands, fists, elbows, knees, and feet, with groups such as hands, fists, forearms, knees, and feet. Targets are a body landmark bone (ankle_left, knee_right…), floor, or a prop anchor (bar, seat, wall). The solve is ROM-constrained: the arm/leg remains within configured joint bounds while chasing a target, so an out-of-reach target yields the closest bounded pose and a residual diagnostic. Author the gross pose (e.g. a pelvis: hinge), then let reach finish the hand placement. Example, touch your toes:
    +
    step "Fold" 2.5s settle:
    +  pelvis: hinge 95
    +  knees: flex 12
    +  reach: hand_left ankle_left
    +  reach: hand_right ankle_right
    +  ground-lock: feet
    +  cue "Hinge and reach toward the ankles"
    +
    • Props: prop chair | wall | bar | box | dip-bars (top level). The chair sits behind the figure (sit-to-stand, box squat), the wall behind that (wall sit), the bar overhead, the box in front (step-ups), and the dip bars either side at hip-press height (grip: hands bars + elbow flex = triceps dips).
    • Pins: pin: <effector> <anchor> moves the whole BODY so the effector sits on the anchor (vs reach, which moves just the limb). Use one primary pin for body translation: pin: foot_right box can support a step-up, and pin: pelvis floor can keep the pelvis on the mat. Use grip, not several simultaneous hand pins, for a bar or rails.
    • Grips: grip: hands bar or grip: hands bars is the dedicated two-hand prop contact. It assigns separate left/right anchors, solves both arms, and closes the fingers. Declare prop bar or prop dip-bars first. Prefer this to multiple pins for a pull-up, hang, or dip.
    • Lying / seated: pose start = supine | prone | seated for floor and mat work (glute bridge, dead bug, cobra, seated forward fold). In a supine exercise whose torso stays down, add ground-lock: back to each phase.
    • Hands: fingers: flex 80 makes a fist; curl individual fingers for shapes (index_right: flex 95). Single-DOF per finger, good for grip and rough gesture, not exact sign language.
    +

    Three-point superhero landing

    +

    A landing is defined by its contacts, not by a dramatic cue. Keep the existing front-foot support planted while reach constraints blend the rear knee and same-side fist down. Once the knee has arrived, hand the whole-body anchor to that knee and solve the foot and fist independently. Repeat the three contacts through the hold; never combine a whole-root pin with ground-lock in one step.

    +
    posecode posture "Superhero Three-Point Landing"
    +  rig humanoid
    +  pose start = standing
    +
    +  step "Drop into the landing" 0.55s flow:
    +    pelvis: hinge 45
    +    spine: flex 50
    +    chest: flex 9
    +    hip_right: flex 84
    +    knee_right: flex 123
    +    ankle_right: dorsiflex 15
    +    hip_left: extend 12
    +    knee_left: flex 105
    +    ankle_left: plantarflex 28
    +    shoulder_left: flex 98
    +    shoulder_left: abduct 2
    +    elbow_left: flex 8
    +    fingers_left: flex 80
    +    shoulder_right: extend 24
    +    shoulder_right: abduct 22
    +    elbow_right: flex 16
    +    ground-lock: foot_right
    +    reach: knee_left floor
    +    reach: fist_left floor
    +    cue "Drop over the planted right foot as the left knee and fist descend"
    +
    +  step "Make three-point contact" 0.3s settle:
    +    neck: extend 25
    +    pin: knee_left floor
    +    reach: foot_right floor
    +    reach: fist_left floor
    +    cue "Set the left knee and left fist on the floor beside the planted right foot"
    +
    +  step "Hold the landing" 0.8s linear:
    +    pin: knee_left floor
    +    reach: foot_right floor
    +    reach: fist_left floor
    +    cue "Hold the three contacts with the free right arm swept behind you"
    +
    +  step "Recover to standing" 0.9s drive:
    +    pelvis: hold neutral
    +    spine: hold neutral
    +    hips: hold neutral
    +    knees: hold neutral
    +    ankles: hold neutral
    +    neck: hold neutral
    +    chest: hold neutral
    +    shoulders: hold neutral
    +    elbows: hold neutral
    +    fingers_left: hold neutral
    +    ground-lock: feet
    +    cue "Press through both feet and return to standing"
    +
    +  repeat 1

    Authoring by domain

    The same grammar covers many fields. A few patterns that read well:

    -
    • Anatomy / education: isolate one joint and sweep it through its range
    -

    (shoulders: abduct 1600). Name the plane in the cue. Great for teaching.

    -
    • Physiotherapy: gentle, single-joint reps; for one-sided work use a
    -

    singular joint (knee_right: flex 95) and skip ground-lock so the standing leg stays planted.

    -
    • Desk / posture: slow stretch documents with ground-lock: feet;
    -

    contrast a "collapsed" phase with a "tall" reset.

    -
    • Sports / martial arts: short phases (0.2–0.6s); use flow through a
    -

    gather/chamber, drive for takeoff, and snap on the strike or release.

    +
    • Anatomy / education: isolate one joint and sweep it through its range (shoulders: abduct 1600). Name the plane in the cue. Great for teaching.
    • Physiotherapy: gentle, single-joint reps; for one-sided work use a singular joint (knee_right: flex 95) and explicitly plant the stance foot, for example ground-lock: foot_left.
    • Desk / posture: slow stretch documents with ground-lock: feet; contrast a "collapsed" phase with a "tall" reset.
    • Sports / martial arts: short phases (0.2–0.6s); use flow through a gather/chamber, drive for takeoff, and snap on the strike or release.

    Dance / choreography

    Posecode shines for showing the movement in your head. Build a phrase as a sequence of phases, one per count or musical beat, and let later phases inherit unset joints:

    -
    • Turnout: hips: rotate-out 25–30.
    • Port de bras (arm positions): first shoulders: flex 30, elbows: flex 35;
    -

    second shoulders: abduct 85, elbows: flex 16; fifth/en haut shoulders: flex 160, elbows: flex 20. Move between them across phases.

    -
    • Plié: hips: flex 18, knees: flex 50, ankles: dorsiflex 12.
    • Relevé: ankles: plantarflex 28 (rise onto the balls of the feet).
    • Turn / pirouette: turn: 360 (yaw in degrees, absolute). Pair with a
    -

    relevé to spin on the balls of the feet; turn: 90 for a quarter-turn.

    -
    • Travel across the floor: travel: <x> <z> (metres from the start spot,
    -

    absolute). A box-step traces a square back home: travel: 0.4 00.4 0.40 0.40 0. A grapevine travels sideways; a walk cycle steps forward in +z. Add the stepping legs with FK on top.

    +
    • Turnout: hips: rotate-out 25–30.
    • Port de bras (arm positions): first shoulders: flex 30, elbows: flex 35; second shoulders: abduct 85, elbows: flex 16; fifth/en haut shoulders: flex 160, elbows: flex 20. Move between them across phases.
    • Plié: hips: flex 18, knees: flex 50, ankles: dorsiflex 12.
    • Relevé: ankles: plantarflex 28 (rise onto the balls of the feet).
    • Turn / pirouette: turn: 360 (yaw in degrees, absolute). Pair with a relevé to spin on the balls of the feet; turn: 90 for a quarter-turn.
    • Travel across the floor: travel: <x> <z> (metres from the start spot, absolute). A box-step traces a square back home: travel: 0.4 00.4 0.40 0.40 0. A grapevine travels sideways; a walk cycle steps forward in +z. Add the stepping legs with FK on top.

    Both turn and travel are absolute and carried forward like joint angles, and both return home on the loop wrap, so phrases resolve cleanly. They work from standing poses only. Name each step by its count ("5-6 - relevé, arms en haut") so the phrase reads like choreography. To extend a phrase, append more steps; the figure carries its pose forward. Use ground-lock: feet for grounded phrases (it still lets the figure turn and travel; it only keeps the feet on the floor vertically).