Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- **Realistic human figure**: the playground, landing hero, and `<posecode-player>` embeds now render a fully rigged, textured human character (hands with articulated fingers, sneakers, face) instead of the procedural capsule mannequin. All solving (FK, ground-lock, pins, reach-IK) still runs on the driver skeleton, rebuilt to the character's exact proportions and retargeted bone-for-bone every frame; the procedural figure remains as an automatic fallback (and via `?figure=classic` / `character="off"`).
- **Self-collision resolution**: a capsule-based de-penetration pass keeps forearms/hands out of the torso, head, and legs (and shins out of each other), clamped to healthy ROM, so limbs no longer pass through the body mid-movement.
- **Solid props**: props now declare blocking faces (the wall's surface, the chair's backrest and seat edge, the box's near face) and a contact pass keeps the body out of them — translating the whole figure along the face normal, or bending the offending leg's hip clear (ROM-clamped). Limbs pinned/gripped/reached to a prop anchor stay exempt as declared support. A new `solid-props` eval invariant (independent geometry re-derivation) guards every prop movement against this bug class.
- `viewer.characterActive`, `createViewer({ characterUrl })`, and the embed `character` attribute.
- `scripts/capture-gifs.mjs` (`npm run gifs`): reproducible headless regeneration of the README movement GIFs from the live renderer.

### Fixed

- Wall sit no longer clips through the wall: the body now translates forward until the back rests on the wall's surface (feet walking out, thighs parallel), the physically correct wall-sit geometry. Sit-to-stand and box-squat land against the chair's backrest instead of sinking into it, a standing figure's calves clear the seat edge, and a step-up's trailing shin bends over the box edge instead of sweeping through it.
- Deadlift arms now hang toward the bar during the hinge (were authored as shoulder extension, flying up behind the back).
- Crunch keeps the feet planted with bent knees (shins previously folded through the floor and jacked the body up).
- Touch-toes folds like a human (hinge depth and knee bend were over-authored, collapsing the figure).
Expand Down
9 changes: 8 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ These are the unlocks, roughly in order of leverage:
Powers `sit-to-stand`, `box-squat`, `wall-sit`, `dead-hang`, `hanging-knee-raise`.
Bar and dip-bar contacts now resolve to independent left/right anchors with
terminal wrist orientation; mocap is contact-corrected after blending.
Props are now **solid**: declared blocking faces (wall surface, chair
backrest + seat edge, box edge) physically stop the body — a wall-sit
slides down the wall instead of through it, a sit lands against the
backrest, a swing leg steps over the box edge — guarded by a `solid-props`
eval invariant on every prop movement.
Next: more props (bench, rings, bands), load cues, arbitrary surface shapes.
4. ~~**Lying & seated base poses**~~: **shipped.** `supine | prone | seated`
start poses (grounded by a bounding-box drop). Powers `glute-bridge`,
Expand Down Expand Up @@ -84,7 +89,9 @@ Each prop is a small scene object + an anchor type; movements then reference it
- A **starter** prop set (chair / wall / bar / box / dip bars): no bench,
rings, bands, or loaded implements yet, and props sit at fixed default
placements.
- Props are visual + reach anchors (no physical sit/lean solve).
- Prop solidity is face-based: each built-in prop declares its blocking
surfaces (wall face, backrest, seat edge, box edge). Arbitrary-shape
collision and load/pressure simulation are future.
- Fingers are **single-DOF** curls, good for grip and rough gesture, not exact
sign language. The head has no facial articulation.

Expand Down
14 changes: 14 additions & 0 deletions packages/posecode-eval/src/checks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
lowestPoint,
palmFloorAngleDeg,
phaseMaxLandmarkSpeed,
propPenetrationDepth,
segmentTiltDeg,
spineCurlDeg,
torsoPitchDeg,
Expand Down Expand Up @@ -137,6 +138,19 @@ export function genericChecks(result: ProbeResult): CheckOutcome[] {
detail: `${clearance.toFixed(3)}m clearance (want > -0.01m)`,
});
}

// Props are solid: no body capsule may sit inside a prop's blocking face
// (the wall-sit-through-the-wall class of bug). Independent re-derivation
// of the face geometry, so it fails loudly if resolvePropContacts or a
// prop's collider declaration regresses.
const penetration = propPenetrationDepth(result, p);
if (Number.isFinite(penetration)) {
out.push({
id: `solid-props:${p.name}`,
pass: penetration < 0.03,
detail: `${penetration.toFixed(3)}m into a solid prop face (want < 0.030)`,
});
}
}

for (let i = 1; i < result.phases.length; i++) {
Expand Down
92 changes: 90 additions & 2 deletions packages/posecode-eval/src/metrics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,90 @@ export function headPropClearance(result: ProbeResult, pose: PhasePose): number
return clearance;
}

interface SolidFace {
point: Vec3;
normal: Vec3;
tangentU: Vec3;
halfU: number;
tangentV: Vec3;
halfV: number;
captureDepth: number;
blocks: readonly string[];
}

/** The solid prop faces, re-derived from the prop geometry independently of
* the renderer's collider declarations so a regression in either is caught. */
function solidFaces(propTypes: readonly string[]): SolidFace[] {
const out: SolidFace[] = [];
const all = ["torso", "head", "thigh", "shin", "arm"];
if (propTypes.includes("wall")) {
out.push({ point: [0, 1.3, -0.29], normal: [0, 0, 1], tangentU: [1, 0, 0], halfU: 1.1, tangentV: [0, 1, 0], halfV: 1.3, captureDepth: 0.8, blocks: all });
}
if (propTypes.includes("chair")) {
out.push(
{ point: [0, 0.78, -0.31], normal: [0, 0, 1], tangentU: [1, 0, 0], halfU: 0.21, tangentV: [0, 1, 0], halfV: 0.25, captureDepth: 0.4, blocks: ["torso", "head"] },
{ point: [0, 0.47, 0.05], normal: [0, 0, 1], tangentU: [1, 0, 0], halfU: 0.21, tangentV: [0, 1, 0], halfV: 0.03, captureDepth: 0.42, blocks: ["shin"] },
);
}
if (propTypes.includes("box")) {
out.push({ point: [0, 0.15, 0.11], normal: [0, 0, -1], tangentU: [1, 0, 0], halfU: 0.25, tangentV: [0, 1, 0], halfV: 0.15, captureDepth: 0.42, blocks: ["shin"] });
}
return out;
}

/** Body capsule radii matching the render mannequin (see mannequin.ts). */
const PART_RADII = { torso: 0.13, head: 0.105, thigh: 0.075, shin: 0.055, arm: 0.038 } as const;

/**
* Worst body penetration into a solid prop face (metres, ≤0 when clear), or
* -Infinity when the document declares no solid-faced prop. Limbs pinned or
* reached to a non-floor anchor are that phase's declared prop support and
* don't count (a foot standing ON the box is not "in" the box).
*/
export function propPenetrationDepth(result: ProbeResult, pose: PhasePose): number {
const faces = solidFaces(result.propTypes);
if (faces.length === 0) return -Infinity;
const exemptLegs = new Set<string>();
const contacts = [
...pose.pins,
...pose.reaches.map((r) => ({ effector: r.effector, anchor: r.target })),
];
for (const c of contacts) {
if (c.anchor === "floor") continue;
if (c.effector === "feet" || c.effector === "foot_left") exemptLegs.add("left");
if (c.effector === "feet" || c.effector === "foot_right") exemptLegs.add("right");
}
const segments: [string, string, keyof typeof PART_RADII][] = [
["pelvis", "neck", "torso"],
["neck", "head", "head"],
];
for (const side of ["left", "right"]) {
segments.push([`shoulder_${side}`, `elbow_${side}`, "arm"], [`elbow_${side}`, `wrist_${side}`, "arm"]);
if (exemptLegs.has(side)) continue;
segments.push([`hip_${side}`, `knee_${side}`, "thigh"], [`knee_${side}`, `ankle_${side}`, "shin"]);
}
let worst = -Infinity;
for (const [aId, bId, part] of segments) {
const a = pose.bones.get(aId);
const b = pose.bones.get(bId);
if (!a || !b) continue;
const r = PART_RADII[part];
for (const t of [0, 0.25, 0.5, 0.75, 1]) {
const p: Vec3 = [a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t, a[2] + (b[2] - a[2]) * t];
for (const f of faces) {
if (!f.blocks.includes(part)) continue;
const rel = sub(p, f.point);
const d = dot(rel, f.normal);
if (d < -f.captureDepth) continue;
if (Math.abs(dot(rel, f.tangentU)) > f.halfU + r) continue;
if (Math.abs(dot(rel, f.tangentV)) > f.halfV + r) continue;
worst = Math.max(worst, r - d);
}
}
}
return worst;
}

/** Fastest landmark's average speed from the previous endpoint into this phase. */
export function phaseMaxLandmarkSpeed(previous: PhasePose | null, pose: PhasePose): number {
if (!previous || pose.durationSec <= 0) return 0;
Expand All @@ -209,8 +293,11 @@ export function phaseMaxLandmarkSpeed(previous: PhasePose | null, pose: PhasePos

export function footSkateDistance(previous: PhasePose, pose: PhasePose, side: "left" | "right"): number {
const id = `ankle_${side}`;
// Authored travel AND the solid-prop contact push both translate the whole
// body deliberately, feet included; skate is what's left after removing them.
const local = (p: Vec3, phase: PhasePose): readonly [number, number] => {
const x = p[0] - phase.rootOffset[0], z = p[2] - phase.rootOffset[2];
const x = p[0] - phase.rootOffset[0] - phase.propPush[0];
const z = p[2] - phase.rootOffset[2] - phase.propPush[2];
const c = Math.cos(-phase.rootYaw), s = Math.sin(-phase.rootYaw);
return [x * c - z * s, x * s + z * c];
};
Expand All @@ -224,7 +311,8 @@ export function feetCenterSkateDistance(previous: PhasePose, pose: PhasePose): n
const id = `ankle_${side}`;
const a = bone(previous, id), b = bone(pose, id);
const unyaw = (p: Vec3, phase: PhasePose) => {
const x = p[0] - phase.rootOffset[0], z = p[2] - phase.rootOffset[2];
const x = p[0] - phase.rootOffset[0] - phase.propPush[0];
const z = p[2] - phase.rootOffset[2] - phase.propPush[2];
const c = Math.cos(-phase.rootYaw), s = Math.sin(-phase.rootYaw);
return [x * c - z * s, x * s + z * c] as const;
};
Expand Down
26 changes: 26 additions & 0 deletions packages/posecode-eval/src/probe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import {
depenetrate,
groundFigure,
levelPlantedFeet,
propContactExemptions,
resolvePropContacts,
} from "posecode-render";

export type Vec3 = readonly [x: number, y: number, z: number];
Expand All @@ -39,6 +41,14 @@ export interface PhasePose {
reaches: readonly ReachTarget[];
rootOffset: Vec3;
rootYaw: number;
/**
* Horizontal body translation applied by the solid-prop contact solve
* (resolvePropContacts): the feet legitimately glide by this much while the
* body is pressed out of a prop (a wall-sit walks the feet forward as the
* back slides down the wall), so skate metrics compensate for it like they
* do for authored travel.
*/
propPush: Vec3;
/** True when the phase relies on pins/reach-IK the probe cannot solve. */
usesSceneIk: boolean;
/** Whether the phase should rest on the floor (no elevated prop/grip support). */
Expand Down Expand Up @@ -89,6 +99,11 @@ export function probeMovement(source: string): ProbeResult {
m.root.updateMatrixWorld(true);
depenetrate(m);
groundFigure(m);
resolvePropContacts(m, propScene.colliders, propContactExemptions([
...(ir.phases[0]?.pins ?? []),
...(ir.phases[0]?.grips ?? []),
...(ir.phases[0]?.reaches ?? []).map((r) => ({ effector: r.effector, anchor: r.target })),
]));
const baseRootPos = m.root.position.clone();
const baseRootQuat = m.root.quaternion.clone();

Expand Down Expand Up @@ -168,6 +183,16 @@ export function probeMovement(source: string): ProbeResult {
m.root.updateMatrixWorld(true);
}
}
// Props are solid (viewer parity): after the root solvers place the body,
// push it back out of any prop face it crossed and bend swing legs clear.
// Limbs pinned/gripped to a prop anchor are declared support, exempt.
const prePush = m.root.position.clone();
resolvePropContacts(m, propScene.colliders, propContactExemptions([
...info.pins,
...info.grips,
...info.reaches.map((r) => ({ effector: r.effector, anchor: r.target })),
]));
const propPush: Vec3 = [m.root.position.x - prePush.x, 0, m.root.position.z - prePush.z];
alignFloorPalms(m, info.reaches, info.pins);
// Plantigrade correction (viewer parity): flatten planted soles. This lifts
// the foot mesh a little, so it must run BEFORE the floor clamp reconciles.
Expand All @@ -192,6 +217,7 @@ export function probeMovement(source: string): ProbeResult {
reaches: [...info.reaches],
rootOffset: [info.rootOffset.x, 0, info.rootOffset.z],
rootYaw: info.rootYaw,
propPush,
usesSceneIk: info.pins.length > 0 || info.reaches.length > 0 || info.grips.length > 0,
floorBound,
meshMinY: Number.isFinite(finalBox.min.y) ? finalBox.min.y : 0,
Expand Down
5 changes: 3 additions & 2 deletions packages/posecode-render/src/depenetrate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,9 @@ function wp(m: Mannequin, id: string, out = new THREE.Vector3()): THREE.Vector3
/**
* Rotate `joint` (world-space axis/angle) and clamp it back into `limits`.
* Mirrors the CCD solver's joint update so corrections obey the same ROM.
* Shared with the prop-contact pass (propcontact.ts).
*/
function rotateJoint(
export function rotateJoint(
joint: THREE.Object3D,
axis: THREE.Vector3,
angle: number,
Expand All @@ -113,7 +114,7 @@ function rotateJoint(
}

/** The joint's ROM (radians), widened to admit its current authored pose. */
function widenedLimits(
export function widenedLimits(
boneId: string,
joint: THREE.Object3D,
): { x: [number, number]; y: [number, number]; z: [number, number] } | null {
Expand Down
29 changes: 28 additions & 1 deletion packages/posecode-render/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import {
type ClipSource,
} from "./clips.js";
import { depenetrate } from "./depenetrate.js";
import { resolvePropContacts, propContactExemptions } from "./propcontact.js";
import { alignFloorPalms, levelPlantedFeet, wrapGrip, relaxHands, swingArms, aimHead } from "./contacts.js";

const DEG = Math.PI / 180;
Expand Down Expand Up @@ -585,6 +586,19 @@ export function createViewer(
aimHead(mannequin, focus.multiplyScalar(1 / pts.length));
}

/** Prop-contact exemptions for a phase: limbs pinned/gripped/reached to props. */
function contactExemptionsOf(info: {
pins?: readonly PinTarget[];
grips?: readonly GripTarget[];
reaches?: readonly ReachTarget[];
}): ReturnType<typeof propContactExemptions> {
return propContactExemptions([
...(info.pins ?? []),
...(info.grips ?? []),
...(info.reaches ?? []).map((r) => ({ effector: r.effector, anchor: r.target })),
]);
}

function frameCamera(): void {
// Auto-frame the figure: fit its bounding box, keep a pleasant angle.
// Include any scene prop too: a pull-up bar sits well above the figure's
Expand Down Expand Up @@ -641,6 +655,15 @@ export function createViewer(
applyGroundLockTo(mannequin, info.groundLock, frameAnchors(info.rootYaw, info.rootOffset));
applyPins(info.pins);
applyGrips(info.grips);
// Props are solid: after the root solvers place the body, push it back
// out of any prop face it crossed (wall-sit slides down the wall's
// surface, not through it) and bend swing legs clear of box edges.
// Before reach-IK so a later root push can't drag reached hands off
// their world targets. Limbs pinned/gripped to a prop anchor are that
// phase's declared support, exempt from clearing.
if (propScene) {
resolvePropContacts(mannequin, propScene.colliders, contactExemptionsOf(info));
}
// Reach-IK BEFORE the floor safety clamp. When authored FK pushes a
// reaching limb through the floor (cobra: prone + shoulders flex 50),
// the limb must bend to meet the floor. Running reaches after the clamp
Expand Down Expand Up @@ -782,6 +805,9 @@ export function createViewer(
mannequin.root.updateMatrixWorld(true);
depenetrate(mannequin);
groundFigureOf(mannequin);
if (propScene) {
resolvePropContacts(mannequin, propScene.colliders, contactExemptionsOf(ir.phases[0] ?? {}));
}
levelPlantedFeet(mannequin, ir.phases[0]?.groundLock ?? []);
authoredFingers = new Set(timeline.bonesUsed.filter(isFingerId));
authoredShoulders = new Set(timeline.bonesUsed.filter((id) => id.startsWith("shoulder_")));
Expand Down Expand Up @@ -1012,7 +1038,8 @@ export { applyGroundLock, groundFigure } from "./groundlock.js";
export type { Mannequin, Proportions, CollisionRadii } from "./mannequin.js";
export { buildTimeline } from "./timeline.js";
export { solveCCD, type IkChain, type JointLimits } from "./ik.js";
export { buildProps, type PropScene } from "./props.js";
export { buildProps, type PropScene, type FaceCollider, type BlockedPart } from "./props.js";
export { resolvePropContacts, propContactExemptions, type PropContactExemptions } from "./propcontact.js";
export { loadCharacter, rigCharacter, type Character } from "./character.js";
export {
loadClipSource,
Expand Down
Loading
Loading