Admin segment management UI + Playwright visual source plugin

[jmjava]

Summary

Add generic Playwright visual source, recording pipeline, and admin segment management to documentation-generator. A new application writes YAML only — no Python code needed.

Extracts and generalizes the working prototype in course-builder tools/courseforge.

Architecture

What moves to documentation-generator (100% generic, no app logic)

File	Role
actions.py	Generic Playwright command executor — interprets YAML act/verify instructions (click, check, hover, drag_rectangle, expect_visible, expect_text, etc.)
scenario.py	YAML loader + AppConfig/ScenarioStep data model + save/reload + queries
demo-capture.py	Recording pipeline: load YAML → TTS per segment → Playwright recording paced to audio → ffmpeg compose. Becomes docgen record
admin_routes.py	FastAPI segment CRUD API (list steps, save steps back to YAML)
demo-picker.html	Admin UI: toggle demo, edit narration, reorder steps, change visual_type

What stays in each application repo (YAML only)

Each app provides one file: scenario.yml. No Python.

app:
  name: CourseForge
  base_url: "http://localhost:3300"
  start_params: {lat: 40.668, lon: -74.902, zoom: 15}
  viewport: {width: 1920, height: 1080}
  ready_selector: ".sidebar-header h1"
  ready_wait_ms: 3000

steps:
  - id: draw
    narration: "Next, we activate the drawing tool..."
    browser: true
    demo: true
    fallback_duration_ms: 5000
    visual_type: playwright
    act:
      - drag_rectangle:
          draw_button: '[data-testid="button-draw-area"]'
          map_container: '[data-testid="leaflet-map-root"]'
          center: [0.5, 0.5]
          size: [0.35, 0.35]
    verify:
      - expect_visible: {selector: '[data-testid="coord-grid"]'}
      - expect_value: {selector: '[data-testid="coord-north"]', truthy: true}

New docgen commands

# Record a demo video from a scenario YAML (TTS + Playwright + ffmpeg)
docgen record --scenario path/to/scenario.yml --out demo-output/

# Print outline from scenario
docgen record --scenario path/to/scenario.yml --print-outline

# Admin UI for editing scenario steps
docgen admin --scenario path/to/scenario.yml --port 8300

Supported YAML commands

Act (browser interactions):
click, check, uncheck, hover, fill, wait, wait_for, drag_rectangle, evaluate

Verify (assertions):
expect_text, expect_visible, expect_checked, expect_enabled, expect_disabled, expect_value, evaluate

Integration with docgen.yaml

visual_map:
  "18":
    type: playwright
    scenario: tools/courseforge/e2e/scenario.yml
    source: courseforge-demo.mp4

docgen record (standalone) or docgen compose (integrated):

1. Load the scenario YAML (app: config + steps:)
2. Filter to demo: true steps
3. Generate TTS per segment (model/voice from tts: config or scenario-level override)
4. Launch Playwright with app.viewport, navigate to app.start_url()
5. Wait for app.ready_selector, then execute act commands paced to TTS duration
6. Compose recording + narration via ffmpeg
7. Optionally append closing still for visual_type: still steps

Admin segment management

docgen admin command or route on wizard server:

• GET /api/admin/demo/steps — list all steps with metadata
• POST /api/admin/demo/steps — save full step list back to YAML
• HTML page — toggle demo, edit narration, reorder, change visual_type, edit act/verify

Recording pipeline detail (from demo-capture.py)

The recording pipeline handles:

• TTS generation: per-segment MP3 via OpenAI TTS, duration measurement via ffprobe
• Audio concatenation: ffmpeg concat of segment MP3s into single narration track
• Browser recording: Playwright with record_video_dir, actions paced to segment audio duration
• Still frame append: for visual_type: still steps, creates H.264 clip from screenshot
• Final mux: video + narration audio → MP4 with AAC audio
• Cleanup: removes intermediate files

All of this is generic — the only input is the scenario YAML.

Extraction plan from course-builder

Once built in documentation-generator:

1. Delete from course-builder: demo-capture.py, admin_routes.py, demo-picker.html, /admin/ route, actions.py, scenario.py
2. CourseForge keeps only: scenario.yml (the data), helpers.py + test_courseforge_gui.py (standalone GUI regression tests)
3. Usage: docgen record --scenario tools/courseforge/e2e/scenario.yml
4. CourseForge backend returns to being purely the LiDAR processing API

Working prototype

All code in jmjava/course-builder:

• Scenario YAML: tools/courseforge/e2e/scenario.yml
• Generic loader: tools/courseforge/e2e/scenario.py
• Generic executor: tools/courseforge/e2e/actions.py
• Recording pipeline: tools/courseforge/demo-capture.py
• Admin API: tools/courseforge/backend/admin_routes.py
• Admin HTML: tools/courseforge/backend/static/admin/demo-picker.html
• Tests: tools/courseforge/tests/test_scenario_demo_selection.py