Skip to content

README.md

Latest commit

 

History

History
339 lines (275 loc) · 8.11 KB

README.md

File metadata and controls

339 lines (275 loc) · 8.11 KB

PawTalk FVR Integration

Integration layer for TFCG-FVR-SUITE to provide anti-drift provenance tracking for all PawTalk AI workflows.

🎯 What is FVR?

FVR (File-based Versioned Replay) is a protocol for capturing and replaying AI interactions with guaranteed anti-drift properties. It ensures that:

  1. All inputs are tracked - Button presses, context, timestamps
  2. All outputs are logged - Predictions, actions, reasoning
  3. All refinements are recorded - Manual corrections, model updates
  4. Everything is replayable - Sessions can be replayed to detect drift
  5. Provenance is guaranteed - Checksums ensure data integrity

🏗️ Architecture

fvr-integration/
├── src/
│   ├── adapters/                   # Protocol adapters
│   │   ├── button-press-adapter.ts # Button press events
│   │   ├── bark-audio-adapter.ts   # Audio analysis (V2)
│   │   └── spatial-er-adapter.ts   # Gemini ER spatial data (V3)
│   ├── core/
│   │   ├── session.ts             # Session management
│   │   ├── storage.ts             # FVR file storage
│   │   └── replay.ts              # Replay engine
│   ├── cli.ts                     # Command-line tool
│   └── index.ts                   # Main exports
├── protocols/                      # Protocol definitions
│   ├── button-press-v1.schema.json
│   ├── bark-audio-v1.schema.json
│   └── spatial-er-v1.schema.json
└── examples/                       # Usage examples

🚀 Getting Started

Installation

# Navigate to FVR integration directory
cd fvr-integration

# Install dependencies
npm install

# Build
npm run build

# Install CLI globally (optional)
npm link

Basic Usage

import { ButtonPressAdapter } from 'pawtalk-fvr-integration';

// Start new session
const adapter = new ButtonPressAdapter();
const session = adapter.startSession({
  sessionType: 'training',
  petId: 0,
  petName: 'LeBron',
  metadata: {
    firmwareVersion: '1.0.0',
    appVersion: '1.0.0',
  },
});

// Log button presses
adapter.logButtonPresses(
  [
    {
      buttonId: 0,
      pressType: 'long',
      durationMs: 1500,
      timestamp: new Date(),
      petProfile: 0,
      deviceId: 'device_abc123',
    },
  ],
  {
    timeOfDay: 'morning',
    dayOfWeek: 'Tuesday',
    timeSinceLastMeal: 120,
    timeSinceLastWalk: 300,
    location: 'door',
  }
);

// Log AI prediction
adapter.logPrediction({
  intent: 'outside',
  confidence: 0.87,
  reasoning: 'Long press indicates urgency',
  modelVersion: 'intent_predictor_v1',
  inferenceTimeMs: 45,
});

// Log manual correction (if needed)
adapter.logRefinement({
  type: 'manual_correction',
  description: 'Corrected intent from "play" to "outside"',
  changes: {
    intent: { from: 'play', to: 'outside' },
  },
  userId: 'user_123',
});

// End session and export
adapter.endSession();
const fvrData = adapter.exportToJSON();

// Save to file
import { writeFileSync } from 'fs';
writeFileSync('sessions/lebron_training_2025-10-16.fvr', fvrData);

📋 FVR Session Format

Session Structure

{
  "id": "sess_abc123xyz",
  "sessionType": "training",
  "petId": 0,
  "petName": "LeBron",
  "startTime": "2025-10-16T08:00:00.000Z",
  "endTime": "2025-10-16T08:30:00.000Z",
  
  "prompts": [
    {
      "type": "button_press_sequence",
      "version": "1.0.0",
      "events": [
        {
          "buttonId": 0,
          "pressType": "long",
          "durationMs": 1500,
          "timestamp": "2025-10-16T08:15:23.000Z",
          "petProfile": 0,
          "deviceId": "device_abc123"
        }
      ],
      "context": {
        "timeOfDay": "morning",
        "dayOfWeek": "Tuesday",
        "timeSinceLastMeal": 120,
        "timeSinceLastWalk": 300,
        "location": "door"
      },
      "checksum": "a1b2c3d4e5f6g7h8"
    }
  ],
  
  "outputs": [
    {
      "type": "intent_prediction",
      "version": "1.0.0",
      "intent": "outside",
      "confidence": 0.87,
      "reasoning": "Long press indicates urgency; Typical outside time",
      "modelVersion": "intent_predictor_v1",
      "inferenceTimeMs": 45,
      "checksum": "x1y2z3a4b5c6d7e8"
    }
  ],
  
  "refinements": [
    {
      "type": "manual_correction",
      "timestamp": "2025-10-16T08:16:00.000Z",
      "description": "Verified prediction was correct - LeBron went outside",
      "changes": {
        "outcome": "successful"
      },
      "userId": "user_123"
    }
  ],
  
  "metadata": {
    "firmwareVersion": "1.0.0",
    "appVersion": "1.0.0",
    "modelVersion": "intent_predictor_v1"
  },
  
  "history": {
    "parent": null,
    "branch": "main",
    "commits": [
      "commit_a1b2c3d4: Session started",
      "commit_e5f6g7h8: Added button press sequence: 1 events",
      "commit_i9j0k1l2: Added prediction: outside (0.87)",
      "commit_m3n4o5p6: Added refinement: manual_correction - Verified prediction",
      "commit_q7r8s9t0: Session ended"
    ]
  }
}

🔄 Anti-Drift Replay

Replay sessions to verify model consistency:

import { ButtonPressAdapter } from 'pawtalk-fvr-integration';
import { readFileSync } from 'fs';

// Load saved session
const adapter = new ButtonPressAdapter();
const sessionJson = readFileSync('sessions/lebron_training.fvr', 'utf-8');
const session = adapter.importFromJSON(sessionJson);

// Replay with current model
const result = await adapter.replaySession(
  session,
  async (events, context) => {
    // Call current AI model
    const response = await fetch('http://localhost:8000/predict', {
      method: 'POST',
      body: JSON.stringify({ press_history: events, context }),
    });
    return response.json();
  }
);

if (result.driftDetected) {
  console.log('⚠️  Drift detected!');
  console.log('Differences:', result.differences);
} else {
  console.log('✅ No drift - model predictions are consistent');
}

🛠️ CLI Tool

# Create new session
pawtalk-fvr create --pet LeBron --type training

# Log button press
pawtalk-fvr log-press --button 0 --type long --duration 1500

# Log prediction
pawtalk-fvr log-prediction --intent outside --confidence 0.87

# End session
pawtalk-fvr end --output sessions/lebron_training.fvr

# Replay session
pawtalk-fvr replay sessions/lebron_training.fvr --ai-url http://localhost:8000

# Validate session
pawtalk-fvr validate sessions/lebron_training.fvr

# Export session summary
pawtalk-fvr export sessions/lebron_training.fvr --format markdown > report.md

📊 Expandable Protocols

Adding New Adapters

Create a PR to add new protocol adapters:

  1. Define schema in protocols/
  2. Implement adapter in src/adapters/
  3. Add tests in tests/
  4. Update docs

Example for bark audio:

// src/adapters/bark-audio-adapter.ts
export interface BarkAudioPrompt extends FVRPrompt {
  type: 'bark_audio_analysis';
  audioData: {
    format: 'wav';
    sampleRate: number;
    duration: number;
    checksum: string;
  };
  audioUrl: string;  // Reference to stored audio file
}

🧪 Testing

# Run tests
npm test

# Test specific adapter
npm test -- button-press-adapter

# Test replay functionality
npm test -- replay

📁 Storage

FVR files are stored in:

data/fvr-sessions/
├── 2025-10/
│   ├── lebron_training_2025-10-16_001.fvr
│   ├── lebron_training_2025-10-16_002.fvr
│   ├── kobe_training_2025-10-17_001.fvr
│   └── ...
└── index.json  # Session index for quick lookup

🔐 Data Integrity

  • Checksums: SHA-256 hashes for all prompts/outputs
  • Validation: Automatic integrity checks on import/replay
  • Immutability: Sessions are append-only (refinements, not edits)
  • Versioning: Protocol versions tracked in each entry

📚 Resources

🎯 Roadmap

  • ✅ V1: Button press adapter
  • 🔄 V2: Bark audio adapter
  • 🎯 V3: Spatial ER adapter (Gemini robotics)
  • 🌟 Future: Community protocol registry

Questions? See main PawTalk README