Overview
The Vector AI Brain at /Volumes/AI-Models/vector/ already has a production-grade autonomous system:
- Priority-based BehaviorArbiter (SAFETY > ONBOARD > AI > IDLE)
- Persistent MemoryBank with Jaccard scene similarity
- LearningEngine for autonomous environment discovery
- MissionController (SecurityPatrol, ObjectCensus, RoomCheck, PresenceWatch)
- SpatialMemory for room mapping
- VoiceHandler for voice commands
Current AI stack: LLaVA (vision) + Llama 3.1 (personality) — two separate models, no memory context injection.
Goal
Replace both models with Gemma 4 (single call) that:
- Sees through Vector's camera (multimodal vision)
- Gets injected with full MemoryBank context — every prior observation, known objects, room history
- Returns unified nav decision + personality reaction + memory note in one JSON response
- Plugs transparently into the existing BehaviorArbiter + MemoryBank + MissionController
Architectural decisions
- Swap mechanism:
GEMMA4=1 env var in rover_brain.py — zero changes to existing code paths
- Model:
gemma4:e4b via Ollama REST at http://localhost:11434
- Context injection: MemoryBank.get_recent(5) + known_objects + SpatialMemory summary prepended to every prompt
- Output contract: JSON with keys: scene, objects, nav_decision, danger_level, reaction, memory_note
- Patcher:
patch_all_models() monkey-patches mission_controller._ask_vision and learning_engine LLaVA calls
- Fallback: if Gemma 4 unavailable, falls back to original AIBrain (LLaVA)
Delivered so far
gemma4_brain.py — Gemma4Brain class + patch_all_models() + standalone CLI
rover_brain.py — GEMMA4=1 swap point added
gemmi_processor.py — Gemma 4 replaces LLaVA as Ollama fallback
gemma4_companion.py — lightweight companion loop using existing SDK config
What remains
See child issues for implementation slices.
Overview
The Vector AI Brain at
/Volumes/AI-Models/vector/already has a production-grade autonomous system:Current AI stack: LLaVA (vision) + Llama 3.1 (personality) — two separate models, no memory context injection.
Goal
Replace both models with Gemma 4 (single call) that:
Architectural decisions
GEMMA4=1env var in rover_brain.py — zero changes to existing code pathsgemma4:e4bvia Ollama REST athttp://localhost:11434patch_all_models()monkey-patches mission_controller._ask_vision and learning_engine LLaVA callsDelivered so far
gemma4_brain.py— Gemma4Brain class + patch_all_models() + standalone CLIrover_brain.py— GEMMA4=1 swap point addedgemmi_processor.py— Gemma 4 replaces LLaVA as Ollama fallbackgemma4_companion.py— lightweight companion loop using existing SDK configWhat remains
See child issues for implementation slices.