Drop perception. Update docs.

[Dreamsorcerer]

Problem

Dropping heavyweight perception from base install. With other PRs, this removes torch from base install.

Also dropping bitsandbytes which doesn't appear to be used anywhere at all?

Also updating the docs now to reflect all those changes.

Breaking Changes

Users wanting perception now need to request it.

[paul-nechifor]

bitsandbytes is needed by langchain-ollama or langchain-huggingface. I don't remember which one, but I added it intentionally (should have left a comment at the time).

[greptile-apps]

Greptile Summary

This PR removes perception from the base extra and drops bitsandbytes from agents, with the goal of eliminating heavy GPU dependencies (primarily PyTorch) from the default install. Documentation and the README quickstart are updated to match the new lighter base definition.

• pyproject.toml: base is now agents + web + visualization (no perception); bitsandbytes is removed from agents.
• docs/requirements.md: install examples and the extras table are updated to separate perception as an opt-in extra; a new whisper row is added to the table.
• README.md: quickstart simplified to dimos[unitree], with perception and simulation as explicit add-on steps.

Confidence Score: 3/5

The core goal — dropping perception from base — is correctly implemented in pyproject.toml, but the install still brings in torch through openai-whisper in agents, and the docs introduce a whisper extra that does not exist in pyproject.toml.

The pyproject.toml change is minimal and correct in isolation: perception is removed from base and bitsandbytes is cleaned up. However, several mismatches between the new docs and the actual package definitions (openai-whisper vs faster-whisper in agents, a documented whisper extra with no matching pyproject.toml entry) were already flagged on this PR and remain unresolved, meaning users following the updated documentation will encounter broken installs or unexpected torch dependencies.

pyproject.toml (agents extra still references openai-whisper) and docs/requirements.md (whisper extra row and faster-whisper description do not match the code).

Important Files Changed

Filename	Overview
pyproject.toml	Removes bitsandbytes from agents and removes perception from base; openai-whisper (torch dependency) remains in agents, contradicting the "no torch" intent
docs/requirements.md	Updated to reflect new base definition (no perception) and adds a whisper row; the whisper extra and faster-whisper descriptions do not match pyproject.toml (pre-existing flagged issues)
README.md	Quickstart simplified to dimos[unitree] (safe, since unitree already depends on base); perception and simulation are now separate optional steps
uv.lock	Lock file updated to remove bitsandbytes entries; mechanical change consistent with the pyproject.toml diff

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    core["**core** (bare install)\ntransport, CLI, blueprints, occupancy maps"]
    agents["**agents**\nlangchain, openai, openai-whisper, anthropic"]
    web["**web**\nfastapi, uvicorn, ffmpeg-python"]
    viz["**visualization**\nrerun-sdk, dimos-viewer"]
    perception["**perception** (opt-in, GPU)\nultralytics, transformers, moondream"]
    sim["**sim** (opt-in)\nmujoco, playground, pygame"]
    unitree["**unitree**\nunitree-webrtc-connect"]
    base["**base** = agents + web + viz"]

    core --> agents
    core --> web
    core --> viz
    core --> perception
    core --> sim
    agents --> base
    web --> base
    viz --> base
    base --> unitree

    style perception fill:#f96,stroke:#c33
    style base fill:#9cf,stroke:#36a

Loading

_{Reviews (2): Last reviewed commit: "Merge branch 'dev' into sam/drop-percept..." | Re-trigger Greptile}

[greptile-apps]

Documented whisper extra does not exist

Line 48 documents a whisper extra (openai-whisper, GPU required), but no such extra is defined in pyproject.toml. A user running pip install 'dimos[whisper]' will get an error. Either the whisper extra needs to be added to pyproject.toml (and openai-whisper removed from agents), or this row should be removed from the table.

[greptile-apps]

agents key-packages column says faster-whisper, code says openai-whisper

The table on line 43 lists faster-whisper as a key package for the agents extra and marks it "No GPU", but pyproject.toml still lists openai-whisper (which requires torch) in agents. This mismatch will mislead users who expect a torch-free agent install.