Skip to content

Python bindings (pyo3) for tree-structured KV cache #2

@BioInfo

Description

@BioInfo

Feature Request: Python bindings

Most LLM agent frameworks (LangGraph, LlamaIndex, AutoGen) are Python-first. Adding Python bindings would let agents embed Dendrite without FFI or IPC overhead.

Scope

  • pyo3-based bindings for the core TreeState and KvCache API
  • dendrite-py crate (new crate in workspace)
  • pip install dendrite via PyPI (or local build)
  • Basic example: fork, append tokens, query KV state from Python

API Target

import dendrite

state = dendrite.TreeState(num_layers=32, num_kv_heads=8, head_dim=128)

parent = state.root()
child1 = state.fork(parent)
child2 = state.fork(parent)

state.append_token(child1, token_id=42)
state.append_token(child2, token_id=99)

Why This Matters

Dendrite's O(1) fork advantage is most valuable to Python agent frameworks doing tree search. Right now there's no clean path to use Dendrite from Python without running it as a subprocess.

References

  • pyo3 docs
  • crates/dendrite-ffi/ — existing FFI crate (currently a stub)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions