feat: add spatialreal avatar plugin

Author: 3DRX

examples/avatar_agents/README.md

@@ -15,6 +15,7 @@ These providers work with pre-configured avatars using unique avatar identifiers
 - **[LemonSlice](./lemonslice/)** - [Platform](https://www.lemonslice.com/) | [Integration Guide](https://lemonslice.com/docs/self-managed/livekit-agent-integration)
 - **[LiveAvatar](./liveavatar/)** - [Platform](https://www.liveavatar.com/)
 - **[Simli](./simli/)** - [Platform](https://app.simli.com/)
+- **[SpatialReal](./spatialreal/)** - [Platform](https://app.spatialreal.ai/) | [Integration Guide](https://docs.spatialreal.ai/guide/rtc-mode)
 - **[Tavus](./tavus/)** - [Platform](https://www.tavus.io/)
 - **[TruGen](./trugen/)** - [Platform](https://app.trugen.ai/)
 

examples/avatar_agents/spatialreal/README.md

@@ -0,0 +1,30 @@
+# LiveKit SpatialReal Avatar Agent
+
+This example demonstrates how to create a lip-synced avatar agent using [SpatialReal](https://www.spatialreal.ai/).
+
+Create your SpatialReal app and avatar [here](https://app.spatialreal.ai/).
+
+## Usage
+
+* Update the environment:
+
+```bash
+# SpatialReal config
+export SPATIALREAL_API_KEY="..."
+export SPATIALREAL_APP_ID="..."
+export SPATIALREAL_AVATAR_ID="..."
+
+# OpenAI config (or other models, tts, stt)
+export OPENAI_API_KEY="..."
+
+# LiveKit config
+export LIVEKIT_API_KEY="..."
+export LIVEKIT_API_SECRET="..."
+export LIVEKIT_URL="..."
+```
+
+* Start the agent worker:
+
+```bash
+python examples/avatar_agents/spatialreal/agent_worker.py dev
+```

examples/avatar_agents/spatialreal/agent_worker.py

@@ -0,0 +1,46 @@
+import logging
+import os
+
+from dotenv import load_dotenv
+
+from livekit.agents import Agent, AgentServer, AgentSession, JobContext, cli
+from livekit.plugins import openai, spatialreal
+
+logger = logging.getLogger("spatialreal-avatar-example")
+logger.setLevel(logging.INFO)
+
+load_dotenv()
+
+server = AgentServer()
+
+
+@server.rtc_session()
+async def entrypoint(ctx: JobContext):
+    session = AgentSession(
+        llm=openai.realtime.RealtimeModel(voice="alloy"),
+        resume_false_interruption=False,
+    )
+
+    if not os.getenv("SPATIALREAL_API_KEY"):
+        raise ValueError("SPATIALREAL_API_KEY is not set")
+
+    if not os.getenv("SPATIALREAL_APP_ID"):
+        raise ValueError("SPATIALREAL_APP_ID is not set")
+
+    if not os.getenv("SPATIALREAL_AVATAR_ID"):
+        raise ValueError("SPATIALREAL_AVATAR_ID is not set")
+
+    spatialreal_avatar = spatialreal.AvatarSession()
+    await spatialreal_avatar.start(session, room=ctx.room)
+
+    # start the agent, it will join the room and wait for the avatar to join
+    await session.start(
+        agent=Agent(instructions="Talk to me!"),
+        room=ctx.room,
+    )
+
+    session.generate_reply(instructions="say hello to the user")
+
+
+if __name__ == "__main__":
+    cli.run_app(server)

livekit-agents/pyproject.toml

@@ -108,6 +108,7 @@ soniox = ["livekit-plugins-soniox>=1.4.4"]
 speechify = ["livekit-plugins-speechify>=1.4.4"]
 speechmatics = ["livekit-plugins-speechmatics>=1.4.4"]
 spitch = ["livekit-plugins-spitch>=1.4.4"]
+spatialreal = ["livekit-plugins-spatialreal>=1.4.4"]
 tavus = ["livekit-plugins-tavus>=1.4.4"]
 trugen = ["livekit-plugins-trugen>=1.3.13"]
 turn-detector = ["livekit-plugins-turn-detector>=1.4.4"]

livekit-plugins/livekit-plugins-spatialreal/README.md

@@ -0,0 +1,104 @@
+# LiveKit Agents Plugin for SpatialReal Avatar
+
+This plugin provides integration with [SpatialReal](https://www.spatialreal.ai/)'s avatar service for lip-synced avatar rendering in LiveKit voice agents.
+
+## Installation
+
+```bash
+pip install livekit-plugins-spatialreal
+```
+
+## Configuration
+
+Set the following environment variables:
+
+```bash
+SPATIALREAL_API_KEY=your-api-key
+SPATIALREAL_APP_ID=your-app-id
+SPATIALREAL_AVATAR_ID=your-avatar-id
+
+LIVEKIT_URL=
+LIVEKIT_API_KEY=
+LIVEKIT_API_SECRET=
+```
+
+## Usage
+
+```python
+from livekit.agents import Agent, AgentSession, JobContext, cli, WorkerOptions
+from livekit.plugins import spatialreal
+
+class VoiceAssistant(Agent):
+    def __init__(self):
+        super().__init__(
+            instructions="You are a helpful voice assistant."
+        )
+
+async def entrypoint(ctx: JobContext):
+    await ctx.connect()
+
+    # Configure your pipeline components (STT, LLM, TTS)
+    session = AgentSession(
+        stt=stt,
+        llm=llm,
+        tts=tts,
+    )
+
+    # Initialize and start the avatar session
+    avatar = spatialreal.AvatarSession()
+    await avatar.start(session, room=ctx.room)
+
+    # Start the agent session
+    await session.start(
+        agent=VoiceAssistant(),
+        room=ctx.room,
+    )
+
+if __name__ == "__main__":
+    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
+```
+
+For production agents, catch `SpatialRealException` so you can decide whether to fail the job or continue without avatar output:
+
+```python
+try:
+    await avatar.start(session, room=ctx.room)
+except spatialreal.SpatialRealException as err:
+    logger.error("Avatar startup failed: %s", err)
+    raise
+```
+
+## API Reference
+
+### `AvatarSession`
+
+Main class for integrating SpatialReal avatars with LiveKit agents.
+
+#### Constructor Parameters
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `api_key` | `str` | SpatialReal API key (or set `SPATIALREAL_API_KEY`) |
+| `app_id` | `str` | SpatialReal application ID (or set `SPATIALREAL_APP_ID`) |
+| `avatar_id` | `str` | Avatar ID to use (or set `SPATIALREAL_AVATAR_ID`) |
+| `console_endpoint_url` | `str` | Custom console endpoint URL |
+| `ingress_endpoint_url` | `str` | Custom ingress endpoint URL |
+| `avatar_participant_identity` | `str` | LiveKit identity for avatar participant |
+| `idle_timeout_seconds` | `int` | LiveKit egress idle timeout in seconds (`0` uses server defaults) |
+| `sample_rate` | `int \| None` | Optional avatar audio sample rate override |
+
+#### Methods
+
+- `start(agent_session, room, *, livekit_url, livekit_api_key, livekit_api_secret)`: Start the avatar session and hook into the agent's audio output. Raises `SpatialRealException` with actionable context if startup fails.
+- `aclose()`: Clean up avatar session resources.
+
+When starting, the plugin automatically sets `lk.publish_on_behalf` to the
+agent participant identity for avatar worker association in LiveKit frontends.
+
+### `SpatialRealException`
+
+Exception raised for SpatialReal-related errors.
+
+## License
+
+MIT

livekit-plugins/livekit-plugins-spatialreal/livekit/plugins/spatialreal/__init__.py

@@ -0,0 +1,37 @@
+"""SpatialReal avatar plugin for LiveKit Agents.
+
+This plugin provides integration with SpatialReal's avatar service for
+lip-synced avatar rendering in LiveKit voice agents.
+
+See https://docs.spatialreal.ai for more information.
+
+Usage:
+    from livekit.plugins.spatialreal import AvatarSession
+
+    avatar = AvatarSession()
+    await avatar.start(agent_session, room=ctx.room)
+"""
+
+from .avatar import AvatarSession, SpatialRealException
+from .version import __version__
+
+__all__ = [
+    "AvatarSession",
+    "SpatialRealException",
+    "__version__",
+]
+
+# Try to register plugin if Plugin class is available (livekit-agents >= 1.3)
+try:
+    from livekit.agents import Plugin
+
+    from .log import logger
+
+    class SpatialRealPlugin(Plugin):
+        def __init__(self) -> None:
+            super().__init__(__name__, __version__, __package__, logger)
+
+    Plugin.register_plugin(SpatialRealPlugin())
+except (ImportError, AttributeError):
+    # Plugin registration not available in older versions
+    pass