Voice agent architecture

The Scenaro voice agent is a Python service built on LiveKit Agents — the current Transport implementation. It joins the Transport session, runs the voice pipeline (STT → LLM → TTS), and communicates with the frontend over the data channel.

Repository

Source: github.com/scenaro/voice-agent Local development is bootstrapped from the projects workspace via ./bootstrap-scenaro-local.sh.

Session lifecycle

Responsibilities

Layer	Responsibility
Platform API	Session creation, agent dispatch, scenario config loading
Voice Agent	Voice pipeline, LLM reasoning, tool selection, protocol messages
@scenaro/sdk	Data channel listen, tool RPC handlers, feature UI
Transport (LiveKit today)	WebRTC realtime engine, session/room management

Config handshake

On join, the agent sends a config payload:

{
  "type": "config",
  "features": [{ "name": "search", "type": "collection:items:search", "tools": [...] }],
  "collections": [...],
  "company_id": 2,
  "restore_state": { "checkpoint": "...", "data": {} }
}

The frontend responds with config_ack. Until ack is received, the agent may wait or retry with config_request.

Tool execution model

Tools fall into two categories:

Type	Executed by	Path
Backend tools	Voice agent	Internal Python functions
Frontend tools	Your React app	Transport RPC (LiveKit today) → SDK session → your tool handler

Frontend tools declare a frontend_method in Cockpit. The agent calls it via Transport RPC; the SDK session routes it to your handler — a tools entry on useScenaroSession or a FeatureProvider registry handler.

State sync

The frontend can send experience checkpoints on state:update. The agent uses this for context restoration across page navigations or session resumes.

Voice stack

Configured per scenario in Cockpit. The agent supports multiple STT/LLM/TTS providers via LiveKit Agents plugins (the current Transport implementation’s agent framework).

​Repository

​Session lifecycle

​Responsibilities

​Config handshake

​Tool execution model

​State sync

​Voice stack

​Related docs