How to use LiveKit Voice Agent to Chat?
A voice agent is an AI program that you can talk to in real time: you speak, it recognises your words (STT, speech-to-text), sends the text to a large language model (LLM), then turns the answer back into speech (TTS, text-to-speech). LiveKit is an open-source framework that ties these three pieces together over the internet, so your agent can run on a server and answer calls from a browser or a phone.
This note walks you through a minimal but complete voice agent in Python. The program greets you and answers any question using the LLM — no custom tools required.
Watch the walkthrough
Open in Google Drive (no account needed).
What you need
- A Mac (the same steps work on Windows and Linux too).
- Homebrew installed and the
uvpackage manager — if you do not haveuv, runbrew install uv. - A LiveKit Cloud account (the free Build plan is enough).
- Your LiveKit Cloud credentials, which you copy into a file called
.env— see below.
Set up LiveKit Cloud
Sign up at cloud.livekit.io. In the dashboard, click Projects → Create Project, then open the project and go to Keys. Copy the three values:
LIVEKIT_URL— looks likewss://your-project.livekit.cloud.LIVEKIT_API_KEY— a short string likeAPIxxxxxxxxxxxxx.LIVEKIT_API_SECRET— a longer string, shown only once.
In your project folder, create a file named .env with these three lines:
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=APIxxxxxxxxxxxxx
LIVEKIT_API_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Important: never put .env on GitHub or any public site — it contains your live credentials.
Running it on your own computer
Download the zip file and
unzip it to a folder. The zip holds the program and a pyproject.toml project file.
Add your own .env file with the keys, as shown above. Then open the folder in
Visual Studio Code and type these two lines in the terminal, one at a time:
uv sync
uv run main.py console
uv sync reads pyproject.toml, downloads Python 3.13 if you do not
already have it, and installs every package the agent needs. uv run main.py console
then starts the agent in your Terminal so you can speak to it directly. You only need to run
uv sync once — only re-run it if you later change main.py or the
dependencies.
The Python files
The zip contains two files: pyproject.toml and main.py. Here they
are for reference.
pyproject.toml
[project]
name = "folder-ai-livekit-agent"
version = "0.1.0"
requires-python = ">=3.13"
dependencies = [
"livekit-agents>=1.8.4,<2",
"python-dotenv",
]
[build-system]
requires = ["uv_build>=0.13.6,<0.14.0"]
build-backend = "uv_build"
main.py
from dotenv import load_dotenv
from livekit.agents import (
Agent,
AgentServer,
AgentSession,
JobContext,
cli,
inference,
)
load_dotenv()
server = AgentServer()
@server.rtc_session()
async def entrypoint(ctx: JobContext):
session = AgentSession(
vad=inference.VAD(),
stt=inference.STT("deepgram/nova-3", language="multi"),
llm=inference.LLM("google/gemma-4-31b-it"),
tts=inference.TTS("cartesia/sonic-3", voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"),
)
agent = Agent(
instructions=(
"You are a helpful and friendly voice assistant built by LiveKit. "
"Use your knowledge to answer any question the user asks, clearly "
"and concisely. Engage naturally in conversation."
),
)
await session.start(agent=agent, room=ctx.room)
await session.generate_reply(instructions="greet the user and ask what they'd like to talk about")
if __name__ == "__main__":
cli.run_app(server)
How main.py works, in plain words
Voice Activity Detection, STT, LLM, TTS. The
AgentSession line wires four pieces together.
vad=inference.VAD() is Voice Activity Detection — it listens to the microphone
stream and tells the agent when you start and stop talking, so the LLM only processes real
speech instead of background noise. Speech is turned to text by Deepgram
(deepgram/nova-3), text is answered by Google Gemma 4
(google/gemma-4-31b-it), and the answer is spoken by Cartesia
(cartesia/sonic-3). All four run on LiveKit Cloud so you do not need separate
API keys for each provider — a single LiveKit account covers them.
No tools needed. Unlike agents that call external APIs (weather, calendar, databases), this agent answers directly from the LLM's built-in knowledge. That means zero setup beyond LiveKit Cloud credentials and zero external dependencies — but also no access to real-time or private data. Tools are optional; you add them only when you need them.
A friendly greeting. After the session starts,
generate_reply(instructions="greet the user and ask what they'd like to talk about")
makes the agent speak the first thing, so callers hear a hello instead of silence.
Talk to your agent
With uv run main.py console running in your Terminal, the program will ask your
Mac for microphone permission — click Allow. Then speak into your
microphone: you should hear the agent greet you back, and you can continue the conversation
freely. Press Ctrl+C when you want to stop.
If the Terminal does not print anything after you press play, or it shows an error about
missing credentials, stop the server (Ctrl+C), check that
.env is in the project folder, and run
uv run main.py console again.
Limitations
- You need a LiveKit Cloud account and real credentials in
.env. The program will not start without them. - The free Build plan has limits (checked 2026-11-04): 1,000 session minutes per month, a $2.50 LiveKit Inference credit (about 50 minutes of calls), cold starts — the first call after an idle period may take a few seconds to answer — and only one deployed agent. Paid plans start at $50/month.
- The LLM answers from its own knowledge only. Without tools or RAG, the agent cannot browse the web, access real-time data, or use information from your files. For that you will need to add tools (see the weather-tool version of this note) or a retrieval pipeline.
Further reading
- livekit.io — the official LiveKit homepage with pricing and docs.
- livekit/agents — the LiveKit Agents Python repository with more examples.
- LiveKit Cloud Agents docs — the official guide for building and deploying agents.
- How to use LiveKit Voice Agent with Weather Tool? — a parallel note showing how to give your agent a custom tool that calls real APIs.
- How to install Homebrew Package Manager for macOS?
— where to get
uvin one command if you have not installed it yet. - How can Retrieval-Augmented Generation (RAG) answer questions without hallucination? — a voice agent's LLM has the same hallucination problem; RAG is how you fix it.