How to use LiveKit Voice Agent with Weather Tool?
A voice agent is an AI program that you can talk to in real time: you speak, it recognises your words (STT, speech-to-text), sends the text to a large language model (LLM), then turns the answer back into speech (TTS, text-to-speech). LiveKit is an open-source framework that ties these three pieces together over the internet, so your agent can run on a server and answer calls from a browser or a phone.
This note walks you through a minimal but complete voice agent in Python. The program greets you, tells you the weather (a demo tool).
Watch the walkthrough
Open in Google Drive (no account needed).
What you need
- A Mac (the same steps work on Windows and Linux too).
- Homebrew installed and the
uvpackage manager — if you do not haveuv, runbrew install uv. - A LiveKit Cloud account (the free Build plan is enough).
- Your LiveKit Cloud credentials, which you copy into a file called
.env— see below.
Set up LiveKit Cloud
Sign up at cloud.livekit.io. In the dashboard, click Projects → Create Project, then open the project and go to Keys. Copy the three values:
LIVEKIT_URL— looks likewss://your-project.livekit.cloud.LIVEKIT_API_KEY— a short string likeAPIxxxxxxxxxxxxx.LIVEKIT_API_SECRET— a longer string, shown only once.
In your project folder, create a file named .env with these three lines:
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=APIxxxxxxxxxxxxx
LIVEKIT_API_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Important: never put .env on GitHub or any public site — it contains your live credentials.
Running it on your own computer
Download the zip file and
unzip it to a folder. The zip holds the program and a pyproject.toml project file.
Add your own .env file with the keys, as shown above. Then open the folder in
Visual Studio Code and type these two lines in the terminal, one at a time:
uv sync
uv run main.py console
uv sync reads pyproject.toml, downloads Python 3.13 if you do not
already have it, and installs every package the agent needs. uv run main.py console
then starts the agent in your Terminal so you can speak to it directly. You only need to run
uv sync once — only re-run it if you later change main.py or the
dependencies.
The Python files
The zip contains two files: pyproject.toml and main.py. Here they
are for reference.
pyproject.toml
[project]
name = "folder-ai-livekit-agent"
version = "0.1.0"
requires-python = ">=3.13"
dependencies = [
"livekit-agents>=1.8.4,<2",
"python-dotenv",
"requests",
]
[build-system]
requires = ["uv_build>=0.13.6,<0.14.0"]
build-backend = "uv_build"
main.py
import requests
from dotenv import load_dotenv
from livekit.agents import (
Agent,
AgentServer,
AgentSession,
JobContext,
RunContext,
cli,
function_tool,
inference,
)
load_dotenv()
def _geocode(city: str) -> tuple[float, float, str] | None:
"""Resolve a city name to (lat, lon, resolved_label) using Open-Meteo geocoding."""
r = requests.get(
"https://geocoding-api.open-meteo.com/v1/search",
params={"name": city, "count": 1, "language": "en", "format": "json"},
timeout=10,
)
results = r.json().get("results")
if not results:
return None
best = results[0]
name = best.get("name", city)
country = best.get("country")
label = f"{name}, {country}" if country else name
return best["latitude"], best["longitude"], label
def _ip_location() -> tuple[float, float, str] | None:
"""Resolve lat/lon AND a human-readable city/region label from the caller's IP."""
try:
info = requests.get("https://ipinfo.io/json", timeout=10).json()
lat_str, lon_str = info["loc"].split(",")
parts = [p for p in (info.get("city"), info.get("region"), info.get("country")) if p]
label = ", ".join(parts) if parts else "your current location"
return float(lat_str), float(lon_str), label
except Exception:
return None
def _fetch_weather(latitude: float, longitude: float) -> dict:
params = {
"latitude": latitude,
"longitude": longitude,
"current": "temperature_2m,weather_code,relative_humidity_2m,apparent_temperature",
"daily": "temperature_2m_max,temperature_2m_min,weather_code",
"temperature_unit": "celsius",
"timezone": "auto",
"forecast_days": 1,
}
data = requests.get(
"https://api.open-meteo.com/v1/forecast",
params=params,
timeout=10,
).json()
return data
# WMO weather code -> short English description
_WMO_CODES = {
0: "clear sky",
1: "mainly clear", 2: "partly cloudy", 3: "overcast",
45: "fog", 48: "freezing fog",
51: "light drizzle", 53: "moderate drizzle", 55: "dense drizzle",
56: "light freezing drizzle", 57: "dense freezing drizzle",
61: "slight rain", 63: "moderate rain", 65: "heavy rain",
66: "light freezing rain", 67: "heavy freezing rain",
71: "slight snow fall", 73: "moderate snow fall", 75: "heavy snow fall",
77: "snow grains",
80: "slight rain showers", 81: "moderate rain showers", 82: "violent rain showers",
85: "slight snow showers", 86: "heavy snow showers",
95: "thunderstorm", 96: "thunderstorm with slight hail", 99: "thunderstorm with heavy hail",
}
@function_tool
async def lookup_weather(
context: RunContext,
location: str = "",
):
"""
Look up weather information. If the user mentions a specific city or
place (e.g. "Tokyo", "Mountain View"), pass it in. Otherwise - when the
user says "here", "my city", "where I am", or does not name any place -
call this tool with no location argument. It will automatically detect
the user's approximate city from their IP address.
"""
lat = lon = None
resolved = ""
if location and location.strip():
geo = _geocode(location.strip())
if geo:
lat, lon, resolved = geo
if lat is None:
ip = _ip_location()
if ip is None:
return {"error": "Could not detect your location automatically."}
lat, lon, resolved = ip
try:
data = _fetch_weather(lat, lon)
except Exception as e:
return {"error": f"Weather service unavailable: {e}"}
cur = data.get("current", {}) or {}
daily = data.get("daily", {}) or {}
code = cur.get("weather_code")
condition = _WMO_CODES.get(code, f"code {code}") if code is not None else "unknown"
return {
"location": resolved,
"temperature_c": cur.get("temperature_2m"),
"feels_like_c": cur.get("apparent_temperature"),
"condition": condition,
"humidity_percent": cur.get("relative_humidity_2m"),
"today_high_c": (daily.get("temperature_2m_max") or [None])[0],
"today_low_c": (daily.get("temperature_2m_min") or [None])[0],
}
server = AgentServer()
@server.rtc_session()
async def entrypoint(ctx: JobContext):
session = AgentSession(
vad=inference.VAD(),
stt=inference.STT("deepgram/nova-3", language="multi"),
llm=inference.LLM("google/gemma-4-31b-it"),
tts=inference.TTS("cartesia/sonic-3", voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"),
)
agent = Agent(
instructions=(
"You are a friendly voice assistant built by LiveKit. "
"You can answer weather questions. You do NOT need to ask the "
"user which city they are in - if they do not name a specific "
"city, just call lookup_weather() with no arguments and it will "
"auto-detect their location. Only pass a location when the user "
"explicitly names one (e.g. 'what's the weather in Tokyo?'). "
"Never ask 'which city?' - always try to answer directly first."
),
tools=[lookup_weather],
)
await session.start(agent=agent, room=ctx.room)
await session.generate_reply(instructions="greet the user and ask about their day")
if __name__ == "__main__":
cli.run_app(server)
How main.py works, in plain words
Voice Activity Detection, STT, LLM, TTS. The
AgentSession line wires four pieces together.
vad=inference.VAD() is Voice Activity Detection — it listens to the microphone
stream and tells the agent when you start and stop talking, so the LLM only processes real
speech instead of background noise. Speech is turned to text by Deepgram
(deepgram/nova-3), text is answered by Google Gemma 4
(google/gemma-4-31b-it), and the answer is spoken by Cartesia
(cartesia/sonic-3). All four run on LiveKit Cloud so you do not need separate
API keys for each provider — a single LiveKit account covers them.
A real weather tool. The lookup_weather function is wrapped with
@function_tool, which tells the LLM “you can call this function when the
user asks about the weather.” The function is fully working: it calls two free
Open-Meteo endpoints
(one for city name → lat/lon geocoding, one for the actual forecast) and falls back to
ipinfo.io to guess the
caller's approximate city from their IP when no place is named. It returns temperature
(current + today's high/low in Celsius), feels-like temperature, humidity, a condition
description (mapped from WMO weather codes like 65 → "heavy rain"), and
the resolved city name. If any API fails, it returns an "error" key so the LLM
can apologise instead of fabricating numbers.
Prompt engineering saves a tool call. The agent instructions are deliberately
long: “You do NOT need to ask the user which city they are in — if they do not
name a specific city, just call lookup_weather() with no arguments and it will
auto-detect their location. Never ask ‘which city?’ — always try to answer
directly first.” Without this paragraph, the LLM would often waste a turn asking for
clarification, which sounds robotic and slows the conversation.
A friendly greeting. After the session starts,
generate_reply(instructions="greet the user and ask about their day") makes the
agent speak the first thing, so callers hear a hello instead of silence.
Talk to your agent
With uv run main.py console running in your Terminal, the program will ask your
Mac for microphone permission — click Allow. Then speak into your
microphone: you should hear the agent greet you back, and you can continue the conversation
freely. Press Ctrl+C when you want to stop.
If the Terminal does not print anything after you press play, or it shows an error about
missing credentials, stop the server (Ctrl+C), check that
.env is in the project folder, and run
uv run main.py console again.
Limitations
- You need a LiveKit Cloud account and real credentials in
.env. The program will not start without them. - The free Build plan has limits (checked 2026-11-04): 1,000 session minutes per month, a $2.50 LiveKit Inference credit (about 50 minutes of calls), cold starts — the first call after an idle period may take a few seconds to answer — and only one deployed agent. Paid plans start at $50/month.
- The weather APIs are free but rate-limited. Open-Meteo and ipinfo.io are both free for personal use, but if you run the agent heavily you may need a paid tier or a different provider.
Further reading
- livekit.io — the official LiveKit homepage with pricing and docs.
- livekit/agents — the LiveKit Agents Python repository with more examples.
- LiveKit Cloud Agents docs — the official guide for building and deploying agents.
- How to install Homebrew Package Manager for macOS?
— where to get
uvin one command if you have not installed it yet. - How can Retrieval-Augmented Generation (RAG) answer questions without hallucination? — a voice agent's LLM has the same hallucination problem; RAG is how you fix it.