Twilio Agent Connect

A powerful SDK for building intelligent, context-aware AI agents with Twilio's communication technologies.

Documentation ◆ API Reference ◆ Python SDK ◆ TypeScript SDK ◆ Examples

Seamlessly integrate with Twilio Conversation Memory and Conversation Orchestrator to build LLM-powered agents with persistent memory and conversation context.

[!TIP] Building AI agents on AWS or Microsoft? Connect them to Twilio's voice, messaging, and conversation context with these dedicated packages: - TAC for AWS — Strands, Bedrock Agents, Bedrock AgentCore - TAC for Microsoft — Microsoft Agent Framework, Azure AI Foundry (incl. Voice Live), Azure OpenAI

Key Features¶

Multi-Channel Support: Built-in handling for Voice (Twilio ConversationRelay, OpenAI GPT-Live, OpenAI Realtime API), SMS, RCS, WhatsApp, and Chat
Outbound Conversations: Agent-initiated conversations across all supported channels
ConversationRelay-Only Mode: Get started quickly with TAC's voice plumbing (TwiML, WebSocket, callbacks) before adding Conversation Orchestrator or Conversation Memory
Memory Management: Automatic integration with Twilio Conversation Memory for persistent user context
Conversation Lifecycle: Automatic tracking of conversation sessions and state
Human Handoff: Built-in tool to route conversations to human agents via Twilio Studio Flows (including Flex)

Installation¶

pip install twilio-agent-connect

For server support (includes FastAPI and uvicorn for TACFastAPIServer):

pip install "twilio-agent-connect[server]"

For speech-to-speech voice over Twilio Media Streams:

pip install "twilio-agent-connect[server,gpt-live]"         # OpenAI GPT-Live
pip install "twilio-agent-connect[server,openai-realtime]"  # OpenAI Realtime

TAC requires Python 3.10 or newer.

Quick Examples¶

Option 1: Use the Setup Wizard

Use the Twilio Setup Wizard to automatically create a Memory Store and Conversation Configuration and generate your .env file:

git clone https://github.com/twilio/twilio-agent-connect-python.git
cd twilio-agent-connect-python
make setup  # Open http://localhost:8080

Option 2: Manual Setup

You can also create a Memory Store and Conversation Configuration manually through the Twilio Console. For a full walkthrough — credentials, Console navigation, and webhook configuration — see the TAC Quickstart.

After completing setup, here's a minimal example to get started:

Multi-Channel with OpenAI SDK¶

Use the OpenAI adapter to automatically inject conversation memory and user context into your OpenAI API calls across Voice, SMS, RCS, WhatsApp, and Chat channels.

First, install the required packages:

pip install openai python-dotenv

Note: python-dotenv is optional — TAC works with environment variables from any source (.env files, Docker, Kubernetes, CI/CD, shell exports, etc.).

Then create your application:

from dotenv import load_dotenv
from openai import AsyncOpenAI
from tac import TAC, TACConfig
from tac.adapters.openai import with_tac_memory
from tac.channels.sms import SMSChannel
from tac.channels.voice import VoiceChannel
from tac.server import TACFastAPIServer

load_dotenv()

tac = TAC(config=TACConfig.from_env())
voice_channel = VoiceChannel(tac)
sms_channel = SMSChannel(tac)
openai_client = AsyncOpenAI()

conversation_history = {}
SYSTEM_INSTRUCTIONS = (
    "You are a customer service agent speaking with a user over voice or SMS. "
    "Keep responses short and conversational — a sentence or two. "
    "Do not use markdown, asterisks, bullets, or emojis; your words will be "
    "spoken aloud or sent as plain text."
)

async def handle_message_ready(user_message, context, memory_response):
    conv_id = context.conversation_id

    if conv_id not in conversation_history:
        conversation_history[conv_id] = []
    conversation_history[conv_id].append({"role": "user", "content": user_message})

    client = with_tac_memory(openai_client, memory_response, context)

    response = await client.responses.create(
        model="gpt-5.4-mini",
        instructions=SYSTEM_INSTRUCTIONS,
        input=conversation_history[conv_id]
    )

    llm_response = response.output_text
    conversation_history[conv_id].append({"role": "assistant", "content": llm_response})

    return llm_response

tac.on_message_ready(handle_message_ready)
TACFastAPIServer(tac=tac, voice_channel=voice_channel, messaging_channels=[sms_channel]).start()

Note: See the getting started guide for complete setup instructions and .env configuration details.

That's it! The server automatically: - Creates FastAPI app with /twiml, /ws, and /webhook endpoints - Handles Voice, SMS, RCS, WhatsApp, and Chat conversations - Routes responses to the appropriate channel - Injects conversation memory and user profile into OpenAI calls

For configuration details and environment variables, see the getting started guide.

How It Works¶

TAC simplifies building AI agents by handling the integration between Twilio's communication channels and your LLM:

Message Flow¶

Webhook/Connection Received: Twilio sends webhook (SMS) or WebSocket connection (Voice) to your server
Channel Processing: Channel validates and processes the incoming event
Memory Retrieval: TAC optionally retrieves user memories and profile from Memory
Callback Invoked: Your on_message_ready callback receives user message, context, and optional memory response
Response Handling: Your callback returns a response string that TAC routes to the appropriate channel

For detailed architecture and advanced usage, see CLAUDE.md.

Telemetry¶

The SDK reports usage events to help improve the product — conversation started and ended, messages received, responses sent, and voice connection and interrupt events. Each event includes your Twilio account SID, the channel, and the SDK name and version, plus a conversation ID once one has been established and timing such as conversation duration.

Message content, transcripts, phone numbers, and other end-user identifiers are never collected.

Telemetry is on by default. To disable it, set the following environment variable:

TAC_ANALYTICS_DISABLED=true

Learn More¶

Examples & Guides: - Getting Started Guide - Setup wizard, examples, and comprehensive documentation - Partner SDK Examples - Integration examples for OpenAI, AWS Bedrock Agent, AWS Bedrock AgentCore, and AWS Strands - Speech-to-Speech Voice - Twilio Media Streams bridged to OpenAI GPT-Live or the OpenAI Realtime API - ConversationRelay-Only Mode - Get started with voice using just ConversationRelay - More examples coming soon

AWS and Microsoft connectors: - TAC for AWS — StrandsConnector, BedrockConnector, BedrockAgentCoreConnector for AWS Strands, Bedrock Agents, and Bedrock AgentCore - TAC for Microsoft — AgentFrameworkConnector and VoiceLiveConnector for Microsoft Agent Framework, Azure AI Foundry (including Voice Live), and Azure OpenAI

Documentation: - API Reference - Full API documentation generated from the source - CLAUDE.md - Architecture, development guide, and API reference - Getting Started Guide - Setup instructions, environment variables, and troubleshooting

TAC Development / Contribution¶

TAC uses uv for package management. Ensure you have it installed:

uv --version

Setup Development Environment¶

# Install all dependencies (including dev tools)
make sync

Running Tests and Checks¶

# Format code
make format

# Run linting
make lint

# Run type checking
make type-check

# Run tests
make test

# Run all checks at once
make check