Voice Providers Configuration

Overview

Petunia supports multiple voice providers with intelligent fallback and automatic provider selection based on available API keys.

Primary Provider: Cartesia (STT/TTS)

Cartesia is Petunia's primary voice provider for voice I/O, offering:

• Cartesia Sonic 3.0: Ultra-realistic text-to-speech (TTS)
• Cartesia Ink: Speech-to-text (STT) transcription

Required Environment Variables

# Required for Cartesia to work
CARTESIA_API_KEY=sk_car_xxxxxxxxxxxxx

# Optional - override default voice ID
CARTESIA_VOICE_ID=6ccbfb76-1fc6-48f7-b71d-91ac6298247b

# Optional - override default provider selection
DEFAULT_VOICE_PROVIDER=cartesia

Fallback Providers

ElevenLabs

ElevenLabs is used as a fallback when Cartesia is unavailable.

# Required for ElevenLabs fallback
ELEVENLABS_API_KEY=xxxxxxxxxxxxx
# OR
ELEVEN_API_KEY=xxxxxxxxxxxxx

# Optional - override default voice ID
NEXT_PUBLIC_ELEVENLABS_VOICE_ID=EXAVITQu4vr4xnSDxMaL

Retell (Optional / Non-onboarding)

Retell is not used for onboarding. We keep it only for optional, client-specific integrations.

Automatic Provider Selection (Telephony vs Voice I/O)

The system automatically selects the best available telephony provider based on what’s configured:

1. Twilio (if Twilio env vars are set)
2. RingCentral (if RingCentral env vars/keys are set)
3. ElevenLabs (TTS fallback / preview)
4. Retell (optional, non-onboarding)

Provider Types

• cartesia: TTS generation using Cartesia Sonic
• cartesia-ink: Speech-to-text using Cartesia Ink
• elevenlabs: TTS and conversational AI
• retell: Phone call capabilities

Usage

API Routes

// Unified voice API - automatically selects best provider
POST /api/communications/voice/unified
{
  "provider": "auto", // or "twilio", "ringcentral", "retell", "elevenlabs"
  "phoneNumber": "+1234567890",
  "text": "Hello, this is Petunia...",
  "contactId": "contact-123"
}

// Cartesia-specific routes
POST /api/voice/cartesia/tts
POST /api/voice/cartesia/ink

Code Usage

import { getUnifiedVoiceService } from '@/lib/services';

const voiceService = await getUnifiedVoiceService();

// Initiate call - automatically uses best available provider
const response = await voiceService.initiateCall({
  phoneNumber: '+1234567890',
  text: 'Hello from Petunia',
  contactId: 'contact-123',
});

Error Handling

The system includes automatic fallback:

• If Cartesia fails, automatically tries ElevenLabs
• If primary provider unavailable, uses first available fallback
• Clear error messages logged when all providers fail

Troubleshooting

"Voice provider not found"

• Check that at least one API key is set
• Verify API keys are correct in environment variables
• Check logs for initialization errors

"Cartesia API key not configured"

• Set CARTESIA_API_KEY in your environment
• System will automatically fallback to other providers if available

Onboarding-Specific Contract (IMPORTANT)

Browser Voice (Talk through your computer)

• STT: Cartesia Ink (POST /api/voice/cartesia/ink)
• Agent brain: backend route (POST /api/voice/onboarding) which uses our onboarding prompt from lib/config/petunia-personality.ts
• TTS: Cartesia Sonic-3 (POST /api/voice/cartesia/tts)

Phone Onboarding (Call by phone)

• Tier A (ship today): turn-based conversational loop using Twilio Voice webhook (/api/webhooks/twilio/voice) + signed <Play> streaming TTS (/api/webhooks/twilio/voice/tts).
• Tier B (real-time duplex): Twilio Media Streams (WebSocket duplex audio) via realtime-server /twilio-media with streaming STT/TTS and interruption (clear). This is behind env flags.

Key point: Petunia’s agent lives on our backend (LLM + memory + tools). Telephony is transport.

Tier B Env Flags

TWILIO_VOICE_MODE=stream
TWILIO_MEDIA_WS_URL=wss://YOUR_PUBLIC_REALTIME_SERVER_HOST
TWILIO_STREAM_STT=cartesia
TWILIO_STREAM_BRAIN=anthropic
TWILIO_STREAM_TTS=elevenlabs

"No voice provider API keys found"

• Set at least one of: CARTESIA_API_KEY, ELEVENLABS_API_KEY, or RETELL_API_KEY
• System will log warnings but attempt to use Cartesia (will fail without key)

Vercel Deployment

All required dependencies are in dependencies (not devDependencies):

• eslint - Required for Next.js builds
• @typescript-eslint/parser - Required for ESLint
• @types/swagger-ui-react - Required for API docs
• swagger-ui-react - Required for API docs

Make sure these are set in Vercel environment variables:

• CARTESIA_API_KEY (recommended)
• ELEVENLABS_API_KEY or ELEVEN_API_KEY (fallback)
• RETELL_API_KEY (optional, for phone calls)