How I Built a Podcast Generator with ElevenLabs API
Disclosure: This post contains affiliate links. If you purchase through these links, I may earn a commission at no extra cost to you. Try ElevenLabs today →
How I Built a Podcast Generator with ElevenLabs API
“Can I build an app with ElevenLabs API?” Absolutely — and I proved it by building a fully automated podcast generator. In this post, I’ll walk through the architecture, code, and lessons learned so you can build your own.
The Concept
A web app where users enter a topic, and the podcast generator:
- Researches the topic via an LLM
- Writes a script with host and guest dialogue
- Generates two distinct voices using ElevenLabs
- Stitches them together with intro/outro music
- Delivers a downloadable MP3 podcast episode
Tech Stack
| Component | Tool |
|---|---|
| Backend | Python (FastAPI) |
| TTS Engine | ElevenLabs API |
| LLM Scriptwriter | OpenAI GPT-4 |
| Audio Processing | pydub + FFmpeg |
| Frontend | React |
Step 1: Script Generation
First, I prompt an LLM to write a podcast script in a structured format:
# Prompt to GPT-4
messages = [
{"role": "system", "content": "You are a podcast script writer. "
"Write a 3-minute podcast about {topic}. "
"Format as: [HOST]: dialog | [GUEST]: dialog"}
]Step 2: Voice Setup
I chose two distinct ElevenLabs voices — one for the host and one for the guest. The host voice is professional and articulate (Voice: Rachel), while the guest voice is warm and conversational (Voice: Domi).
from elevenlabs import generate, save
HOST_VOICE = "21m00Tcm4TlvDq8ikWAM" # Rachel
GUEST_VOICE = "AZnzlk1XvdvUeBnXmlld" # Domi
def generate_segment(text, voice_id, filename):
audio = generate(
text=text,
voice=voice_id,
model="eleven_multilingual_v2",
voice_settings={
"stability": 0.4, # More expressive
"similarity_boost": 0.8
}
)
save(audio, filename)
return filenameStep 3: Audio Assembly
I parse the script, generate each segment with the correct voice, then stitch everything together with pydub:
from pydub import AudioSegment
import os
def assemble_podcast(script_path, output_path):
segments = []
with open(script_path) as f:
for line in f:
if line.startswith("[HOST]:"):
voice = HOST_VOICE
text = line.replace("[HOST]:", "").strip()
elif line.startswith("[GUEST]:"):
voice = GUEST_VOICE
text = line.replace("[GUEST]:", "").strip()
else:
continue
seg_file = generate_segment(text, voice, "seg_temp.mp3")
segments.append(AudioSegment.from_mp3(seg_file))
# Add intro music
intro = AudioSegment.from_mp3("intro.mp3")
final = intro + sum(segments)
# Fade out
final = final.fade_out(3000)
final.export(output_path, format="mp3")
print(f"Podcast saved: {output_path}")Step 4: Full API Endpoint
from fastapi import FastAPI, Query
from fastapi.responses import FileResponse
app = FastAPI()
@app.get("/generate-podcast")
async def generate_podcast(topic: str = Query(...)):
script = write_script(topic) # Calls GPT-4
path = f"podcasts/{topic}.mp3"
assemble_podcast(script, path)
return FileResponse(path, media_type="audio/mpeg")What It Sounds Like
The result is surprisingly natural. The host-guest back-and-forth with different voices creates genuine conversation dynamics. Listeners can’t tell it’s AI-generated — they just hear a polished podcast episode.
Challenges & Solutions
| Challenge | Solution |
|---|---|
| Voice consistency across segments | Use same voice settings for each speaker |
| Natural pacing | Insert 300-500ms silence between turns |
| Scripts feel canned | Add conversational fillers like “Well…”, “Actually…”, “That’s a great question” |
| Audio stuttering | Prewarm ElevenLabs connection, batch generation |
Ideas to Extend This
- Multi-language podcasts — use ElevenLabs multilingual model
- Listener Q&A episodes — let users submit questions
- Daily news briefing — RSS-to-podcast pipeline
- Interview with historical figures — clone voices of historical recordings
Cost Breakdown
| Component | Cost (per 3-min episode) |
|---|---|
| ElevenLabs TTS (~1500 chars) | $0.07 |
| GPT-4 API | $0.05 |
| Total | $0.12 per episode |
Want to build your own podcast generator? Start with ElevenLabs free →
Related Free Resources
Pair this guide with the free assets in our Free Library – tools, prompt packs and templates we actually use.
