Podcast Smart Companion

Overview

A local Streamlit app that turns any podcast episode into a structured knowledge object. Drop in a YouTube URL or an audio file and the pipeline transcribes it with Whisper, chunks the transcript, embeds every chunk into ChromaDB, generates a map-reduce summary, builds an emotion profile, extracts keywords, clusters topics, and indexes it all for semantic search and episode-to-episode recommendations.

The Problem

Podcast content is notoriously hard to search, skim, or cross-reference. Hours of audio become opaque once the episode ends. The goal was a fully local pipeline that could process any podcast and make its content queryable, summarizable, and comparable to other episodes — without sending anything to a cloud API.

My Role

Designed and built the entire pipeline solo — ingestion, transcription, chunking, embedding, summarization, emotion analysis, keyword extraction, topic clustering, and the recommendation engine. Built the Streamlit UI across all five modes.

Key Features

• —YouTube audio download via yt-dlp with creator + title slug naming
• —Local audio file upload (MP3, WAV, M4A) with configurable processing
• —Faster-Whisper ASR with VAD filtering — selectable model size (small / medium / large-v3)
• —Configurable chunk size with timestamp-aligned chunk storage in Parquet
• —ChromaDB vector index with sentence-transformers embeddings for semantic search
• —Map-reduce summarization producing TL;DR and bullet points per episode
• —Per-chunk emotion profiling with mean emotion vector across the episode
• —Keyword extraction and topic clustering (k-means, 6 clusters) per episode
• —Hybrid recommendation engine — alpha-weighted blend of semantic cosine similarity and emotion vector distance
• —Episode Viewer with timeline, summary, and emotion profile panels

Challenges

Building the recommendation system to be useful with a small local library. A pure semantic similarity approach collapses when episodes cover overlapping topics. Blending emotion profile distance with semantic centroid distance — weighted by a tunable alpha — gives meaningfully different results across episode types and made the recommendations actually interesting to explore.

What I Learned

Audio ML pipelines have a lot of moving parts that need to stay in sync — chunk boundaries, timestamp alignment, embedding indices, and feature files all referencing the same episode ID. Getting that coordination right early made the rest of the pipeline much easier to build and debug.