Speaker diarization
Also written: who spoke when
Speaker diarization identifies who spoke when in an audio recording by segmenting speech by speaker.
Speaker diarization is like putting labels on an audio recording to show who is talking at different times. Imagine listening to a podcast with two hosts. Diarization would help a computer figure out which parts of the conversation were spoken by Host A and which parts by Host B.
This technology is useful for many applications. For example, it can help automatically create transcripts that show who said what, making it easier to follow long meetings or interviews. It can also be used to analyze conversations, like identifying the main speakers in a customer service call.
Speaker diarization is the process of partitioning an audio stream into homogeneous segments according to the identity of the speaker. It aims to answer the question 'who spoke when?' by segmenting the audio and assigning each segment to a specific speaker. This is typically achieved by extracting acoustic features from the audio, clustering these features to identify distinct speaker characteristics, and then assigning speaker labels to the corresponding time segments.
Tradeoffs involve accuracy versus computational cost. More sophisticated models can achieve higher accuracy but require more processing power. Real-world performance can be affected by factors such as overlapping speech, background noise, and the number of speakers.
Recent mentions of speaker diarization are tied to advancements in speech-to-text technologies, particularly from Google Gemini. These updates focus on improving transcription accuracy, language support, and integrating AI analysis for audio data, making it easier for developers to build applications that understand and process spoken content.
- speech_to_text Gemini 3.5 TranscribeGoogle · Gemini Models ·
- speech_to_text Gemini 3.5 TranscribeGoogle · Gemini Models ·
- A Developer’s Guide to Gemini 3.5 TranscribeGoogle · Google Cloud Tech on YouTube ·
- Turn Audio into Action with Gemini 3.5 TranscribeGoogle · Google Cloud Tech on YouTube ·
- speech_to_text Gemini 3.5 TranscribeGoogle · Gemini Models ·
- speech_to_text Gemini 3.5 TranscribeGoogle · Gemini Models ·
- Gemini API Cookbook: Add Gemini Transcribe quickstart and update READMEs (#1348)Google · Gemini API Cookbook ·
- Voice OverviewxAI · xAI Docs: Guides ·
- Google Gemini
- Google Gemini's Transcribe product is highlighted for its advanced speech-to-text features, including speaker diarization, which helps developers accurately transcribe and analyze audio.
- xAI Grok
- xAI Grok offers Voice APIs that include transcription capabilities, implying support for features like speaker diarization to distinguish different voices in audio.
- Automatic Speech Recognition
- ASR converts spoken audio into text, but does not
- Speaker Recognition
- Speaker recognition identifies a specific person from their
- Voice Activity Detection
- VAD detects the presence of speech in an audio signal,
The concept of speaker diarization emerged from research in speech processing and pattern recognition, with early work dating back to the 1990s. The goal was to enable machines to understand and process human speech more effectively by distinguishing between different speakers in an audio recording. Source ↗
Record a short conversation with a friend using your phone. Upload the audio to a service that offers speaker diarization, if available, and observe how it segments the speech by speaker. Alternatively, listen to a podcast with multiple hosts and try to mentally segment who is speaking when, mimicking the function of diarization.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- Code snippetTranscribe audio in 25 languages with batch or streaming modesTranscribe audio files or streams into text across 25 languages, supporting batch and real-time processing. · 5 min
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
Questions people ask
- What is the difference between speaker diarization and transcription?
- Transcription converts spoken words into text. Speaker diarization adds another layer by identifying which speaker is associated with each piece of transcribed text.
- How does speaker diarization work?
- It typically involves analyzing audio features to group segments belonging to the same speaker and then assigning labels to these segments.
First seen on Subvolts Jul 28, 2026. Updated Oct 19, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.