--- source: 'https://howaiworks.ai/glossary/voice-recognition' section: glossary title: Voice Recognition description: >- How AI turns spoken words into text (speech recognition / ASR) — and how that differs from identifying who is speaking (speaker recognition). tags: - Speech Recognition - audio processing - NLP - Deep Learning category: Natural Language Processing datePublished: '2025-08-20' lastUpdated: '2026-07-23' --- # Voice Recognition > How AI turns spoken words into text (speech recognition / ASR) — and how that differs from identifying who is speaking (speaker recognition). ## Definition Voice recognition is the technology that turns spoken language into text — you talk, and a computer writes down the words. This is what powers dictation, live captions, meeting transcripts and the part of a voice assistant that figures out what you said. Its technical name is **automatic speech recognition (ASR)**, or speech-to-text, and answering *what was said* is what most people searching for "voice recognition" want. There is a second, genuinely different task that shares the same words, and the two are worth keeping straight. **Speech recognition (ASR)** answers *what was said*. **Speaker recognition**, also called voice biometrics, answers *who said it* — matching a voice against known speakers to verify an identity, the way a bank might confirm a caller by their voiceprint. A system can do one, the other, or both, but they are distinct problems: transcribing a meeting does not require knowing who anyone is, and confirming your identity does not require transcribing what you said. The rest of this page is about ASR unless it says otherwise. ## How It Works Speech is a continuous pressure wave, and text is a string of discrete symbols. ASR is the job of getting from the first to the second, and every system does it in roughly the same three moves: capture the sound, boil it down to features, and decode those features into words. **Capture.** A microphone samples the sound wave thousands of times a second. Speech recognition almost always uses a 16 kHz sampling rate, and that number is not arbitrary. The Nyquist-Shannon theorem says a sampled signal can only faithfully represent frequencies up to half its sampling rate, so 16 kHz captures everything below 8 kHz — which covers essentially all of the frequency content that makes speech intelligible. Sampling faster mostly adds data without adding useful signal for this task. **Features.** The raw waveform is redundant and noisy, so it is compressed into a compact time-frequency representation — historically mel-frequency cepstral coefficients (MFCCs), and in modern systems a mel-spectrogram or features learned by the network itself. The effect is the same: a stream of vectors describing how energy is distributed across pitch, several times per word, that a model can consume. **Decode.** The model maps that feature stream to text. This is where the field changed shape. For decades the standard was a *pipeline* of separate hand-built parts: an acoustic model (long built on hidden Markov models with Gaussian mixtures, the "HMM-GMM" era) that scored which speech sounds — phonemes, of which English has around 40 — were present, a pronunciation dictionary mapping those sounds to words, and a language model scoring which word sequences are plausible. A search algorithm then stitched the pieces into the most likely sentence. The modern approach is **end-to-end**: one [deep neural network](https://howaiworks.ai/glossary/deep-learning), usually a [Transformer](https://howaiworks.ai/glossary/transformer), trained to go straight from audio to text with no separate pronunciation dictionary. OpenAI's Whisper (Radford et al., 2022) is the well-known example — a transformer encoder-decoder trained on 680,000 hours of audio collected from the web, with the largest model at 1550M (1.55 billion) parameters. Learning acoustics, pronunciation and language jointly from that much data is what lets one model transcribe many languages and shrug off accents and noise that broke the old hand-tuned pipelines. ### Measuring accuracy: Word Error Rate You cannot say a transcript is "95% right" just by eyeballing it, because there are three distinct ways to be wrong: swap a word (**substitution**), drop one (**deletion**) or hallucinate an extra one (**insertion**). The standard metric, **Word Error Rate (WER)**, counts all three against the length of the correct reference transcript: ``` WER = (S + I + D) / N ``` where S, I and D are the number of substitutions, insertions and deletions needed to turn the system's output into the reference, and N is the number of words in the reference. Lower is better; 0% is a perfect transcript, and WER can exceed 100% when insertions pile up. A worked example. Take a 10-word reference and a hypothesis with one of each error type: ``` reference: the meeting is scheduled for three thirty on friday afternoon hypothesis: the meeting scheduled four three thirty PM on friday afternoon (del "is") (sub for→four) (ins "PM") ``` That is one substitution, one deletion and one insertion against ten reference words, so WER = (1 + 1 + 1) / 10 = **30%**. The number to anchor against is human performance: professional transcribers make roughly 5% word errors on conversational telephone speech (IBM measured 5.1% on the Switchboard benchmark; Saon et al., 2017), and that ~5% is the bar ASR is judged against. ## Types The honest top-level split is the one from the Definition — the two tasks people call "voice recognition": - **Speech recognition (ASR / speech-to-text)** answers *what was said* and produces a transcript. This is the dominant meaning. - **Speaker recognition (voice biometrics)** answers *who said it*, matching a voice against enrolled speakers to verify or identify a person. Within ASR, the other real distinction is how much a system knows about the voice in advance. **Speaker-dependent** systems are tuned to one person's voice and were common when compute was scarce and enrollment (reading a training script aloud) bought a lot of accuracy. **Speaker-independent** systems work for anyone out of the box, which is what every modern assistant and dictation service has to be — and reaching that generality without losing accuracy is exactly what large-scale training on diverse voices buys. ## Real-World Applications The clearest place ASR shows up is **dictation and transcription**: talk-to-type on phones, automatic captions on YouTube and video calls, and meeting-notes tools that produce a searchable transcript. Open models like Whisper made this cheap enough to embed almost anywhere, which is why transcription features appeared across so many apps at once. **Voice assistants** — Siri, Alexa, Google Assistant — use ASR as their front door: the speech-to-text step converts your words before any [natural language processing](https://howaiworks.ai/glossary/natural-language-processing) works out what you meant and a [text-to-speech](https://howaiworks.ai/glossary/text-to-speech) system speaks the reply. **Accessibility** is a long-standing use, from live captioning for deaf and hard-of-hearing users to voice control for people who cannot use a keyboard. In **healthcare**, ambient documentation tools listen to a clinician-patient conversation and draft the visit notes, saving time spent typing into records. The speaker-recognition sense has its own deployments, mostly in **authentication**: call centers and banks that confirm a caller by their voiceprint rather than a password. It is the same audio input as ASR but a completely different question — identity, not content. ## Challenges A single WER number is seductive and misleading, because the conditions it was measured under matter enormously. A system reported at a few percent WER on clean, read-aloud speech can be several times worse in the situations people actually use it: - **Noise and distance.** Background chatter, traffic, or a phone held across the room degrade the signal the model never cleanly hears. Far-field audio (a smart speaker across a kitchen) is much harder than a microphone at your mouth. - **Accents and dialects.** A model trained mostly on one accent under-performs on others, which makes WER a fairness issue, not just an accuracy one — the technology works less well for some groups of speakers than others. - **Code-switching.** Speakers who mix languages mid-sentence break systems that assume one language at a time. - **Domain jargon and names.** Medical terms, product names and unusual proper nouns are rare in training data, so the model falls back to a common-sounding word it has seen more often. Two problems are intrinsic to the audio rather than the conditions. **Homophones** — "their" versus "there," "to" versus "two" — are acoustically identical, so only surrounding context can disambiguate them, and a weak language model will guess wrong. And raw speech carries **no punctuation or capitalization**: sentence boundaries, commas and proper-noun capitals all have to be inferred, which is why unpunctuated ASR output can be correct word-for-word yet hard to read. ## Code Example Here is WER computed the standard way — as a word-level edit distance between reference and hypothesis — recovering the substitution, insertion and deletion counts by walking the dynamic-programming table back. Running it on the worked example above reproduces the 30% by machine rather than by hand: ```python def word_error_rate(reference, hypothesis): ref, hyp = reference.split(), hypothesis.split() n, m = len(ref), len(hyp) # edit-distance table; each substitution, insertion or deletion costs 1 d = [[0] * (m + 1) for _ in range(n + 1)] for i in range(n + 1): d[i][0] = i for j in range(m + 1): d[0][j] = j for i in range(1, n + 1): for j in range(1, m + 1): if ref[i - 1] == hyp[j - 1]: d[i][j] = d[i - 1][j - 1] else: d[i][j] = 1 + min(d[i - 1][j - 1], # substitution d[i][j - 1], # insertion d[i - 1][j]) # deletion # walk the table back to count each error type i, j, S, I, D = n, m, 0, 0, 0 while i > 0 or j > 0: if i > 0 and j > 0 and ref[i - 1] == hyp[j - 1]: i, j = i - 1, j - 1 elif i > 0 and j > 0 and d[i][j] == d[i - 1][j - 1] + 1: S += 1; i, j = i - 1, j - 1 elif j > 0 and d[i][j] == d[i][j - 1] + 1: I += 1; j -= 1 else: D += 1; i -= 1 return (S + I + D) / n, S, I, D, n reference = "the meeting is scheduled for three thirty on friday afternoon" hypothesis = "the meeting scheduled four three thirty PM on friday afternoon" wer, S, I, D, N = word_error_rate(reference, hypothesis) print(f"S={S} I={I} D={D} N={N}") print(f"WER = ({S} + {I} + {D}) / {N} = {wer:.0%}") ``` Output: ``` S=1 I=1 D=1 N=10 WER = (1 + 1 + 1) / 10 = 30% ``` Production systems use a library such as `jiwer` and normalize text first — lower-casing, expanding numbers, stripping punctuation — because otherwise "30%" versus "thirty percent" or "Friday" versus "friday" would count as errors the listener would never notice. ## Frequently Asked Questions ### Is voice recognition the same as speech recognition? Usually yes. Most people who type "voice recognition" mean automatic speech recognition (ASR) — turning spoken words into text. The phrase is also used for speaker recognition, which identifies who is speaking rather than what was said. They are different tasks that happen to share the word "voice." ### What is Word Error Rate (WER)? WER is the standard accuracy metric for speech-to-text. It counts the substitutions, deletions and insertions needed to turn the system's output into the correct reference transcript, divided by the number of words in the reference. Lower is better; 0% is a perfect transcript. ### How accurate is speech recognition? On clean, read-aloud audio the best systems are close to human transcribers, who make roughly 5% word errors on conversational telephone speech. Accuracy drops sharply with background noise, strong accents, overlapping speakers, and specialized vocabulary, so a headline number rarely holds on your own audio. ### What was the big change in how ASR works? Older systems chained together separate hand-built components (an acoustic model, a pronunciation dictionary and a language model). Modern systems such as OpenAI's Whisper are end-to-end neural networks — usually [Transformers](https://howaiworks.ai/glossary/transformer) — trained to map audio directly to text, which is simpler and more robust. ### Why does it mis-hear specific words? Homophones ("their" vs "there"), proper nouns, brand names and domain jargon are the usual culprits, because they sound identical or near-identical to more common words and the language model guesses the frequent option. Raw audio also has no punctuation, so sentence boundaries are inferred. ## Related ### Related terms - [Audio Processing (AP)](https://howaiworks.ai/glossary/audio-processing) - [Natural Language Processing (NLP)](https://howaiworks.ai/glossary/natural-language-processing) - [Text-to-Speech (TTS)](https://howaiworks.ai/glossary/text-to-speech) - [Transformer](https://howaiworks.ai/glossary/transformer) - [Deep Learning](https://howaiworks.ai/glossary/deep-learning) --- Source: https://howaiworks.ai/glossary/voice-recognition — HowAIWorks.ai