Play a recording of “eight” faster, and you shorten the word—but you also raise its pitch and shift the vocal-tract resonances. That will not give a synthesizer every voice, pitch, and speaking rate it needs. The problem captures a basic difficulty of early speech synthesis: linguistic content, timing, pitch, and sound quality all have to be controlled, yet a recorded waveform bundles them together.
Before large neural models generated speech directly from text, researchers assembled recordings, modeled an artificial vocal tract, or predicted acoustic parameters statistically. Each approach had to turn written words into a timed sequence of intelligible sounds. What changed was the material the system worked with—and how much control it gained or lost by choosing it.
What a text-to-speech system had to decide
A text-to-speech system must decide what to say before it can produce a waveform. It has to expand numbers and abbreviations, choose pronunciations, and resolve ambiguous spellings. The English word “read,” for example, has different pronunciations in present- and past-tense contexts. Once the system makes the wrong choice, waveform generation cannot fix it.
Next come the sound units, usually phonemes or context-sensitive variants, along with durations, pauses, stress, and a pitch contour. An acoustic method then turns those decisions into playable samples. The stages did not always divide neatly: stored speech units could carry some of their original prosody, while a parameter-driven system could change pitch explicitly.
One useful way to distinguish the early approaches is to ask: What does the synthesizer keep as its raw material?
- Stored words or phrases: complete recordings selected from a limited inventory.
- Short speech units: recorded segments joined to build unfamiliar utterances.
- Acoustic or articulatory parameters: numerical controls used to create a waveform.
- Statistical speech models: learned distributions that predict acoustic controls from linguistic context.
These categories are not a strict timeline. Techniques coexisted and were often combined. A telephone service, for example, could use recordings for fixed prompts and synthesize names or numbers it could not predict in advance.
Mechanical and electronic voices: demonstrating the mechanism
Long before digital text-to-speech, speech-making devices tested whether controllable physical processes could produce a human-like voice. Mechanical arrangements imitated parts of the vocal tract. Later electronic demonstrations shaped an excitation source with resonant circuits. They could not read arbitrary text, but they helped show that speech could be treated as a source of energy passing through a changing filter.
That source–filter model became a lasting engineering tool. For many voiced sounds, periodic vibration at the vocal folds supplies the excitation. The vocal tract emphasizes some frequencies and weakens others, creating resonances called formants. An unvoiced consonant such as /s/ calls for a noise-like source instead. A synthesizer can generate the appropriate excitation and pass it through a filter whose settings change over time.
The model leaves important details out. Stops involve rapid closures and releases; nasal sounds introduce further resonances and anti-resonances; neighboring sounds overlap in articulation. A simple source–filter device might produce recognizable vowels yet struggle with the transitions that make continuous speech convincing.
Formant synthesis: building speech from controls
Formant synthesis puts the source–filter idea into software. Rather than store many recordings, it describes an utterance with parameters such as fundamental frequency, voicing strength, noise level, and the frequencies and bandwidths of prominent resonances. Rules change those values as the phoneme sequence unfolds.
Take a vowel followed by a nasal consonant. Holding the vowel formants steady until the phoneme boundary, then switching abruptly, will not sound much like natural speech. The resonances shift, energy moves between frequency bands, and nasal quality starts to emerge before the neat boundary implied by a transcription. Rules for this overlap—coarticulation—were essential to intelligible parameter-driven speech.
Formant synthesizers could say words absent from any recording library. They needed relatively little stored audio and let designers change pitch or rate directly, advantages when memory was scarce or output had to be adjusted programmatically. The weakness was often audible: a small parameter set and hand-built rules missed the fine irregularities of a real voice. Listeners could understand every word and still hear a thin or buzzy speaker.
Why a phoneme label was not enough
A phoneme inventory records meaningful sound distinctions, not complete waveform instructions. The /t/ in “top” differs acoustically from the /t/ in “stop”; speaking rate and neighboring vowels change it further. Rule-based systems therefore needed context-sensitive timing and transition rules, not a table assigning one sound to each letter or phoneme. Covering more contexts also meant writing and managing more rules.
Concatenation: letting recordings carry the detail
Concatenative synthesis made a different trade-off. It kept the acoustic detail of a human performance by recording a speaker and assembling new utterances from the recordings. Whole words or phrases worked well for fixed announcements: the result was an actual performance. But recording every possible sentence was impractical.
Smaller units gave the system broader coverage. Researchers used phonemes, syllables, and diphones—segments running from the middle of one phoneme to the middle of the next. Diphones preserved human-produced transitions, where much of the information that distinguishes sounds lies, while placing joins near comparatively stable parts of the phonemes.
The joins did not disappear; they moved. Two clear units can sound wrong together if their pitch, loudness, vocal quality, or surrounding contexts differ. Sometimes there is a click. More often, a vowel changes character midway or a consonant seems detached from the next syllable. Consistent recording conditions and a single speaker helped. Signal processing could adjust pitch and timing, though large adjustments brought artifacts of their own.
Bigger inventories made unit selection possible. Instead of retaining one example of each unit and reshaping it to fit, a system stored many candidates recorded in different contexts. It then searched for a sequence that matched the intended linguistic and prosodic context while connecting well to adjacent segments. The choice balanced two costs:
- Target cost: how closely a candidate fits the intended sound, stress, duration, and place in the utterance.
- Join cost: how smoothly it connects to the units on either side.
Picking the best recording for each sound in isolation could make a poor sentence. A sequence-level search might choose a slightly less exact candidate because it joined far better with its neighbors. Long stretches taken from the same original recording could be especially convincing: their timing and voice quality already belonged together.
All those choices required a substantial, carefully prepared corpus. Recordings had to be consistent, aligned with their phonetic labels, and varied enough to cover useful contexts. Rare names, borrowed words, and unusual sound sequences could still produce awkward selections. Style or emotion was hard to change when the recordings offered no suitable examples. A voice fluent on common sentences might falter on one short, unfamiliar phrase.
Prosody was more than pitch
Correct phonemes do not guarantee a convincing utterance. Prosody includes duration, prominence, pauses, and pitch across a phrase. Compare “She borrowed the blue folder” with “She borrowed the blue folder.” The words stay the same, but the emphasis changes what a listener is likely to infer. A text-to-speech system has to make such decisions from punctuation, syntax, and limited context—even when the speaker’s intended emphasis is not evident.
Early systems commonly relied on rules for phrase boundaries, stressed syllables, and broad intonation patterns. They might lengthen a sound before a pause or raise pitch near a particular boundary. That helped avoid flat delivery, but the rules could be too general. A comma does not always call for the same pause, and questions do not all share one pitch shape.
The acoustic method imposed its own limits. A recorded unit already has a pitch and duration, and changing either too far can distort the voice. Parameter-driven synthesis permits bigger changes, but that freedom cannot tell the system which contour a person would choose. Producing the planned waveform and planning an appropriate performance are separate problems.
Statistical parametric synthesis: a compact learned voice
Statistical parametric methods learned from recordings how acoustic features varied with linguistic context. They generated speech from predicted features rather than joining waveform fragments. Hidden Markov model, or HMM, based synthesis became influential before neural waveform generation was widespread. It was not speech recognition run backward: the fields shared some modeling tools, but the inference tasks differed.
Training used paired text and speech from a speaker. The text became context-rich labels covering such details as neighboring phonemes, syllable positions, stress, and phrase location. Acoustic analysis extracted features over short frames, often including a spectral representation and fundamental frequency where voicing could be estimated. Models connected those linguistic contexts to distributions over acoustic features and durations.
For new text, the system generated labels, predicted unit durations, and produced trajectories of acoustic parameters. A vocoder converted those trajectories into a waveform. The voice model could be smaller than a large unit-selection database and made changes to rate or pitch relatively straightforward. Some methods could also adapt a voice with limited speaker-specific data, though the outcome depended heavily on the data and method.
There was an audible price for that compactness. Predicting smooth averages of many possible acoustic realizations could make speech sound muffled or over-smoothed; the vocoder could add another synthetic quality. HMM-based voices could be consistent and intelligible without retaining the detail of well-chosen recordings. Statistical and concatenative methods therefore remained useful for different jobs rather than one simply displacing the other.
For a closer look at constructing a voice from recorded acoustic material, How Early Speech Researchers Built Voices from Acoustic Patterns examines that recorded-pattern perspective.
How the techniques compared in practice
There was no single winner independent of the job. Recorded phrases suited a limited set of prompts. Parameterized generation could handle open vocabulary with little stored audio. Unit selection benefited from a carefully recorded single-speaker corpus; statistical modeling suited compact voices whose pitch or rate needed adjustment. Those are design constraints, not a ranking of old and new.
| Approach | Main material | Typical strength | Typical limitation |
|---|---|---|---|
| Recorded phrases | Complete utterances | Natural delivery for known prompts | Cannot freely express new sentences |
| Formant synthesis | Rules and acoustic controls | Small footprint and direct control | Fine vocal detail is difficult to reproduce |
| Concatenative synthesis | Recorded speech units | Retains real speech detail | Audible mismatches and corpus demands |
| Statistical parametric synthesis | Learned acoustic models | Compact, consistent, adjustable output | Over-smoothed or vocoder-colored sound |
Judging a voice also meant asking more than whether it sounded pleasant. Intelligibility concerned whether listeners recovered the words; naturalness concerned how closely the delivery resembled human speech. Listening tests with isolated sentences could measure both, but results depended on the text, language, speaker, recording quality, and playback conditions. A voice designed for predictable prompts should not be assessed as though it were meant to read unrestricted prose.
When the language had little recorded data
Corpus-heavy methods were harder to build for languages without large, prepared speech collections. Even a modest voice required decisions about spelling, pronunciation, sound-unit coverage, and recording prompts. A small corpus might miss important consonant combinations or prosodic patterns; a borrowed pronunciation scheme might represent local contrasts poorly. More audio helped only when recordings and text labels were consistent enough to train models or select units reliably.
Researchers could choose prompts to cover many useful sound contexts, rely on linguistic rules where data were thin, or share some modeling resources across related varieties when justified. None was a complete substitute for good data. A phoneme inventory does not provide natural intonation, and similar spellings do not guarantee the same pronunciation. A modest, accurately labeled corpus could be worth more than a larger collection with uncertain transcripts.
A useful way to hear the differences
Consider the sentence “Mara ordered 12 tickets, then called home.” It puts several decisions in one short test: expand “12,” pronounce “Mara,” and place a plausible pause after “tickets.” A phrase-recording system needs an exact or near-exact recording to keep it continuous. A diphone system can assemble it but may reveal its joins. A formant system builds unfamiliar sounds from rules, while a statistical model predicts acoustic parameters through the sentence. If “twelve” sounds right but the pause falls inside “tickets,” listen for a timing or phrasing error before blaming the waveform generator.