SmollerTTS, a 2.81M-parameter English TTS model
SmollerTTS is a 2.81M-parameter English voice that runs on your browser’s CPU. The weights take up 11.34 MB. Try it below.
Starting small
I originally wanted a model under a million parameters that spoke several languages. The early versions sounded robotic and skipped words. I kept changing things, but it was hard to tell whether pronunciation, the acoustic model or the waveform decoder was responsible.
I narrowed it to one English speaker and a pretrained HiFi-GAN V2 vocoder. That made debugging easier. I could compare audio reconstructed from a real recording with audio made from my model’s predictions. The vocoder was the same in both, so the difference pointed back to the acoustic model.
The result is my own 1.88M-parameter acoustic network, trained from scratch on LJSpeech, paired with a 0.93M-parameter vocoder. Both run locally; your text stays on your device.
From text to a waveform
Text becomes phonemes, the model predicts their timing and acoustic features, and the vocoder turns those features into audio. Each chunk of speech is predicted in parallel.
Text and pronunciation
The frontend expands dates, times and money into words before converting them to phonemes. “$12.99” needs a spoken reading such as “twelve dollars and ninety-nine cents.” Stress markers tell the model which syllables should receive emphasis.
This was one place where an apparently simple fix backfired. Splitting difficult words into familiar pieces helped some examples but made ordinary words worse. The current frontend keeps dictionary pronunciations, only splits tested compounds automatically, and lets you supply an explicit pronunciation when needed.
Timing and acoustic features
Phoneme and stress embeddings are summed at a width of 160, with positional encoding added. Four encoder blocks combine four-head self-attention, depthwise convolutions and small feed-forward layers. Attention gives each phone context from the sentence; the convolutions capture local patterns.
A duration head predicts how many frames each phone should occupy. The length regulator repeats the N encoded phones in H by their predicted durations, giving T audio frames in total. A position signal within each phone lets its beginning and ending sound different.
Six convolutional frame blocks then predict an 80-band mel spectrogram, a compact description of how sound energy changes over time. Pitch, voicing and energy predictions condition this stage. The acoustic network contains 1,882,807 parameters.
Each frame block uses a depthwise convolution to mix nearby frames, followed by two linear layers that expand the features from 160 to 320 channels and bring them back to 160.
Here, X is the input frame sequence, LN is layer normalization, and W₁ and W₂ are learned linear layers including their biases. The learned channel-wise scale α controls how much the block adds to its input. Padding masks are omitted from the equation.
The waveform decoder
The frozen HiFi-GAN LJ_V2 generator converts those mel features into a 22,050 Hz mono waveform. Its four upsampling stages expand each frame by 8 × 8 × 2 × 2, or 256 audio samples. Residual branches with different receptive fields help model the waveform at several scales.
I use the published vocoder weights, which contribute 925,985 parameters. Its training discriminators are not needed for playback. Together, the acoustic model and vocoder total 2,808,792 parameters.
The acoustic model learns normalized mel features. Before sending its predictions to HiFi-GAN, I restore their original scale using statistics calculated from the training set.
M contains the log-mel features and Z their normalized values. The indices t and f identify a frame and mel band; μ and σ are the per-band training mean and standard deviation. Hats mark predictions, and x̂ is the generated waveform.
Training the acoustic model
I used LJSpeech, roughly 24 hours of recordings from one English speaker. The released run used 12,134 accepted training clips, about 22.12 hours, with separate development recordings and reserved test IDs. A single speaker kept the voice consistent without needing a speaker-conditioning system.
Forced alignment estimates where each phoneme occurs in the recordings, giving the model a duration target. WORLD extracts pitch and voicing. The preparation code computes energy and mel features, then calculates normalization statistics from training frames only.
The mel features have to match what the vocoder was trained to consume. That includes the magnitude convention, frequency range, window, hop, padding and log scale. Getting the tensor shape right is not enough. I also compare the exported graphs with the Python model to catch changes introduced on the way to the browser.
The acoustic model starts from random weights. The release recipe uses 10,000 updates with AdamW, complete utterances and a learning-rate warm-up. The loss combines mel reconstruction, changes between neighbouring mel frames, phoneme duration and prosody prediction. The frozen vocoder does not participate in this training loop.
Training settings
| Initialization | Random; seed 2718 |
|---|---|
| Batch | Up to 24 utterances; 12,000 padded frames |
| Optimizer | AdamW; β = (0.9, 0.98); weight decay 0.01 |
| Learning rate | 5 × 10⁻⁴ base; 300-step warm-up; 20,000-step half-life |
| Mel features | 80 bands; FFT 1024; hop 256; 0–8,000 Hz |
| Precision | FP32; TF32 enabled on CUDA |
The loss uses absolute error for mel reconstruction, frame-to-frame differences for the delta term, and squared error on log durations. Prosody combines pitch, energy and voicing. The full configuration is in the repository.
I still want to address a mismatch between training and inference. The decoder receives recorded prosody during training and predicted prosody when generating speech. Training it to cope with its own imperfect predictions is one of the next experiments I want to run.
Getting it into the browser
The model is exported as three FP32 ONNX graphs for the encoder, frame decoder and vocoder. JavaScript handles text processing and length regulation between them. A Web Worker runs inference on a single WebAssembly CPU thread, keeping the page responsive while audio is generated.
| Graph | File size | Job |
|---|---|---|
| Encoder | 5.00 MB | Phones, durations, coarse prosody |
| Frame decoder | 2.59 MB | Frame features and mel prediction |
| Vocoder | 3.74 MB | Mel features to waveform |
| Total weights | 11.34 MB | 2.81M parameters, FP32 |
Including the runtime and pronunciation files, the download is about 30 MB. The weights are FP32, with no quantization yet. I checked the exported model against the Python version to catch differences in the browser output.
The full demo plays longer text in chunks, starting as soon as the first is ready. This article’s player waits for the complete output. Repeated requests can reuse cached audio.
Speech quality vs. model size (rough)
I scored SmollerTTS on the four English prompts used by tts-bench and added it to their published results.
SmollerTTS scores 3.55 UTMOS and 6.69% mean prompt WER here. The WER uses Whisper large-v3. Different voices, runtimes and some evaluator revisions make this a rough overview, and four sentences cannot establish a reliable ranking.
A closer comparison with tiny-voca
I also rendered a separate set of 48 prompts with SmollerTTS and two tiny-voca voices. The set covers everyday speech, narration, dialogue, numbers, technical terms and longer passages. All three voices completed every prompt.
| Model / voice | Parameters | UTMOS | WER |
|---|---|---|---|
| SmollerTTS / LJ | 2.81M | 3.77 | 0.92% |
| tiny-voca F2 | 4.54M | 4.43 | 0.26% |
| tiny-voca M2 | 4.54M | 4.33 | 0.39% |
This test uses Whisper small.en and corpus-level WER, so it should be read separately from the graph. I normalized time and unit formatting for all models after inspecting the transcripts; SmollerTTS scored 1.71% WER before that change. The full results include the details.
SmollerTTS is smaller, but tiny-voca scores better on both metrics. That matches what I hear. The words are often clear, but the delivery can still sound flat. A low word error rate does not mean a voice is pleasant to listen to.
What I would improve next
Timing and prosody come first. Longer training runs and bigger decoders did not consistently help in the early experiments. I want to test changes against fixed prompts, with people listening as well as automated scores. After that, I would like to try quantization to reduce the download.
Try it, or help to hillclimb it
The demo, source and weights are public. The repository includes data preparation, training, checkpoint resume, listening tools and ONNX export.
The training guide documents how to train from random initialization. It reproduces the released architecture and recipe, but the complete accepted-recording list from the original run has not been recovered, so it does not promise identical training data selection or checkpoint bytes.
If you’re interested in helping out, send me an email, open a pull request, or file an issue in the repository.
References
Keith Ito and Linda Johnson. The LJ Speech Dataset. 2017.
Jungil Kong, Jaehyeon Kim and Jaekyoung Bae. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. NeurIPS, 2020. Code and pretrained weights.
Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi and Hiroshi Saruwatari. UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022. Interspeech, 2022. Implementation.
5uck1ess and contributors. tts-bench. Benchmark snapshot
1dd3a81, accessed 19 November 2026.
Cite this post
@misc{chan2026smollertts,
title = {{SmollerTTS}, a 2.81M-parameter English text-to-speech model},
author = {Chan, Dun Li},
year = {2026},
month = {nov},
howpublished = {\url{https://dunlichan.com/writings/smollertts/}},
note = {Blog post}
}