FermionResearch/Phonon-2

TL;DR: Phonon-2 is a 164 MB English ASR model distilled from NVIDIA's Parakeet TDT 0.6B v3, with an encoder quantized to five learned weight levels (~2.1 bits/weight) via quantization-aware training. It averages 5.21% WER on the Open ASR Leaderboard's seven English sets — within 0.25 points of its 2.5 GB full-precision teacher, 15x smaller in bytes — and runs at 174x realtime on an M5 MacBook Air via MLX. The interesting part isn't the architecture (it's Parakeet's); it's the demonstration that sub-3-bit QAT can hold near-teacher accuracy on a production-grade ASR encoder, with a genuinely cross-platform inference stack.

What it is and why it matters

Extreme low-bit quantization usually comes with visible accuracy decay, and most published ASR compression stops at int8 or 4-bit. Phonon-2 claims each encoder weight is held at one of five learned levels — about 2.1 bits of information per weight — and the resulting 164 MB model lands at 5.21% average WER vs. 4.96% for the 2,508 MB teacher. On two of the seven sets it actually beats the teacher (AMI meetings: 9.37 vs 9.42; VoxPopuli: 2.46 vs 3.19). That's an unusually clean result for this compression ratio, and the kind of thing worth noting if you care about on-device speech.

The "five values" detail (the repo topic is parakeet_tdt_five_value) suggests something like learned ternary-plus codebook quantization rather than uniform low-bit — five levels is log2(5) ≈ 2.32 bits nominal, and the README's "about 2.1 bits" implies entropy coding on top. The README gives no training recipe, no paper, and no detail on how the levels are learned, so treat the method as a black box for now.

The numbers

Per the README (leaderboard code, full test sets; † rows from the published leaderboard):

Speed claims: 174x realtime on an M5 MacBook Air, 143x on 8 Zen 5 cores, 6,680x on one H100 at batch 128. The headline — "most accurate open English ASR under 900 MB" — holds against the models in the table, and it clearly beats Whisper large-v3-turbo everywhere except LibriSpeech. Note the teacher still wins the average, and Canary 180M Flash remains better on clean read speech and financial calls (SPGISpeech 2.04 vs 3.70).

What's actually there, and caveats

• Weights on HF under CC-BY-4.0 (inheriting Parakeet's license), MLX format; CLI/server code at github.com/fermionresearch/phonon, Apache 2.0. Docker images for CPU and CUDA.
• No paper, no training code, no QAT details. The model card is marketing-flavored (it's the engine inside a commercial dictation app, Detta). The compression method is the scientifically interesting bit and it's undocumented.
• English only. Decoder/tokenizer and output conventions are Parakeet's; presumably only the encoder is quantized, and the TDT decoder is small anyway.
• Benchmark mix: some rows are the leaderboard's published numbers, others re-run locally — reasonable, but not independently verified. Nothing on accented, far-field, or noisy speech beyond what AMI/Earnings-22 cover.
• The Phonon-1 → Phonon-2 jump (6.56 → 5.21 at 2.5x smaller) suggests the QAT recipe improved substantially between versions.

Try it

On Apple silicon:

pip install fermion-research mlx mlx-audio mlx-lm soundfile scipy zstandard
phonon transcribe recording.wav --json   # word-level timestamps

Anywhere else, via Docker:

docker run --rm -v "$PWD":/audio -v phonon-cache:/home/phonon/.cache \
  ghcr.io/fermionresearch/phonon-cpu:2.0.6 transcribe phonon-2 /audio/recording.wav

Model card: https://huggingface.co/FermionResearch/Phonon-2 · docs at fermionresearch.com/docs/speech. If sub-3-bit QAT for encoders is what interests you, watch for whether they publish the method — right now you can only verify the artifact, not the recipe.