A Speech Prosthesis Employing a Speech Synthesizer-Vowel Discrimination from Perioral Muscle Activities and Vowel Production

TL;DR. In 1985, Sugie and Tsunoda strapped a few surface electrodes around a person's mouth, classified which vowel the lips were shaping from the raw muscle activity, and drove a speech synthesizer with the result — speech without sound, pitched as a prosthesis for people who had lost their larynx. The idea was right and forty years early: it needed dense electrode arrays, sequence models, and neural vocoders that didn't exist. All three now do, wrist-EMG has shipped as a consumer product, and full-sentence silent speech from facial EMG has been demonstrated in the lab. This is one of the cleanest "ripe" papers in the BCI-adjacent literature.

The idea: speech is a motor act, so tap the motors

Speech is downstream of muscle. Before any sound exists, the brain sends motor commands through cranial nerves to the tongue, lips, jaw, and larynx; the acoustics are just the mechanical consequence. Surface electromyography (sEMG) picks up the electrical signature of those motor commands at the skin — a noninvasive tap on the efference copy of speech, one stage upstream of audio.

Sugie and Tsunoda's insight was that this tap is enough to reconstruct speech content, at least partially, and that you can close the loop: decode articulation from EMG, then re-synthesize the sound the mouth would have made. Crucially, this works even if the sound source is gone. A laryngectomy removes the vibrating source but usually leaves the articulators — lips, tongue, jaw — intact and innervated. The patient can still "mouth" words perfectly. The prosthesis, in their framing, replaces the lost source while the patient keeps supplying the articulation.

Their concrete system, as I understand it (bibliographic details are verified; the technical specifics below are from my knowledge of the paper and secondary literature, so treat exact figures with caution): a small number of surface electrode pairs — roughly three channels — over perioral muscles such as orbicularis oris and its antagonists, analog filtering and rectification of the EMG envelopes, and a simple discrimination scheme that mapped the pattern of muscle activation to one of the five Japanese vowels /a i u e o/, which then triggered a hardware speech synthesizer. The physiological logic is clean: Japanese vowels have distinct labial signatures — rounding for /u/ and /o/, spreading for /i/, wide opening for /a/ — and rounding versus spreading versus opening recruit different perioral muscles. Reported accuracies were well above chance but nowhere near usable; my recollection is on the order of 60–90% over a handful of vowels in constrained conditions, and I would not stake more precision than that.

Against the 1985 state of the art, this was genuinely novel. "Silent speech interface" did not exist as a term (Denby et al. coined the modern framing in their 2010 survey). Speech prostheses meant the electrolarynx — a buzzer held to the throat — or esophageal speech. Nobody was seriously proposing to decode linguistic content from muscle electricity in real time and re-voice it.

Why it could not work in 1985

Every box in the pipeline was the bottleneck simultaneously.

Sensing. Three bipolar surface channels sample a low-dimensional projection of a high-dimensional motor system. Speech involves dozens of muscles, and the ones carrying most of the phonetic load — the intrinsic and extrinsic tongue muscles — are barely visible from the face surface at all; you get them indirectly via submental (under-chin) channels and crosstalk. With three perioral channels you can hope to read lip rounding and jaw opening, which is why vowels were the right and roughly the only feasible target. Consonants, which depend on precise tongue place and manner, were out of reach. Today's high-density sEMG grids put 32–128 channels on a few square centimeters and, with modern source-separation, can resolve individual motor unit firings — a fundamentally richer signal, closer to reading the motor code than its blurred envelope.

Decoding. The classifier had to run on mid-1980s hardware: a microprocessor doing roughly 10^6 operations per second with kilobytes of RAM. That buys you thresholds and linear discriminants on hand-designed envelope features, evaluated frame by frame with no temporal model. But speech is a sequence problem — coarticulation means the EMG for a phoneme depends heavily on its neighbors — and sequence models worth having (HMM-GMM systems at scale, later RNNs and Transformers trained with CTC-style alignment-free losses) were years to decades away. A phone-class NPU today delivers on the order of 10^{12} ops/s, a millionfold gap.

Synthesis. Even a perfect decoder would have fed a formant synthesizer that sounded like a robot spelling out phonemes. There was no path from "sequence of discrete classes" to natural, expressive, personalized speech. Neural vocoders (WaveNet 2016, HiFi-GAN 2020) and discrete speech-token codecs closed exactly this gap: today you can map a coarse articulatory representation to natural audio in the user's own cloned voice, streaming, on-device.

Data and wearability. No datasets, no way to collect them at scale, and electrodes meant gel, tape, and wires on the face. There was no plausible product.

What changed

Three curves crossed usability thresholds independently, and the paper sits at their intersection.

First, the sensor: high-density sEMG arrays with active electrodes and dry-contact designs, plus decomposition methods that recover motor-unit spike trains from the surface. Meta's wrist-EMG band — grown out of the CTRL-labs acquisition, shipping with their AR glasses and backed by a 2025 Nature paper showing generic (cross-user, no-calibration) decoding of handwriting and gestures from thousands of participants — settled the question of whether noninvasive EMG interfaces work outside the lab. That result matters here specifically: the historical curse of EMG decoding was per-session, per-user calibration, and large-scale pretraining appears to break it.

Second, the decoder: deep sequence models trained end-to-end. Gaddy and Klein's "Digital Voicing of Silent Speech" (EMNLP 2020, best paper) is the direct modern realization of Sugie and Tsunoda: facial/submental EMG in, audible speech out, open vocabulary. Their key technical contribution addressed the field's hardest problem — EMG during silent articulation differs systematically from EMG during vocalized speech (no laryngeal activity, attenuated effort), so you can't naively train on vocalized data and test silent. They aligned silent-EMG recordings to audio from parallel vocalized recordings to transfer the target signal. Earlier, Tanja Schultz's group had spent a decade on EMG-based speech recognition, and DARPA-funded work (Meltzner et al.) showed sEMG recognition over vocabularies of a couple thousand words at low error rates. Word error rates for open-vocabulary silent EMG-to-speech remain substantial — tens of percent in the published work I know — but the trend line is steep and the datasets are tiny (tens of hours) compared to what made ASR work.

Third, the synthesizer: streaming neural vocoders and discrete speech tokens mean the output can be fluent, prosodic, and in the user's own voice, reconstructed from pre-laryngectomy recordings. For the prosthetic use case this is not cosmetic — voice identity is much of what patients report losing.

What a serious 2026 revival looks like

Keep the paper's architecture and its target population; replace every component.

Hardware: a wearable with two or three dry HD-sEMG patches — perioral, submental (for tongue), and one on the neck/cheek — totaling 64–128 channels at ~2 kHz, plus an IMU for jaw motion, streaming to a phone. The submental array is the important addition over 1985: that's where the tongue leaks to the surface, and tongue is where the consonants live.

Model: a streaming encoder (conformer-style, ~100 ms chunks) mapping multichannel EMG to discrete speech tokens, decoded by an on-device vocoder. Pretrain self-supervised on large unlabeled EMG from hundreds of users doing vocalized speech — where you get free supervision, since the audio is the label — then adapt to silent articulation per user with a short calibration, using the Gaddy-style vocalized→silent alignment trick. The Meta wrist result suggests cross-user generalization is achievable with enough pretraining scale; nobody has run that experiment at scale on the face.

Experiments: two arms. (1) Laryngectomy patients: they articulate normally, so the silent/vocalized domain gap is their permanent condition — measure intelligibility (human transcription WER) and voice-identity preservation against electrolarynx baselines. (2) Healthy users mouthing silently for private voice input to AR glasses: measure WER and latency against whisper-to-microphone. A defensible target given current trajectories: sub-15% WER on open vocabulary with under 200 ms end-to-end latency. That is a guess, not a citation.

What to reuse from 1985: the framing (prosthesis first, interface second), the perioral electrode logic, and the real-time closed loop — users adapt to the decoder if feedback is immediate, which frame-by-frame streaming enables and offline pipelines forfeit.

Already vindicated? Largely — but not finished

The scientific claim — speech content is recoverable from surface EMG — is vindicated. Gaddy & Klein produced intelligible open-vocabulary sentences from silent EMG and released their dataset; Schultz's and Meltzner's groups showed usable recognition vocabularies; adjacent invasive work (Chang lab's ECoG speech neuroprostheses, Nature 2023) proved the full decode-to-voice loop clinically, one level further upstream.

What's open: the silent/vocalized domain gap has workarounds but no principled solution; cross-user, calibration-free facial EMG at scale is undemonstrated; total public data is a few tens of hours where ASR needed hundreds of thousands; and nobody has built face-worn hardware people would actually wear — the wrist won consumer EMG partly because it's not the face, which may push the interface use case toward minimal, glasses-integrated contact points while the prosthetic use case tolerates more hardware.

Where to read it

The paper: doi.org/10.1109/tbme.1985.325564 (IEEE TBME, 1985; paywalled, short, and worth the historical read — the block diagram is the modern one). Read alongside: Denby et al., "Silent Speech Interfaces" (Speech Communication, 2010) for the field's map; Gaddy & Klein, "Digital Voicing of Silent Speech" (EMNLP 2020) plus their open dataset as the direct descendant; and the Meta/CTRL-labs wrist-sEMG Nature paper (2025) for the proof that generic noninvasive EMG decoding ships. If you want the invasive upper bound, Metzger et al. and Willett et al. (both Nature, 2023) on speech neuroprostheses show what the same pipeline achieves when you tap cortex instead of muscle.