Analog Versus Digital: Extrapolating from Electronics to Neurobiology

TL;DR — In 1998 Rahul Sarpeshkar worked out, with actual noise physics, when analog computation beats digital: analog wins at low precision because its power cost grows roughly exponentially with bits, digital wins at high precision because its cost grows only polynomially, and the true optimum is a hybrid — many low-precision analog channels with periodic digital restoration, which is plausibly what the brain does on 12 W. Nobody had a workload that lived in the analog-favorable regime, so the theory sat in the hearing-aid niche for two decades. Then neural network inference converged on exactly that regime — error-tolerant, 4–8 bit, matrix-multiply-dominated — and today's analog in-memory compute chips are, mostly without saying so, building his architecture. What's still missing is the part he actually gave us: a principled, quantitative allocator that decides per layer and per operation where analog pays.

The idea as he had it

Sarpeshkar came out of Carver Mead's neuromorphic lab at Caltech, where the ambient belief was that subthreshold analog VLSI was the road to brain-like efficiency. His 1998 Neural Computation paper is notable for refusing to be a partisan of either side. Instead he asked the engineering question quantitatively: what does it cost, in power and area, to compute at a given signal-to-noise ratio in each technology?

The core physics is simple enough to hold in your head. An analog signal on a wire carries many bits at once, and one device (a transistor's exponential I–V curve, charge summing on a node, Kirchhoff's current law) performs what would take digital logic dozens of gates. That's the analog win: computation is nearly free because physics does it. The analog loss is noise. To gain one extra bit of precision you must double the signal-to-noise amplitude ratio, which quadruples signal power relative to thermal noise. Concretely, the energy stored on a capacitor is CV^2 and its thermal noise is kT/C, so the achievable SNR is roughly CV^2/kT — energy per operation scales like kT \cdot \mathrm{SNR}, i.e. like kT \cdot 2^{2b} for b bits. Exponential in precision.

Digital pays a fixed overhead per bit — more full-adder cells, more wires, more switching — but noise is erased at every gate by signal restoration. Cost grows roughly linearly in b for adders and roughly quadratically for multipliers. Polynomial in precision.

Exponential versus polynomial means a crossover. Sarpeshkar's numbers put it somewhere around 8–10 bits (technology-dependent; take that as order-of-magnitude). Below it, analog is dramatically cheaper — often by 100–10,000× in power at 4–6 bits. Above it, analog becomes hopeless.

The hybrid prescription

If analog only wins at low SNR, how do you build a system with useful end-to-end precision? Two moves, and this is the genuinely original part.

Distribute. Channel capacity over a noisy wire grows only logarithmically with SNR, while analog power grows exponentially with it. So for a fixed information rate, it is far cheaper to use many wires at low SNR than one wire at high SNR. There is an optimal SNR per wire — modest, single-digit-bits territory — and you should fan the computation out across many such channels.

Restore periodically. Cascaded analog stages accumulate noise; after N stages your precision has degraded and keeps degrading. Digital restoration (discretize, clean up, re-emit) resets the noise accumulation, but a converter costs real energy — in modern terms, the ADC is expensive. So there is an optimal restoration interval: do as much cheap analog work as your noise budget allows, then pay for one restoration, and repeat.

He then pointed at the brain: dendrites integrate synaptic currents in graded, analog fashion; the spike at the soma is a discrete, restorative event; the axon transmits it losslessly over distance; the synapse converts back to analog. Massively distributed, low precision per component (synapses are perhaps 3–5 bits, with a lot of hedging in the literature), restoration built into the signaling. On his accounting, this hybridness is a load-bearing reason a ~10^{15}-synapse machine runs on roughly 12–20 W — around 10–100 femtojoules per synaptic event, give or take an order of magnitude.

Why it couldn't work in 1998

The theory was correct; the market conditions were maximally hostile.

Digital was free-riding on scaling. 1998 was mid-Dennard: every process node (then ~0.35–0.25 µm) delivered digital logic that was faster, denser, and lower-power simultaneously. Any advantage a hand-crafted analog chip demonstrated evaporated in two process generations. Analog, meanwhile, scales badly — device mismatch worsens as transistors shrink, supply voltages drop and eat your dynamic range, and there was (and largely still is) no synthesis toolchain: analog design was artisanal, one PhD per chip.

No workload lived in the analog-favorable regime. The crossover says analog wins below roughly 8 bits. In 1998 nothing important tolerated 4-bit arithmetic. Scientific computing wanted 64-bit floats; DSP wanted 16–24 bits; even the neural networks of the day were trained and run in floating point on workstations. The applications that genuinely fit — hearing aids, cochlear implants, sensor front-ends, all low-power, low-precision, always-on — were exactly where the ideas found refuge, including in Sarpeshkar's own later work (his analog bionic-ear processor ran on the order of hundreds of microwatts). A niche, not a paradigm.

No way to absorb analog's flaws. Analog computation comes with offsets, gain errors, temperature drift, and mismatch. In 1998 you compensated with careful design and trimming. The idea that a learning algorithm could be trained around a specific chip's imperfections — hardware-in-the-loop or noise-aware training — was not available at any useful scale.

What changed

Every one of those three conditions flipped.

Dennard scaling ended around 2005; the free lunch is over, and energy per operation, not transistors, is the binding constraint on AI compute. Meanwhile the dominant workload on Earth became neural network inference: overwhelmingly matrix–vector multiplies, provably tolerant of low precision. INT8 inference is routine, 4-bit weights are standard practice for LLM serving, and serious work exists below that. The industry spent a decade sliding down the precision axis and landed, without ever citing him, in the exact region Sarpeshkar's curves label "analog wins."

And the analog substrate improved: in-memory computing performs the matvec inside the memory array itself — conductances as weights, voltages as activations, Kirchhoff's law as the multiply-accumulate — in flash (Mythic), phase-change memory (IBM's analog-AI chips, including a 14 nm PCM chip demonstrated on speech recognition in Nature in 2023), ReRAM crossbars, and switched-capacitor/charge-domain SRAM (EnCharge and academic precursors). Crucially, hardware-aware training now exists: inject the measured noise and mismatch of the array into the training loop and the network learns to route information around the flaws. The learning algorithm became the trimming pot.

Tellingly, the headline problem in every one of these chips is the ADC — conversion energy and area dominate the budget, and everyone amortizes ADCs across many analog operations. That is not a new discovery. It is Sarpeshkar's optimal-restoration-interval problem, being re-solved empirically, tile by tile.

What a serious 2026 revival looks like

The piece nobody has built is the allocator. Current analog-AI chips make one global choice — this array is analog PCM, that block is digital — mostly by intuition and what the fab offers. Sarpeshkar's framework is quantitative enough to do better.

Take his cost model — analog energy \propto kT \cdot \mathrm{SNR} per operation, digital energy roughly \propto b^2 per multiply, restoration cost per ADC bit and channel — and recalibrate its constants against a modern process. Then treat a transformer as a graph of operations, each with a measured precision requirement (per-layer quantization sensitivity is already routinely computed, e.g. via Hessian-based metrics). Solve the joint optimization: for each operation, analog or digital; if analog, what SNR per wire, how many wires, and where the restoration points go. Co-train the network with the resulting noise model in the loop, so the software absorbs what the hardware can't fix.

The physics predicts a clean split that matches engineering intuition: weight-stationary matvecs against static weights (FFN blocks, projections) at 3–6 bit effective precision go analog; anything requiring dynamic operands — attention's activation-by-activation products, the KV cache — plus accumulation across tiles, softmax, and normalization stays digital. Reuse from the paper: the entire cost framework, the distributed-low-SNR argument, the restoration-interval optimization. Replace: subthreshold transconductors with charge-domain or PCM arrays, and hand-tuning with hardware-aware training. The deliverable is a design tool, not a chip — the thing that turns analog AI from a bet into an engineering discipline.

Already vindicated? Partly.

The qualitative prediction is vindicated: the hybrid analog-compute/digital-restoration architecture is exactly what Mythic, IBM's analog-AI program, EnCharge, and the memristor-crossbar literature build, and mixed-signal neuromorphic systems (the Boahen lab's Braindrop, the DYNAP family) descend from it directly. Sarpeshkar himself extended the program in his 2010 book Ultra Low Power Bioelectronics, which develops the analog-digital tradeoff theory in depth.

What remains open: no commercial analog accelerator has yet displaced digital at datacenter scale — conductance drift, batch-1-only weight-stationarity, and ADC overhead are still fighting the theory's promised factor of 10–100× in energy. Whether analog can serve training (which needs higher precision and weight updates) is genuinely unresolved. And the per-layer allocation theory sketched above exists nowhere in principled form; the field is doing it by folklore. On the neuroscience side, the claim that the brain sits at his hybrid optimum remains a compelling consistency argument rather than a demonstrated fact — dendritic computation has only grown in estimated importance since 1998, which cuts in his favor, but nobody has measured a cortical "SNR per wire" against his optimum.

Where to read it

The paper is at doi.org/10.1162/089976698300017052 (Neural Computation 10(7), 1998; bibliographic details verified). Read it alongside Mead's 1990 Proceedings of the IEEE piece "Neuromorphic electronic systems" for the worldview it's responding to; Mark Horowitz's ISSCC 2014 keynote "Computing's energy problem" for the modern digital energy numbers; and IBM's 2023 Nature analog-AI speech-recognition paper to see the prescription running in silicon. If you want the full theory, Sarpeshkar's Ultra Low Power Bioelectronics is the book-length version. The 1998 paper is the rare case where the right quantitative answer arrived 25 years before the question became urgent — and the question is now very urgent.