Design and Implementation of a VLSI Tactile Sensing Computer

TL;DR — In 1982, Marc Raibert (later of leg-lab and Boston Dynamics fame) and John Tanner (from Carver Mead's VLSI group at Caltech) built a robot fingertip in which the silicon under the skin was the computer: conductive rubber transduced pressure onto electrodes on the surface of a custom chip, and an array of processors in that same chip ran programmable 2D convolutions on the touch image before a single bit left the finger. They could only fit 6×3 taxels and trivial logic, and nothing downstream knew what to do with a tactile image anyway. Both constraints are gone: modern CMOS puts millions of transistors under every square millimeter of skin, and manipulation policies are now visibly starved for exactly the compressed contact features this architecture was designed to produce.

The idea: sensing and computing in one substrate

Every tactile sensor before this one was a passive transducer: resistive or capacitive elements, wired out row by row to a computer somewhere else. Raibert and Tanner's move was to invert the packaging. Instead of putting a sensor on a substrate and a computer at the end of a cable, they made the substrate a computer. The top metal layer of a custom NMOS chip served as the electrode array; a sheet of pressure-sensitive conductive rubber pressed onto the bare die completed the transduction; and beneath each electrode sat a small processing element.

The processing elements formed a parallel array machine. Each cell thresholded its local pressure reading into a binary value, and the array could compute two-dimensional convolutions between the binary tactile image and programmable filter masks — edge detection, smoothing, local feature extraction — with results read out serially to the host. In modern language: an in-sensor convolutional front end, forty years before "in-sensor computing" became a term of art.

The paper also took seriously a problem that killed most large-array proposals of the era: yield. Using a Poisson defect model, they analyzed how much per-cell redundancy would be needed to get working arrays of ~1,000 cells, and tested a 3×3 array with duplicated computing elements per taxel. They were explicitly designing for a scaled future they could not yet fabricate.

Why it could not work in 1982

Transistor budget. They were working in roughly 4–5 μm NMOS, an era when a whole ambitious chip held tens of thousands of transistors (the Motorola 68000, 1979, had about 68,000). Each 1 mm² taxel had to contain its electrode and its processing element, so the logic per cell was necessarily minimal — a binary threshold and a small slice of a convolution engine. The working device was 18 cells. Today a leading-edge process fits on the order of 10^8 transistors per mm²; even a cheap mature 28 nm node gives millions per mm². The per-taxel compute budget grew by roughly five to six orders of magnitude.

Yield and packaging. A bare die with rubber pressed onto its face, wire-bonded at the edges, mounted on a fingertip that collides with the world — every part of that was fragile in 1982. Defect densities made large dies uneconomical (hence the redundancy analysis), and there was no flip-chip, no wafer-level packaging, no flexible interposers.

No consumer for the data. This is the deepest limit. In 1982 there was no algorithmic machinery that could exploit tactile images. Manipulation control was analytical; there was no learning to discover which local filters mattered, and the programmable masks had to be hand-designed. A convolution engine in the fingertip is only as useful as the downstream system's appetite for its output, and the appetite was near zero.

What changed

Three things, and they compound.

Silicon. The density argument above means a fingertip-sized die can now hold thousands of taxels at sub-millimeter pitch plus a real neural network — a small CNN or spiking encoder running at milliwatts. In-sensor and near-sensor computing is now an active field for vision (Sony stacks DSPs under image sensors in commercial parts); the tactile version is strictly easier in bandwidth terms.

Learning. The masks no longer need to be designed. Convolutional encoders trained end-to-end can discover what contact features matter for slip prediction, pose estimation, or policy conditioning. And learned encoders are naturally robust to dead taxels — Raibert and Tanner's redundancy problem partially dissolves into training-time dropout.

Demand. Vision-language-action policies (RT-2, OpenVLA, π0 and successors) have made dexterous manipulation a learning problem, and their most conspicuous weakness is contact: occluded grasps, slip, insertion, in-hand reorientation. Multiple 2023–2025 lines of work bolt tactile tokens onto transformer policies and report gains. The consumer of tactile features finally exists, and it is hungry.

Meanwhile the field's dominant tactile sensors — GelSight-style vision-based fingertips like Meta's DIGIT — are architecturally the opposite of this paper: a camera streams megapixel images of a deformed gel to an external GPU. They won on manufacturability and resolution, but they carry the costs Raibert and Tanner designed away: bulk, latency (camera frame rates, typically 30–90 Hz), power, and a fat data pipe per finger. Slip events unfold in milliseconds; human fast-adapting afferents respond at kHz-scale bandwidth. A frame-based camera in the fingertip is the wrong point in the design space for reflexes.

What a serious 2026 revival looks like

A fingertip system-on-chip: a 32×32 or 64×64 piezoresistive or capacitive taxel array (sub-mm pitch, matching human fingertip acuity of roughly 1 mm) fabricated over or flip-chip-bonded to a die containing a learned encoder. The encoder runs at 1–5 kHz on contact events, not frames — event-driven readout so that a static grasp costs almost nothing and a slip transient generates an immediate burst. Output: a few hundred bits per event over a two-wire bus. Per-finger power in the low milliwatts.

Training: distill from high-resolution vision-based tactile. Collect paired data with a GelSight-class sensor and the taxel chip on the same interactions; train the on-chip encoder to predict the features a large offline tactile model extracts from the gel images. Then freeze the encoder and feed its tokens into a VLA policy alongside vision and proprioception, fine-tuning the policy on contact-rich tasks (insertion, cable routing, regrasping) where vision-only baselines demonstrably fail.

Reuse from the paper: the core architectural bet (compute co-located with transduction, only features leave the finger); the serial feature link; the redundancy/defect-tolerance mindset, now expressed as encoders trained to tolerate dead cells. Replace: conductive rubber with modern MEMS or piezoelectric transduction (the rubber's hysteresis and drift were notorious); binary thresholding with multi-bit or event-coded amplitude; hand-programmed masks with trained weights; raster serial scan with asynchronous event readout.

The honest hard part is not silicon — it is the skin. Durable, replaceable, manufacturable elastomer-over-die packaging that survives months of collisions remains genuinely unsolved, and it is the same packaging problem the 1982 paper faced, just at a higher standard.

Descendants, vindication, and what's open

The lineage is real. John Tanner's milieu — Carver Mead's Caltech group — went on to found neuromorphic engineering, and this paper reads as a tactile sibling of Mead's silicon retina program: put the early processing in the sensor, as biology does. Modern descendants include Gordon Cheng's group at TUM (large-area robot skin built from hexagonal cells, each with local processing and event-driven communication — arguably the most direct heir), the NUS ACES asynchronously-coded electronic skin (Science Robotics, 2019, to my recollection), Chiara Bartolozzi's neuromorphic tactile work at IIT, and the broader in-sensor-computing literature. The principle is vindicated; the product does not exist. No mass-produced fingertip with embedded neural inference is deployed on any commercial manipulator today, as far as I know.

What's open: (1) the packaging/durability problem above; (2) what to compute in the skin — fixed distilled encoders versus updatable ones, and how much task-specificity belongs at the edge; (3) the interface standard — tactile has no equivalent of MIPI, and without one, every skin is a bespoke integration; (4) whether event-driven tactile actually buys policy-level wins over 100 Hz frame-based touch, which is an empirical question nobody has cleanly answered.

The 1982 paper's quiet thesis — that a fingertip should export meaning, not pressure maps — was untestable then. It is cheaply testable now, and the manipulation results of the last two years suggest the answer will matter.

Where to read it

The paper is at doi.org/10.1177/027836498200100301, IJRR vol. 1, no. 3, 1982 (bibliographic details verified; my account of internal circuit details is from memory of the literature and should be checked against the text). Read it alongside Mead's Analog VLSI and Neural Systems (1989) for where the sensing-computing fusion idea went; the GelSight and DIGIT papers for the architecture that won on manufacturability; the NUS ACES paper for the event-driven counterargument; and any recent tactile-VLA paper (e.g., work adding touch tokens to π0-class policies) to see the demand side that finally arrived.