Shared and idiosyncratic coding regimes coexist in macaque IT

TL;DR: Using Neuropixels 2.0 probes in an fMRI-localized face patch of macaque anterior IT, the authors recorded well-isolated single neurons while monkeys viewed 3,000+ natural images — and found that the famous "smooth, low-dimensional, DNN-predictable" picture of IT is largely a property of pooled activity, not of the neurons themselves. Many single units are sparse, mutually uncorrelated, and — most provocatively — reliably feature-random: their image preferences replicate across repeated presentations but look like noise from the perspective of every DNN feature space tested. If this holds, a core assumption behind a decade of model–brain comparison work (that DNN-predictable population structure reflects what individual IT neurons compute) needs revision. Note: only the abstract was available to me, so this explainer builds the conceptual picture and flags what to verify in the full text rather than reporting detailed numbers.

The assumption this paper attacks

The modern story of ventral-stream vision goes roughly like this: images flow through V1 → V2 → V4 → IT, and by the time you reach inferotemporal cortex, neurons represent objects in a format that supports invariant recognition. Since ~2014, the field's strongest quantitative claim has been that deep networks trained on object recognition predict IT responses remarkably well — well enough that "DNN predictivity" became the de facto benchmark for models of the ventral stream (Brain-Score and its descendants). Alongside this, population-level analyses have painted IT as smooth (nearby images in perceptual space evoke nearby responses), low-dimensional (a modest number of latent factors explain most stimulus-driven variance), and — in face patches specifically — almost startlingly linear. The Chang & Tsao "face code" result is the canonical example: individual face-patch neurons appeared to compute roughly linear projections of a ~50-dimensional face space, so much so that faces could be reconstructed from a couple hundred neurons.

But there's a quiet methodological caveat under all of this. Much of the foundational data comes from multi-unit activity (MUA) — the pooled spiking of several unsorted neurons near an electrode — or from single units recorded one at a time with selection biases toward responsive, tuned cells, or from fMRI voxels averaging hundreds of thousands of neurons. Pooling is not innocent. Averaging over neurons acts as a low-pass filter on the representational geometry: idiosyncratic, sparse, high-frequency components of individual tuning curves cancel out, and what survives is precisely the shared, correlated, low-dimensional structure. If DNN features happen to align with that shared structure (plausible, since both DNNs and pooled neural signals emphasize smooth, generalizable features), then DNN predictivity of MUA tells you little about what individual neurons are doing.

The question this paper asks is the obvious-in-retrospect one: is the population-level picture representative of the constituent neurons? Neuropixels 2.0 makes it answerable, because you can record hundreds of well-isolated single units simultaneously from a targeted patch — no pooling, no cherry-picking responsive cells, and enough units to characterize the full distribution.

What they did

The setup, per the abstract: Neuropixels 2.0 recordings from an fMRI-localized anterior IT face patch in macaques viewing more than 3,000 natural images, with repeated presentations (essential — reliability across repeats is the crux of the whole argument). They then compared, side by side, the coding properties of single units versus MUA constructed from the same recordings: sparseness, pairwise correlations, dimensionality, response latency, decodability, and predictability by DNN feature spaces.

The findings, and the picture to hold in your head

Three results stack on top of each other.

1. Single units are sparse and heterogeneous; MUA is dense and correlated. Single-unit responses were "substantially sparser and more heterogeneous" than MUA. This alone is not shocking — pooling always densifies — but the magnitude of the gap matters, and it sets up the dimensionality claim: the sparse neurons collectively form a higher-dimensional code than the MUA suggests, and they support "efficient image identification," i.e., decoding which specific image was shown, not just which category. Sparse neurons also responded later than broadly tuned neurons — a timing signature that hints the sparse code is computed downstream of, or on top of, the shared code, possibly via recurrence or local computation.

2. Some neurons are reliably feature-random. This is the headline. Partly independent of sparseness, certain neurons had stimulus preferences that were reproducible across repeated presentations — so this is genuine, stimulus-locked signal, not trial-to-trial noise — yet discontinuous across DNN feature spaces. Concretely: take the images a neuron fires to, embed them in a DNN's feature space, and they don't cluster. No layer, no architecture (presumably; the full text should say which models were tested) gives a smooth tuning function over its features. The neuron behaves as if it computes something like a hash of the image: deterministic, repeatable, but orthogonal to every "sensible" feature axis we know how to construct.

3. These neurons are decoupled from their neighbors. The feature-random neurons were "virtually uncorrelated with the surrounding population despite being located within the same face patch." That's a striking anatomical fact. Face patches are usually described as functionally homogeneous modules — that's what makes them patches. Finding cells embedded in the patch that ignore the patch's shared signal suggests fine-grained functional heterogeneity that fMRI and MUA are blind to by construction.

Why "reliably feature-random" is the interesting claim

There's a well-known failure mode in single-unit electrophysiology: a neuron looks like it prefers a weird grab-bag of images, and the boring explanation is noise — with enough neurons and finite trials, some will spuriously "prefer" arbitrary image sets. The authors' defense, per the abstract, is reproducibility: preferences are stable across repeated presentations. The right mental test is split-half reliability — rank the 3,000 images by response on odd trials, check that even trials agree — versus DNN explained variance normalized by that reliability ceiling. A neuron with high split-half reliability but near-zero noise-corrected DNN predictivity is genuinely computing something outside DNN feature spaces. That's the plot to look for first in the full paper.

Assuming that plot holds up, what could a reliably feature-random neuron be doing? A few live hypotheses, and the paper's framing points at one:

The hash-function / expander interpretation (the authors' favored one). Think of it like the two halves of a modern retrieval system. A learned embedding (the shared, low-dimensional code) puts similar things near each other — great for generalization, categorization, transfer. But an embedding is terrible for exact identification, because similar items collide. For that you want something closer to a hash: high-dimensional, sparse, decorrelated codes where even similar inputs map to distinguishable patterns. This is exactly the architecture theorists have long attributed to the hippocampus (pattern separation via sparse dentate gyrus coding) and to cerebellum-like structures — the Marr/Albus expansion, the Drosophila mushroom body with its literally random Kenyon-cell connectivity. The provocative move here is claiming that this regime exists inside sensory cortex itself, interleaved with the smooth code, in the same face patch. Sparse coding supporting "efficient image identification" plus later response latencies fits: a fast, shared, feed-forward code for "what kind of thing is this," and a slower, expanded, idiosyncratic code for "which exact thing is this."

The mundane alternative. "Random with respect to DNN features" means random with respect to the feature spaces tested. These neurons could be smoothly tuned to variables DNNs don't represent: retinal-position-conjoined features, memory or familiarity signals, reward associations, temporal context, or fine texture statistics that ImageNet-style training discards. "Feature-random" is then a statement about our models' blind spots, not about randomness in the brain. This is still an important result — it bounds what DNN benchmarking can see — but it's a different result. The abstract doesn't say which DNNs were tested; whether they included self-supervised models, adversarially robust models, and video models matters a lot for how far "random across DNN feature spaces" generalizes.

How this sits against prior work

This isn't coming out of nowhere; it lands in the middle of several live debates. Stringer et al. (2019) showed mouse V1 population codes are much higher-dimensional than classically thought, with a power-law eigenspectrum — but that was a population-geometry claim, not a claim about individual neurons defying feature spaces. There's a long-running argument about whether DNN predictivity of IT is saturating and what the unexplained variance is (noise? wrong features? wrong objective?). And the Chang & Tsao linear face code has been contested (the Bardon/Chang exchanges over "axis coding" vs. sparse identity coding). This paper's contribution to those debates is methodological sharpness: by contrasting single units against MUA from the same recordings, it argues that the disagreement is partly about measurement granularity. The smooth-linear-code camp and the sparse-high-dimensional camp may both be right — about different components of the same circuit.

It also rhymes with an old observation the field half-buried: single-unit physiologists have always known about neurons with inexplicable preferences, which typically got excluded as "unresponsive" or "not visually driven" or simply never made it past selection criteria. Unbiased dense sampling with Neuropixels forces those cells back into the picture.

What to be skeptical about

Because only the abstract is available, all of the following are things to check, not criticisms I can substantiate:

• Spike sorting. "Well-isolated" is doing heavy lifting. A poorly isolated unit that mixes two real neurons could look sparse-and-weird while being an artifact. Reliability across repeats helps but doesn't fully rule out sorting artifacts that are stable within a session. Cross-session identification of the same units would be much stronger evidence.
• The reliability ceiling. Feature-randomness claims live or die by noise-corrected DNN predictivity. If the feature-random neurons have low absolute reliability, "reproducible" may mean weakly reproducible, and the DNN failure less meaningful.
• Which DNNs. If the model zoo was a handful of supervised CNNs, "discontinuous across DNN feature spaces" is a weak claim in 2026. It should include large self-supervised ViTs (DINOv2-class), CLIP-style models, and ideally models with retinotopy/eye-position inputs.
• Behavioral support. The abstract offers no behavioral evidence that the sparse/feature-random code is used for identification — "supports efficient image identification" is a decoding claim, not a causal one. The dual-coding interpretation is, for now, an interpretation.
• Generality. One face patch, presumably a small number of animals. Whether this holds across IT (and outside face patches) is open.

Why it matters if it holds

For the model–brain comparison enterprise, the implication is uncomfortable: DNN predictivity benchmarks scored against pooled signals may be systematically blind to a whole coding regime. A model could hit ceiling on MUA-based Brain-Score-style metrics while explaining essentially nothing about a substantial fraction of real IT neurons. Conversely, the "unexplained variance" in single-unit fits may not be noise to be corrected away — it may be reliable, structured signal that our feature spaces simply don't contain. That reframes what "a better model of IT" needs: not just better shared features, but a mechanism that generates sparse, decorrelated, expanded codes on top of them (random projections plus thresholding would be a natural first model, and would make testable predictions about the statistics of feature-random tuning).

For ML, the analogy runs the other way too: the brain appears to keep an embedding and a hash in the same representation, computed at different latencies. Retrieval-augmented systems do something similar with separate modules; sensory cortex may do it in one population.

When you read the full paper, go straight to the feature-randomness analyses — the split-half reliability versus noise-corrected DNN predictivity comparison, and the controls establishing that these responses aren't eye-movement, adaptation, or sorting artifacts. That's where the paper's central claim is either nailed down or not.