Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors

TL;DR — In 1976 Stephen Grossberg published the mathematical core of what became Adaptive Resonance Theory: a competitive-learning network that self-organizes feature detectors, plus a proof that such learning is provably unstable in rich environments — new inputs relentlessly overwrite old categories — unless top-down expectations gate when plasticity is allowed. He named this the stability–plasticity dilemma, thirteen years before "catastrophic forgetting" got its modern name. It's now the central unsolved obstacle for continually-trained foundation models, and Grossberg's specific remedy — match-based, vigilance-gated plasticity — has never been seriously tried at scale.

The idea as Grossberg had it

The paper (Part I of a two-part series in Biological Cybernetics) asks how a sheet of neurons can develop feature detectors from raw input, unsupervised, in parallel, with purely local learning rules. The state of the art it was written against: Hubel and Wiesel had shown that cortical feature detectors are shaped by experience; von der Malsburg (1973) had simulated small self-organizing orientation maps; and mainstream AI had largely abandoned neural networks after Minsky and Papert (1969). Kohonen's self-organizing maps were still a few years away.

Grossberg's architecture is two layers. A field F_1 registers an input pattern x = (x_1, \dots, x_n). Bottom-up adaptive weights z_{ij} project to a field F_2 whose cells compete through a recurrent on-center off-surround shunting network — effectively a winner-take-all (Grossberg had just published the analysis of when such recurrent competitive dynamics contrast-enhance and normalize, versus amplify noise). The winning F_2 cell is the "category," and only its weights learn, via an instar rule:

where x_j is the postsynaptic (category) activity gating the update. The winner's weight vector drifts toward a time-average of the patterns it wins. This is competitive learning / online vector quantization, derived from neural dynamics — Rumelhart and Zipser's better-known competitive learning paper came ten years later, and Grossberg's priority here is generally acknowledged.

What Part I actually proves — and why the negative result is the point

The paper's positive result: if the environment is "sparse" — roughly, if there are not too many pattern classes relative to the number of category cells, and the classes are sufficiently separated — then the code stabilizes. Each category cell's weights converge, classification becomes consistent, and the network has learned a stable set of feature detectors with purely local, unsupervised rules. This was a real theorem about a nonlinear learning system, at a time when almost nothing in neural network theory came with proofs.

The negative result is the one that matters historically: in a dense environment — arbitrary sequences of arbitrarily many patterns — the code can be recoded indefinitely. A category cell tuned to pattern A gets captured by pattern B; when A returns it either gets misclassified or captures yet another cell; memories churn forever. This is catastrophic forgetting, identified analytically rather than empirically, in 1976. McCloskey and Cohen's famous demonstration in backprop networks came in 1989.

Grossberg's framing was sharper than "forgetting is bad." He posed it as a dilemma: a system plastic enough to learn new patterns fast is, by the same mechanism, plastic enough to be overwritten fast. You cannot fix it by turning down the learning rate — that just trades one horn for the other. The fix must be structural: the system needs a way to decide, per input, whether this is a familiar pattern (resonate with the existing category and refine it) or a genuinely novel one (protect existing memories and recruit fresh capacity). Part II of the series (same journal, same year) supplies the mechanism: learned top-down expectations fed back from F_2 to F_1, a match computed between expectation and input, and a mismatch-triggered reset that shuts off learning and drives a search. The vigilance parameter \rho — the match threshold — sets the granularity of categories. This matured into Adaptive Resonance Theory (Grossberg 1980; Carpenter & Grossberg's ART1, 1987).

Why it could not work then

Compute. The model is a system of coupled nonlinear ODEs — shunting dynamics at both fields plus weight dynamics — that must be numerically integrated per input presentation. In 1976 a well-equipped lab had a PDP-11-class minicomputer: roughly 10^5–10^6 instructions per second and tens to a few hundred kilobytes of core memory. Simulations in this literature ran with tens of units and dozens of patterns. A modern GPU delivers roughly 10^{14}–10^{15} ops/s — about nine to ten orders of magnitude more — and the ODEs can anyway be replaced by their discrete fixed points (softmax/top-k) at almost no cost in the theory.

No data, no task. There were no large labeled or unlabeled corpora, no sensors streaming nonstationary data, and no deployed systems that needed to learn continually. The dilemma was real but hypothetical.

No field. This was the depth of the post-Perceptrons winter. Neural networks were a fringe topic; and when they returned in the late 1980s, the community standardized on i.i.d. batch training with backpropagation — a regime that defines away the stability–plasticity problem by shuffling the data. Continual learning stayed a niche for another three decades. Grossberg's work, dense with dynamical-systems mathematics and published in a biology journal, sat largely outside the ML canon despite ~1,400 citations.

What changed

Foundation models made the dilemma unavoidable. Pretrained models must absorb new knowledge — new code APIs, new events, new domains — and full retraining costs millions of dollars, while naive fine-tuning measurably degrades prior capabilities. The mainstream toolkit is: replay buffers (store old data — expensive, sometimes prohibited), regularization like EWC (Kirkpatrick et al., 2017 — anchor important weights, degrades over long sequences), and parameter isolation (LoRA adapters, progressive nets). All three are workarounds for a missing primitive: a per-input decision about whether to learn at all, and where.

That primitive is exactly what Grossberg proposed. Note also that pieces of the machinery already exist at scale under other names: mixture-of-experts routing is competitive coding among category units; retrieval and key–value memory layers compute match scores between inputs and stored prototypes; recent test-time-memory architectures gate writes by "surprise," which is a mismatch signal in all but name. What's missing is the closed loop: match → gate plasticity → mismatch → recruit capacity.

A serious 2026 revival

Take a pretrained transformer and attach a bank of LoRA adapters (or MoE experts) at selected layers; each adapter k carries a prototype key p_k in representation space.

1. Match. For an input with pooled representation h, compute m_k = \text{sim}(h, p_k). This is the top-down expectation test, done in learned feature space rather than pixel space — the right modernization, since raw-input matching is ART's historical weakness.
2. Resonance. If \max_k m_k \ge \rho: route through adapter k^\*, and allow gradient updates only to that adapter and its key (nudging p_{k^\*} toward h — literally the instar rule). The frozen backbone plus untouched adapters are the protected memories.
3. Mismatch/reset. If \max_k m_k < \rho: recruit a fresh adapter initialized near zero, set its key to h, and train it on this data. No gradient reaches old adapters.
4. Vigilance control. Make \rho adaptive — e.g., raise it when a resonant update increases loss on a small self-generated probe set, which is the ARTMAP "match tracking" trick translated to LLMs.

Evaluate with genuine continual pretraining: a sequence of disjoint domains (code → legal → biomedical → a new year of news), no replay buffer, measuring backward transfer on held-out tasks from earlier domains and forward transfer to later ones. The honest baselines are sequential full fine-tuning, EWC, and replay; the claim to test is that match-gated plasticity beats regularization without needing stored data.

Reuse from 1976: the match/mismatch decision, vigilance, winner-gated local learning, and capacity recruitment on novelty. Replace: shunting ODEs with attention/top-k routing (their equilibrium behavior); instar dynamics with gated SGD; input-space matching with representation-space matching; and add a slow consolidation path (merging adapters into the backbone offline) that Grossberg's framework never needed but a bounded-parameter deployment does.

Has it been tried? What's still open?

ART itself was fully developed (ART1/2/3, fuzzy ART, ARTMAP) and used through the 1990s–2000s in clustering, anomaly detection, and some industrial applications — but always at small scale, and it acquired a reputation for category proliferation: with noisy inputs and fixed vigilance, it recruits categories without bound. That is the known failure mode any revival must handle, plausibly via representation-space matching and adapter merging. Modern continual-learning work has independently rediscovered fragments — dynamically expandable networks, Expert Gate, CN-DPM's novelty-triggered expert allocation, prompt-pool methods like L2P that route by key matching — but none, to my knowledge, implements the full resonance loop (match → gated plasticity → reset → recruit) in a foundation-model training run. The conceptual vindication is broader still: top-down expectation matching against bottom-up input is the core of predictive coding (Rao & Ballard 1999) and figures in modern theories of cortical function; mismatch-gated plasticity has experimental support in neuromodulatory gating of learning by novelty and surprise.

Open questions: whether a single scalar vigilance suffices or must be layer- and task-dependent; whether learned representations drift enough under continual training to invalidate stored keys (a stability problem one level up); and whether the discrete recruit-or-resonate decision can be made differentiable enough to co-train with everything else.

Where to read it

The paper is at doi.org/10.1007/BF00344744 (bibliographic details verified via OpenAlex). Read it together with Part II — Grossberg (1976), "Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions," Biological Cybernetics — where the top-down matching mechanism actually appears; then Carpenter & Grossberg (1987) for ART1 as a concrete algorithm, McCloskey & Cohen (1989) for the empirical rediscovery of the instability theorem, and Kirkpatrick et al. (2017, EWC) plus a recent continual-learning-for-LLMs survey to see how far the modern field still is from the mechanism Grossberg specified fifty years ago. Fair warning: the 1976 papers are written in a dense dynamical-systems idiom; the payoff is watching someone derive, from first principles, the exact problem your training pipeline has today.