Nine Emotion Centroids: A Label-Free Valence Axis That Transfers Across Four Modalities

TL;DR: Average the hidden states of ~50 short stories for each of nine emotions, take the top principal component of those nine centroids, and you get a single direction in the encoder's representation space that scores sentiment nearly as well as a fully supervised probe — with roughly 1,500× fewer labels. The same three-line recipe produces a working valence direction in a text LLM, CLIP's image encoder, CLAP's audio encoder, and an EEG foundation model, none of which share pretraining. Projecting the direction out of an LLM's residual stream tanks sentiment accuracy by up to 37 points while matched random directions cost under 1 point. The paper is also unusually honest about its boundaries: the trick works for continuous attributes like valence and fails flat on seven categorical-concept tests, the EEG axis quietly requires supervision, and steering works in Llama/Mistral but not Qwen/Gemma. One headline claim — the "universal cross-modal classifier" — is, on close reading, close to a tautology of the AUC metric; I'll explain why below.

The problem: concept directions usually cost labels

A recurring finding in interpretability is that high-level attributes live along linear directions in a model's residual stream: refusal, truthfulness, sentiment, "is about France." The standard way to find such a direction is supervised — collect thousands of labeled examples, fit a linear probe on frozen activations, and call the weight vector the axis. AxBench, the current reference point for this game, uses 100–1,000 labeled examples per concept, and it's text-only.

This paper asks: what's the minimum supervision needed to find the valence axis — the pleasant/unpleasant dimension of affect — and does the same procedure find it in encoders trained on completely different data? The answer to the first question is startlingly small: nine emotion category names and nine writing prompts, which the author counts as 18 "supervision events." The answer to the second is yes, with caveats worth understanding.

There's also a subtle design point in why nine categories rather than the obvious two. If you fit a direction contrasting "positive" vs. "negative" examples, you impose the axis by construction and you tangle valence with arousal (excited-positive and terrified-negative are both high-arousal). Spreading nine emotion centroids across Russell/Ekman affect space — anger, disgust, fear, sadness, amusement, joy, inspiration, tenderness, neutral — and letting PCA find the direction of maximum spread means the data decides which axis dominates. Valence wins because it's the biggest source of variance among emotion categories in these representations.

The recipe

Three steps, no training, one forward pass per sentence:

1. Author ~50 paragraph-length narrative sentences per emotion ("Footsteps in the hallway, but she was supposed to be alone" for fear). No polarity labels, no curation beyond the emotion assignment.
2. Embed each through a frozen encoder f_\theta at a fixed layer \ell, final-token residual stream, and average within emotion to get nine centroids \mu_c \in \mathbb{R}^d.
3. Center the 9 \times d centroid matrix and take the top right-singular vector v_1. That's the V-axis. Score any new input x as the scalar projection \langle x, v_1 \rangle.

Two implementation facts matter. First, single words don't work: using just the nine emotion words (N_c=1) gives chance AUC (0.50) on SST-2; you need ≥20 paragraph-length narratives per emotion, plateauing around 50. The direction lives in narrative context representations, not lexical embeddings. Second, layer choice matters and reveals something: mid-depth (\ell \approx L/2) is optimal for standard Llama/Mistral/Qwen models, but reasoning-distilled models (DeepSeek-R1-Distill, Qwen3-thinking) suppress the axis at mid-depth and only recover it near the last layer — a depth shift that's specific to the valence axis, not categorical concepts. The author speculates chain-of-thought distillation repurposes intermediate layers for reasoning and pushes affect readout toward the unembedding.

The evidence, modality by modality

The same construction, swapping stories for modality-native stimuli:

• Text: Llama-3-8B-Instruct, block 20: AUC 0.772 on SST-2 vs. 0.828 for a probe trained on 6,920 labels (93%). Qwen3-8B: 0.787 vs. 0.840, but only at block 31 (depth 0.86) — the depth shift again.
• Vision: CLIP-image, axis built from ~50 emotion-correlated images per category: Pearson r = +0.636 against crowdworker valence ratings on 11,811 EmoSet images (random-direction null: |r| ≤ 0.112).
• Audio: CLAP-audio on ESC-50 environmental sounds: mean AUC 0.906 across 50 one-vs-rest splits — the strongest result, plausibly because CLAP's text-audio contrastive objective imports language-shaped affect structure directly.
• EEG: CBraMod foundation model on FACED (123 subjects watching emotional videos): AUC 0.720 ± 0.055, subject-stratified splits, pooled p < 4×10⁻⁸.

The EEG entry deserves an immediate asterisk that the paper itself supplies: the unsupervised recipe fails on EEG. PC1 of the nine EEG centroids picks up arousal, not valence — joy (+0.71) and fear (+0.69) both project positive. The reported EEG axis is a Fisher LDA fit on binary FACED valence labels, i.e., supervised. So the "label-free axis in four modalities" claim is really three modalities; for EEG only the downstream classifier head is label-free. This is a genuinely interesting negative result in its own right — arousal, not valence, dominates class variance in EEG representations, which matches decades of affective-neuroscience findings that arousal is the easier physiological signal — but it means the brain leg of the story is weaker than the abstract's rhythm suggests.

A related soft spot for vision and audio: the per-emotion stimulus pools are "class-correlated" images from EmoSet itself and clips from ESC-50 itself. Choosing 50 disgust-evoking images requires knowing which images are disgusting — implicit valence supervision smuggled in through curation, and a mild train/test-distribution overlap with the evaluation sets. The label-count arithmetic (1,546× fewer labels) is cleanest for text.

Deleting the direction

The strongest section is the causal one (§4). Using Arditi et al.'s directional ablation, at every layer and token position the residual stream h is replaced by h - \langle h, \hat{v}_1\rangle \hat{v}_1 — the V-axis is orthogonally projected out, forcing the model to compute without that dimension available. Then sentiment classification accuracy is read off. The control is identical ablation of three random directions of matched norm.

The gap is enormous and replicates across two model families. The 37-point collapse in Qwen3-8B is consistent with the depth-shift story: if the valence readout is concentrated late, there's no redundant earlier copy to fall back on. The paper is careful about scope — this is inference-time projection evidence, not counterfactual circuit intervention, and the ablation is sign-insensitive by construction so no sign-search inflates it. One caveat the paper doesn't dwell on: the readout head was trained on unablated activations, so part of the drop measures "the head relied on this direction," not only "the model computes sentiment through it." Still, the matched-norm random null cleanly kills the "any big perturbation hurts" objection.

Curiously, the causal story splits by operation: ablation works everywhere, steering doesn't. Adding a scaled V-axis back to generation shifts output sentiment in Llama and Mistral (Spearman ρ up to 0.45, p < 10⁻⁹) but is dead flat in Qwen and Gemma (|ρ| < 0.05) — and base-vs-instruct doesn't predict it, family does. Something about pretraining lineage determines whether a probeable direction is also a usable control knob. That's a small, odd, potentially important observation for anyone doing activation steering.

The "universal classifier" — and why it's less than it sounds

Section 5's headline: a 2-parameter logistic head trained only on SST-2 text scores achieves AUC 0.961 on images, 0.764 on audio, 0.828 on EEG, with all 12 off-diagonal cells of a 4×4 source→target matrix ≥ 0.70, while a generic 16-D cross-encoder shared subspace sits at chance (0.525).

Here's the thing to notice, and the paper's own Table 3 gives it away: the columns of the transfer matrix are nearly constant. Every source modality gets 0.961 on images, 0.828 on EEG, 0.764 on audio, 0.725 on text. Why? AUC is invariant under monotone transforms. A logistic head \sigma(as+b) with positive slope is a monotone function of the 1-D projection score s, so the target-modality AUC depends only on the target's own V-axis projections and the sign convention — the trained slope and intercept are irrelevant to ranking. And the sign is fixed per modality using that modality's own positive class. So the off-diagonal cells contain essentially no information beyond the diagonal: the "classifier transfers across modalities" framing is nearly a tautology of the metric.

What does survive this critique is meaningful: (a) the same nine-anchor recipe yields a discriminative valence direction in four independently trained encoders, (b) the generic-substrate baseline shows that cross-encoder representational agreement à la the Platonic Representation Hypothesis captures object identity, not affect — you need the task-relevant direction, and one dimension of it beats sixteen generic ones by 0.18–0.44 AUC, and (c) calibrated CKA tests show the V-axis projections across text/image/audio are statistically aligned (p ≤ 0.032), though the paper concedes the alignment magnitude is small. The right summary is "the same cheap recipe finds valence everywhere," not "one classifier reads brains."

Where it breaks, on purpose

The paper's most credibility-building material is its negative results. Seven independent tests show the recipe does nothing for categorical concepts: AxBench's 500 concepts (recovery statistically indistinguishable from a random-direction null, KS p = 0.41), CIFAR-100 superclass centroids, Park–Choe–Veitch word-pair directions (which are near-orthogonal to the V-axis, mean |cos| = 0.038), multi-concept probes, long-tail retrieval. The proposed boundary: PC1-of-centroids works only for continuous attributes where categories naturally order along a dominant variance axis. Valence may be nearly unique in being both simple enough (1-D) and universal enough (present in any affect-bearing data distribution) for nine anchors to pin it down. The obvious falsifiable follow-ups — toxicity, formality, intensity — are named but not run.

How much to believe, and what it would mean

Strengths: the text result is clean and cheap to replicate (~2 minutes of A100 time for 450 forward passes; total paper compute < 50 A100-hours); the ablation effect sizes are far beyond any plausible noise; the failure boundary is mapped with unusual rigor; and every headline caveat appears in the paper's own text rather than a reviewer's rebuttal.

Weaknesses: single author, and the EEG and cross-modal claims are softer than the abstract's cadence implies — supervised EEG axis, curation-based implicit supervision in vision/audio, same-dataset axis construction and evaluation for EmoSet/ESC-50, and the AUC-invariance issue hollowing out the transfer matrix. The CKA alignment (the one genuinely cross-modal statistic) is "small-magnitude but reliably above chance," which is a much more modest claim than "the same direction appears in brains." ESC-50 "valence" labels for sounds like washing machine are also a fairly loose ground truth.

If the core result holds, the useful takeaways are: (1) for continuous attributes, concept directions may be recoverable from a handful of category anchors rather than labeled contrasts — a practical, near-free tool worth adding to the interpretability kit; (2) valence structure is convergently encoded across wildly different training regimes, extending the platonic-convergence story from object identity to affect; (3) probeable ≠ steerable, and the family-specificity of steering plus the reasoning-distillation depth shift are two clues about how post-training reshapes where and how affect is represented.

What to read: §2 (the recipe — you can reimplement it in an afternoon) and §4 (the ablation panel, the paper's strongest evidence). Read §5 with the AUC-invariance point above in mind, and Appendix H if you care about the EEG arousal-vs-valence story, which is quietly one of the more interesting findings here.