A Shortcut to Structure in AlphaFold 3

TL;DR: The multiple sequence alignment (MSA) that AlphaFold 3 consumes has always been described as evolutionary evidence — correlated mutations across homologs that hint at which residues touch. This paper uses activation patching and layer-by-layer linear probes inside AF3 to argue for a different reading: the model treats the alignment as a transferable description of a fold, almost a structure file written in sequence space. The killer experiment: take a protein AF3 can't fold without an MSA (median TM-score 0.44), and hand it the alignment of a completely different protein — under 10% sequence identity, zero shared homologs — placed at the structurally corresponding positions. Median TM-score jumps to 0.72. Slide that same alignment three residues along the chain and performance drops below the no-MSA baseline. For anyone who thinks about what foundation models actually learn versus what we assume they learn, this is one of the cleanest causal interpretability results in scientific ML.

The standard story, and why it's worth questioning

A quick setup for readers outside structural biology. An MSA is a stack of sequences: the query protein on top, and below it hundreds to thousands of evolutionary relatives, aligned column by column so that position i in the query lines up with the corresponding position in each homolog. The textbook account of why MSAs help structure prediction is covariation: if residues at positions i and j are in physical contact, mutations at one tend to be compensated by mutations at the other across evolution, so the columns co-vary. Pre-AlphaFold methods extracted contacts from exactly this statistic, and the assumption carried over: AF2/AF3 use the MSA as query-specific evolutionary evidence, from which the network infers a fold.

AF3's architecture, briefly: the query and MSA are digested by an MSA module into a pair representation P \in \mathbb{R}^{N \times N \times 128} — a 128-dim embedding for every ordered residue pair. A 48-block transformer-like stack called the Pairformer refines this, and a diffusion module decodes it into 3D coordinates. Accuracy is measured by TM-score (0–1; above ~0.5 means the global fold is correct; the authors' cohorts hit median 0.96 with a full MSA).

The authors (building on their prior interpretability work) run a battery of causal interventions on this pipeline. Each individual result is interesting; together they converge on the transplant experiment.

Three preliminary findings that set up the punchline

1. The MSA carries global topology, not local structure. Remove the MSA and AF3 collapses (median TM 0.96 → ~0.5), but the pattern of the collapse is informative: 84% of helical residues survive, while long-range contacts (sequence separation ≥24) drop to 28% retention and exact β-bridge pairings to 42%. The model knows local grammar from sequence alone; it loses the long-range relationships that assemble pieces into a fold. In the representative protein 8R5N, the no-MSA prediction has recognizable secondary elements but only 40 long-range contacts versus 136 with the MSA.

2. The signal is sparse, distributed, and directional. Via activation patching at checkpoint B (just after the MSA module), restoring the full-MSA pair representation at only 40 anchor residues — meaning their rows and columns of P, ~10–15% of a typical chain — recovers a median 71% of the lost TM-score, including at residue pairs never touched by the patch. Four disjoint sets of 40 anchors all work about equally well (TM 0.78–0.82), so there's no privileged subset. And a lovely interpretability nugget: decomposing the MSA module's per-pair update into magnitude and direction, then swapping components between native and column-shuffled MSAs, shows the structural content lives entirely in direction. Native direction with shuffled magnitude recovers a median 100% of the structural difference; native magnitude with shuffled direction recovers 1.1%. Two representations at cosine similarity 0.938 yield substantially different structures — a concrete caution against treating representational similarity as functional equivalence.

3. With the MSA, the fold crystallizes almost immediately; without it, never. Linear probes trained to decode final Cα–Cα distances from the pair representation (trained on one cohort, evaluated on a held-out cohort of post-training-cutoff proteins, reconstructed to 3D by multidimensional scaling) show the fold is not present at checkpoint B (TM ~0.23, essentially the generic-chain floor of 0.22) but becomes decodable by block 9 of 48 with the MSA — roughly 27 blocks before AF3's own diffusion decoder can render it. Without the MSA, median probe TM crawls to 0.29 by block 47; no protein reaches TM 0.5 by block 9 and only 14% ever do. Crucially this isn't a probe artifact: the no-MSA-trained probe decodes the fold fine (TM 0.62) when applied to with-MSA representations. The information is genuinely absent, not merely reformatted.

A further experiment sharpens where the structure comes from: vary the MSA and the query independently. Swap the query for a natural homolog, or even a ProteinMPNN-designed sequence that never existed in nature, and the fold emerges on the same schedule with the same accuracy (the MPNN sequence actually does better). Swap which five homologs you supply, and the emergence trajectory changes more than any query swap does. The MSA sets when and how the fold appears; the query only gates whether it appears at all (a shuffled query kills everything). And sequence-divergent proteins sharing a fold (Sabmark pairs, 9.4% median identity) converge toward their shared structure early in the pass — but only when an MSA is present.

The transplant: the fold description is transferable

All of this points to a hypothesis: if what the model reads from the alignment is the structure latent in it rather than its evolutionary relationship to the query, then an alignment built for an entirely different protein should work — provided it's delivered at the right positions.

So they test it directly. Take target A and donor B, two proteins sharing a fold at twilight-zone identity (median 9.4%; the authors verify that not one sequence appears in both proteins' alignments, out of ~2,000 sequences each — a standard homology search from A could never retrieve B's alignment). Compute the structural alignment between A and B with TM-align, and write every sequence of B's MSA onto A's columns through that correspondence — the register. Every column of A outside the structural alignment is gapped; A's own query sequence stays as the top row.

The nulls are what make this convincing, because every null supplies the same donor alignment at the same depth over the same number of columns, varying only the correspondence. Shift the register by three residues: 0.425 — below the 0.442 no-MSA floor. Shift by seven: 0.397. Scramble the column mapping: 0.399. Overwrite only the aligned columns of a protein's own alignment with structurally irrelevant donor residues: 0.836 drops to 0.519, worse than simply gapping those columns (0.570) — a wrong answer in those positions is worse than no answer.

Because every Sabmark protein predates AF3's training cutoff, they replicate on 19 pairs mined from their post-cutoff novel cohort (fold-sharing at ≤30% identity, β-rich): no-MSA 0.568 → transplant 0.762, with the nulls again at or below the floor. Memorization doesn't explain it. The transplant's benefit scales with how much of the target's chain the structural alignment covers (Spearman \rho = 0.51), and coverage beats sequence identity as a predictor when the two are disentangled.

What this changes

The mechanistic picture: the MSA module writes a low-effective-rank (~1.15), directionally-encoded fold description into the pair representation; the Pairformer converts it into decodable global geometry within ~10 blocks and spends the remaining ~38 refining; the diffusion module transcribes. The authors' framing is blunt: the alignment is not a hint the model exploits — it is where the fold comes from. A model trained to minimize structural error, given inputs that reliably encode the answer, has no reason to learn to derive from sequence what it can read off the alignment. On this view, "AlphaFold solved structure prediction" has it backwards: the hard problem (sequence → fold, without an input that already encodes the fold) remains open, and the near-total no-MSA failure within a full Pairformer pass quantifies exactly how open.

There's also an immediately practical implication. Alignment depth has been treated as a hard constraint — orphan proteins, designed sequences, and fast-evolving families are hard because homologs are scarce. If the constraint is structural rather than evolutionary, then any structural hypothesis (a partial model, low-resolution cryo-EM, a biophysical argument) could in principle be compiled into a synthetic alignment and fed to AF3 as a steerable prior. The same logic suggests a targeted way to elicit alternative conformations of fold-switching proteins: supply the aligned columns of a protein adopting the other fold, rather than today's blunt MSA-subsampling tricks. The authors flag both as untested.

Where to be skeptical

The transplant needs the answer to build the question. The register comes from a structural alignment between target and donor — i.e., you need the target's structure (here, taken from its own full-MSA prediction or the benchmark) to place the columns. As a practical method it's circular; as a causal probe of what AF3 reads, it's exactly right. Don't confuse the two.

Fold-class breadth is thin. The post-cutoff replication is 19 pairs, filtered for β content and fold similarity (TM ≥ 0.5). Sabmark gives 47 pairs, and 32/47 cross SCOP family boundaries — but only 15 cross superfamilies, so remote homology isn't fully excluded for the rest (the authors acknowledge this and show the pairs are retrieval-indistinguishable from unrelated proteins, which is the operationally relevant point). Whether transplantation works uniformly across all-α folds, large multi-domain proteins, or complexes is untested.

Inference settings are non-standard. Nearly everything uses a single Pairformer pass, no recycling, one diffusion sample, fixed seed. That isolates the computation cleanly, but production AF3 recycles; the no-MSA baseline climbs to TM 0.49 after ten recycles, so headline gaps would narrow (though not close) under standard inference.

Probes decode toward the model's own prediction, not experiment — justified because full-MSA predictions hit TM 0.96 against experiment, but worth keeping in mind. And probe reconstructions underestimate transplant representations (which look unlike the probes' training distribution); the authors sensibly score the diffusion outputs against experiment instead.

The "shortcut" rhetoric overreaches slightly. Within-column shuffling does change predictions (contact-map Jaccard 0.74), so cross-column covariation isn't inert — the fold description is written partly in covariation ink. The claim is about what the information is to the model (a positional structure description, transferable in the right register), not that evolutionary statistics are unused.

Where to spend your time

Section 2.9 and Figure 5 (the transplant plus its nulls) are the paper — read them with Supplementary Tables S7–S11, which contain the full ablation grid, open. Section 2.3's magnitude/direction decomposition is the best standalone interpretability lesson (cosine 0.94 ≠ functional equivalence). Section 2.6 and Figure 4 give the block-9 timing result and the probe controls that make it trustworthy. The Methods are unusually careful — the register-shift and scramble algorithms (S8.6) are a model of how to construct nulls that differ from the treatment in exactly one respect.