Are neural networks already clean? Nick Cammarata's case against superposition

Neural networks may not be the black boxes we think they are. In a long new essay, "Neural Networks Are Already Clean", interpretability researcher Nick Cammarata argues that most neurons in today's models already do one readable job, and that we can study them directly. This article explains his argument, the classic example he overturns, and what it could mean for AI safety research.

Cammarata co-wrote "Zoom In: An Introduction to Circuits" (Distill, 2020), one of the founding papers of the field, and started the automated interpretability project at OpenAI. He now works at Goodfire, which published the essay on October 15, 2026. He calls it "a working hypothesis", and the essay is clear that the picture is not proven.

The usual view: superposition

Most interpretability work rests on one idea: superposition. The theory says a network has more concepts than neurons, so it packs several unrelated concepts into each neuron. Looking at single neurons is then mostly useless.

So researchers first "untangle" the activations with a second model, usually a sparse autoencoder (SAE), and only then study the features it finds.

Cammarata writes that he believed this for years: "For years I was a staunch superposition believer." He now argues that this first step may not be needed: "we can study networks directly, instead of first untangling them." He does not throw out SAEs and probes. In his view they are still useful, but "they belong at the end, not the start."

The cat and car neuron, looked at again

The classic example of a "mixed" neuron is neuron 4e:55 in the image model InceptionV1 (GoogLeNet). It fires for cat faces and for the fronts of cars. It was discussed in "Feature Visualization" (2017) and again in "Zoom In" (2020), which Cammarata co-wrote. For years it served as proof that unrelated concepts share neurons.

The essay gives a different reading: "it's not a cat and car neuron, it's a V-shape neuron." Pointed shapes make it fire and round shapes push it negative. That is why it also responds to V-shaped flames, water jets and fox faces. Cats and cars were simply the examples people noticed.

Small models are coarse, not messy

Cammarata finds that neurons stay readable across a 300x range in model size. What changes is how fine the concepts are:

• Small models. With less room, the network does not fold unrelated things together. Its neurons cover coarser concepts, which can be harder for people to name.
• Large models. Bigger models learn narrower neurons that are easier to recognise.

He sums this up as "generalization is fractal": the same kind of structure repeats at every scale, with more detail in bigger models.

The numbers from language models

The essay reports early audits of a language model:

A few notes on these figures:

• The unit is "neuron poles" (each neuron's positive and negative side), not whole neurons.
• The audits are preliminary, and a footnote says the numbers depend on the thresholds used.
• Dense and Mixture of Experts models "seem roughly similar".
• Goodfire says it found similar results in models up to trillions of parameters, such as Kimi K3. A companion paper will cover them.

A bitter lesson for interpretability

The essay ends with a parallel to Rich Sutton's "The Bitter Lesson", the 2019 essay arguing that simple methods that scale beat clever hand-built ones. Cammarata suggests the same may hold here: "scaling ordinary investigation beats engineering a clever new decomposition." If he is right, reading neurons one by one, at scale and with automated tools, could beat the complex methods the field has bet on.

What to do next

• Read the essay: it is a 65-minute read, with a plain text version.
• If you work with SAEs, read the section on why he now puts decomposition "at the end, not the start".
• Treat the numbers as early results until the companion paper is out.

Sources

• Neural Networks Are Already Clean, Nick Cammarata, Goodfire (text version)
• Goodfire research index (publication date)
• Zoom In: An Introduction to Circuits, Distill, 2020
• Feature Visualization, Distill, 2017
• Language Models Can Explain Neurons in Language Models, OpenAI, 2023
• The Bitter Lesson, Rich Sutton, 2019