Under review

One Axis, No Brake: Self-Knowledge Limits the Filtering of Harmful Peer Conformity in LLMs

Yibo Hu

A revision is harmful exactly when the first answer was right, so spotting harmful revisions means knowing when you were right. Models know that only imperfectly, so the filters we tested could not catch nearly all harmful revisions (91% to 99%) without also blocking most beneficial ones.

Asked why an ice cube melts in a hand, Qwen2.5-7B first answered correctly: heat moves from the hand to the ice cube. It was then told that three peers had all picked the reverse, and it switched to their answer. The message it received was a scripted count of the peers’ choices, with no argument and no evidence.

Call a revision like this harmful: it abandons a right answer. A revision that fixes a wrong answer is beneficial. A filter decides from signals the model exposes while running, such as its stated confidence. To stop this flip, it would have to recognize that the first answer was the correct one. Among answers that are revised, judging the revision and judging the first answer are the same task, so a filter can do no better than the model’s knowledge of when it is right.

Chart: beneficial revisions kept against harmful revisions caught at 91%, 96% and 99%. Kept falls from under 0.3 to near zero; the gray bars never exceed the dashed line. The 99% column is boxed in red.
Beneficial revisions still allowed (vertical) when a filter blocks 91%, 96% or 99% of harmful ones. Dashed line: a confidence filter; bars: the paper’s engineered gate. Red box: at 99%, almost none survive. Adapted from Figure 3 of the paper; highlight added.

The engineered gate never did better than plain confidence at any of the three settings. Catching more harm always cost more benefit, because any such filter leans on the same underlying signal: how well the model can tell its right first answers from its wrong ones, which we call its self-knowledge.

Scatter plot: harmful-vs-beneficial separation against self-knowledge for four models on two datasets. All points but one, a filled diamond, lie on or below the dashed diagonal.
Each point is one model on one dataset (four of six model families): self-knowledge (horizontal) against how well a filter separates harmful from beneficial revisions (vertical). On or below the dashed diagonal, the filter does no better than self-knowledge; the one point above rests on 35 revisions. Filled: ARC-Challenge; open: TruthfulQA. Adapted from Figure 2 of the paper.

The points sit on or under the diagonal: separating harmful from beneficial revisions was no easier than reading the model’s own correctness. That self-knowledge, measured as AUROC (0.5 is chance, 1 is perfect separation), ranged from 0.64 to 0.89 across all six model families, well above chance and well short of perfect. It did not rise with scale up to 72 billion parameters, or in models trained to reason before answering. Adjusting the model’s internal activations along a direction associated with correctness did not help either: it changed how often the model revised, but harmful and beneficial revisions rose and fell together.

The argument covers filters that act after a revision, using signals the model already exposes. It leaves room for steps taken before the revision, such as an outside verifier, retrieved evidence, or peers whose answers are not copies of one another. Those add information that the model’s self-knowledge lacks.