Research Blog
Social Influence and Collective Decision-Making
Language models now read one another's answers, vote together and grade each other's work. We measure how this contact changes their answers, and when agreement among models is no longer a reliable signal.

Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks
Most answer changes counted as conformity happen with no speaker at all: repeating a wrong answer is enough to move the model.
Read more →
Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity
Unanimous wrong peers pull a model off a correct answer far more than unanimous correct peers pull it off a wrong one.
Read more →
The Judge is Part of the Experiment: Auditing Peer Context Sensitivity in LLM-Based Evaluation
An AI grader can score the same answer differently after reading the peer discussion behind it, and graders disagree on the direction.
Read more →
Social Pressure Breaks Majority Voting in LLM Safety Panels
Majority voting usually cuts false alarms; if every AI reviewer reads one misleading message, the panel can flag every harmless item.
Read more →
One Axis, No Brake: Self-Knowledge Limits the Filtering of Harmful Peer Conformity in LLMs
Blocking harmful answer changes needs the model to know when it was right; models know too little to block harm without blocking help.
Read more →
Conformity Breaks Conformal Prediction
A shortlist calibrated to contain the right answer 90% of the time stops keeping that promise after the model reads unanimous wrong peers.
Read more →Monitoring and Security in Multi-Agent Systems
When AI agents pass work to one another, an attack can be split across many steps. We study attacks and failures that no single agent or single check can detect, and how monitoring can catch them.

Early Detection of Distributed Backdoors in Multi-Agent LLM Systems
An attack hidden in fragments across AI agents’ tool outputs can be flagged before the fragments are assembled, in time to stop it.
Read more →
Silence Is Endorsement: Verification-Status Laundering in LLM Agent Pipelines
A safety monitor can approve a risky action when a summary keeps the claim that it was authorized and drops the note that nobody checked.
Read more →
When Local Monitors Miss Compositional Harm
Checking each agent message on its own can pass every piece of an attack; only a view of the assembled, decoded result blocked it.
Read more →
Controllable Fake Document Infilling for Cyber Deception
Decoys that rewrite only chosen phrases of a real document are as hard to spot as freely generated fakes, and stay closer to the original.
Read more →Reliability under Uncertainty and Distribution Shift
Does an AI model know what it does not know? We study when a model's confidence matches how often it is right, how that confidence breaks down under distribution shift, when new data differ from the data the model was trained on, and how to build models that report their own uncertainty.

Safety-Flag: A Unified Benchmark for the Reliability and Calibration of LLM Content Moderators
AI moderators fail in opposite directions, flagging harmless messages or missing harmful ones, and accuracy alone does not say which.
Read more →
Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shift
On a new variety of a familiar category, an image classifier loses far more accuracy than confidence.
Read more →
Uncertainty-Aware Reliable Text Classification
A text classifier can be unsure because an input is hard to decide or because it is unlike its training text; one confidence score mixes the two.
Read more →
Multidimensional Uncertainty-Aware Evidential Neural Networks
A model can report conflicting evidence and missing evidence as separate numbers; missing evidence is what flags unfamiliar inputs.
Read more →AI for Health and Public-Interest Domains
People ask chatbots about medication, and social scientists use AI to measure conflict and politics. We evaluate whether these systems are reliable enough for that use, and build tools for it.

When Better Codebooks Are Not Enough: Predictive Performance and Behavioral Reliability in LLM Political Event Coding
A richer codebook raised a model’s score at coding political news, but not once category names were replaced by neutral placeholders.
Read more →
Can I Take Another Dose? DoseBench: Evaluating LLM Decision-Making in OTC Dosing QA
Two chatbots with about the same accuracy on dosing questions differ sharply in how often they approve a dose the label rules out.
Read more →
Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification
Rewritten as short statements a model can check, a political science codebook classifies political relations without labeled training data.
Read more →
Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries
Asked the same health questions in four languages, GPT-3.5 answered less completely and less consistently outside English, worst in Hindi.
Read more →
A Survey on the Role of Crowds in Combating Online Misinformation: Annotators, Evaluators, and Creators
Ordinary users fight misinformation in three roles: labeling it, showing by their reactions whether corrections work, and writing them.
Read more →
ConfliBERT: A Pre-trained Language Model for Political Conflict and Violence
Trained on about 7 billion words about conflict and politics, ConfliBERT scored higher than BERT on all 18 conflict-research tasks tested.
Read more →