Early Detection of Distributed Backdoors in Multi-Agent LLM Systems
The attack we studied hides an encrypted program in fragments spread across the outputs of AI agents’ tools; the fragments are joined only after the run. A detector that reads the run as it unfolds flagged nearly every successful attack while there was still time to stop it, but how early it fired depended on how conspicuous the fragments were.
No agent ever holds the whole malicious program, and no single tool output contains it, so a check that judges each step on its own has no complete attack to find. The paper calls this a distributed backdoor. We built a working version on a team of one manager agent and three sub-agents, ran it with clean and with tampered tools on five language models and two tool environments, and recorded the step at which each fragment arrived. The question was whether a defender reading the shared record step by step could tell that a run was going bad before it ended.
Before the first fragment arrives there is nothing to detect, and almost no run was flagged that early. Once fragments appear, detection is quick. A classifier reading the run so far flagged 99.5% of the 428 successful attacks, with a median of twelve more steps of the run still to come, while raising false alarms on 1.1% of runs that never received a fragment. Because assembly happens only after the run, an alarm at that point leaves time to abort. Runs that received fragments but never managed to assemble them were flagged far more often. Those alarms have a cost: they would stop runs whose attack was never going to do harm.
Much of that early signal comes from the surface of the planted text. Encrypted fragments are long, random-looking strings, unlike ordinary tool output. To test how much this mattered, we rewrote the same fragments as ordinary English words, so they still decode to the same program. The two panels above show the difference at the same false-alarm rate. In their legend, “trajectory prefix” detectors read the whole run so far and “single observation” detectors only the current tool output; “length available” or “withheld” says whether they may use how long the text is. At a permissive threshold with no false-alarm budget, a detector trained on whole runs still flagged nearly every rewritten attack before the run ended, so the warning is not only a reaction to random-looking text. Without the length of the text, its median first alarm moved from the 7th step of the run to the 12th, and only 22.3% of runs were flagged before their fourth and final fragment arrived.
The warning window exists because assembly happens after the run, at a point the system can block; an attack that put itself together during the run would remove that margin. The study covers one attack family in two tool environments. Within that scope, such an attack leaves a usable trace well before it is assembled, and a near-perfect detection rate on one payload format should not be read as protection against a better-disguised one.




