Privacy Leakage in Sequential Multi-Agent Pipelines: An Information-Theoretic View

When you chain multiple LLM agents together, sensitive information doesn’t just pass through each hop — it compounds. A detail that seems innocuous after the first agent can become individually identifiable after the second, and fully de-anonymized after the third. This compositional privacy leakage is one of the least-discussed failure modes in multi-agent system design, and it’s worth understanding before your pipeline reaches production.

What Compositional Privacy Leakage Actually Means

In a single-agent setting, privacy is relatively easy to reason about: the model sees input X, produces output Y, and you can ask whether Y reveals more about a sensitive variable S than you intended. The threat model is bounded.

Sequential pipelines break that bound. Each agent’s output becomes the next agent’s context. If Agent 1 leaks a small amount of information about S into its output, Agent 2 conditions on that output — and may amplify, combine, or re-express that signal alongside new retrieved information. By the time the chain reaches Agent N, the cumulative mutual information between the final output and S can be substantially larger than any single agent contributed.

This isn’t hypothetical. Consider a pipeline where Agent 1 summarizes customer support tickets, Agent 2 categorizes them by issue type, and Agent 3 generates a report with statistics. Each step seems innocuous. But if the tickets contain health-related complaints, the final report might allow inference about individual users’ medical histories even though no single agent explicitly disclosed them.

Measuring Leakage: Mutual Information as a Privacy Metric

The right tool for quantifying this kind of leakage is mutual information (MI): I(Output; S), where S is the sensitive variable you want to protect. MI measures how much knowing the output reduces uncertainty about S. A value of zero means the output tells you nothing about S; higher values indicate leakage.

For a sequential pipeline with agents A₁ → A₂ → A₃, the leakage at each stage is:

Stage 1: I(Y₁; S)  where Y₁ = A₁(X)
Stage 2: I(Y₂; S)  where Y₂ = A₂(Y₁, context₂)
Stage 3: I(Y₃; S)  where Y₃ = A₃(Y₂, context₃)

Because of the data processing inequality, I(Y₃; S) ≤ I(X; S) if the pipeline is a simple Markov chain — but real multi-agent pipelines often violate the Markov assumption. Agents frequently inject new context (retrieved documents, tool outputs, user inputs) at each step, which can re-introduce or amplify signals about S that earlier stages had partially suppressed.

This makes end-to-end MI measurement essential. You can’t bound leakage just by measuring each agent individually.

Where Leakage Enters and Amplifies

Three architectural patterns are particularly prone to compositional leakage:

Context injection at each hop. When a RAG step or tool call adds new context at every agent in the chain, each piece of retrieved content is an opportunity to correlate with S. A retriever that returns documents mentioning geographic locations, employer names, or medical terms can inadvertently de-anonymize inputs that were already partially sanitized.

Output verbosity gradients. Early agents in a chain are often summarizers or classifiers — their outputs are terse and low-leakage. Later agents are often generators — they produce prose conditioned on accumulated context, which is far more expressive. Leakage that was compressed in stage 1 can be expanded and made explicit in stage 3.

Fan-in patterns. Multi-agent architectures that merge outputs from parallel agents before passing to a synthesizer are especially risky. Even if each parallel branch independently leaks little about S, a synthesizer that sees all branches simultaneously can combine them into a highly identifying signal.

Input (contains S)
       │
       ▼
  ┌─────────┐
  │ Agent 1 │  I(Y₁;S) = low
  └────┬────┘
       │ Y₁
       ▼
  ┌─────────┐  ◄── RAG injection (may re-introduce S signal)
  │ Agent 2 │  I(Y₂;S) = medium
  └────┬────┘
       │ Y₂
       ▼
  ┌─────────┐
  │ Agent 3 │  I(Y₃;S) = HIGH  ← compositional amplification
  └────┬────┘
       │
   Final Output

Engineering Controls for Sequential Privacy

Understanding the threat is step one. The more important question is what you can actually do about it in a production system.

Output filtering between hops. Apply a lightweight classifier or redaction layer to each agent’s output before it’s forwarded to the next agent. This is a coarse control — it adds latency and can degrade task performance — but it’s the simplest defense and composable with any existing pipeline.

Privacy-regularized fine-tuning. If you control the model weights, you can add a regularization term to the training objective that penalizes high mutual information between outputs and a designated sensitive variable. This bakes the privacy constraint into the model rather than enforcing it externally. The trade-off is that it requires labeled data identifying which variables are sensitive and access to the training loop.

Differential privacy at the pipeline level. Rather than applying DP to individual model weights, you can calibrate noise injection to the pipeline’s end-to-end sensitivity — how much any single input record can change the final output. This is harder to implement than per-model DP but provides a tighter guarantee for the full chain.

Architectural decomposition. Sometimes the cleanest fix is structural: break the pipeline so that agents handling sensitive inputs never produce outputs that flow into agents handling public or cross-user contexts. This separation of concerns — sometimes called a privacy firewall — trades pipeline flexibility for a hard information-flow boundary.

Practical Implications for Pipeline Design

For engineers building sequential multi-agent systems today, the key shifts in mindset are:

First, model your pipeline’s information flow before you deploy it. Draw the graph, label each edge with what kinds of signals could propagate along it, and identify where context injection points create opportunities for leakage amplification.

Second, test privacy at the pipeline level, not the model level. Membership inference attacks and attribute inference attacks should be run against the full pipeline’s outputs, not against individual agents in isolation.

Third, make privacy budget a first-class design constraint. Just as you track token costs and latency budgets per hop, tracking an MI budget — how much leakage each stage is allowed to contribute — forces explicit trade-off decisions rather than letting leakage accumulate by default.

Sequential multi-agent pipelines are increasingly the default architecture for complex tasks. Getting privacy right in this setting requires treating the pipeline as the unit of analysis, not the individual model.