From 70995fcc55340b5553dd7917fa798842fee058ed Mon Sep 17 00:00:00 2001
From: Max Rossi <113717219+maxr0ssi@users.noreply.github.com>
Date: Thu, 1 Oct 2026 13:09:41 -0400
Subject: [PATCH 1/3] Simplify the CacheBack project page
---
README.md | 7 +-
demo/index.html | 2 +-
index.html | 96 +++++-------------
website/assets/app.js | 1 -
.../figures/communication-overview.webp | Bin 0 -> 12200 bytes
website/assets/styles.css | 88 +++-------------
website/check.mjs | 4 +-
7 files changed, 48 insertions(+), 150 deletions(-)
create mode 100644 website/assets/figures/communication-overview.webp
diff --git a/README.md b/README.md
index aa0c3ca..e74a755 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@
Website:
-Paper and library:
+Paper and library:
GitHub Pages serves the root of `main`. This is a static website with no build
step or backend. `.nojekyll` disables Jekyll processing.
@@ -15,10 +15,13 @@ node website/check.mjs
python3 -m http.server 4174 --bind 127.0.0.1
```
+The page shows the paper’s Figure 2 beside a short TL;DR, both benchmark plot
+groups, recorded demos, and an always-visible citation.
+
The homepage is `index.html`; its styles, scripts and paper figures are in
`website/assets/`. The main coding video is in `demo/coding/`. The booking
replay reads `demo/trace-w4.json`. Preserve the original video and recorded
measurements; the checks verify displayed results against those recordings.
-Code links point to RCLC. The canonical URL and social preview use the
+Code links point to CacheBack. The canonical URL and social preview use the
organization website address.
diff --git a/demo/index.html b/demo/index.html
index 290bc82..69b46c9 100644
--- a/demo/index.html
+++ b/demo/index.html
@@ -93,7 +93,7 @@
Where should Maya collect her pass on Friday, and what ID shou
Written messages between agents can lose evidence; passing their entire internal state becomes expensive. Receiver-conditioning lets the receiver describe what it needs, so the sender can select what to pass on. CacheBack is our simplest, robust, training-free implementation.
-
On FanOutQA with Qwen 3 8B, CacheBack sends 75% fewer positions than full-state transfer. Compared with same-size text communication, strict accuracy improves by 14.7 percentage points and median completion is 3.2× faster.
-
- Read the full abstract
-
-
Multi-agent systems distribute large contexts across agents that communicate to solve a task. Text messages are compact but require decoding and may omit evidence the receiving agent needs. Recent latent communication instead transfers KV caches. This avoids text generation and can improve accuracy and latency. However, a full KV cache grows linearly with both the context an individual agent processes, and the number of agents that coordinate together.
-
This raises memory and context costs, often far exceeding available GPU resources and context window sizes. Our key observation is that agents need only send what the receiving agent requires for its local task -- which we call receiver-conditioned communication. The receiver agent passes the sender a small description of its information needs, which serves to filter and compress the sender agent's KV cache.
-
CacheBack is a simple, robust, training-free instance of receiver conditioning based on the sender's attention weights. On FanOutQA, CacheBack with Qwen 3 removes 75% of the state the agent would otherwise receive, improving accuracy by 14.7 percentage points and reducing median task-completion latency by 3.2× relative to text communication. We show comparable improvements across model families that span dense Transformers, Mamba-attention hybrids, and sliding-window attention.
-
A KV cache stores intermediate state from the text a model has processed. Abstract from the published paper, by Rossi et al. CC BY 4.0; emphasis added.
-
-
-
-
-
-
-
How it works
-
The receiver describes what it needs for its next task. CacheBack uses that request to select part of the sender’s internal numerical state, or latent state, to send back.
-
-
-
-
Sender
Select state using the request
-
-
01 Request
-
02 Selected latent state
-
-
Receiver
Has the next task
-
-
-
CacheBack
-
Our simplest implementation. Robust and training-free.
-
Uses attention to the receiver’s request to score sender positions, then selects a subset within the message budget.
-
-
- What crosses the handoff?
CacheBack selects positions using the sender’s attention to the receiver’s request. In the paper’s selected-embedding channel, the corresponding input vectors are sent and the receiver rebuilds its KV cache. A position is not an isolated fact, and an attention score is not a guarantee of relevance. The experiments use matching models; cross-model translation is outside this work. Read the method and limitations.
+ Figure 2. Select what the receiver needs, instead of accumulating every sender’s context.
+
TL;DR
Agents can share internal model state instead of writing reports. Sending all of it, however, overwhelms the receiver.
The receiver asks for what it needs. CacheBack uses attention to that request to select which state gets sent. Simple, robust, and training-free.
-
Results
On FanOutQA, three agents read separate sources and send state to one answering agent. These curves compare accuracy and completion time across message budgets and text-sender sizes.
+
Results
-
Accuracy and completion time
Higher is more accurate. Further left is faster. Each point is a measured setting; lines join the best observed tradeoffs within each method. Text points vary sender size; CacheBack points vary compression or message budget. The receiver stays fixed.
Reading r4, r8, r16: rX means keeping about 1/X of the sender’s sequence positions. For example, r4 keeps 25%, r8 keeps 12.5%, and r16 keeps 6.25%. Larger numbers mean more compression and a smaller message.
+
FanOutQA
Three senders read separate sources; one receiver answers. Higher is more accurate; left is faster. CacheBack points vary message size; text points vary sender size. rX keeps 1/X of the sender’s positions: r4 keeps 25%.
● ━ CacheBack▲ ┄ Text communication× Question only
- Qwen 3: r4 improves strict accuracy from 40.7% to 55.3% and median completion from 612 to 192 seconds.
- Nemotron: the benefit also appears in a model combining Mamba and attention. A smaller text sender is faster, but less accurate.
+
+
- Repeated handoffs: LongBench v2
On LongBench v2 Easy, four agents read successive sections of a long document. A final receiver answers from the last handoff. Evidence from the beginning must survive the chain.
A relative budget uses the rX fraction above, so message size grows with the sender’s context. A fixed budget caps the number of message positions, as in the labels “32K fixed” and “64K fixed”.
+
LongBench v2 Easy
Four agents read a long document in sequence, passing state to a final answering agent. Relative budgets (rX) grow with context; fixed budgets (32K, 64K) cap message positions. Stronger compression can reduce accuracy.
- Qwen 3: both message-budget schemes improve the measured tradeoff. More aggressive compression eventually reduces accuracy.
- Nemotron: a fixed budget caps message positions; a relative budget grows with the material read. Neither denotes total GPU memory.
+
+
-
-
Redrawn from the paper’s plotting data for Qwen and Nemotron. Each benchmark uses 50 tasks submitted together on eight H100s; completion includes queueing. Accuracy averages three receiver draws. Qwen uses a 0 to 1,000 s time axis, Nemotron 0 to 400 s; each model keeps its scale across benchmarks. Figures 5, 11, and 12 in the paper.
-
About “94% CacheBack”: r16 retains one sender position in sixteen, removing 93.75%. The selected Qwen result above uses r4, or 75% fewer positions. Position reduction is not total GPU memory saved.
Seven workers help a coordinator fix a Django bug. Compare latent handoffs with written reports.
+
Watch the demo
36 seconds · 4K · Recorded coding task
Fixing a Django bug with seven coding agents
Qwen3-8B produces the patch 4.41× faster with CacheBack than with written reports on this case. Both runs make the same one-line code change and pass all 88 tests.
-
Qwen3-8B · one A100 · r4 · one warm run per method. Times include three handoffs and final answer generation, excluding model loading, tokenization, and warmup. This recording is an example, not a benchmark. Raw recording · Setup
+
Qwen3-8B · one A100 · r4 · one warm run per method. Times include three handoffs and final answer generation, excluding model loading, tokenization, and warmup. This recording is an example, not a benchmark. Raw recording · Setup
Bind a sender and receiver to matching models, then pass the receiver’s request to rcc.transfer. CacheBack is the default selector. Continue generation with your model’s native API.
