From 57f441bdd6f9913d4002e9bbdfec68aedb16f7bd Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 16 Nov 2026 17:38:18 +0800 Subject: [PATCH 1/2] docs(blog): qualify the reported Context ON/OFF scores Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- .../public/blog/context-delivery/index.html | 96 ++-- .../public/blog/data/long-horizon-audit.json | 30 +- .../stride-context-on-vs-off-en.svg | 501 +++++++++--------- .../stride-context-on-vs-off-zh.svg | 501 +++++++++--------- apps/presentation/site/public/blog/index.html | 2 +- .../blog/zh/context-delivery/index.html | 74 ++- .../site/public/blog/zh/index.html | 2 +- .../scripts/generate-long-horizon-audit.py | 253 +++++---- examples/blog-bilingual-index-smoke.mjs | 236 +++++++-- 9 files changed, 918 insertions(+), 777 deletions(-) diff --git a/apps/presentation/site/public/blog/context-delivery/index.html b/apps/presentation/site/public/blog/context-delivery/index.html index b97d221904..40cc7986dc 100644 --- a/apps/presentation/site/public/blog/context-delivery/index.html +++ b/apps/presentation/site/public/blog/context-delivery/index.html @@ -7,7 +7,7 @@
← All articles

Context delivery

An Agent needs a way to look things up, too

Read necessary facts first, reuse verified structures within a compaction cycle, and recover the complete baseline when continuity is lost.

-

Long-running work accumulates material: the original goal, later constraints, attempted routes, failed experiments and current progress. Sending everything again on each continuation repeats more and more content; keeping only a summary can make the original basis of a decision difficult to recover.

+

Long-running work accumulates material: the original goal, later constraints, attempted routes, failed experiments and current progress. Sending everything again on each continuation repeats more and more content; keeping only a summary can make the original basis of a decision difficult to recover.

Selective JSON Pointer lookup by itself is straightforward—it folds long-tail history not needed for the immediate decision. The core mechanism in LoopX's context delivery is compaction-cycle-bounded structured delta context: complete sources are saved first, unchanged typed subtrees are reused exactly against a verified baseline within the active context_epoch, and any host compaction or digest mismatch atomically resets to a fresh complete baseline rather than chaining dangling deltas.

Evidence: separate context comparisons, cycle deltas and task ablation

Four mechanisms, with separate measurement baselines

@@ -111,101 +111,99 @@

Eight tasks: single-layer payloads (Progr

Basis and source: the published eight task rows sum to 7,285,951 B / 22,408,690 B = 32.5% for Progressive across all 213 wakes. Delta and first-screen shares of approximately 35.8% and 3.0% / 5.0% cover only the 112 in-cycle wakes. Their Full denominator was published rounded as 12.53 MB; it is not the 213-wake total. This is an aggregate recalculation, not a new experiment or an independent audit of per-wake logs. Public aggregate data and provenance.

Implementation status: #8266 remains an open C1 proposal. The final merged scope of #8304 preserves precision while reusing ordering and a verified digest. These cohort bytes do not establish a shipped unified Controller/Worker gain.

-

Ten-task controlled comparison: Context ON vs Context OFF under fixed Explore + Stride

-

Cutting packet bytes only matters if long-horizon task performance holds up or improves. From the EdgeBench 10-task evaluation (2026-11-12 22:40 CST snapshot), we isolate the controlled two-arm contrast holding the same Explore + Stride harness fixed: Explore + context-on (progressive reading + compaction-cycle structured delta enabled) vs Explore + context-off (both disabled), testing only what switching Context Delivery on vs off changes within that chassis.

+

EdgeBench scores reported for Context ON and OFF

+

The November 12, 2026 snapshot (22:40 CST) brings together scores labeled Context ON and OFF on ten EdgeBench tasks. The recorded scores favor ON on 7 of 10 tasks and OFF on 3. The public record is missing per-arm evidence for the actual configuration, code revision, evaluator and budget across these pairs, so the tally cannot isolate the effect of Context Delivery.

    -
  • Ten-task tally (7 ahead or slight edge vs 3 OFF ahead): under fixed Explore + Stride, enabling Context Delivery puts Explore + context-on ahead of or slightly ahead of context-off on 7 of 10 tasks—including Borden (80 vs 45, +77.8%), ANN (7,159 vs 3,746 QPS, +91.1%), and Grid (2,197 vs 2,311 cost, −114)—while explicitly preserving all 3 counterexamples where context-off scores higher.
  • -
  • Strictly duration-matched ~18h pairs (4 tasks within the 15-minute tolerance): 3 wins to 1—Borden rises from 45 to 80 (+35 pts / +77.8%); Grid total cost (lower is better) improves from 2,311 to 2,197 (−114 / −4.9%, +2.35 normalized pts); Rust (both 17h57m) shows a slight native edge for context-on (27.1801 vs 27.1757); the matched 18h counterexample is Portfolio (both 18h00m, 13.62 vs 39.44, OFF wins).
  • -
  • Historical Stride slice & incomplete-duration pairs (6 tasks, runtimes stated explicitly): using the matching Stride pair from the historical seven-arm ANN cohort, context-on reaches 7,159 QPS vs context-off 3,746 QPS (+91.1%, +33.51 normalized pts); FFmpeg reaches 2.7086 vs 1.9929 (+35.9%, OFF stopped early pending rerun); VLIW cycles (lower is better) improve to 1,042 vs 1,097 (−55 cycles / −5.0%, OFF historical 14h45m); Ad reaches 49.67B vs 49.63B (ON 18h00m vs OFF 8h47m host-startup hang). The two remaining counterexamples are Codec (ON full 18h00m 68.89 vs OFF 70.79 in 12h38m despite a model-service block) and OpenTTD (both Explore arms <18h, ON 30,266,729 vs OFF 33,090,430).
  • +
  • Four pairs list runtimes close to 18 hours. Both listed durations fall within 15 minutes of 18 hours and within 15 minutes of each other. ON leads on Borden and Grid (lower cost is better). On Rust the native scores are 27.1801 and 27.1757, displayed as 27.18 on both sides of the /100 chart. OFF leads on Portfolio.
  • +
  • The other six include historical and incomplete runs. Four favor ON and two favor OFF (Codec and OpenTTD). The aggregate reports historical ANN at 7,159 vs 3,746 QPS. The earlier public ANN table shows 7,159 QPS, and leaves the 3,746 QPS value and both active durations unverified. The 6,020 and 5,766 QPS farther down belong to a different Explore pair without Stride. The older Grid results below also come from a different pair.
-

Reconciling two figures with the ablation section below: (1) this section's ANN pair (3,746 vs 7,159 QPS) comes from the two Stride-enabled Explore arms in the seven-arm cohort (Explore + Stride + context-off / on); #ablation-evidence separately lists the without-Stride Explore baseline (Explore · Context off 5,766 and Explore · Context on 6,020) from that study, so the two tables describe distinct arm pairs. (2) Grid here (2,197 vs 2,311) reports the completed ~18h pair within 15-minute tolerance (with context-on via same-session continuation), whereas #ablation-evidence retains the earlier unmatched-duration “Historical Grid” counterexample (Off 2054 < On+Stride 2088) for historical continuity.

-

EdgeBench 10-task controlled score comparison (fixed Explore + Stride: Context ON vs OFF)

EdgeBench 10-task controlled Explore + Stride Context ON vs OFF scores: 7 tasks ahead or slight edge (ANN +91.1%, Borden +77.8%, Grid −114), retaining Portfolio, Codec, and OpenTTD counterexamples
A: Nine tasks with normalized EdgeBench scores (/100, direction-normalized so higher is better), ordered by ON − OFF delta, showing arm runtime status. B: Native-point summary including task #10 Borden (no /100 scale, matched ~18h: 80 vs 45), key native deltas, and counterexample disclosures.
-
Expand the 10-task controlled score and runtime table -
+

Reported EdgeBench scores for Context ON and OFF

Reported Context ON and OFF scores. Panel A shows nine normalized pairs. Panel B shows Borden in native points and highlights Portfolio, Codec, and OpenTTD.
A shows nine pairs with /100 scores, ordered by ON minus OFF after direction is normalized (higher is better). B shows Borden in its native units and highlights the three tasks where OFF leads.
+
Expand the ten-task score and runtime table +
- - - - - + + + + + - - + + - + - - + + - + - - + + - + - - + + - + - - - - + + + + - - + + - - + + - - + + - + - - + + - + - - + + - + - - + + - - + +
Task (metric & direction)Duration parityExplore + context-on
Native (/100 · duration · valid)
Explore + context-off
Native (/100 · duration · valid)
ON vs OFF deltaTask (metric and direction)Reported pairContext ON
Native (/100 · duration · valid)
Context OFF
Native (/100 · duration · valid)
ON − OFF (native and /100)
Borden (score ↑)Matched ~18hBorden (Native score ↑)Near 18h 80 (— · 18h02m · 17 valid) 45 (— · 17h59m · 17 valid)+35 (+77.8%)+35 / —
Grid (total cost ↓)Matched ~18hGrid (Total cost ↓)Near 18h 2,197 (67.20 · 17h57m · 17 valid) 2,311 (64.85 · 17h59m · 13 valid)−114 (−4.9%, +2.35 pts)−114 / +2.35
Rust (score ↑)Matched ~18hRust (Native score ↑)Near 18h 27.1801 (27.18 · 17h57m · 10 valid) 27.1757 (27.18 · 17h57m · 10 valid)+0.0044 (tied / slight edge)+0.0044 / 0
Portfolio (score / 100 ↑)Matched 18h counterex.Portfolio (Score / 100 ↑)Near 18h · OFF leads 13.62 (13.62 · 18h00m · 17 valid) 39.44 (39.44 · 18h00m · 17 valid)−25.82 pts (OFF wins)−25.82 / −25.82
ANN (QPS ↑)7-arm Stride cohort7,159 (43.17 · 6 valid)3,746 (9.66 · 4 valid)+3,413 (+91.1%, +33.51 pts)Historical7,159 (43.17 · unverified · 6 valid)3,746 (9.66 · unverified · 4 valid)+3,413 / +33.51
FFmpeg (score ↑)OFF early stopFFmpeg (Native score ↑)Shorter or early-stop OFF 2.7086 (16.17 · 18h01m · 16 valid)1.9929 (11.19 · early stop · 6 valid)+0.7157 (+35.9%, +4.98 pts)1.9929 (11.19 · stopped early · 6 valid)+0.7157 / +4.98
VLIW (cycles ↓)OFF 14h45m hist.VLIW (Cycles ↓)Shorter or early-stop OFF 1,042 (97.25 · 17h57m · 14 valid) 1,097 (93.82 · 14h45m · 12 valid)−55 cycles (−5.0%, +3.43 pts)−55 / +3.43
Ad (total score ↑)OFF 8h47m hangAd (Total score ↑)Shorter or early-stop OFF 49,673,040,827 (79.30 · 18h00m · 16 valid) 49,631,904,944 (78.33 · 8h47m · 7 valid)+41.14M (+0.97 pts)+41,135,883 / +0.97
Codec (score ↑)OFF 12h38m counterex.Codec (Native score ↑)Shorter or early-stop OFF · OFF leads 68.89 (68.89 · 18h00m · 13 valid) 70.79 (70.79 · 12h38m · 11 valid)−1.90 pts (OFF wins)−1.9 / −1.9
OpenTTD (score ↑)Both <18h counterex.OpenTTD (Native score ↑)Both below 18h · OFF leads 30,266,729 (62.02 · 10h47m · 10 valid)33,090,430 (62.99 · early stop · 7 valid)−2.82M (−0.97 pts, OFF wins)33,090,430 (62.99 · stopped early · 7 valid)−2,823,701 / −0.97
-

Measurement and scope boundary: these are C1 mixed-cohort exploratory observations (active solver duration accumulated after excluding downtime). This section holds Explore + Stride constant solely to examine the directional effect of toggling context delivery (Context ON vs Context OFF) within the harness; because resumed sessions and incomplete durations remain, it does not establish an isolated C2 causal proof. Whole-system iso-budget Pareto comparisons against zero-overhead external baselines belong to a separate question addressed in Layer 2 of EdgeBench feedback and memory. Structured records live in public aggregate data.

+

The original published aggregate preserves the score and runtime records. The structured data documents their limits and sources. See EdgeBench feedback and memory for whole-system comparisons under comparable budgets.

Multi-task and layered ablation: benefit chain and boundary counterexamples

In EdgeBench ANN vector search optimization (gpt-6.1-sol / xhigh, 18h, subject to Recall@10 ≥ 0.95) and cross-task ablations, we separate Context Delivery (progressive selection and cycle deltas) from Stride (evidence-conditioned experiment sizing), auditing selected control-packet bytes, main-session model input tokens, and local/early-window counterexamples side by side (reconciled from archived trajectories and receipts by @shangzh0):

Multi-task ablation and layered accounting

Progressive Context Delivery and Stride Ablation: ANN 18h Peak QPS Progression and Layered Token/Byte Savings vs Boundary Counterexamples
Top (A): ANN 18h six-arm peak valid throughput QPS (Recall@10 ≥ 0.95). Bottom (B): Layered control-packet byte and main-session input token savings vs Context off (left, negative) alongside disclosed local/early-window overhead and cross-task counterexamples (right, positive). Open image for full size.
diff --git a/apps/presentation/site/public/blog/data/long-horizon-audit.json b/apps/presentation/site/public/blog/data/long-horizon-audit.json index 1557febc5b..2da9f0cc4b 100644 --- a/apps/presentation/site/public/blog/data/long-horizon-audit.json +++ b/apps/presentation/site/public/blog/data/long-horizon-audit.json @@ -160,7 +160,9 @@ "stride_context_on_off_comparison": { "snapshot_cst": "2026-11-12 22:40 CST", "benchmark": "EdgeBench", - "design": "Controlled two-arm comparison under fixed Explore + Stride: Explore + Stride + context-on (progressive reading and compaction-cycle structured delta enabled) vs Explore + Stride + context-off (both disabled).", + "design": "Descriptive snapshot of scores reported as Context ON and OFF. Four pairs list durations within 15 minutes of 18 hours, with no more than 15 minutes between arms. Six pairs are historical or have incomplete or unverified durations. Public evidence does not establish common treatment, budget, version or evaluator across the ten pairs.", + "source_url": "https://github.com/loopx-project/loopx/blob/4a15d6e6fe0148f801ed40d7bfd9d5e5025274a5/apps/presentation/site/public/blog/data/long-horizon-audit.json#L160", + "prior_public_ann_url": "https://github.com/loopx-project/loopx/blob/9ff3b5a7628da7f7c6776bb4e1eeb011aa47eca0/apps/presentation/site/public/blog/context-delivery/index.html#L225-L239", "summary": { "total_tasks": 10, "context_on_ahead_or_edge": 7, @@ -170,8 +172,8 @@ "duration_matched_context_off_wins": 1 }, "reconciliation_notes": [ - "ANN uses the Stride-enabled pair from the historical seven-arm cohort (Explore + Stride + context-on 7,159 QPS vs Explore + Stride + context-off 3,746 QPS); the ablation section separately records the without-Stride Explore pair (6,020 vs 5,766 QPS).", - "Grid reflects the completed ~18h Explore pair within 15m tolerance (context-on 2,197 at 17h57m vs context-off 2,311 at 17h59m); the ablation section retains an earlier unmatched-duration Historical Grid counterexample (Off 2054 < On+Stride 2088)." + "The aggregate reports historical ANN as ON 7,159 and OFF 3,746 QPS with both active durations unverified. The earlier public table confirms the ON value but does not list the OFF value. Its 6,020 and 5,766 QPS belong to a different Explore pair without Stride.", + "This table reports the near-18h Grid pair (ON 2,197 at 17h57m, OFF 2,311 at 17h59m). The ablation section retains an earlier unequal-duration Grid observation (OFF 2,054, ON plus Stride 2,088)." ], "tasks": [ { @@ -190,14 +192,14 @@ "context_off_valid": 17, "winner_explore": "context-on", "delta_label": "+35 (+77.8%)", - "note_en": "Matched ~18h (both within 15m tolerance).", - "note_zh": "等时长约 18h(均在 15m 容差内)。" + "note_en": "Both listed durations are within 15 minutes of 18 hours.", + "note_zh": "两臂列出的时长均在 18 小时前后 15 分钟内。" }, { "task": "Grid", "direction": "lower", "metric_label_en": "Total cost", - "metric_label_zh": "原生总分", + "metric_label_zh": "总代价", "cohort_group": "matched_18h", "context_on_duration": "17h57m", "context_off_duration": "17h59m", @@ -209,8 +211,8 @@ "context_off_valid": 13, "winner_explore": "context-on", "delta_label": "-114 (-4.9%, +2.35 pts)", - "note_en": "Matched ~18h within 15m tolerance (same-session continuation on context-on).", - "note_zh": "等时长约 18h(容差内免补;context-on 沿同会话续跑)。" + "note_en": "Two near-18h reported durations. The aggregate records a same-session continuation on the ON side.", + "note_zh": "两臂列出的时长均接近 18 小时,汇总记录 ON 在原会话续跑。" }, { "task": "Rust", @@ -228,8 +230,8 @@ "context_off_valid": 10, "winner_explore": "context-on-edge", "delta_label": "+0.0044 (tied / slight edge)", - "note_en": "Matched 17h57m each; context-on holds slight native edge.", - "note_zh": "等时长均为 17h57m;context-on 原生分微弱占先。" + "note_en": "Both durations are listed as 17h57m. ON leads in native score and ties at two decimal places on the /100 scale.", + "note_zh": "两臂均列为 17h57m。ON 原生分微弱占先,百分制保留两位小数时持平。" }, { "task": "Portfolio", @@ -247,8 +249,8 @@ "context_off_valid": 17, "winner_explore": "context-off", "delta_label": "-25.82 pts (OFF wins)", - "note_en": "Matched 18h00m counterexample with corrected evaluator; all evaluations drained.", - "note_zh": "修正评测器下的等时长 18h 反例;两 Explore 臂全部评分排空。" + "note_en": "OFF leads. The aggregate lists 18h00m on both sides and reports a corrected evaluator with all evaluations drained.", + "note_zh": "OFF 领先。汇总列出两臂均为 18h00m,并记录修正后的评测器与全部已完成的评分。" }, { "task": "ANN", @@ -266,8 +268,8 @@ "context_off_valid": 4, "winner_explore": "context-on", "delta_label": "+3,413 QPS (+91.1%, +33.51 pts)", - "note_en": "Corresponding Explore + Stride pair from the historical seven-arm ANN cohort; active solver intervals unverified.", - "note_zh": "历史七臂中同开 Stride 的对应两臂配置;原生执行区间尚待核实。" + "note_en": "Historical ANN pair as labeled in the aggregate. Both durations and the OFF score remain unverified by the cited earlier public table.", + "note_zh": "沿用汇总中的历史 ANN 配对标签。所引旧版公开表无法核对两臂时长及 OFF 分数。" }, { "task": "FFmpeg", diff --git a/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-en.svg b/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-en.svg index 1fba6fdf83..4ccd6e62b4 100644 --- a/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-en.svg +++ b/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-en.svg @@ -29,406 +29,405 @@ z - - + - - 0 + + 0 - + - - 25 + + 25 - + - - 50 + + 50 - + - - 75 + + 75 - + - - 100 + + 100 - - EdgeBench normalized score (/100, higher is better) + + EdgeBench normalized score (/100, higher is better) - - ANN (QPS↑ · 7-arm Stride) + + ANN · historical, time unverified - - FFmpeg (score↑ · OFF early stop) + + FFmpeg · shorter OFF - - VLIW (cycles↓ · OFF 14h45m) + + VLIW · shorter OFF - - Grid (cost↓ · ~18h matched) + + Grid · near 18h - - Ad (score↑ · OFF 8h47m) + + Ad · shorter OFF - - Rust (score↑ · ~18h matched) + + Rust · near 18h - - OpenTTD (score↑ · both <18h) + + OpenTTD · both under 18h - - Codec (score↑ · OFF 12h38m) + + Codec · shorter OFF - - Portfolio (score↑ · 18h matched) + + Portfolio · near 18h - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - +33.51 (43.17 vs 9.66) + + 43.17 / 9.66 (+33.51) - - +4.98 (16.17 vs 11.19) + + 16.17 / 11.19 (+4.98) - - +3.43 (97.25 vs 93.82) + + 97.25 / 93.82 (+3.43) - - +2.35 (67.20 vs 64.85) + + 67.20 / 64.85 (+2.35) - - +0.97 (79.30 vs 78.33) + + 79.30 / 78.33 (+0.97) - - 27.18 vs 27.18 (+0.0044) + + 27.18 / 27.18 (+0.00) - - -0.97 (62.02 vs 62.99) + + 62.02 / 62.99 (-0.97) - - -1.90 (68.89 vs 70.79) + + 68.89 / 70.79 (-1.90) - - -25.82 (13.62 vs 39.44) + + 13.62 / 39.44 (-25.82) - - A. Fixed Stride: context-on vs context-off (/100) + + A. Nine normalized pairs (ON / OFF) - - - + + - Task #10 (no /100 scale): Borden (~18h matched) - • context-on (18h02m): 80 (17 valid) - • context-off (17h59m): 45 (17 valid) - → +35 pts (+77.8%, duration-matched gain). - - - - Borden has no /100 score + Borden (Native score): 80 / 45 + ON 18h02m / OFF 17h59m + + + + - Key native deltas under fixed Explore + Stride - • ANN (QPS ↑): 7,159 vs 3,746 (+91.1%, +33.51 pts) - • FFmpeg (score ↑): 2.7086 vs 1.9929 (+35.9%, OFF early stop) - • VLIW (cycles ↓): 1,042 vs 1,097 (−55 cycles / −5.0%) - • Grid (cost ↓): 2,197 vs 2,311 (−114 / −4.9%, ~18h) - - - - Examples (ANN OFF unverified) + ANN (QPS): 7,159 / 3,746 + FFmpeg (Native score): 2.7086 / 1.9929 + VLIW (Cycles): 1,042 / 1,097 + Grid (Total cost): 2,197 / 2,311 + + + + - 10-task summary: 7 ON ahead/edge vs 3 OFF ahead - • Matched ~18h (4): 3 ON wins (Borden/Grid/Rust), - 1 counterexample (Portfolio 13.62 vs 39.44) - • Historical / <18h (6): 4 ON ahead, 2 OFF ahead - (Codec 68.89 vs 70.79; OpenTTD 30.27M vs 33.09M) + 10 reported pairs: ON 7 / OFF 3 + 4 near-18h pairs: ON 3 / OFF 1 + Tasks where OFF leads + Portfolio (Score / 100): 13.62 / 39.44 + Codec (Native score): 68.89 / 70.79 + OpenTTD (Native score): 30,266,729 / 33,090,430 - - B. Native score deltas (Borden) & counterexamples + + B. Native scores (ON / OFF) - - EdgeBench 10-task controlled comparison: Context ON vs OFF under Explore + Stride + + EdgeBench scores reported for Context ON and OFF - - Holding Explore + Stride constant (context-on adds progressive reads + cycle deltas). Ahead on 7/10 tasks, retaining 3 counterexamples and runtime gaps. + + 4 pairs list runtimes near 18 hours. The other 6 include historical or incomplete runs. - - Basis: EdgeBench controlled two-arm snapshot at 2026-11-12 22:40 CST. Borden has no /100 scale and appears in native points in Panel B (80 vs 45). - Grid, Borden, Rust, and Portfolio are ~18h duration-matched (within 15m); ANN uses the matching Stride pair from the 7-arm study; remaining 5 pairs have <18h runs. Checks within-harness direction only. + + Snapshot 2026-11-12 22:40 CST. The public records cannot isolate the effect of Context Delivery. - - + - - Explore + Stride + context-on + + Context ON - - + - - Explore + Stride + context-off + + Context OFF - - + + diff --git a/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-zh.svg b/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-zh.svg index fcea8cf5f0..dda4debee7 100644 --- a/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-zh.svg +++ b/apps/presentation/site/public/blog/images/context-delivery/stride-context-on-vs-off-zh.svg @@ -29,406 +29,405 @@ z - - + - - 0 + + 0 - + - - 25 + + 25 - + - - 50 + + 50 - + - - 75 + + 75 - + - - 100 + + 100 - - EdgeBench 百分制归一分 (/100,越高越优;含 ↓ 指标已方向归一化) + + EdgeBench 百分制分数(越高越好) - - ANN (QPS↑ · 历史7臂Stride) + + ANN · 历史时长待核 - - FFmpeg (原生↑ · OFF早停) + + FFmpeg · OFF 时长较短 - - VLIW (周期↓ · OFF 14h45m) + + VLIW · OFF 时长较短 - - Grid (总分↓ · 等时长 ~18h) + + Grid · 接近 18h - - Ad (总分↑ · OFF 8h47m) + + Ad · OFF 时长较短 - - Rust (原生↑ · 等时长 ~18h) + + Rust · 接近 18h - - OpenTTD (原生↑ · 均<18h) + + OpenTTD · 均未满 18h - - Codec (原生↑ · OFF 12h38m) + + Codec · OFF 时长较短 - - Portfolio (分↑ · 等时长 18h) + + Portfolio · 接近 18h - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #0f766e"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - + +" clip-path="url(#pd81db1c1d1)" style="fill: #94a3b8"/> - - +33.51 (43.17 vs 9.66) + + 43.17 / 9.66 (+33.51) - - +4.98 (16.17 vs 11.19) + + 16.17 / 11.19 (+4.98) - - +3.43 (97.25 vs 93.82) + + 97.25 / 93.82 (+3.43) - - +2.35 (67.20 vs 64.85) + + 67.20 / 64.85 (+2.35) - - +0.97 (79.30 vs 78.33) + + 79.30 / 78.33 (+0.97) - - 27.18 vs 27.18 (+0.0044) + + 27.18 / 27.18 (+0.00) - - -0.97 (62.02 vs 62.99) + + 62.02 / 62.99 (-0.97) - - -1.90 (68.89 vs 70.79) + + 68.89 / 70.79 (-1.90) - - -25.82 (13.62 vs 39.44) + + 13.62 / 39.44 (-25.82) - - A. 固定 Stride 下 context-on vs context-off 百分制对照 + + A. 九组百分制分数(ON / OFF) - - - + + - 第 10 题(无 /100 标度):Borden 等时长 ~18h - • context-on(18h02m):80(17 次有效) - • context-off(17h59m):45(17 次有效) - → +35 分(+77.8%,同构等时长显著拉升)。 - - - - Borden 无百分制分数 + Borden (原生分): 80 / 45 + ON 18h02m / OFF 17h59m + + + + - 关键原生分数增幅(固定 Explore + Stride) - • ANN (QPS ↑):7,159 vs 3,746(+91.1%, +33.51 百分分) - • FFmpeg (原生 ↑):2.7086 vs 1.9929(+35.9%, OFF 早停) - • VLIW (周期 ↓):1,042 vs 1,097(−55 周期 / −5.0%) - • Grid (代价 ↓):2,197 vs 2,311(−114 / −4.9%, ~18h) - - - - 部分原生分(ANN 的 OFF 分数待核) + ANN (QPS): 7,159 / 3,746 + FFmpeg (原生分): 2.7086 / 1.9929 + VLIW (周期数): 1,042 / 1,097 + Grid (总代价): 2,197 / 2,311 + + + + - 10 题总盘:ON 7 项占先 vs OFF 3 项领先 - • 等时长 ~18h(4 题):3 胜(Borden/Grid/Rust 微先), - 1 反例(Portfolio 13.62 vs 39.44) - • 非等时长或历史对照(6 题):ON 领先 4 题, - OFF 领先 2 题(Codec 70.79 / OpenTTD 33.09M) + 10 组记录:ON 7 / OFF 3 + 接近 18h 的 4 组:ON 3 / OFF 1 + OFF 占先的题目 + Portfolio (原生分 / 100): 13.62 / 39.44 + Codec (原生分): 68.89 / 70.79 + OpenTTD (原生分): 30,266,729 / 33,090,430 - - B. 原生关键差值(含 Borden)与反例边界 + + B. 原生分(ON / OFF) - - EdgeBench 十题同构效果对照:固定 Explore + Stride 下的 Context ON vs OFF + + EdgeBench 十题记录中的 Context ON 与 OFF 分数 - - 固定 Explore + Stride 框架(context-on 开启 Progressive 与周期内差分)。10 题中 context-on 7 题占先,完整保留 3 项反例与真实时长差。 + + 4 组列出的时长接近 18 小时,其余 6 组涉及历史运行或未跑满的记录。 - - 口径说明:截取自 EdgeBench 同构两臂实测(2026-11-12 22:40 CST 快照)。Borden 无百分制换算故在右栏 B 单列原生分(80 vs 45)。 - Grid、Borden、Rust、Portfolio 四题双臂均已跑满约 18h(15m 容差内);ANN 取自历史七臂对应 Stride 配置;其余 5 题存在至少一臂未满 18h。本图仅检验开关的组内方向性效果。 + + 快照 2026-11-12 22:40 CST。现有逐臂记录不足以单独衡量 Context Delivery 的影响。 - - + - - Explore + Stride + context-on(开启上下文交付) + + Context ON - - + - - Explore + Stride + context-off(关闭上下文交付) + + Context OFF - - + + diff --git a/apps/presentation/site/public/blog/index.html b/apps/presentation/site/public/blog/index.html index 7fffa21f9f..eaa47487cd 100644 --- a/apps/presentation/site/public/blog/index.html +++ b/apps/presentation/site/public/blog/index.html @@ -36,7 +36,7 @@

Context delivery · Mechanisms and examples

· Updated November 14

-

An Agent needs a way to look things up, too

Compare 10-task controlled Explore + Stride scores under Context ON vs OFF (7/10 ahead or edged out—Borden 80 vs 45, ANN 7,159 vs 3,746—with Portfolio/Codec/OpenTTD counterexamples retained), alongside separate 32.5% Progressive, −74.25% ANN in-cycle delta, and ≈ 3.0% / 5.0% first-read baselines.

Read article
+

An Agent needs a way to look things up, too

Ten reported EdgeBench score pairs include four with runtimes close to 18 hours, plus historical and incomplete runs. The article also traces the separate baselines for the 32.5% progressive view, the 74.25% in-cycle ANN reduction and the 3.0% / 5.0% first read.

Read article

Research notes · EdgeBench and LHTB

diff --git a/apps/presentation/site/public/blog/zh/context-delivery/index.html b/apps/presentation/site/public/blog/zh/context-delivery/index.html index 33f1f5e126..c630067518 100644 --- a/apps/presentation/site/public/blog/zh/context-delivery/index.html +++ b/apps/presentation/site/public/blog/zh/context-delivery/index.html @@ -7,7 +7,7 @@
← 全部文章

Context delivery

Agent 读上下文,也需要一套“查书”的方法

先读必要事实,在有效 compaction 周期内复用已验证的结构;连续性丢失时恢复完整基线。

-

Agent 做长任务,资料会越来越多:原目标、后来追加的约束、试过的方案、失败的实验、当前做到哪里。每次继续都把整份材料再塞一遍,重复会不断累积;只留一段摘要,又可能找不回决定所依赖的原话。

+

Agent 做长任务,资料会越来越多:原目标、后来追加的约束、试过的方案、失败的实验、当前做到哪里。每次继续都把整份材料再塞一遍,重复会不断累积;只留一段摘要,又可能找不回决定所依赖的原话。

按 JSON Pointer 渐进选读本身并不复杂,主要用来折叠当前决定暂不需要的历史长尾;LoopX 上下文交付真正的机制重心,是受 compaction 周期约束的结构化差分上下文(compaction-cycle structured delta):完整来源先落盘存证,在同一个已核验的 context_epoch 内精确复用未变化的结构子树、只编码本轮结构增删改;一旦宿主发生上下文压缩或基线摘要失配,立即原子切换回新的完整基线,绝不悬空叠加旧差分。

先看数据:独立上下文对照、周期内结构化差分与任务消融

四种机制,分别核对测量基线

@@ -111,101 +111,99 @@

八题对照:区分单层载荷(Progr

口径与来源:上表逐题数据复算自已公开的八题表,Progressive 合计为 7,285,951 B / 22,408,690 B = 32.5%,覆盖全部 213 次唤醒。Delta 与首屏的 35.8%、3.0% / 5.0% 仅对应 112 次周期内唤醒;其 Full 分母原文仅报告约 12.53 MB,不能与 213 次总量混用。这里没有新增实验,也未独立核对逐 wake 原始记录。公开聚合数据与来源。

实现状态:#8266 仍是开放的 C1 提案;#8304 的最终合入范围是保留精度的排序与摘要复用。上述字节数据不归因为统一 Controller/Worker 已上线的收益。

-

十题同构两臂效果对比:固定 Explore + Stride 下的 Context ON 与 Context OFF

-

在压降上下文传输载荷的同时,我们进一步检验这套交付机制在长程求解中的实际得分表现:从 EdgeBench 十题实测(2026-11-12 22:40 CST 快照)中取出固定同一 Explore + Stride 框架的两臂对照:Explore + context-on(开启 Progressive 按需读取与周期内结构化 Delta)对比 Explore + context-off(两项均关闭),专门检验 Context Delivery 单一开关在同构底盘下的组内表现。

+

EdgeBench 十题记录中的 Context ON 与 OFF 分数

+

这份快照汇集了截至 2026 年 11 月 12 日 22:40(北京时间)标为 Context ON 和 OFF 的十道 EdgeBench 题目。按记录的原生分及指标方向计算,7 题 ON 占先,3 题 OFF 占先。公开资料没有逐题给出两臂的实际配置、代码版本、评测器和预算,因此这组数据无法单独衡量 Context Delivery 的影响。

    -
  • 十题整体盘面(7 题领先或微弱占先 vs 3 题 OFF 领先):固定 Explore + Stride 开启 Context Delivery 后,10 题中有 7 题取得更高或微弱占先的成绩(包括 Borden 80 vs 45 [+77.8%]、ANN 7,159 vs 3,746 QPS [+91.1%]、Grid 2,197 vs 2,311 [−114 代价]);同时完整保留 3 项 Context OFF 领先的反例,不做选择性截取。
  • -
  • 严格等时长约 18h 配对(4 题,15 分钟容差内):3 胜 1 负——Borden 从 45 升至 80(+35 分 / +77.8%);Grid 代价(越低越好)从 2,311 降至 2,197(−114 / −4.9%,+2.35 百分分);Rust(均 17h57m)维持在 27.1801 vs 27.1757 的微弱占先;等时长反例则是 Portfolio(均为 18h00m,13.62 vs 39.44,OFF 领先)。
  • -
  • 历史同构与非等时长对照(6 题,逐题标明实际运行时间):ANN 取历史七臂中同开 Stride 的两臂配置,context-on 达 7,159 QPS 对 context-off 3,746 QPS(+91.1%,+33.51 百分分);FFmpeg 2.7086 vs 1.9929(+35.9%,OFF 早停待重跑);VLIW 周期数(越低越好)1,042 vs 1,097(−55 周期 / −5.0%,OFF 沿用 14h45m 历史轮);Ad 49.67B vs 49.63B(ON 满 18h00m,OFF 因宿主启动空挂仅累计 8h47m)。另外两项反例为 Codec(ON 满 18h00m 得 68.89,不敌模型服务拦截下仅跑 12h38m 的 OFF 70.79)与 OpenTTD(两臂均未满 18h,ON 30,266,729 vs OFF 33,090,430)。
  • +
  • 四组列出的双臂时长接近 18 小时。每臂距 18 小时不超过 15 分钟,双臂相差也不超过 15 分钟。Borden 和 Grid 的 ON 占先(Grid 的代价越低越好)。Rust 两臂原生分是 27.1801 和 27.1757,百分制图上都显示为 27.18。Portfolio 的 OFF 占先。
  • +
  • 其余六组涉及历史运行或未跑满的记录。其中四题 ON 占先,两题 OFF 占先(Codec 和 OpenTTD)。汇总记录的历史 ANN 分数为 7,159 和 3,746 QPS。旧版公开 ANN 表能核对 7,159 QPS,3,746 QPS 和两臂实际运行时长仍待核实。后文的 6,020 和 5,766 QPS 来自未开启 Stride 的另一组 Explore 运行,较早的 Grid 分数也来自另一组。
-

与下方消融表两处口径衔接说明:(1)本节 ANN 对照 3,746 vs 7,159 QPS 来自历史七臂里同开 Stride 的两臂(Explore + Stride + context-off / on);下文 #ablation-evidence 另行列出该研究中未开 Stride 的 Explore 基线(Explore · Context off 5,766 与 Explore · Context on 6,020),二者对照条件不同,互不冲突。(2)本节 Grid(2,197 vs 2,311)反映 15 分钟容差内均跑满 ~18h 的最新完成对比(其中 context-on 沿同一会话续跑);下文仍保留更早一轮未匹配时长与资源的「Historical Grid」反例(Off 2054 < On+Stride 2088)作历史记录。

-

EdgeBench 十题同构两臂效果对照(固定 Explore + Stride:Context ON vs OFF)

EdgeBench 十题同构 Explore + Stride 下 Context ON vs OFF 分数对照:7 题领先或占先(ANN +91.1%、Borden +77.8%、Grid −114),完整保留 Portfolio、Codec、OpenTTD 三项反例
A:9 道有 EdgeBench 百分制归一分的题目按 ON − OFF 分差排序(已方向归一化为越高越优),标出各自时长状态。B:无百分制标度的第 10 题 Borden(等时长 ~18h,80 vs 45)、关键原生分增幅,以及 3 项负向反例口径。
-
展开十题同构两臂逐题分数与运行时长明细 -
+

EdgeBench 记录中的 Context ON 与 OFF 分数

记录中的 Context ON 与 OFF 分数。左图显示 9 组百分制分数。右图单列 Borden 原生分,标出 Portfolio、Codec 和 OpenTTD。
A 显示有百分制分数的九组记录,按 ON 减 OFF 排序(方向已归一为越高越好)。B 单列 Borden 原生分,标出 OFF 占先的三题。
+
展开十题分数与时长 +
- - - - + + + + - + - + - - + + - + - + - + - + - + - - - - + + + + - + - + - + - + - + - + - + - + - + - +
题目(指标与方向)时长配对状态Explore + context-on
原生(/100 · 时长 · 有效次)
Explore + context-off
原生(/100 · 时长 · 有效次)
ON vs OFF 差值记录类别Context ON
原生(/100 · 时长 · 有效次数)
Context OFF
原生(/100 · 时长 · 有效次数)
ON − OFF(原生与百分制)
Borden(原生分 ↑)等时长 ~18h接近 18 小时 80(— · 18h02m · 17 次) 45(— · 17h59m · 17 次)+35(+77.8%)+35 / —
Grid(原生总分 ↓)等时长 ~18hGrid(总代价 ↓)接近 18 小时 2,197(67.20 · 17h57m · 17 次) 2,311(64.85 · 17h59m · 13 次)−114(−4.9%, +2.35 分)−114 / +2.35
Rust(原生分 ↑)等时长 ~18h接近 18 小时 27.1801(27.18 · 17h57m · 10 次) 27.1757(27.18 · 17h57m · 10 次)+0.0044(微先 / 持平)+0.0044 / 0
Portfolio(原生分 / 100 ↑)等时长 18h 反例接近 18 小时 · OFF 占先 13.62(13.62 · 18h00m · 17 次) 39.44(39.44 · 18h00m · 17 次)−25.82 分(OFF 胜)−25.82 / −25.82
ANN(QPS ↑)历史七臂 Stride7,159(43.17 · 6 次)3,746(9.66 · 4 次)+3,413(+91.1%, +33.51 分)历史记录7,159(43.17 · 待核实 · 6 次)3,746(9.66 · 待核实 · 4 次)+3,413 / +33.51
FFmpeg(原生分 ↑)OFF 早停待重跑OFF 时长较短或早停 2.7086(16.17 · 18h01m · 16 次) 1.9929(11.19 · 早停 · 6 次)+0.7157(+35.9%, +4.98 分)+0.7157 / +4.98
VLIW(周期数 ↓)OFF 14h45m 历史OFF 时长较短或早停 1,042(97.25 · 17h57m · 14 次) 1,097(93.82 · 14h45m · 12 次)−55 周期(−5.0%, +3.43 分)−55 / +3.43
Ad(原生总分 ↑)OFF 8h47m 空挂OFF 时长较短或早停 49,673,040,827(79.30 · 18h00m · 16 次) 49,631,904,944(78.33 · 8h47m · 7 次)+41.14M(+0.97 分)+41,135,883 / +0.97
Codec(原生分 ↑)OFF 12h38m 反例OFF 时长较短或早停 · OFF 占先 68.89(68.89 · 18h00m · 13 次) 70.79(70.79 · 12h38m · 11 次)−1.90 分(OFF 胜)−1.9 / −1.9
OpenTTD(原生分 ↑)两臂均未满 反例两臂均未满 18 小时 · OFF 占先 30,266,729(62.02 · 10h47m · 10 次) 33,090,430(62.99 · 早停 · 7 次)−2.82M(−0.97 分, OFF 胜)−2,823,701 / −0.97
-

实验口径与边界说明:以上属于 C1 多轮探索观测(有效运行时间按求解进程扣除停机后累计)。本节固定 Explore + Stride 框架,仅隔离检验开启上下文交付(Context ON vs Context OFF)在组内的方向性表现;因存在续跑会话与部分未补满时长差异,不单独构成剥离全部交互项后的纯 C2 因果证明。控制面整体相对零协议税外部基线的等预算帕累托对比属另一层独立议题,详见 《EdgeBench 反馈与记忆》中的第二层目标讨论。原始聚合结构化数据见 公开聚合数据。

+

原来的公开汇总快照保留了比分和时长记录。结构化数据列明取舍与出处。全系统在等预算下的比较另见EdgeBench 反馈与记忆。

真实模型与多题消融实证:收益链路与边界反例

在 EdgeBench ANN 向量检索优化(gpt-6.1-sol / xhigh,18h,约束 Recall@10 ≥ 0.95)与跨题消融中,我们将 Context Delivery(渐进读取与周期差分)与 Stride(基于证据差分的步长控制)拆开对照,同时统计控制包选中字节、主会话模型输入 Token 与局部/早窗反例(基于 @shangzh0 归档轨迹与控制面回执复算):

多题消融与分层计量审计

Context 渐进式交付与 Stride 步幅控制消融:ANN 18h 峰值吞吐 QPS 对比与分层 Token/字节节省及边界反例
上图(A):ANN 18h 六臂峰值有效吞吐 QPS(Recall@10 ≥ 0.95)。下图(B):相对 Context off 的分层字节与主会话输入 Token 节省(左,负值)及诚实披露的局部/早窗开销与跨题反例(右,正值)。点击查看原图。
diff --git a/apps/presentation/site/public/blog/zh/index.html b/apps/presentation/site/public/blog/zh/index.html index f2b74e838c..6110174755 100644 --- a/apps/presentation/site/public/blog/zh/index.html +++ b/apps/presentation/site/public/blog/zh/index.html @@ -36,7 +36,7 @@

Context delivery · 机制与实例

· 11 月 14 日更新

-

Agent 读上下文,也需要一套“查书”的方法

拆解十题同构两臂(固定 Explore + Stride)下的 Context ON vs OFF 跑分对比(7/10 题占先、Borden 80 vs 45、ANN 7,159 vs 3,746,并保留 Portfolio/Codec/OpenTTD 反例),同时核对渐进视图 32.5%、ANN 周期内增量轮 −74.25% 与首读 3.0% / 5.0% 的载荷基线。

阅读全文
+

Agent 读上下文,也需要一套“查书”的方法

核对 EdgeBench 的十组记录分数,其中四组的时长接近 18 小时,其余涉及历史运行或未跑满的记录。文章也分别交代渐进视图 32.5%、ANN 周期内增量轮缩减 74.25% 和首读 3.0% / 5.0% 各自的测量基线。

阅读全文

研究札记 · EdgeBench 与 LHTB

diff --git a/apps/presentation/site/scripts/generate-long-horizon-audit.py b/apps/presentation/site/scripts/generate-long-horizon-audit.py index 48d837c18a..1fb4c26288 100644 --- a/apps/presentation/site/scripts/generate-long-horizon-audit.py +++ b/apps/presentation/site/scripts/generate-long-horizon-audit.py @@ -5,6 +5,7 @@ """ from pathlib import Path import json +import re import matplotlib as mpl mpl.use("Agg") @@ -30,6 +31,23 @@ def save(fig, relative): plt.close(fig) +def listed_times_near_18_hours(row): + times = [] + for side in ("on", "off"): + match = re.fullmatch(r"(\d+)h([0-5]\d)m", row[f"context_{side}_duration"]) + if match is None: + return False + times.append(int(match[1]) * 60 + int(match[2])) + return all(abs(minutes - 18 * 60) <= 15 for minutes in times) and abs(times[0] - times[1]) <= 15 + + +def reported_side_ahead(row): + on, off = row["context_on_native"], row["context_off_native"] + if on == off: + return None + return "on" if (on > off) == (row["direction"] == "higher") else "off" + + for language in ("en", "zh"): zh = language == "zh" # 1. Independent archived tutorial comparisons, not one measured cascade. @@ -324,165 +342,132 @@ def save(fig, relative): fig.text(.03,.025,footer,fontsize=10,color="#475569",linespacing=1.6) save(fig, f"images/edgebench-feedback-and-memory/stride-yardstick-closed-loop-{language}.svg") - # 4. EdgeBench Controlled Two-Arm Explore + Stride: context-on vs context-off score comparison (10 tasks) - two_arm = DATA["stride_context_on_off_comparison"] - arm_tasks = {row["task"]: row for row in two_arm["tasks"]} - ordered_norm_names = ["ANN", "FFmpeg", "VLIW", "Grid", "Ad", "Rust", "OpenTTD", "Codec", "Portfolio"] - norm_rows = [arm_tasks[name] for name in ordered_norm_names] + # 4. Reported EdgeBench score pairs and their observation limits. + comparison = DATA["stride_context_on_off_comparison"] + rows = comparison["tasks"] + norm_rows = sorted( + (row for row in rows if row["context_on_norm"] is not None), + key=lambda row: row["context_on_norm"] - row["context_off_norm"], + reverse=True, + ) + native_rows = {row["task"]: row for row in rows} fig = plt.figure(figsize=(14.6, 7.8)) gs = fig.add_gridspec( 1, 2, width_ratios=[1.25, 1.05], - left=.18, right=.98, bottom=.175, top=.805, wspace=.11, + left=.17, right=.97, bottom=.175, top=.805, wspace=.14, ) left = fig.add_subplot(gs[0, 0]) right = fig.add_subplot(gs[0, 1]) - - y_pos = list(range(len(norm_rows))) - on_vals = [r["context_on_norm"] for r in norm_rows] - off_vals = [r["context_off_norm"] for r in norm_rows] - off_bar_color = "#94a3b8" - - left.barh([y - .18 for y in y_pos], on_vals, height=.32, - color=TEAL, label=("Explore + Stride + context-on" if not zh else "Explore + Stride + context-on(开启上下文交付)")) - left.barh([y + .18 for y in y_pos], off_vals, height=.32, - color=off_bar_color, label=("Explore + Stride + context-off" if not zh else "Explore + Stride + context-off(关闭上下文交付)")) - - tag_map_zh = { - "ANN": "ANN (QPS↑ · 历史7臂Stride)", - "FFmpeg": "FFmpeg (原生↑ · OFF早停)", - "VLIW": "VLIW (周期↓ · OFF 14h45m)", - "Grid": "Grid (总分↓ · 等时长 ~18h)", - "Ad": "Ad (总分↑ · OFF 8h47m)", - "Rust": "Rust (原生↑ · 等时长 ~18h)", - "OpenTTD": "OpenTTD (原生↑ · 均<18h)", - "Codec": "Codec (原生↑ · OFF 12h38m)", - "Portfolio": "Portfolio (分↑ · 等时长 18h)", - } - tag_map_en = { - "ANN": "ANN (QPS↑ · 7-arm Stride)", - "FFmpeg": "FFmpeg (score↑ · OFF early stop)", - "VLIW": "VLIW (cycles↓ · OFF 14h45m)", - "Grid": "Grid (cost↓ · ~18h matched)", - "Ad": "Ad (score↑ · OFF 8h47m)", - "Rust": "Rust (score↑ · ~18h matched)", - "OpenTTD": "OpenTTD (score↑ · both <18h)", - "Codec": "Codec (score↑ · OFF 12h38m)", - "Portfolio": "Portfolio (score↑ · 18h matched)", + positions = list(range(len(norm_rows))) + on_bars = left.barh( + [y - .18 for y in positions], [row["context_on_norm"] for row in norm_rows], + height=.32, color=TEAL, label="Context ON", + ) + off_bars = left.barh( + [y + .18 for y in positions], [row["context_off_norm"] for row in norm_rows], + height=.32, color="#94a3b8", label="Context OFF", + ) + tags = { + "matched_18h": "接近 18h" if zh else "near 18h", + "historical_stride": "历史时长待核" if zh else "historical, time unverified", + "incomplete_off": "OFF 时长较短" if zh else "shorter OFF", + "both_incomplete": "均未满 18h" if zh else "both under 18h", } - left.set_yticks(y_pos, [(tag_map_zh if zh else tag_map_en)[r["task"]] for r in norm_rows], fontsize=9.7) + suffix = "zh" if zh else "en" + left.set_yticks( + positions, + [f"{row['task']} · {tags[row['cohort_group']]}" for row in norm_rows], + fontsize=9.5, + ) left.invert_yaxis() - left.set_xlim(0, 134) - left.set_xticks([0, 25, 50, 75, 100]) - left.set_xlabel("EdgeBench 百分制归一分 (/100,越高越优;含 ↓ 指标已方向归一化)" if zh else "EdgeBench normalized score (/100, higher is better)", fontsize=9.5) + left.set_xlim(0, 137) + ticks = [0, 25, 50, 75, 100] + left.set_xticks(ticks) + for tick_value, text in zip(ticks, left.get_xticklabels()): + text.set_gid(f"context-axis-{tick_value}") + left.set_xlabel("EdgeBench 百分制分数(越高越好)" if zh else "EdgeBench normalized score (/100, higher is better)", fontsize=9.5) left.set_title( - "A. 固定 Stride 下 context-on vs context-off 百分制对照" if zh else "A. Fixed Stride: context-on vs context-off (/100)", - loc="left", pad=14, fontsize=11.0, fontweight="bold", + "A. 九组百分制分数(ON / OFF)" if zh else "A. Nine normalized pairs (ON / OFF)", + loc="left", pad=14, fontsize=11, fontweight="bold", ) left.set_axisbelow(True) left.grid(axis="x", color="#e2e8f0", linewidth=.7) left.tick_params(axis="both", length=0) - for idx, r in enumerate(norm_rows): - diff = r["context_on_norm"] - r["context_off_norm"] - anchor = max(r["context_on_norm"], r["context_off_norm"]) - if r["task"] == "Rust": - txt = f"{r['context_on_norm']:.2f} vs {r['context_off_norm']:.2f} (+0.0044)" - col = TEAL - fw = "bold" - elif diff > 0: - txt = f"+{diff:.2f} ({r['context_on_norm']:.2f} vs {r['context_off_norm']:.2f})" - col = TEAL - fw = "bold" - else: - txt = f"{diff:.2f} ({r['context_on_norm']:.2f} vs {r['context_off_norm']:.2f})" - col = "#b45309" - fw = "bold" - left.text(anchor + 2.2, idx, txt, va="center", fontsize=8.8, fontweight=fw, color=col) + for index, (row, on_bar, off_bar) in enumerate(zip(norm_rows, on_bars, off_bars)): + task = row["task"] + on_bar.set_gid(f"context-bar-{task}-on") + off_bar.set_gid(f"context-bar-{task}-off") + on, off = row["context_on_norm"], row["context_off_norm"] + delta = on - off + color = TEAL if delta > 0 else ("#b45309" if delta < 0 else SLATE) + left.text(max(on, off) + 2.2, index, f"{on:.2f} / {off:.2f} ({delta:+.2f})", + va="center", fontsize=8.8, fontweight="bold", color=color).set_gid(f"context-score-{task}") right.axis("off") right.set_title( - "B. 原生关键差值(含 Borden)与反例边界" if zh else "B. Native score deltas (Borden) & counterexamples", - loc="left", pad=14, fontsize=11.0, fontweight="bold", + "B. 原生分(ON / OFF)" if zh else "B. Native scores (ON / OFF)", + loc="left", pad=14, fontsize=11, fontweight="bold", ) - borden = arm_tasks["Borden"] - cards_two_arm = ( - [ - ( - "第 10 题(无 /100 标度):Borden 等时长 ~18h\n" - f"• context-on(18h02m):{borden['context_on_native']}(17 次有效)\n" - f"• context-off(17h59m):{borden['context_off_native']}(17 次有效)\n" - "→ +35 分(+77.8%,同构等时长显著拉升)。", - "#f0fdfa", "#99f6e4", - ), - ( - "关键原生分数增幅(固定 Explore + Stride)\n" - "• ANN (QPS ↑):7,159 vs 3,746(+91.1%, +33.51 百分分)\n" - "• FFmpeg (原生 ↑):2.7086 vs 1.9929(+35.9%, OFF 早停)\n" - "• VLIW (周期 ↓):1,042 vs 1,097(−55 周期 / −5.0%)\n" - "• Grid (代价 ↓):2,197 vs 2,311(−114 / −4.9%, ~18h)", - "#f8fafc", "#cbd5e1", - ), - ( - "10 题总盘:ON 7 项占先 vs OFF 3 项领先\n" - "• 等时长 ~18h(4 题):3 胜(Borden/Grid/Rust 微先),\n" - " 1 反例(Portfolio 13.62 vs 39.44)\n" - "• 非等时长或历史对照(6 题):ON 领先 4 题,\n" - " OFF 领先 2 题(Codec 70.79 / OpenTTD 33.09M)", - "#fffbeb", "#fcd34d", - ), - ] - if zh else - [ - ( - "Task #10 (no /100 scale): Borden (~18h matched)\n" - f"• context-on (18h02m): {borden['context_on_native']} (17 valid)\n" - f"• context-off (17h59m): {borden['context_off_native']} (17 valid)\n" - "→ +35 pts (+77.8%, duration-matched gain).", - "#f0fdfa", "#99f6e4", - ), - ( - "Key native deltas under fixed Explore + Stride\n" - "• ANN (QPS ↑): 7,159 vs 3,746 (+91.1%, +33.51 pts)\n" - "• FFmpeg (score ↑): 2.7086 vs 1.9929 (+35.9%, OFF early stop)\n" - "• VLIW (cycles ↓): 1,042 vs 1,097 (−55 cycles / −5.0%)\n" - "• Grid (cost ↓): 2,197 vs 2,311 (−114 / −4.9%, ~18h)", - "#f8fafc", "#cbd5e1", - ), - ( - "10-task summary: 7 ON ahead/edge vs 3 OFF ahead\n" - "• Matched ~18h (4): 3 ON wins (Borden/Grid/Rust),\n" - " 1 counterexample (Portfolio 13.62 vs 39.44)\n" - "• Historical / <18h (6): 4 ON ahead, 2 OFF ahead\n" - " (Codec 68.89 vs 70.79; OpenTTD 30.27M vs 33.09M)", - "#fffbeb", "#fcd34d", - ), - ] - ) - y_starts = [.98, .63, .28] - for y_box, (txt, face, edge) in zip(y_starts, cards_two_arm): - right.text(.01, y_box, txt, transform=right.transAxes, va="top", fontsize=9.1, - linespacing=1.50, bbox={"boxstyle": "round,pad=.60", "facecolor": face, "edgecolor": edge}) + borden = native_rows["Borden"] + winners = {row["task"]: reported_side_ahead(row) for row in rows} + near = [row for row in rows if listed_times_near_18_hours(row)] + on_count = list(winners.values()).count("on") + off_count = list(winners.values()).count("off") + near_on = sum(winners[row["task"]] == "on" for row in near) + near_off = sum(winners[row["task"]] == "off" for row in near) + examples = [native_rows[task] for task in ("ANN", "FFmpeg", "VLIW", "Grid")] + counterexamples = [row for row in rows if winners[row["task"]] == "off"] + + def native_line(row): + return f"{row['task']} ({row[f'metric_label_{suffix}']}): {row['context_on_native']:,} / {row['context_off_native']:,}" + + borden_label = "Borden 无百分制分数" if zh else "Borden has no /100 score" + near_line = (f"接近 18h 的 {len(near)} 组:ON {near_on} / OFF {near_off}" if zh + else f"{len(near)} near-18h pairs: ON {near_on} / OFF {near_off}") + total_line = (f"{len(rows)} 组记录:ON {on_count} / OFF {off_count}" if zh + else f"{len(rows)} reported pairs: ON {on_count} / OFF {off_count}") + cards = [ + ("context-native-borden", "\n".join([ + borden_label, native_line(borden), + f"ON {borden['context_on_duration']} / OFF {borden['context_off_duration']}", + ]), "#f0fdfa", "#99f6e4"), + ("context-native-examples", "\n".join([ + "部分原生分(ANN 的 OFF 分数待核)" if zh else "Examples (ANN OFF unverified)", + *(native_line(row) for row in examples), + ]), "#f8fafc", "#cbd5e1"), + ("context-native-off", "\n".join([ + total_line, near_line, "OFF 占先的题目" if zh else "Tasks where OFF leads", + *(native_line(row) for row in counterexamples), + ]), "#fffbeb", "#fcd34d"), + ] + for top, (group_id, text, face, edge) in zip([.98, .72, .36], cards): + right.text( + .01, top, text, transform=right.transAxes, va="top", fontsize=9.1, + linespacing=1.45, bbox={"boxstyle": "round,pad=.60", "facecolor": face, "edgecolor": edge}, + ).set_gid(group_id) fig.suptitle( - "EdgeBench 十题同构效果对照:固定 Explore + Stride 下的 Context ON vs OFF" - if zh else - "EdgeBench 10-task controlled comparison: Context ON vs OFF under Explore + Stride", + "EdgeBench 十题记录中的 Context ON 与 OFF 分数" if zh else + "EdgeBench scores reported for Context ON and OFF", fontsize=14.6, fontweight="bold", y=.955, ) - sub_two_arm = ( - "固定 Explore + Stride 框架(context-on 开启 Progressive 与周期内差分)。10 题中 context-on 7 题占先,完整保留 3 项反例与真实时长差。" + other_count = len(rows) - len(near) + fig.text( + .5, .88, + f"{len(near)} 组列出的时长接近 18 小时,其余 {other_count} 组涉及历史运行或未跑满的记录。" if zh else - "Holding Explore + Stride constant (context-on adds progressive reads + cycle deltas). Ahead on 7/10 tasks, retaining 3 counterexamples and runtime gaps." + f"{len(near)} pairs list runtimes near 18 hours. The other {other_count} include historical or incomplete runs.", + ha="center", fontsize=10, color="#475569", ) - fig.text(.5, .88, sub_two_arm, ha="center", fontsize=9.9, color="#475569") - fig.legend(*left.get_legend_handles_labels(), loc="lower center", ncol=2, frameon=False, bbox_to_anchor=(.45, .078), fontsize=9.6) - footer_two_arm = ( - "口径说明:截取自 EdgeBench 同构两臂实测(2026-11-12 22:40 CST 快照)。Borden 无百分制换算故在右栏 B 单列原生分(80 vs 45)。\n" - "Grid、Borden、Rust、Portfolio 四题双臂均已跑满约 18h(15m 容差内);ANN 取自历史七臂对应 Stride 配置;其余 5 题存在至少一臂未满 18h。本图仅检验开关的组内方向性效果。" + fig.legend(*left.get_legend_handles_labels(), loc="lower center", ncol=2, frameon=False, + bbox_to_anchor=(.45, .081), fontsize=9.6) + fig.text( + .03, .028, + f"快照 {comparison['snapshot_cst']}。现有逐臂记录不足以单独衡量 Context Delivery 的影响。" if zh else - "Basis: EdgeBench controlled two-arm snapshot at 2026-11-12 22:40 CST. Borden has no /100 scale and appears in native points in Panel B (80 vs 45).\n" - "Grid, Borden, Rust, and Portfolio are ~18h duration-matched (within 15m); ANN uses the matching Stride pair from the 7-arm study; remaining 5 pairs have <18h runs. Checks within-harness direction only." + f"Snapshot {comparison['snapshot_cst']}. The public records cannot isolate the effect of Context Delivery.", + fontsize=8.9, color="#475569", ) - fig.text(.03, .018, footer_two_arm, fontsize=8.9, color="#475569", linespacing=1.45) save(fig, f"images/context-delivery/stride-context-on-vs-off-{language}.svg") diff --git a/examples/blog-bilingual-index-smoke.mjs b/examples/blog-bilingual-index-smoke.mjs index 1c2235fdb6..f32ace408c 100644 --- a/examples/blog-bilingual-index-smoke.mjs +++ b/examples/blog-bilingual-index-smoke.mjs @@ -76,10 +76,161 @@ function listingDates(html, language, articleSlugs) { return dates; } +const SNAPSHOT_TASKS = ["Borden", "Grid", "Rust", "Portfolio", "ANN", "FFmpeg", "VLIW", "Ad", "Codec", "OpenTTD"]; +const SNAPSHOT_SCOPES = { + matched_18h: ["Near 18h", "接近 18 小时"], + historical_stride: ["Historical", "历史记录"], + incomplete_off: ["Shorter or early-stop OFF", "OFF 时长较短或早停"], + both_incomplete: ["Both below 18h", "两臂均未满 18 小时"], +}; +const reportedNumber = value => value.toLocaleString("en-US", { maximumFractionDigits: 4 }); +const signed = value => (value < 0 ? "−" : value > 0 ? "+" : "") + reportedNumber(Math.abs(value)); +const delta = (on, off) => Number((on - off).toFixed(4)); +const durationParts = value => value.match(/^(\d+)h([0-5]\d)m$/)?.slice(1).map(Number); +const plainText = svgOrHtml => svgOrHtml.replace(/<[^>]*>/g, "").replaceAll("&", "&").replaceAll("<", "<").trim(); + +function scoreSnapshotRows(snapshot) { + for (const source of [snapshot.source_url, snapshot.prior_public_ann_url]) { + if (!/^https:\/\/github\.com\/loopx-project\/loopx\/blob\/[a-f0-9]{40}\//.test(source)) { + throw new Error("Score snapshot must link immutable public sources"); + } + } + deepStrictEqual(snapshot.tasks.map(row => row.task), SNAPSHOT_TASKS, "Score snapshot task order and coverage"); + const pairs = snapshot.tasks.map(row => { + const { task, direction } = row; + const arms = ["on", "off"].map(side => ({ + native: row["context_" + side + "_native"], + norm: row["context_" + side + "_norm"], + duration: row["context_" + side + "_duration"], + valid: row["context_" + side + "_valid"], + })); + const [on, off] = arms; + if (!["higher", "lower"].includes(direction) || !Object.hasOwn(SNAPSHOT_SCOPES, row.cohort_group) || + !arms.every(arm => Number.isFinite(arm.native) && + (task === "Borden" ? arm.norm === null : Number.isFinite(arm.norm) && arm.norm >= 0 && arm.norm <= 100) && + typeof arm.duration === "string" && + (durationParts(arm.duration) || ["unverified", "stopped early"].includes(arm.duration)) && + Number.isInteger(arm.valid) && arm.valid >= 0)) { + throw new Error("Invalid score snapshot row: " + task); + } + const sign = direction === "lower" ? -1 : 1; + const nativeWinner = Math.sign((on.native - off.native) * sign); + const normWinner = on.norm === null ? 0 : Math.sign(on.norm - off.norm); + const onLeads = nativeWinner > 0; + const winner = onLeads ? (on.norm !== null && normWinner === 0 ? "context-on-edge" : "context-on") : "context-off"; + if (nativeWinner === 0 || (normWinner !== 0 && normWinner !== nativeWinner) || row.winner_explore !== winner) { + throw new Error("Score snapshot direction or winner conflicts with the reported values: " + task); + } + const reportedMinutes = arms.map(arm => { + const time = durationParts(arm.duration); + return time ? time[0] * 60 + time[1] : Infinity; + }); + const near = reportedMinutes.every(time => Math.abs(time - 1080) <= 15) && + Math.abs(reportedMinutes[0] - reportedMinutes[1]) <= 15; + deepStrictEqual(near, row.cohort_group === "matched_18h", "Near-18h classification: " + task); + return { row, on, off, onLeads, near }; + }); + const near = pairs.filter(pair => pair.near); + const ahead = pairs.filter(pair => pair.onLeads); + const offTasks = pairs.filter(pair => !pair.onLeads).map(pair => pair.row.task); + deepStrictEqual(offTasks, ["Portfolio", "Codec", "OpenTTD"], "Keep the three reported OFF counterexamples"); + deepStrictEqual(snapshot.summary, { + total_tasks: pairs.length, + context_on_ahead_or_edge: ahead.length, + context_off_ahead: offTasks.length, + duration_matched_18h_pairs: near.length, + duration_matched_context_on_wins: near.filter(pair => pair.onLeads).length, + duration_matched_context_off_wins: near.filter(pair => !pair.onLeads).length, + }, "Score snapshot tally must follow native direction and the listed runtimes"); + return pairs; +} + +function assertScoreTable(section, pairs, lang) { + const zh = lang === "zh"; + const table = section.match(/([\s\S]*?)<\/tbody>/)?.[1] ?? ""; + const rows = [...table.matchAll(/([\s\S]*?)<\/tr>/g)]; + deepStrictEqual(rows.length, pairs.length, "Score table must keep all ten task rows: " + lang); + const armText = ({ native, norm, duration, valid }) => { + const time = zh ? ({ unverified: "待核实", "stopped early": "早停" }[duration] ?? duration) : duration; + const cells = (norm?.toFixed(2) ?? "—") + " · " + time + " · " + valid; + return reportedNumber(native) + (zh ? "(" + cells + " 次)" : " (" + cells + " valid)"); + }; + for (const [index, { row, on, off, onLeads }] of pairs.entries()) { + const raw = [...rows[index][1].matchAll(/]*>([\s\S]*?)<\/td>/g)].map(match => match[1]); + const metric = row["metric_label_" + lang] + (row.direction === "higher" ? " ↑" : " ↓"); + const title = row.task + (zh ? "(" + metric + ")" : " (" + metric + ")"); + const scope = SNAPSHOT_SCOPES[row.cohort_group][Number(zh)] + (onLeads ? "" : (zh ? " · OFF 占先" : " · OFF leads")); + const changes = signed(delta(on.native, off.native)) + " / " + + (on.norm === null ? "—" : signed(delta(on.norm, off.norm))); + deepStrictEqual(raw.map(plainText), [title, scope, armText(on), armText(off), changes], + "Score table row differs from the published aggregate: " + lang + "/" + row.task); + deepStrictEqual([raw[2].startsWith(""), raw[3].startsWith("")], + [onLeads, !onLeads], "Score table emphasizes the correct side: " + lang + "/" + row.task); + } +} + +function assertScoreFigure(svg, pairs, lang) { + // Matplotlib emits named groups for each actual label and bar, with child groups indented. + const group = id => { + const contents = svg.match(new RegExp('^( *)\\n([\\s\\S]*?)^\\1$', "m"))?.[2]; + if (!contents) throw new Error("Score chart is missing " + lang + "/" + id); + return contents; + }; + const labels = value => [...value.matchAll(/]*>([\s\S]*?)<\/text>/g)].map(match => plainText(match[1])); + const tickX = n => Number(group("context-axis-" + n).match(/]*\bx="([^"]+)"/)?.[1]); + const origin = tickX(0); + const unit = (tickX(100) - origin) / 100; + const normalized = pairs.filter(pair => pair.on.norm !== null) + .sort((a, b) => delta(b.on.norm, b.off.norm) - delta(a.on.norm, a.off.norm)); + deepStrictEqual([...svg.matchAll(//g)].map(match => match[1]), + normalized.map(pair => pair.row.task), "Score chart must retain the normalized delta order: " + lang); + for (const { row, on, off } of normalized) { + const change = delta(on.norm, off.norm); + const label = on.norm.toFixed(2) + " / " + off.norm.toFixed(2) + + " (" + (change >= 0 ? "+" : "") + change.toFixed(2) + ")"; + deepStrictEqual(labels(group("context-score-" + row.task)), [label], + "Score chart label differs from the published aggregate: " + lang + "/" + row.task); + for (const [side, arm] of [["on", on], ["off", off]]) { + const points = [...group("context-bar-" + row.task + "-" + side).matchAll(/[ML] ([\d.]+) /g)] + .map(match => Number(match[1])); + if (!points.length || !(Math.abs(Math.min(...points) - origin) < 0.02) || + !(Math.abs(Math.max(...points) - (origin + arm.norm * unit)) < 0.02)) { + throw new Error("Score chart bar differs from the published aggregate: " + lang + "/" + row.task + "/" + side); + } + } + } + + const nativeLine = ({ row, on, off }) => + row.task + " (" + row["metric_label_" + lang] + "): " + reportedNumber(on.native) + " / " + reportedNumber(off.native); + const borden = pairs[0]; + deepStrictEqual(labels(group("context-native-borden")).slice(1), + [nativeLine(borden), "ON " + borden.on.duration + " / OFF " + borden.off.duration], + "Score chart Borden native pair differs: " + lang); + const examples = ["ANN", "FFmpeg", "VLIW", "Grid"].map(task => pairs.find(pair => pair.row.task === task)); + deepStrictEqual(labels(group("context-native-examples")).slice(1), examples.map(nativeLine), + "Score chart native example differs: " + lang); + const counters = pairs.filter(pair => !pair.onLeads); + const near = pairs.filter(pair => pair.near); + const nearOn = near.filter(pair => pair.onLeads).length; + const counts = (total, off) => "ON " + (total - off) + " / OFF " + off; + const total = counts(pairs.length, counters.length); + const nearTotal = counts(near.length, near.length - nearOn); + const expectedCard = lang === "zh" + ? [pairs.length + " 组记录:" + total, "接近 18h 的 " + near.length + " 组:" + nearTotal, "OFF 占先的题目"] + : [pairs.length + " reported pairs: " + total, near.length + " near-18h pairs: " + nearTotal, "Tasks where OFF leads"]; + deepStrictEqual(labels(group("context-native-off")), [...expectedCard, ...counters.map(nativeLine)], + "Score chart observed tally or native counterexample differs: " + lang); +} + export async function validateBilingualBlog(blogDir) { const auditPath = resolve(blogDir, "data/long-horizon-audit.json"); + if (!existsSync(auditPath) && existsSync(resolve(blogDir, "context-delivery/index.html"))) { + throw new Error("The context delivery article requires its public audit data"); + } if (existsSync(auditPath)) { const audit = JSON.parse(await readFile(auditPath, "utf8")); + const snapshot = audit.stride_context_on_off_comparison; + const snapshotRows = scoreSnapshotRows(snapshot); const tasks = audit.context.tasks; const sum = key => tasks.reduce((total, task) => total + task[key], 0); deepStrictEqual([tasks.length, sum("wakes"), sum("baseline_wakes"), sum("delta_wakes")], @@ -132,48 +283,25 @@ export async function validateBilingualBlog(blogDir) { "utf8", ); assertIncludes(multitaskSvg, "100.0%", `Eight-task byte audit chart must anchor at 100% Full: ${langSuffix}`); - const twoArm = audit.stride_context_on_off_comparison; - deepStrictEqual( - [ - twoArm?.summary?.total_tasks, - twoArm?.summary?.context_on_ahead_or_edge, - twoArm?.summary?.context_off_ahead, - twoArm?.tasks?.length, - ], - [10, 7, 3, 10], - "Controlled two-arm comparison summary must match the 10-task tally (7 ahead/edge, 3 OFF ahead)", + const snapshotSection = article.split('

')[1]?.split('

')[0] ?? ""; + const snapshotSvg = await readFile( + resolve(blogDir, "images/context-delivery/stride-context-on-vs-off-" + langSuffix + ".svg"), "utf8", ); - assertIncludes( - article, - '

', - `Both published editions must expose the controlled Context ON vs OFF score section: ${locale}`, - ); - assertIncludes( - article, - `stride-context-on-vs-off-${langSuffix}.svg`, - `Both published editions must embed the controlled Context ON vs OFF chart: ${langSuffix}`, - ); - const twoArmSvg = await readFile( - resolve(blogDir, `images/context-delivery/stride-context-on-vs-off-${langSuffix}.svg`), - "utf8", - ); - const twoArmSection = article.split('

')[1]?.split('

')[0] ?? ""; - const twoArmTable = twoArmSection.match(/([\s\S]*?)<\/tbody>/)?.[1]; - const twoArmRows = [...(twoArmTable ?? "").matchAll(/([\s\S]*?)<\/tr>/g)]; - deepStrictEqual( - twoArmRows.length, - twoArm.tasks.length, - `Controlled two-arm score table must retain all 10 task rows including counterexamples: ${locale}`, - ); - for (const requiredToken of ["7,159", "3,746", "2,197", "2,311", "27.1801", "13.62", "39.44", "68.89", "70.79"]) { - assertIncludes(twoArmSection, requiredToken, `Controlled score section must preserve ${requiredToken} (${locale})`); + assertIncludes(snapshotSection, "stride-context-on-vs-off-" + langSuffix + ".svg", "Both editions link the reported score figure"); + for (const source of [snapshot.source_url, snapshot.prior_public_ann_url]) { + assertIncludes(snapshotSection, source, "Both editions distinguish the original snapshot from earlier ANN evidence"); } - for (const isolatedText of [twoArmSection, twoArmSvg, JSON.stringify(twoArm)]) { - if (/four-arm|four arm|四臂/i.test(isolatedText)) { - throw new Error(`Controlled two-arm section must remain isolated without four-arm framing (${locale})`); + assertScoreTable(snapshotSection, snapshotRows, langSuffix); + assertScoreFigure(snapshotSvg, snapshotRows, langSuffix); + const index = await readFile(resolve(blogDir, locale, "index.html"), "utf8"); + const listing = index.match(/

[^<]+<\/a><\/h2>

([^<]+)<\/p>/)?.[1] ?? ""; + assertIncludes(listing, langSuffix === "zh" ? "记录分数" : "reported EdgeBench", "Blog catalog describes the recorded score pair scope"); + for (const observedText of [snapshotSection, snapshotSvg, JSON.stringify(snapshot), listing]) { + if (/controlled|fixed Explore|holding Explore|同构|受控|固定 Explore|four.arm|四臂/i.test(observedText)) { + throw new Error("Reported score snapshot restored a controlled or mixed-arm claim: " + locale); } } - for (const publicText of [article, twoArmSvg, JSON.stringify(audit)]) { + for (const publicText of [article, snapshotSvg, JSON.stringify(audit)]) { if (/sforge/i.test(publicText)) { throw new Error(`Internal benchmark name leaked into public context-delivery blog artifact (${locale})`); } @@ -466,6 +594,37 @@ async function assertClockLabelRejected(blogDir, { figure, declared, regressed } } } +async function assertScoreMismatchRejected(blogDir) { + const temporaryRoot = await mkdtemp(join(tmpdir(), "loopx-blog-score-parity-")); + const fixtureBlog = join(temporaryRoot, "blog"); + try { + await cp(blogDir, fixtureBlog, { recursive: true }); + // Each mutation changes just one published representation (or its source provenance). + const data = "data/long-horizon-audit.json"; + const en = "context-delivery/index.html"; + const zh = "zh/context-delivery/index.html"; + const svg = "images/context-delivery/stride-context-on-vs-off-en.svg"; + for (const [path, before, after, expected] of [ + [data, '"context_on_native": 80,', '"context_on_native": 5,', /snapshot direction or winner/], + [data, '"context_on_native": 80,', '"context_on_native": 81,', /table row differs.*Borden/], + [svg, "7,159 / 3,746", "7,159 / 3,747", /chart native example differs/], + [en, "33,090,430 (62.99", "13,090,430 (62.99", /table row differs.*OpenTTD/], + [zh, "(27.18 · 17h57m · 10 次)", "(27.18 · 17h57m · 1 次)", /table row differs.*Rust/], + [data, '"prior_public_ann_url": "https://github.com/', '"prior_public_ann_url": "https://example.invalid/', /immutable public sources/], + [data, '"direction": "lower",\n "metric_label_en": "Total cost"', '"direction": "higher",\n "metric_label_en": "Total cost"', /snapshot direction or winner/], + ]) { + const file = resolve(fixtureBlog, path); + const source = await readFile(file, "utf8"); + assertIncludes(source, before, "Negative fixture must contain the original value: " + path); + await writeFile(file, source.replace(before, after)); + await rejects(() => validateBilingualBlog(fixtureBlog), expected); + await writeFile(file, source); + } + } finally { + await rm(temporaryRoot, { recursive: true, force: true }); + } +} + const modulePath = fileURLToPath(import.meta.url); if (process.argv[1] && resolve(process.argv[1]) === modulePath) { const articleUrl = "https://loopx-project.github.io/loopx/blog/example/"; @@ -552,5 +711,6 @@ if (process.argv[1] && resolve(process.argv[1]) === modulePath) { }, "中文主图恢复反馈口径也必须被配对标签检查拦下", ); + await assertScoreMismatchRejected(blogDir); console.log(`blog-bilingual-index-smoke: ok (${articleSlugs.length} paired articles)`); } From 6faee8b2591a99aa6abab516f774cf92d8d98e77 Mon Sep 17 00:00:00 2001 From: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> Date: Mon, 16 Nov 2026 23:38:45 +0800 Subject: [PATCH 2/2] docs(issue-fix): keep publication link labels on one line Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com> --- loopx/capabilities/issue_fix/README.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/loopx/capabilities/issue_fix/README.md b/loopx/capabilities/issue_fix/README.md index 3338cccdbe..c8b4d3c8de 100644 --- a/loopx/capabilities/issue_fix/README.md +++ b/loopx/capabilities/issue_fix/README.md @@ -345,14 +345,11 @@ candidate so a maintainer can audit why that person was selected. `CODEOWNERS` remains the strongest repository-native signal. Commit volume is only evidence of familiarity; it is not proof of maintainership, availability, -or review authority. See the [reviewer recommendation -contract](docs/protocols/issue-fix-reviewer-recommendation-v0.md) for scoring, -identity, and future-signal details, and the [reviewer request -contract](docs/protocols/issue-fix-reviewer-request-v0.md) for the external-write, +or review authority. See the [reviewer recommendation contract](docs/protocols/issue-fix-reviewer-recommendation-v0.md) for scoring, +identity, and future-signal details, and the [reviewer request contract](docs/protocols/issue-fix-reviewer-request-v0.md) for the external-write, idempotency, and verification rules. Secondary delivery, dedicated-bot isolation, local-private identity mapping, and readback are defined by the -[reviewer notification sink -contract](docs/protocols/issue-fix-reviewer-notification-sinks-v0.md). +[reviewer notification sink contract](docs/protocols/issue-fix-reviewer-notification-sinks-v0.md). ### 6. PR publication and public-write boundary