diff --git a/CHANGELOG.md b/CHANGELOG.md index 2d5ebb2c..ee82ab78 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- fix(progress-report, build-plan-status): **HARNESS rows are no longer counted as built rows, and a module's `verify-module.sh` row shows once, as that module's sign-off status on its phase card, instead of as a row.** A HARNESS step is a module's sign-off, not a build, so it is out of every "x of y built" count: the phase cards, the plan rollup and the Stage 5 card in `progress-report.html`, and the lede of `build-plan-status.sh --html` ("N build rows", plus a "Not counted: N HARNESS rows" note). The verify row's evidence is read exactly as #501 reads it (`harness_verify`'s rule, now `verify_evidence` + `module_status` in `progress-report.py`; nothing new measured, no extra script run, no mxcli or mxbuild call) and shown as one line per module: `Verified ✓ (LOOK n/n)`, `Verify owed - LOOK + CONFIRM`, `Verify stale (N builds since)` (or `(a script was marked done- after it)`), `Verify not run`, `built? no verify summary`, `Verify not clean ()` for a summary with a FAULT, FINDING or SKIPPED row, or `Verify clean , LOOK n/n - held: State says ` when State is blocked or pending a person (held is not signed off, as #501 never promoted it); a State cell that disagrees is flagged under it, the rollup says how many modules are verified, and the Stage 5 card, KPI and owed text add ", k of n modules signed off". The Phase state badge is read from the build rows only (a phase of HARNESS rows alone says "sign-off only" and its card "Sign-off phase: no build rows"). A module with no verify row shows no status. Other HARNESS rows (a coherence pass, a harness run) stay in the row list, marked "(not counted)". `build-plan.md`, `build-plan.json` and the text output are unchanged, so agents still see every HARNESS step. **Behaviour change to check in your project:** the progress report's built counts drop the HARNESS rows (a plan of 10 rows with 2 HARNESS reads "x of 8"), and the verify rows leave the row tables. Fixture: `test-progress-harness-verify.sh` (H1-H10, N1-N6, both layouts). — MendixMau (decision 2026-10-21) - fix(exec): **a script restated from a stale DESCRIBE now WARNs instead of silently reverting an earlier exec** (#261). A script drafted from a DESCRIBE taken before an earlier script of the batch ran put four old messages back, with check, mxbuild and lint green. New `project-bin/describe-baseline.sh stamp` writes `-- baseline: sha256=` (plus `-- baseline-mxcli: `) from a DESCRIBE at draft time; `exec.sh` runs `verify` only for a script that carries the header, so other scripts cost one grep and no mxcli call. It reports DRIFT with the first lines of the diff, STALEHEADER, and RESTAMP after an mxcli upgrade (DESCRIBE text may change, so that is not called drift). A re-run by the script's own last recorded write (the #269 ledger) and a fresh model are not drift. Nothing is refused and the exit codes do not change. Preflight STOP 21 gains one clause (`-- supersedes:`, `stamp`, exec.sh only warns) and its trigger names `create or modify` (+24 words). Not-found DESCRIBE and `--version` output are captured from mxcli v0.26.0. Lands after the #269 PR; with it, supersedes draft #311. **Behaviour change to check in your project:** `exec.sh` output can carry `WARN: stale draft: …` lines, and the BUILD-LOG detail cell `WARN stale baseline: `; `MXTK_RESTATE_GUARD=off` skips the check. — AlvinnYeo (#261) - fix(exec): **re-running an older `mdlsource/` script over a newer owner's live work now WARNs instead of passing silently** (#269). Three field apps had 99/579, 130/357 and 38/159 documents created by more than one script, and a replay of an older one reverted the newer with exit 0 and every gate green. New `project-bin/script-owners.sh` maps each microflow, nanoflow, page and snippet to the scripts that create or restate it, in run order (digit runs compared as numbers, so `2-core/` runs before `11-fixes/`; `done-` ignored), and reads `-- supersedes:` / `-- superseded-by:` headers. `exec.sh` records every kept write in a machine-local last-writer ledger (`.claude/loop/script-owners/`, self-gitignored, compacted per write); `restore-mpr.sh` marks it on a manual restore. A replay WARNs only when the newer owner's write is live: ledger first, else one DESCRIBE per document, at most 5 per exec (the rest print `CONFLICT (not checked)`). A new owner without a supersedes header gets one `NOTE:` line on its first run, kept out of BUILD-LOG. Nothing is refused and the exit codes do not change. With no co-owner, `exec.sh` runs one grep over `mdlsource/` and no mxcli call. `gate-check.sh` prints the co-owned count when it is above 0. Part of the rebuilt draft #311 (with the #261 PR). **Behaviour change to check in your project:** `exec.sh` output can carry `WARN: restatement: …` and `NOTE: new owner of …` lines; the BUILD-LOG detail cell can carry `WARN reverts: `. Set `MXTK_RESTATE_GUARD=off` to skip the check. — MendixMau (#269) - fix(exec): **re-running an older `mdlsource/` script over a newer owner's live work now WARNs instead of passing silently** (#269). Three field apps had 99/579, 130/357 and 38/159 documents created by more than one script, and a replay of an older one reverted the newer with exit 0 and every gate green. New `project-bin/script-owners.sh` maps each microflow, nanoflow, page and snippet to the scripts that create or restate it, in run order (digit runs compared as numbers, so `2-core/` runs before `11-fixes/`; `done-` ignored), and reads `-- supersedes:` / `-- superseded-by:` headers. `exec.sh` records every kept write in a machine-local last-writer ledger (`.claude/loop/script-owners/`, self-gitignored, compacted per write); `restore-mpr.sh` marks it on a manual restore. A script counts as run when it has a ledger row, a `done-` name, or an applied `docs/BUILD-LOG.md` row, so a fresh clone or cloud session (empty ledger) still warns on a replay over committed newer work; a script nobody ran yet (draft-ahead) stays quiet. The ledger cannot see Studio Pro edits or a `git pull`/`checkout` of the `.mpr`. A replay WARNs only when the newer owner's write is live: ledger first, else one DESCRIBE per document, at most 5 per exec (the rest print `CONFLICT (not checked)`). A new owner without a supersedes header gets one `NOTE:` line on its first run, kept out of BUILD-LOG. Nothing is refused and the exit codes do not change. With no co-owner, `exec.sh` runs one grep over `mdlsource/` and no mxcli call. `gate-check.sh` prints the co-owned count when it is above 0. Part of the rebuilt draft #311 (with the #261 PR). **Behaviour change to check in your project:** `exec.sh` output can carry `WARN: restatement: …` and `NOTE: new owner of …` lines; the BUILD-LOG detail cell can carry `WARN reverts: `. Set `MXTK_RESTATE_GUARD=off` to skip the check. — MendixMau (#269) diff --git a/project-bin/build-plan-status.sh b/project-bin/build-plan-status.sh index b165b648..4fc5e038 100755 --- a/project-bin/build-plan-status.sh +++ b/project-bin/build-plan-status.sh @@ -536,18 +536,17 @@ render_plan_section() { # stdout: the plan section's HTML import fnmatch, html, json, os, re, sys src, mdl, detail_out, proj, gen = sys.argv[1:6] -# A HARNESS row that runs verify-module.sh is read from that module's verify summary as -# well as from State; the rule lives in progress-report.py (harness_verify), so the page's -# counts and these rows cannot disagree. Without it, State alone decides, as before. -harness_verify = lambda *a, **k: None -_logs = [] +# HARNESS rows are not build rows (MendixMau decision 2026-10-21): they are out of the tally, and a +# HARNESS row that runs verify-module.sh is not listed here at all, because +# progress-report.py shows it once as that module's sign-off status on the phase card +# (verify_evidence / module_status there). build-plan.json and build-plan.md keep every row. +VERIFY_STEP = re.compile(r"verify-module\.sh`?\s+[`\"']?([A-Za-z_]\w*)") # same as progress-report.py's try: import importlib.util sys.dont_write_bytecode = True # no bin/__pycache__/ left in the project _spec = importlib.util.spec_from_file_location("progress_report", gen) _m = importlib.util.module_from_spec(_spec); _spec.loader.exec_module(_m) - harness_verify = _m.harness_verify - _logs = _m.build_log_rows(proj) + VERIFY_STEP = _m.VERIFY_STEP except (Exception, SystemExit): # a broken or old generator: the generator's own run reports it pass d = json.load(open(src, encoding="utf-8")) @@ -591,24 +590,36 @@ def classify(state): # the parser's rollup reading, per row (keep in step with CLS = {"built": "st-built", "not built": "st-todo", "pending a person": "st-person", "partial": "st-progress", "blocked": "st-todo", "skipped": "st-todo", - "in progress": "st-progress", "unknown": "st-unknown", None: "st-unknown"} + "in progress": "st-progress", "unknown": "st-unknown", "sign-off only": "st-person", None: "st-unknown"} phases = d.get("phases", []) all_steps = [s for p in phases for s in p["steps"]] tally = {"built": 0, "not built": 0, "pending a person": 0, "partial": 0, "blocked": 0, "skipped": 0, None: 0} -by_n = {str(s["number"]): s for s in all_steps} -for s in all_steps: - try: hv = harness_verify(proj, s, by_n, _logs) - except Exception: hv = None # an older progress-report.py beside this script: State decides - # (class for the tally, State cell text, drift flag); only "built" overrides the State reading - s["_ev"] = (("built" if hv[0] == "built" else classify(s["state"])),) + tuple(hv[1:]) if hv else None - tally[s["_ev"][0] if s["_ev"] else classify(s["state"])] += 1 -ev_rows = sum(1 for s in all_steps if s["_ev"] and s["_ev"][2]) +is_harness = lambda s: (s.get("kind") or "").strip().upper() == "HARNESS" +is_signoff = lambda s: is_harness(s) and bool(VERIFY_STEP.search(s.get("step") or "")) +build_steps = [s for s in all_steps if not is_harness(s)] +n_signoff = sum(1 for s in all_steps if is_signoff(s)); n_harness = len(all_steps) - len(build_steps) +signoff_mods = {VERIFY_STEP.search(s["step"] or "").group(1) for s in all_steps if is_signoff(s)} # modules, as the card's "k of n modules" + +def disp_rollup(steps): # the badge: parse_plan's rollup over the build rows only (build-plan.json keeps its own) + rows = [s for s in steps if not is_harness(s)] + if not rows: return "sign-off only" if steps else "unknown" + kinds = [classify(s["state"]) for s in rows] + if None in kinds: return "unknown" + kinds = [k for k in kinds if k != "skipped"] + if not kinds: return "unknown" + if all(k == "built" for k in kinds): return "built" + if all(k in ("not built", "blocked") for k in kinds): return "not built" + if all(k in ("built", "pending a person") for k in kinds): return "pending a person" + return "in progress" +for s in build_steps: + tally[classify(s["state"])] += 1 named_total = named_done = drift_rows = 0 rows_html = [] for p in phases: out = [] for s in p["steps"]: + if is_signoff(s): continue # shown once, as the module's sign-off status on the phase card names = named_scripts(s["step"], s["produces"]) k = classify(s["state"]) disk = "" @@ -633,11 +644,9 @@ for p in phases: drift_rows += 1 disk += '
⚠ %s
' % e(drift) st = e(s["state"]) - if s["_ev"]: - k = s["_ev"][0] - st = e(s["_ev"][1]) + ('
⚠ %s
' % e(s["_ev"][2]) if s["_ev"][2] else "") + kind = e(s["kind"]) + (' (not counted)' if is_harness(s) else "") out.append('%s%s%s%s%s%s%s' % ( - e(s["number"]), e(s["kind"]), ec(s["step"]), ec(s["produces"]), + e(s["number"]), kind, ec(s["step"]), ec(s["produces"]), e(", ".join(s["dependsOn"])), CLS[k], st, disk)) rows_html.append(out) @@ -645,13 +654,14 @@ plural = lambda n, w: "%d %s%s" % (n, w, "" if n == 1 else "s") print("

The plan

") print("

From architecture/build-plan.md: %s, %s: " "%d built, %d partial, %d not built%s, %d pending a person%s%s.

" % ( - plural(len(phases), "phase"), plural(len(all_steps), "row"), + plural(len(phases), "phase"), plural(len(build_steps), "build row"), tally["built"], tally["partial"], tally["not built"] + tally["blocked"], (" (%d of them blocked)" % tally["blocked"]) if tally["blocked"] else "", tally["pending a person"], (", %d not needed" % tally["skipped"]) if tally["skipped"] else "", (", %d with a State cell this page cannot read" % tally[None]) if tally[None] else "") - + ("

⚠ %s where State and the verify evidence (summary, LOOK report) disagree, or the evidence is incomplete (shown in State).

" % plural(ev_rows, "HARNESS row") if ev_rows else "")) + + ("

Not counted: %s (%s). A module's sign-off shows as its verify status on the phase card, not as a row.

" % ( + plural(n_harness, "HARNESS row"), ", ".join(x for x in ((("sign-offs of " + plural(len(signoff_mods), "module")) if n_signoff else ""), (plural(n_harness - n_signoff, "harness pass") if n_harness - n_signoff else "")) if x)) if n_harness else "")) if named_total: print("

Scripts named in the plan: %d, of which %d are renamed done- under mdlsource/.%s

" % ( named_total, named_done, @@ -661,7 +671,7 @@ if named_total: det = [] for p, out in zip(phases, rows_html): h = ['' % e(p["id"]), - '

Phase state: %s

' % (CLS.get(p["state"], "st-unknown"), e(p["state"]))] + '

Phase state: %s

' % (CLS.get(disp_rollup(p["steps"]), "st-unknown"), e(disp_rollup(p["steps"])))] if p.get("note"): h.append("

%s

" % e(p["note"])) if p["steps"]: diff --git a/project-bin/progress-report.py b/project-bin/progress-report.py index 015fa20b..8583b530 100755 --- a/project-bin/progress-report.py +++ b/project-bin/progress-report.py @@ -989,22 +989,22 @@ def _dep_scripts(x, by_n): todo.extend(r.get("dependsOn") or []) return names -def harness_verify(proj, x, by_n=None, log_rows=None, done_files=None): - """(cls, shown, flag) for a HARNESS row that runs verify-module.sh , None for any other - row. cls is what the row counts as; shown the State cell text; flag the drift line or ''.""" +def verify_evidence(proj, x, by_n=None, log_rows=None, done_files=None): + """The evidence for a HARNESS row that runs verify-module.sh , None for any other row: + {mod, state, cl, form, day, why, since, n, tot}. form: none (no summary), unclean, stale, owed + (clean and fresh, LOOK not discharged), verified. module_status reads it (the rule #501 put in harness_verify).""" m = VERIFY_STEP.search(x.get("step") or "") if (x.get("kind") or "").strip().upper() == "HARNESS" else None if not m: return None mod, state = m.group(1), x.get("state") or "" - cl = classify_state(state); raw = state.strip().lower().replace("**", "") + ev = {"mod": mod, "state": state, "cl": classify_state(state), "form": "none", "day": "", "why": "", "since": 0, "n": 0, "tot": 0} f = os.path.join(proj, ".claude", "loop", "verify", mod, "summary.tsv") try: rows = [l.split("\t") for l in open(f, encoding="utf-8", errors="replace").read().splitlines()[1:] if l.strip()] - t = os.path.getmtime(f); day = datetime.datetime.fromtimestamp(t).strftime("%Y-%m-%d") + t = os.path.getmtime(f); ev["day"] = datetime.datetime.fromtimestamp(t).strftime("%Y-%m-%d") except OSError: - return ("built", "built? no verify summary", "State says built, but .claude/loop/verify/%s/summary.tsv does not exist" % mod) if cl == "built" else None - why = _summary_state(rows) - if why: - return (cl, state, "State says built, but the %s verify summary of %s is not clean (%s)" % (mod, day, why)) if cl == "built" else None + return ev + ev["why"] = _summary_state(rows) + if ev["why"]: ev["form"] = "unclean"; return ev names = _dep_scripts(x, by_n or {}) match = lambda s_: any(fnmatch.fnmatchcase(os.path.basename(s_), n) or fnmatch.fnmatchcase(os.path.basename(s_), "done-" + n) for n in names) if log_rows is None: log_rows = build_log_rows(proj) @@ -1014,15 +1014,35 @@ def harness_verify(proj, x, by_n=None, log_rows=None, done_files=None): done_files = _EV_CACHE[("done", proj)] since = [r for r in log_rows if r[0] > t and match(r[1])] if names else [] newer = [d_ for d_ in done_files if names and match(d_) and os.path.getmtime(d_) > t] - if since or newer: - return cl, state, "verify stale (summary %s, %d execs since)" % (day, len(since)) - n, tot, lwhy = look_evidence(proj, mod) - if lwhy: - return cl, state, "verify clean %s — LOOK + CONFIRM owed (%s)" % (day, lwhy) - shown = "built (verified %s, LOOK %d/%d)" % (day, n, tot) - if cl == "built": return "built", shown, "" - if raw.startswith(("blocked", "pending a person")): return cl, state, "verify clean %s and LOOK %d/%d, but State says %s" % (day, n, tot, state) - return "built", shown, "State says %s, but the %s summary of %s is clean and fresh and LOOK is %d/%d: update State" % (state or "nothing", mod, day, n, tot) + if since or newer: ev["form"] = "stale"; ev["since"] = len(since); return ev + ev["n"], ev["tot"], ev["why"] = look_evidence(proj, mod) + ev["form"] = "owed" if ev["why"] else "verified" + return ev + +# A module's sign-off status (MendixMau decision 2026-10-21): the HARNESS verify row is shown once, +# as its module's status on the phase card, never as a build row. Same evidence, nothing measured. +SIGNOFF_CLS = {"verified": "st-built", "owed": "st-progress", "stale": "st-progress", "unclean": "st-todo", "none": "st-todo", "claimed": "drift", "held": "st-person"} +def module_status(ev): + """(form, label, detail) for the module of a verify_evidence result. detail is the evidence + or the State disagreement, '' when there is nothing to add.""" + cl, day, st = ev["cl"], ev["day"], ev["state"] or "nothing" + raw = (ev["state"] or "").strip().lower().replace("**", "") + if ev["form"] == "none": + if cl == "built": return "claimed", "built? no verify summary", "State says built, but .claude/loop/verify/%s/summary.tsv does not exist" % ev["mod"] + return "none", "Verify not run", "" + if ev["form"] == "unclean": + return "unclean", "Verify not clean %s (%s)" % (day, ev["why"]), ("State says built, but the summary is not clean" if cl == "built" else "") + if ev["form"] == "stale": + lbl = ("Verify stale (%d build%s since)" % (ev["since"], "" if ev["since"] == 1 else "s")) if ev["since"] else "Verify stale (a script was marked done- after it)" + return "stale", lbl, "summary %s is older than a script this row depends on" % day + if ev["form"] == "owed": + return "owed", "Verify owed - LOOK + CONFIRM", "verify clean %s; %s" % (day, ev["why"]) + lbl = "Verified \u2713 %s (LOOK %d/%d)" % (day, ev["n"], ev["tot"]) + if cl == "built": return "verified", lbl, "" + # #501 never promoted a row a person holds: blocked / pending a person is not a sign-off + if raw.startswith(("blocked", "pending a person")): + return "held", "Verify clean %s, LOOK %d/%d - held: State says %s" % (day, ev["n"], ev["tot"], ev["state"]), "" + return "verified", lbl, "State says %s: update State" % st def _cells(line): c = re.split(r"(?%s%s%s%s%s" % (esc(p["phase"]), esc(p["start"]), esc(p["min_txt"] or "-"), esc(p["tok_txt"] or "-"), esc(p["runs"] or "-")) for p in lg["phases"]) + "") w("") +def signoff_html(p): + """The phase card's module sign-off line(s): one per module whose last verify row is here.""" + out = [] + for m in p.get("signoffs") or []: + drift = m["form"] == "claimed" or m["detail"].startswith("State says") + out.append("
Module sign-off, %s (row %s): %s%s
" % ( + esc(m["mod"]), esc(m["mod"]), esc(m["n"]), SIGNOFF_CLS.get(m["form"], "st-todo"), esc(m["label"]), + ("
⚠ %s
" if drift else " (%s)") % esc(m["detail"]) if m["detail"] else "")) + return "".join(out) + +def signoff_suffix(bp): + """', k of n modules signed off' after a build count, '' when the plan has no verify row. Held is not signed off.""" + so = (bp or {}).get("signoffs") or [] + return ", %d of %d module%s signed off" % (sum(1 for m in so if m["form"] == "verified"), len(so), "" if len(so) == 1 else "s") if so else "" + +def phase_bar(p, runs): + """The phase card's bar and count; a phase of HARNESS rows only has no build rows to count.""" + if not p["n"] and p.get("harness"): return "
Sign-off phase: no build rows
" + return "
%d of %d rows built%s%s
" % ( + pct(p["built"], p["n"]), pct(p["partial"], p["n"]), p["built"], p["n"], (", %d partial" % p["partial"]) if p["partial"] else "", (" · %d runs" % runs) if runs else "") + +def signoff_rollup(bp): + """One line under the plan counts: module sign-offs and other HARNESS rows, never counted as built rows.""" + so = bp.get("signoffs") or [] + other = sum(1 for p in bp["phases"] for x in p["steps"] if x["cls"] == "harness") + if not so and not other: return "" + v = sum(1 for m in so if m["form"] == "verified") + txt = ("Module sign-offs: %d of %d module%s verified, each shown on its phase card." % (v, len(so), "" if len(so) == 1 else "s")) if so else "" + if other: txt += (" " if txt else "") + "%d other HARNESS row%s listed in %s phase, not counted." % (other, "" if other == 1 else "s", "its" if other == 1 else "their") + return "
%s HARNESS rows are a module's sign-off or a harness pass, not build rows, so the counts above leave them out.
" % txt + def render(proj, info, gate, manifest, arts, main, agents, rates, bp, an, adv, tot, peak, args): L = args.has_log facts = exec_facts(proj, bp) @@ -1613,7 +1683,7 @@ def href(f): return urllib.parse.quote(os.path.relpath(f, args.outdir).replace(o elif states[s] == "progress": prog[s] = min(pct(have, n), 95.0) if n else 0.0 else: prog[s] = 0.0 owed_txt[s] = "%d of %d promised artifacts present" % (have, n) if n else "no manifest rows" - if s == "5" and bp and bp["total"] and states[s] not in ("passed", "waived"): owed_txt[s] = "%d of %d plan rows built%s" % (bp["built"], bp["total"], (", %d partial" % bp["partial"]) if bp["partial"] else "") + if s == "5" and bp and bp["total"] and states[s] not in ("passed", "waived"): owed_txt[s] = "%d of %d plan rows built%s%s" % (bp["built"], bp["total"], (", %d partial" % bp["partial"]) if bp["partial"] else "", signoff_suffix(bp)) overall = sum(prog.values()) / len(STAGES) npass = sum(1 for s in STAGES if states[s] == "passed") dec = {s: collections.Counter(d[2] for d in info["decisions"] if d[0] == s) for s in STAGES} @@ -1666,7 +1736,7 @@ def money(c): return "n/a" if unp else "$%.0f" % c cwin, cnote = context_window(main["msgs"], main["compacts"], main.get("flag_1m", False)) pw = lambda a_: ctx_pct_txt(a_, cwin) nart = sum(len(arts[s_]["files"]) for s_ in STAGES); ndec = len(info["decisions"]) - kp = [("Gates passed", "%d of 8" % npass, "stages P to 6, from PROJECT.md"), ("Plan rows built", ("%d of %d" % (bp["built"], bp["total"])) if bp and bp["total"] else "none yet", "from architecture/build-plan.md"), + kp = [("Gates passed", "%d of 8" % npass, "stages P to 6, from PROJECT.md"), ("Plan rows built", ("%d of %d" % (bp["built"], bp["total"])) if bp and bp["total"] else "none yet", "from architecture/build-plan.md" + signoff_suffix(bp)), ("Artifacts present", "%d" % nart, "files found for the eight stages"), ("Decisions", "%d" % ndec, "rows in the PROJECT.md register")] if not L else [("Wall time", fmt_dur(wall), "first to last event in the session"), ("Waiting for you", fmt_dur(idle_s), "gaps over %d min" % (IDLE_GAP_S // 60)), ("New tokens", fmt_tok(head(tot)), "input + output + cache write"), ("Cache reads", fmt_tok(tot["cr"]), "billed at a fraction"), ("Est. cost", money(cost_all), cost_note), ("Sub-agents", "%d" % nsub, "%d of them in %d workflow run(s)" % (nwf, len({a["wf_id"] for a in agents if a["wf"]}))), @@ -1724,7 +1794,7 @@ def money(c): return "n/a" if unp else "$%.0f" % c w("
New tokens
%s · est. %s
" % (fmt_tok(head(tk)) if tk["n"] or head(tk) else "none", money(tk["cost"]))) w("
Main context
%s
" % ("avg %s, peak %s (%s of %s)" % (fmt_tok(sum(cl) / len(cl)), fmt_tok(max(cl)), pw(max(cl)), fmt_tok(cwin) if cwin else "unknown window") if cl else "no requests")) w("
Sub-agents
%d + %d workflow · %s tokens
" % (subn, wfn, fmt_tok(subh) if sd["agents"] else "0")) - if s == "5" and bp: w("
Build progress
%d of %d rows built%s
" % (bp["built"], bp["total"], (", %d partial" % bp["partial"]) if bp["partial"] else "")) + if s == "5" and bp: w("
Build progress
%d of %d rows built%s%s
" % (bp["built"], bp["total"], (", %d partial" % bp["partial"]) if bp["partial"] else "", signoff_suffix(bp))) if s != "pre": c = dec[s]; w("
Decisions
%d confirmed%d assumed%d raised
" % (c["CONFIRMED"], c["ASSUMED"], c["RAISED"])) if s == "2" and gate["extra"].get("raised"): w("
BRD questions
%d raised
" % gate["extra"]["raised"]) @@ -1863,15 +1933,15 @@ def chip(f, cls=""): if lb: w("
%s. Last build step: %s (%s), %s at %s.
" % (esc(facts_line(facts)), esc(lb[1][:90]), esc(re.sub(r"[^\w .,/()+-]", "", lb[2])[:40].strip() or "no result"), esc(ago(lb[0])), esc(datetime.datetime.fromtimestamp(lb[0]).strftime("%Y-%m-%d %H:%M")))) if facts and facts["score"]: w("
Latest model score %.1f (%s).
" % (facts["score"]["score"], esc(facts["score"]["script"][:40]))) if bp.get("note"): w("
%s.
" % esc(bp["note"])) - w("
Kinds are the plan's own Kind column; row state is the plan's State column read by prefix (built or done = built, partial or in progress = partial, anything else = not built); a HARNESS row running verify-module.sh <Module> is also read from that module's verify summary and LOOK report: built only when the summary is clean and newer than the scripts it depends on and the LOOK is N of N, flagged otherwise. Plan read %s from %s.
" % (esc(bp.get("generated") or "unknown"), "architecture/build-plan.md" if bp.get("source") == "md" else "architecture/build-plan.json")) + w("
Kinds are the plan's own Kind column; row state is the plan's State column read by prefix (built or done = built, partial or in progress = partial, anything else = not built); HARNESS rows are not build rows and are not counted. A HARNESS row running verify-module.sh <Module> is that module's sign-off and shows on its phase card as one status, read from the module's verify summary and LOOK report: verified only when the summary is clean and newer than the scripts it depends on and the LOOK is N of N. The gate's own record of the look is the look obligation, in the obligations table of gate-check.sh <project> 5 (index.html). Plan read %s from %s.
%s" % (esc(bp.get("generated") or "unknown"), "architecture/build-plan.md" if bp.get("source") == "md" else "architecture/build-plan.json", signoff_rollup(bp))) w("
") for p in bp["phases"]: now = p is cur - w("
Phase %s: %s%s
%d of %d rows built%s%s
%s
module files: %s
Click to show or hide the rows
" % - (" now" if now else "", esc(p["id"]), " open" if now else "", esc(p["id"]), esc(p["name"]), " building now" if now else "", pct(p["built"], p["n"]), pct(p["partial"], p["n"]), p["built"], p["n"], (", %d partial" % p["partial"]) if p["partial"] else "", (" · %d runs" % runs_by.get(str(p["id"]), 0)) if runs_by.get(str(p["id"])) else "", esc(" ".join("%s %d" % kv for kv in p["kinds"].items())), - esc(" ".join(("%s %s" % (n, "yes" if v else "no")) for n, v in (("definition", p["def"]), ("brief", p["brief"]), ("ledger", p["ledger"])))))) + w("
Phase %s: %s%s%s
%s
module files: %s
%s
Click to show or hide the rows
" % + (" now" if now else "", esc(p["id"]), " open" if now else "", esc(p["id"]), esc(p["name"]), " building now" if now else "", phase_bar(p, runs_by.get(str(p["id"]))), esc(" ".join("%s %d" % kv for kv in p["kinds"].items())), + esc(" ".join(("%s %s" % (n, "yes" if v else "no")) for n, v in (("definition", p["def"]), ("brief", p["brief"]), ("ledger", p["ledger"])))), signoff_html(p))) if str(p["id"]) in phase_detail: w(phase_detail[str(p["id"])]) - else: w("
" + "".join("" % (x["cls"], esc(x["n"]), esc(x["kind"]), esc(x["step"]), esc(x["state"][:90])) for x in p["steps"]) + "
#KindRowState
%s%s%s%s
") + else: w("
" + "".join("" % (x["cls"], esc(x["n"]), esc(x["kind"]) + (" (not counted)" if x["cls"] == "harness" else ""), esc(x["step"]), esc(x["state"][:90])) for x in p["steps"]) + "
#KindRowState
%s%s%s%s
" if p["steps"] else "") w("
") w("
") else: w("
No plan to summarise yet: %s
" % esc(args.plan_msg or "architecture/build-plan.md was not read.")) diff --git a/tests/wave2/test-progress-harness-verify.sh b/tests/wave2/test-progress-harness-verify.sh index 1d655ab6..a6f0210d 100644 --- a/tests/wave2/test-progress-harness-verify.sh +++ b/tests/wave2/test-progress-harness-verify.sh @@ -1,21 +1,37 @@ #!/usr/bin/env bash -# test-progress-harness-verify.sh: a HARNESS row whose Step runs `verify-module.sh ` is read -# from evidence as well as from its State cell, in architecture/progress-report.html -# (project-bin/build-plan-status.sh --html, which embeds project-bin/progress-report.py). +# test-progress-harness-verify.sh: a HARNESS row whose Step runs `verify-module.sh ` is the +# module's sign-off. In architecture/progress-report.html (project-bin/build-plan-status.sh --html, +# which embeds project-bin/progress-report.py) it is read from evidence as well as from its State +# cell, and shown ONCE, as that module's status on the phase card, never as a build row. # # Review case (MendixMau, 2026-10-21): the agent ran `bin/verify-module.sh ` and never # edited State. The row's Pass (brd-to-build-plan.md, the module-close row) is a clean summary AND -# LOOK + CONFIRM, so the report promotes only on both, and flags everything short of that. +# LOOK + CONFIRM, so the status says Verified only on both, and names everything short of that. +# Decision (MendixMau, 2026-10-21): HARNESS rows are not counted as built rows, and the verify row +# is a module status, not a row of the list. build-plan.json keeps every row for the agents. # +# Every case runs on a single-tree and a two-tree checkout (.mpr under app/): # H1 clean + fresh summary and a LOOK report "2 of 2" with 2 PROOF-OF-LOOK screenshots: -# "built (verified 2026-10-20, LOOK 2/2)", flagged against State, counted (4 of 9 built) +# "Verified ✓ 2026-10-20 (LOOK 2/2)", flagged against State (not built) # H2 State built, no summary: "built? no verify summary" -# H3 a summary with a FINDING: not promoted -# H4 a --skip-journeys summary: not promoted +# H3 a summary with a FINDING: "Verify not clean", never Verified +# H4 a --skip-journeys summary: "Verify not clean", never Verified # H5 a clean summary older than a BUILD-LOG row of a script the row depends on: -# "verify stale (summary 2026-10-20, 1 execs since)", not promoted -# H6 clean and fresh, but the module's LOOK report says 1 of 2: "LOOK + CONFIRM owed", not promoted -# H7 H1 on a two-tree checkout (.mpr under app/) +# "Verify stale (1 build since)" +# H6 clean and fresh, but the module's LOOK report says 1 of 2: "Verify owed - LOOK + CONFIRM" +# (the page writes every em dash as " - ") +# H8 no summary, State not built: "Verify not run" +# H9 clean + fresh + LOOK 2/2 but State "pending a person": "held", never Verified, not counted +# H10 stale only by a done- file newer than the summary (no BUILD-LOG row): +# "Verify stale (a script was marked done- after it)", never "0 builds since" +# N1 counts leave HARNESS rows out: 4 of 5 rows built (plan card, lede, Phase 1 2 of 2) +# N2 one status per module, 9 in all, and none for Catalog, a module with no HARNESS row +# N3 no verify-module.sh row in the rendered row lists; the coherence HARNESS row stays, "(not counted)" +# N4 agent-facing output keeps the step: build-plan.json lists all 10 HARNESS rows with their Step, +# and its phase state still counts them (the JSON is not changed by the display rule) +# N5 the Phase state badge reads the build rows only: Phase 1 built, Phase 2 in progress, +# Phase 3 (HARNESS only) "sign-off only"; its card says "Sign-off phase: no build rows" +# N6 the Stage 5 counts name the sign-offs: ", 1 of 9 modules signed off" (Stage 5 card, KPI) # # Captured inputs (fixtures/verify-446/): the summaries were written by project-bin/verify-module.sh # itself (instruments stubbed, CAPTURE= bash test-verify-module-446.sh); BUILD-LOG.captured.md @@ -30,7 +46,7 @@ case "$SCRIPT" in /*) ;; *) SCRIPT="$PWD/$SCRIPT" ;; esac TK="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" FX="$TK/tests/wave2/fixtures/verify-446" WORK="$(mktemp -d "${TMPDIR:-/tmp}/progress-harness.XXXXXX")" -trap 'rm -rf "$WORK"' EXIT +trap '[ -n "${KEEP:-}" ] || rm -rf "$WORK"' EXIT export TZ=UTC # shellcheck disable=SC1091 . "$TK/bin/lib/portable.sh" @@ -71,40 +87,112 @@ mkproj() { # $1 root, $2 mpr relpath -> writes the page, sets H | 7 | HARNESS | `bin/verify-module.sh Invoices`, then LOOK + CONFIRM per `module-review.md` | summary has no FAULT; N of N pages reviewed | 6 | `module-review.md` | not built | | 8 | HARNESS | `bin/verify-module.sh Returns`, then LOOK + CONFIRM per `module-review.md` | summary has no FAULT; N of N pages reviewed | 1 | `module-review.md` | not built | | 9 | HARNESS | `process-coherence-pass.md` on Orders, Stock | chains hold | 2, 4 | `process-coherence-pass.md` | not built | +| 10 | HARNESS | `bin/verify-module.sh Payments`, then LOOK + CONFIRM per `module-review.md` | summary has no FAULT; N of N pages reviewed | 1 | `module-review.md` | not built | + +### Phase 2 - Catalog + +| # | Kind | Step | Produces / Proves | Depends on | Skills | State | +|---|---|---|---|---|---|---| +| 11 | BUILD | `11-catalog.mdl` | catalog entities | - | - | built | +| 12 | BUILD | `12-catalog-pages.mdl` | catalog pages | 11 | - | not built | +| 14 | BUILD | `14-pricing.mdl` | pricing entities | - | - | built | +| 15 | HARNESS | `bin/verify-module.sh Pricing`, then LOOK + CONFIRM per `module-review.md` | summary has no FAULT; N of N pages reviewed | 14 | `module-review.md` | not built | + +### Phase 3 - Close + +| # | Kind | Step | Produces / Proves | Depends on | Skills | State | +|---|---|---|---|---|---|---| +| 13 | HARNESS | `bin/verify-module.sh Holds`, then LOOK + CONFIRM per `module-review.md` | summary has no FAULT; N of N pages reviewed | 11 | `module-review.md` | pending a person | MD summary "$R" Orders clean; summary "$R" Stock finding; summary "$R" Shipping skip-journeys - summary "$R" Invoices clean; summary "$R" Returns clean + summary "$R" Invoices clean; summary "$R" Returns clean; summary "$R" Holds clean; summary "$R" Pricing clean + mkdir -p "$R/mdlsource"; : > "$R/mdlsource/done-14-pricing.mdl"; touch -t 202610201000 "$R/mdlsource/done-14-pricing.mdl" { sed -n '1,/^|---|/p' "$FX/BUILD-LOG.captured.md" logrow 2026-10-19T09:00:00+0000 01-orders.mdl logrow 2026-10-20T10:00:00+0000 05-invoices.mdl; } > "$R/docs/BUILD-LOG.md" local U="$R/design/ui-reviews" - shot "$U/shots/orders-a.png"; shot "$U/shots/orders-b.png"; shot "$U/shots/returns-a.png" + shot "$U/shots/orders-a.png"; shot "$U/shots/orders-b.png"; shot "$U/shots/returns-a.png"; shot "$U/shots/holds-a.png" + printf '

1 of 1 pages reviewed: Holds. VALID AT: 1234567

\n

PROOF-OF-LOOK: Holds.Hold_List = shots/holds-a.png

\n' > "$U/ui-review-2026-10-19-holds.html" printf '

2 of 2 pages reviewed: Orders. VALID AT: 1234567

\n

PROOF-OF-LOOK: Orders.Order_List = shots/orders-a.png

\n

PROOF-OF-LOOK: Orders.Order_Edit = shots/orders-b.png

\n' > "$U/ui-review-2026-10-20.html" printf '

1 of 2 pages reviewed: Returns. VALID AT: 1234567

\n

PROOF-OF-LOOK: Returns.Return_List = shots/returns-a.png

\n' > "$U/ui-review-2026-10-20-returns.html" - CLAUDE_CONFIG_DIR="$EMPTY" bash "$SCRIPT" "$R" --html --quiet >/dev/null 2>&1 + CLAUDE_CONFIG_DIR="$EMPTY" bash "$SCRIPT" "$R" --html --json --quiet >/dev/null 2>&1 H="$(cat "$R/architecture/progress-report.html" 2>/dev/null)" } -cell() { printf '%s' "$H" | grep -o "$1HARNESS.*" | head -1; } -nopromo() { ! printf '%s' "$1" | grep -q 'verified\|st-built'; } +# the module's sign-off line on its phase card (one div per module, data-module=) +status() { printf '%s' "$H" | grep -o "
.*" | sed 's#
Click.*##' | head -1; } +nstatus() { printf '%s' "$H" | grep -o "data-module='$1'" | wc -l | tr -d ' '; } +neverok() { ! printf '%s' "$1" | grep -q 'Verified\|st-built'; } +# the rendered row lists: the phase cards' rows +rowlist() { printf '%s' "$H" | sed -n '/
/,/<\/table>/p'; } for layout in single two; do R="$WORK/$layout"; [ "$layout" = single ] && mkproj "$R" App.mpr || mkproj "$R" app/App.mpr - c2="$(cell 2)" + L="($layout tree)" + s="$(status Orders)" id=H1; [ "$layout" = two ] && id=H7 - { printf '%s' "$c2" | grep -q 'built (verified 2026-10-20, LOOK 2/2)' && printf '%s' "$c2" | grep -q 'drift.*State says not built' \ - && printf '%s' "$H" | grep -q '4 of 9 rows built'; } \ - && ok "$id ($layout tree): clean + fresh + LOOK 2/2 promotes the close row, flagged, 4 of 9 built" \ - || bad "$id ($layout tree): row 2" "$c2 $(printf '%s' "$H" | grep -o '[0-9]* of 9 rows built' | head -1)" - [ "$layout" = single ] || continue - c3="$(cell 3)"; c4="$(cell 4)"; c5="$(cell 5)"; c7="$(cell 7)"; c8="$(cell 8)" - printf '%s' "$c3" | grep -q 'built? no verify summary' \ - && ok "H2: State built with no summary reads 'built? no verify summary'" || bad "H2: row 3" "$c3" - nopromo "$c4" && ok "H3: a FINDING summary does not promote" || bad "H3: row 4 promoted" "$c4" - nopromo "$c5" && ok "H4: a --skip-journeys summary does not promote" || bad "H4: row 5 promoted" "$c5" - { nopromo "$c7" && printf '%s' "$c7" | grep -q 'verify stale (summary 2026-10-20, 1 execs since)'; } \ - && ok "H5: a summary older than its dependency's exec reads 'verify stale', not promoted" || bad "H5: row 7" "$c7" - { nopromo "$c8" && printf '%s' "$c8" | grep -q 'verify clean 2026-10-20 .* LOOK + CONFIRM owed'; } \ - && ok "H6: clean and fresh, LOOK 1 of 2: 'LOOK + CONFIRM owed', not promoted" || bad "H6: row 8" "$c8" + { printf '%s' "$s" | grep -q 'Verified ✓ 2026-10-20 (LOOK 2/2)' && printf '%s' "$s" | grep -q 'drift.*State says not built: update State'; } \ + && ok "$id $L: clean + fresh + LOOK 2/2: Orders 'Verified ✓ 2026-10-20 (LOOK 2/2)', flagged against State" || bad "$id $L: Orders" "$s" + s="$(status Billing)" + printf '%s' "$s" | grep -q 'built? no verify summary' \ + && ok "H2 $L: State built with no summary: 'built? no verify summary'" || bad "H2 $L: Billing" "$s" + s="$(status Stock)" + { neverok "$s" && printf '%s' "$s" | grep -q 'Verify not clean 2026-10-20 (FINDING'; } \ + && ok "H3 $L: a FINDING summary: 'Verify not clean', never Verified" || bad "H3 $L: Stock" "$s" + s="$(status Shipping)" + { neverok "$s" && printf '%s' "$s" | grep -q 'Verify not clean 2026-10-20 (SKIPPED'; } \ + && ok "H4 $L: a --skip-journeys summary: 'Verify not clean', never Verified" || bad "H4 $L: Shipping" "$s" + s="$(status Invoices)" + { neverok "$s" && printf '%s' "$s" | grep -q 'Verify stale (1 build since)'; } \ + && ok "H5 $L: a summary older than its dependency's exec: 'Verify stale (1 build since)'" || bad "H5 $L: Invoices" "$s" + s="$(status Returns)" + { neverok "$s" && printf '%s' "$s" | grep -q 'Verify owed - LOOK + CONFIRM' && printf '%s' "$s" | grep -q 'reviewed 1 of 2'; } \ + && ok "H6 $L: clean and fresh, LOOK 1 of 2: 'Verify owed - LOOK + CONFIRM'" || bad "H6 $L: Returns" "$s" + s="$(status Payments)" + { neverok "$s" && printf '%s' "$s" | grep -q 'Verify not run'; } \ + && ok "H8 $L: no summary, State not built: 'Verify not run'" || bad "H8 $L: Payments" "$s" + s="$(status Holds)" + { neverok "$s" && printf '%s' "$s" | grep -q "Verify clean 2026-10-20, LOOK 1/1 - held: State says pending a person"; } \ + && ok "H9 $L: clean + LOOK 1/1, State pending a person: 'held', never Verified" || bad "H9 $L: Holds" "$s" + s="$(status Pricing)" + { neverok "$s" && printf '%s' "$s" | grep -q 'Verify stale (a script was marked done- after it)' && ! printf '%s' "$s" | grep -q '0 builds'; } \ + && ok "H10 $L: stale by a newer done- file only: 'a script was marked done- after it'" || bad "H10 $L: Pricing" "$s" + + # N1: 4 build rows (1, 6, 11, 12), 3 built; the 8 HARNESS rows are in no count + { printf '%s' "$H" | grep -q '4 of 5 rows built' && printf '%s' "$H" | grep -q '2 of 2 rows built' \ + && printf '%s' "$H" | grep -q '2 of 3 rows built' && printf '%s' "$H" | grep -q '5 build rows: 4 built' \ + && printf '%s' "$H" | grep -q 'Not counted: 10 HARNESS rows (sign-offs of 9 modules, 1 harness pass)' \ + && ! printf '%s' "$H" | grep -q 'of 15 rows built\|of 14 rows built\|0 of 0 rows built'; } \ + && ok "N1 $L: counts leave HARNESS out: 4 of 5 rows built, Phase 1 2 of 2, Phase 2 2 of 3, lede 5 build rows (sign-offs of 9 modules)" \ + || bad "N1 $L: counts" "$(printf '%s' "$H" | grep -o '[0-9]* of [0-9]* rows built\|[0-9]* build rows[^.]*\|Not counted[^.]*' | sort -u)" + # N2: one status per module, 7 in all, none for the module without a HARNESS row + n=0; for m in Orders Billing Stock Shipping Invoices Returns Payments Holds Pricing; do [ "$(nstatus "$m")" = 1 ] && n=$((n+1)); done + { [ "$n" = 9 ] && [ "$(nstatus Catalog)" = 0 ] && [ "$(printf '%s' "$H" | grep -o "class='k signoff'" | wc -l | tr -d ' ')" = 9 ] \ + && printf '%s' "$H" | grep -q 'Module sign-offs: 1 of 9 modules verified'; } \ + && ok "N2 $L: one sign-off status per module (9), none for Catalog (no HARNESS row), rollup 1 of 9 verified (held not counted)" \ + || bad "N2 $L: per-module status" "$n of 9 once; Catalog $(nstatus Catalog); $(printf '%s' "$H" | grep -o 'Module sign-offs[^.]*')" + # N3: no verify-module.sh row in the row lists; the harness pass stays, marked not counted + RL="$(rowlist)" + { [ -n "$RL" ] && ! printf '%s' "$RL" | grep -q 'verify-module' \ + && printf '%s' "$RL" | grep -q '' \ + && printf '%s' "$RL" | grep -q ''; } \ + && ok "N3 $L: no verify-module.sh row in the row lists; row 9 (coherence) listed, '(not counted)'" \ + || bad "N3 $L: row lists" "$(printf '%s' "$RL" | grep -o '
9HARNESS (not counted)11BUILD[0-9]*HARNESS[^/]*' | head)" + # N4: the agents' copy of the plan still carries every HARNESS row and its Step + J="$R/architecture/build-plan.json" + nj="$("$PY" -c 'import json,sys; d=json.load(open(sys.argv[1])); print(sum(1 for p in d["phases"] for x in p["steps"] if x["kind"]=="HARNESS" and ("verify-module.sh" in x["step"] or "coherence" in x["step"])), d["phases"][0]["state"])' "$J" 2>/dev/null)" + { [ "$nj" = "10 in progress" ] && grep -q 'bin/verify-module.sh Orders' "$J" && grep -q '| 2 | HARNESS | `bin/verify-module.sh Orders`' "$R/architecture/build-plan.md"; } \ + && ok "N4 $L: build-plan.json lists all 10 HARNESS rows with their Step, Phase 1 state still in progress; build-plan.md untouched" \ + || bad "N4 $L: agent-facing plan" "json HARNESS rows, phase 1 state: ${nj:-none}" + # N5: the badge reads build rows only; a HARNESS-only phase is a sign-off phase + bd="$(printf '%s' "$H" | grep -o 'Phase state: [^<]*' | sed 's/.*">//' | tr '\n' ',')" + { [ "$bd" = "built,in progress,sign-off only," ] && printf '%s' "$H" | grep -q 'Sign-off phase: no build rows'; } \ + && ok "N5 $L: Phase state badges built / in progress / sign-off only; Phase 3 card 'Sign-off phase: no build rows'" \ + || bad "N5 $L: badges" "$bd $(printf '%s' "$H" | grep -o 'Sign-off phase[^<]*')" + # N6: the Stage 5 counts say how many modules are signed off (held is not) + { printf '%s' "$H" | grep -q "4 of 5 rows built, 1 of 9 modules signed off" \ + && printf '%s' "$H" | grep -q "from architecture/build-plan.md, 1 of 9 modules signed off"; } \ + && ok "N6 $L: Stage 5 card and KPI: ', 1 of 9 modules signed off'" \ + || bad "N6 $L: sign-off suffix" "$(printf '%s' "$H" | grep -o '[^>]*modules signed off' | sort -u)" done echo "PASS=$PASS FAIL=$FAIL"; [ "$FAIL" -eq 0 ]