diff --git a/CHANGELOG.md b/CHANGELOG.md index 27fa96ab..e40cb54f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,29 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m ## Unreleased +- learn(iterative-build-loop): **a `LINT ROSE` exit 1 after a clean mxbuild is now a row in the refusal table (expected in a growing app, the write is kept so do not re-apply, only the person accepts a rise with `--reason`), and the 1.0 notes list it under known gaps; a softer exit is tracked in #443** — MendixMau +- fix(windows): **Git Bash MSYS paths no longer break `brd-report.sh` and the progress report's session lookup.** A native Windows Python cannot open `/c/work/proj`; `brd-report.sh` now converts the project dir and `--html` through a new `native_path` (`cygpath -m` on MINGW/MSYS, a no-op elsewhere), and `progress-report.py` turns an MSYS path into `C:/...`, builds the log-folder slug from the Windows path, reads `"cwd"` records written with escaped backslashes or forward slashes, and compares case-insensitively on Windows, so the page shows time, token and cost figures again. Test: `tests/wave2/test-windows-msys-paths.sh` (Fixes #460) — MendixMau +- fix(process-map): answered and consented questions no longer show as open; answers, consent, options and CONFIRMED register rows reach the map, the header counts only what is still open, an Open only filter is added and long decision titles are cut at a word (#458) — MendixMau + +- fix(doctor): **`--install` picks the native arm64 mxcli on Windows ARM64 (falling back to amd64 for a pinned release without it), and the docs say `--install --yes` when there is no terminal.** Git Bash there runs emulated so `uname -m` says x86_64 and doctor fetched the amd64 .exe; the arch now also reads the `uname -s` -ARM64 suffix and PROCESSOR_ARCHITEW6432/PROCESSOR_ARCHITECTURE. Agents following the bare `--install` examples got "nothing downloaded" (#459) — MendixMau +- fix(exec): **exec.sh restores the toolkit lint rules itself when mxcli's session-start sync has reverted them to stock, then lints, instead of ending the run BLIND.** mxcli's SessionStart hook runs `mxcli init --sync-skills`, which rewrites its bundled lint rules on every compaction and subagent start with no opt-out; a long Stage 5 lost lint each time until someone reran `sync-project.sh`. exec.sh now runs the installer's own classification (stock or receipt-matching copies restored, a hand edit never lost), re-runs lint once and says so on screen and in the BUILD-LOG row; `MXTK_NO_LINT_RESTORE=1` keeps the old BLIND behaviour. Test: `tests/wave2/test-exec-lint-autorestore.sh` (Fixes #461) — MendixMau +- fix(exec): **the gate keeps mxbuild's full output in a per-run log and names it on screen and in the BUILD-LOG row.** exec.sh printed `head -20` of the output and deleted the rest, so the evidence of a failed gate (a CE1613 detail among it) was lost in a field migration project (12 modules). A gate that does not pass now writes `.mpr-snapshots/.mxbuild-logs/.log` (newest 30 kept; a dot dir, so `restore-mpr.sh` and the snapshot pruning skip it; a clean or delta-gate pass writes none, and a retry always keeps the first attempt's output next to the second); the screen shows the top lines plus the path and the row ends with `mxbuild log: `. Test: `tests/wave2/test-exec-gate-log.sh` (Refs #442, #448, #171, #285, #424) — field migration project +- fix(exec): **the gate pauses this project's `mxcli run --local --watch` while mxbuild runs and retries once on an infra exit.** The gate's mxbuild and a live watcher both write `deployment/`; in a field migration project (12 modules) 17 of 27 failed gates were "gate could not run" (mxbuild exit 1, no CE code), against 0 of 22 in watcher-free copies. The watcher is stopped (SIGSTOP) for the gate and resumed (SIGCONT, also in the EXIT trap); an exit 1 or 3 with no CE code and no errors file is retried once and the BUILD-LOG row says `retry after infra exit`; a final failure still takes the #424 rollback. The match is exact (the watcher's project dir must equal this project's root, so a sibling `-copy` or a nested worktree watcher is never paused), and the paused pids are recorded in `.mpr-snapshots/.watcher-paused` so the next exec.sh resumes a watcher left stopped by a SIGKILLed one and says so. Test: `tests/wave2/test-exec-gate-retry.sh` (Refs #441, #444, #171, #285, #424) — field migration project +- learn(load-test): **trap 9, an unlicensed local runtime caps concurrent sessions at 6 ("Maximum number of sessions exceeded!") and stops itself after a random 2 to 4 hours, both reading as the app failing under load; more than 6 users needs a licensed runtime or a cloud sandbox.** Text only, ondemand skill. Source: a personal licence-budget gate written after parallel journey runs hit both — MendixMau +- fix(progress-report): **the full plan tables (one table per phase plus views A and B) now sit in one closed `
` card below the phase cards, so a large plan no longer pushes the rest of the report off screen; the phase cards above stay the summary, and nothing is written when the project has no plan fragment.** Test: P6 in `test-progress-report-polish.sh` — MendixMau +- fix(build-plan-status): **every State the build-plan skill lists (built, partial, not built, blocked) and every spelling the phase card already counts (complete, verified, in progress, not needed and the like) is now read the same way by the plan header and the phase badge, so card, header and badge give one count; blocked counts as not built and is named in the header, genuinely unreadable values are still reported as unreadable; the later-stage placeholder FAIL is relabelled PENDING only while that stage is pending in PROJECT.md, with tests that a real FAIL stays FAIL and that FAIL stays when the current stage is unknown** — MendixMau +- docs(release): **`docs/releases/v1.0.md` rewritten to the maintainer's structure: performance first (testing at key moments with the measured numbers), the settings a project owner controls, context packs, everything else, how to update.** Docs only; no routed or always-loaded file touched. — MendixMau +- fix(release-review): **the v1.0 release review: the multilingual skill loses its em and en dashes (wording only, content unchanged; its routing row keeps the trigger separator the agent stubs cut on), `doctor.sh` warns when a hook copy in `~/.claude/hooks` calls a toolkit script that no longer exists (it names the script and says to re-run `bin/install-claude-hooks.sh --full`; a warning, never a failure), and the `.gitignore` line for `.claude/progress-report-sessions.txt` now also reaches a project checked out as a git worktree (`.git` is a file there) in `doctor.sh` and `sync-project.sh`; `check-ignore --no-index` stops the line being appended again when the file is already tracked, and a `git rm --cached` hint is printed.** Tests: new `test-doctor-stale-hooks.sh`; worktree and tracked-file cases added to `test-progress-report-gitignore.sh`. CHANGELOG tidy (a stray blank line, one credit). — MendixMau +- fix(progress-report): **polish on a fresh and an updated demo project: a "done " plan row counts as built in the card, the plan header and the phase badge alike; a placeholder gate failure on a stage after the current one reads PENDING, not FAIL; the no-session-log notice is one calm line with the regenerate hint inside "How this was measured"; the test-moments table says unit tests run for the flows chosen at CAC-5, not "when switched on".** — field review of the #434 report +- fix(progress-report): **round 6 of the review: the session cache is machine-local and tied to its project, the first cached run is not slower than a cold one, and the build ledger decides the Stage 4 to 5 switch.** (1) `.claude/progress-report-sessions.txt` holds absolute session-log paths, so `init-project.sh`, `doctor.sh` and `sync-project.sh` now put `/.claude/progress-report-sessions.txt` in the project's `.gitignore`, the way they do for the doctor receipt and `toolkit.env`. (2) The cache header records the project's real path and the attribution rules (25% of tool calls, 20 calls); a copied or moved project, or a changed threshold, reads it as cold, so a copy never lists the original's sessions. (3) Files skipped by the 300-file cap are recorded as `no`, so a cached run never reads more files than a cold one. (4) With rows in `metrics-ledger.tsv` (else BUILD-LOG) the switch comes from the ledger; an earlier exec.sh call or build skill in a session log is ignored, the timed sign-off rule stays, and the footer says how the switch is chosen. Tests: F15 to F17 in `test-progress-report-field-fixes.sh`, new `test-progress-report-gitignore.sh` — review of PR #434 +- fix(progress-report): **round 5 of the field test: "building now" marks the phase of the last successful build step (its mdlsource folder, else the script name matched to the plan, else the highest active phase; blocked steps and earlier amber rows do not move it); discovery of sessions from other folders counts only tool calls that operate on the project (at least 25% of a session's calls and at least 20, gaps over 30 minutes cut, sessions that run progress-report.py excluded, text mentions never count) and lists the sessions used in a collapsed element; a Stage 4 sign-off row with a date but no time puts the Stage 5 switch at the first build activity on or after that date (marked inferred) instead of midnight; the exec log is now the primary source for build activity: `.claude/loop/metrics-ledger.tsv` (written by exec.sh, UTC shown as local time) gives the Stage 5 build facts with or without a session log (runs, applied, failed, blocked, restored, exec time, wall span, runs per phase, latest model score, share of exec time lost to failed or restored runs), the last build step, the building-now phase and the Stage 4 to 5 switch, with BUILD-LOG and then session activity as fallbacks; docs/progress/TIME-LEDGER.md is an optional hand-written extra and its reader maps columns by each table's header.** Tests F10 (existing-project fixture) and F11 (older toolkit, then sync-project.sh) added. — MendixMau +- fix(progress-report): **field-test round on two real projects: one source of truth for stage state, a plan read from build-plan.md when the JSON is stale, wider log discovery with a ledger fallback, and a Build progress section.** (1) PROJECT.md (the stage line and the Decisions sign-off rows) beats a gate-check page that is older than it: the stage line's `· manual: 5` tail no longer swallows the last passed stage, the counter always equals the list it sits beside ("6 of 8" for P,0,1,2,3,4, plus "N waived"), and a passed stage the page does not list shows an amber "Gate-check page is N days old, re-run gate-check". (2) The report never lists itself as a stage artifact, nor any file outside the project folder. (3) `build-plan.json` older than `build-plan.md` or than the newest `docs/BUILD-LOG.md` row is no longer trusted: row state is read from the md State column by prefix (`not built` first; `built (...)` and `done ...` are built; `partial` and `in progress` are partial; `not needed` rows are not counted) and the page says "plan JSON was stale, counted from build-plan.md". (4) With no log in the project's own folder, discovery also scans other `~/.claude/projects/*` folders (30 days, newest 300 files): a log counts when its cwd is the project or when the project path appears in it at least 5 times, in which case only the stretch from the first to the last mention (padded 30 min) is counted, and the page says so; with no log at all, `docs/progress/TIME-LEDGER.md`, `docs/BUILD-LOG.md` and `.claude/loop/metrics-ledger.tsv` feed a labelled ledger card, and the banner says how to pass `--session-log`. (5) The Stage 4 to 5 switch for time and tokens is the earliest evidenced of the Stage 4 sign-off row in PROJECT.md, the first `exec.sh` call and the build skill, so a stale gate page no longer puts all build work in Stage 4. (6) The build detail section is now "Build progress", linked from the Stage 5 card: stacked built / partial / not built bars per phase, the headline "N of M rows built, K partial", the phase being built now marked, and the last build step with how long ago it ran. Test: `tests/wave2/test-progress-report-field-fixes.sh` (39 checks). — MendixMau +- new(progress-report): **`architecture/progress-report.html` replaces `architecture/build-plan.html`, and adds time, token and tips views from the session log.** `build-plan-status.sh --html` now writes one self-contained page: every phase and the built versus proven views the old page had, stages P to 6 with artifact links, a progress bar and stage nav, and, when a Claude Code log for the project folder exists, time per stage, sub-agent use, context size, an estimated token cost (rates in an optional file, placeholder defaults, labelled as estimates), a Run health strip and a speed panel. Tips and tricks sit in a narrow side column (after the title, headline and stage cards on a phone), in one collapsed box per theme (Context, Tokens and cost, Speed, Health, Sub-agents), empty themes hidden. Only patterns with a concrete action in the project are shown, ranked by time or tokens lost, each with what was seen, why it matters and one fix. Severity is calm: red only for a key script (doctor, gate-check, exec, tests) failing every run with at least 3 runs, or a hang over 30 minutes; context is never red (a Context tip is amber when the window was reached or a compaction happened). Required gates and builds on the inherent list never produce a hang or slow tip and never change Run health; tips that lose under a minute are dropped; tips name real scripts or commands only. Costs that come from how the toolkit and mxcli work (build time, required gates and reviews, waiting on builds or the user, the mandated review model) are flagged inherent in the script and never shown. The context window is 1M only when the log records it (a `[1m]` model id or a context-1m beta); otherwise 200k, and a context above 200k with auto compactions caps the window at the compaction point and is labelled inferred. Never a percentage over 100. Every threshold sits in one block at the top of `project-bin/progress-report.py`, and the footer records the project folder and session log paths used. Only numbers, stage names, script names, file paths and sub-agent short descriptions are read from the log, never message text. Without a log the project sections render and the page says time and token data are unavailable. The page refreshes after every `exec.sh` slice and reloads itself every 30 s only while the build is active. PROJECT.md `Progress report: off | ask | auto` (default `ask`) controls it, and `--open` opens it. An existing `build-plan.html` is never deleted, and gate-check and status.sh still accept it. `--json` is unchanged. Skill words fell by about 100 (Step 10 and step 16 shortened). Tests: `tests/wave2/test-progress-report.sh`, `test-build-plan-json.sh`, `test-surface-column.sh`. — MendixMau +- fix(process-map): migrate no longer trusts a regex for the model key: it checks the top-level `anchors` changed and otherwise re-writes the model, and says so instead of claiming "only that key was changed"; a string `targets` in an old file counts as a one-item list instead of crashing; unfilled entries already in the current anchors file are listed, not dropped silently (follow-up to #438 review) — MendixMau +- new(brd-report): the Stage 2 report opens with the pass-1 process map (drawn by `process-map.js build --embed`, not a copy) and lists requirements and open questions under it, each tied to its step; a click on a step filters the list, what no step covers stays under "whole app". `open-questions.html` and `bin/questions-report.sh` are retired: the gate and the interview hook write `analysis/brd-report.html`, and an old project's `docs/open-questions.html` is never an error, only pointed away from. Stage 3: the prototype links to the full map and the map links back — a business owner could not tell where in the work a question fits (field run, 2026-10-16) — MendixMau +- learn(multilingual-app): **a second language is four decisions the translations never see — source language, enabled languages, the user's starting language, and a switch a normal role can use.** New `skills/learned-multilingual-app.md` (routed on demand, stages 3/5/6, architect, mdl + review). Field run 2026-10-09 (Mendix 11.15.0, mxcli v0.25.0, an app authored in Spanish gaining ES + PT): 418 strings per language written correctly, and still the language picker raised an error, showed a blank label and changed nothing, with check, exec.sh and lint green. Causes: captions authored under the default `en_US` (so `en_US` is the authored language); demo users on that hidden source language; the Marketplace LanguageSelector writing `System.User_Language` from the browser, which this project's non-admin role could not, with no grant route found (mxcli `check` passes the grant, `exec` refuses it; Studio Pro does not offer the member, so the first fix — a Studio Pro hand step — was impossible; a probe on the `System.User` module role now comes first); `login.html` outside the translation system. Recipe: a microflow sets `$currentUser`'s language, a nanoflow reloads, CSS highlights from ``; seeded users start in a visible language. `learned-mdl-cannot-express.md` gains the LanguageSelector row. — CarlesZ +- process(inbox): **five field notes from an existing-app legacy-widget rebuild** — exec.sh keeps an MDL write when mxbuild crashes on it and the next exec runs on "Baseline NOT measured"; a signal stop of `run --test-endpoint` leaves AfterStartupMicroflow on `MxTest.RegisterEndpoint` (second field case for #244); Unit-row delete refused as ORPHAN, rename-then-drop works; a durable-rebuild kit with a "restate from DESCRIBE" STOP row (DESCRIBE omits module-role and layout-row visibility: 97 role locks in 16 documents, none shown); learned entries for File Uploader image mode (CE0463), native-mapped `radiobuttons`/`image`, tab-page `CaptionParams`/`Visible:` silently dropped, BUG-39 on widget XPath, and two-hop data-source paths. Three further notes (restore-mpr usage, build-plan-status flag order, another project's mxcli test blocking exec.sh) were dropped because master already fixes them. Mendix 11.14.0 Beta, mxcli v0.25.0, toolkit c1efcb3 — AlvinnYeo +- fix(build-plan-status): --refresh keeps an existing build-plan.json current, so a day of building no longer reads as 0 built — MendixMau - learn(unit-test): the unit-test skill names the per-flow `Unit flows:` list next to `Unit modules:`, so whether a flow gets a test reads the same in both places (follow-up to #435) — MendixMau - learn(load-test): **new on-demand skill for load testing a page without correlating Mendix's client protocol by hand: Playwright records the real `/xas/` calls, a probe maps each opaque `runtimeOperation` to its microflow through Jaeger, and the chosen call is replayed with the session's own cookie and CSRF token.** Ten field traps (HTTP 200 on a failed action, caught REST errors under a green microflow span, the wrong call hammered, a recording that still held the previous page, a tracing check that failed open, a harness that throttled itself, harness vs app limits, requests vs users, sampled trace counts, empty data) and a four-line report that says what the run does not prove. Routed ondemand for lead, test and review; no instrument ships yet and the roadmap's "no load testing" line is reworded. Field evidence: a warehouse demo app, Mendix 11, 16 runs 2026-09-09 to 2026-10-08, bursts of 1000 and 2000 concurrent calls on one session — MendixMau - new(process-map): **Process map: one swimlane map per business process, each step on its wireframe once screens exist, every open question and ASSUMED decision pinned to the step it is about; drawn twice, a draft at Stage 2 close without screens (pass 1, shown at CAC-3) and the full map at Stage 3 with screens (pass 2, walked at the Stage 3 gate close-out).** `skills/process-map.md` (when, model, anchoring, build, verify by looking) and `project-bin/process-map.js` (`validate`, `anchor`, `build`; Node stdlib, deterministic) with its template. Named "map" because "process" in the progress report means the toolkit's own stages, while this shows the app's business process. With no wireframes, a step may name a screen that is not drawn yet or have none, and a person step stays a person step; once a wireframe exists `validate` fails a step page with no wireframe, a use case on no process and not standalone, a decision with one outcome, two steps in one cell. `anchor` places items mechanically (named microflow, listed or named use case, named screen, overlapping source lines) and keeps judged placements by fingerprint, so a reworded question is unplaced again instead of silently wrong; global buckets are allowed and counted. Wired into the Stage 3 Agent produces and Surface rows, a `process-map` opt-in manifest row (an existing model counts as opting in), the `swimlanes` opt-in row (the model discharges it), CAC-3 and CAC-4, and a check_stage_3 FAIL when the map is older than its model, its anchors or a wireframe. `tests/wave2/test-process-map.sh` on a synthetic order approval fixture with its own committed BRD, covering both passes. Field evidence: a requirements-driven rebuild POC, 2026-10-16, where the reviewer could not tell where the sixty questions fit; the map placed 102 items with 0 unplaced. — MendixMau diff --git a/README.md b/README.md index 59d5cbae..13ae7f8a 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,8 @@ git clone https://github.com/MendixMau/mxcli-project-toolkit.git ~/Mendix/mxcli- 1. **`doctor.sh`** names any missing prerequisite (bash, Python 3, sqlite3, line endings, mxcli, mxbuild) and exits non-zero if a stage would break. `doctor.sh --install ` fetches - mxcli and mxbuild after showing the plan and asking. Windows and Linux: read + mxcli and mxbuild after showing the plan and asking (an agent or any shell with no terminal + must add `--yes`, or nothing is downloaded). Windows and Linux: read [Platform support](#platform-support) first. 2. **`init-project.sh`** scaffolds the project folder: `intake.md`, `PROJECT.md`, `CLAUDE.local.md` (the wiring every session reads), the six agent stubs, `bin/` instruments @@ -251,7 +252,8 @@ they would have saved, restarted or reopened it. Everything either side of that > ``` > > `bin/doctor.sh --install ` closes the gap without Studio Pro: it fetches a -> version-matched mxbuild that `exec.sh` then finds on its own. +> version-matched mxbuild that `exec.sh` then finds on its own. With no terminal to answer the +> prompt (an agent, CI), add `--yes`: `bin/doctor.sh --install --yes `. **Existing clone from before 2026-08-18?** You predate `.gitattributes`, so if git checked the scripts out with CRLF they will not run. Fix once: @@ -605,7 +607,8 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini |---|---| | Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: 25-38% fewer tool calls and 30-41% less time, same quality; the token saving is not proven | `project-bin/context-pack.sh` | | An exec.sh guard refused a write (not a permission prompt), or building a module with mxcli — verified, iterative, coverage-checklist gated; on a refusal: commit, wait or stop, never FORCE_EXEC | `skills/iterative-build-loop.md` | -| After marking a module done, or any time "how much is built vs proven" is asked — renders build-plan.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file) | `project-bin/build-plan-status.sh` | +| Adding a second language, a language picker or a translation script — and the moment a language picker errors, shows a blank label or changes nothing: source language, enabled languages, the user's starting language, a switch a non-admin role can use | `skills/learned-multilingual-app.md` | +| After marking a module done, or any time "how much is built vs proven" is asked — renders progress-report.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file); --refresh keeps an existing build-plan.json current but never creates one | `project-bin/build-plan-status.sh` | | Turning a client-derived Mendix app into a clean, shareable demo with zero client fingerprint — branding, data, custom widgets | `skills/anonymize-client-app-for-demo.md` | | Handing a headless-built model to a person — opening it in Studio Pro, a free sandbox, or a colleague's machine: the model travels, the demo data and runtime config (keys, an agent's bound model) do not, and each needs its own re-establish step | `skills/handoff-to-studio-pro.md` | | Stage 5 start, before the first module of any entry mode — one entity, flow, page, nav, demo user, journey and screenshot proven in the running app, so build/run/look/test are known to work before a module depends on them | `skills/walking-skeleton.md` | diff --git a/ROUTING.md b/ROUTING.md index e2fe31e1..bb872da6 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -144,7 +144,8 @@ picks the row up. That is the whole procedure — there is no second list to rem | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | `bin/exec-approval.sh` | mdl,gate | 5,6 | baseline | | Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: 25-38% fewer tool calls and 30-41% less time, same quality; the token saving is not proven | `project-bin/context-pack.sh` | mdl | 5 | ondemand | | An exec.sh guard refused a write (not a permission prompt), or building a module with mxcli — verified, iterative, coverage-checklist gated; on a refusal: commit, wait or stop, never FORCE_EXEC | `skills/iterative-build-loop.md` | mdl,gate | 5 | ondemand | -| After marking a module done, or any time "how much is built vs proven" is asked — renders build-plan.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file) | `project-bin/build-plan-status.sh` | architect,gate,review | 4,5,6 | ondemand | +| Adding a second language, a language picker or a translation script — and the moment a language picker errors, shows a blank label or changes nothing: source language, enabled languages, the user's starting language, a switch a non-admin role can use | `skills/learned-multilingual-app.md` | architect,mdl,review | 3,5,6 | ondemand | +| After marking a module done, or any time "how much is built vs proven" is asked — renders progress-report.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file); --refresh keeps an existing build-plan.json current but never creates one | `project-bin/build-plan-status.sh` | architect,gate,review | 4,5,6 | ondemand | | Turning a client-derived Mendix app into a clean, shareable demo with zero client fingerprint — branding, data, custom widgets | `skills/anonymize-client-app-for-demo.md` | mdl,review | 6 | ondemand | | Handing a headless-built model to a person — opening it in Studio Pro, a free sandbox, or a colleague's machine: the model travels, the demo data and runtime config (keys, an agent's bound model) do not, and each needs its own re-establish step | `skills/handoff-to-studio-pro.md` | test,review | 6,7 | ondemand | | Sharing the project with a colleague on Mendix Team Server while the build loop stays on GitHub — content-transplant snapshots between the two clones (status/push/pull); SUPERSEDED by platform-link.md §3 (adopt without rewriting history, field-proven) — only if that path fails; UNPROVEN against a real Team Server | `project-bin/ts-sync.sh` | mdl,gate | 6,7 | experimental | diff --git a/agents/architect-agent.md b/agents/architect-agent.md index a651dda6..de060966 100644 --- a/agents/architect-agent.md +++ b/agents/architect-agent.md @@ -73,6 +73,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/brd-to-build-plan.md` | Turning BRDs plus architecture into a numbered, dependency-ordered build plan | | `skills/coverage-ledger.md` | Building the Stage 4 coverage ledger | | `bin/coverage-check.sh` | Checking a coverage ledger against its BRD | +| `skills/learned-multilingual-app.md` | Adding a second language, a language picker or a translation script | | `skills/mendix-agents.md` | Building a Mendix AI agent | | `skills/company-brain.md` | Setting up or wiring a COMPANY BRAIN | | `skills/close-the-loop.md` | Cutover and retrospective | diff --git a/agents/ba-agent.md b/agents/ba-agent.md index 6bf934d2..3febbc36 100644 --- a/agents/ba-agent.md +++ b/agents/ba-agent.md @@ -105,7 +105,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. - Run the full interview protocol from `conversion-runbook.md` §1 at every gate: homework first, 2-4 options with evidence, assumptions stated out loud, user answers in the terminal, decision written to the stage HTML *and* `PROJECT.md`, unknowns default + `ASSUMED` + proceed. - Never invent legacy intent from code alone — that's what Path C (the SME interview) is for. Log it as an open question if no SME is available, don't guess. - **Logging an open question is half the job. Raising it is the other half** — see the Open-questions batch below. A question written to a BRD and never put in the chat is not "logged", it is buried. -- **Read `interview-protocol.md` before raising anything.** Every question you hand over carries two named options and your recommendation; a bare question moves your analysis onto the user. Generate `docs/open-questions.html` with `bin/questions-report.sh` and give them the path. +- **Read `interview-protocol.md` before raising anything.** Every question you hand over carries two named options and your recommendation; a bare question moves your analysis onto the user. Generate `analysis/brd-report.html` with `bin/brd-report.sh` and give them the path. - `✋` gates (Stages 0 and 7) do not resolve to `ASSUMED` — they need an explicit `CONFIRMED` answer before the run proceeds. ## Workflow diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index ccdd7709..8d769e90 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -77,6 +77,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/mdl-cookbook-microflows.md` | Writing MDL microflow scripts | | `skills/build/mdl/oneshot-mdl-method.md` | Writing a single MDL script that takes a project from nothing to a working vertical slice | | `skills/learned-page-patterns.md` | Building and auditing Mendix pages, or starting a UI fix round across many pages | +| `skills/learned-multilingual-app.md` | Adding a second language, a language picker or a translation script | | `skills/oneshot-page-structure-patterns.md` | Generating a whole page tree in one script | | `skills/mendix-agents.md` | Building a Mendix AI agent | | `skills/mendix-agent-ui.md` | Embedding a copilot chat panel | diff --git a/agents/review-agent.md b/agents/review-agent.md index 21a2677c..27fbd5a4 100644 --- a/agents/review-agent.md +++ b/agents/review-agent.md @@ -65,6 +65,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/module-dependency-review.md` | Judging module tangles, bidirectional pairs, cohesion and hubs from analysis/app-facts/dependencies.json; stock graph_module_* views split names on the first dot and are not trusted | | `skills/microflow-loop-antipatterns.md` | Reading what loop bodies do (LOOP_TQ, deferred commit, nested loop, REST in loop, transaction control per item, scheduled-event reachability) from described MDL; the catalog holds top-level activities only and cannot see inside a loop | | `skills/process-map.md` | At Stage 2 close (draft, before the questions are raised) and again after the wireframes (Stage 3 re-render), when a reviewer cannot tell where the open questions and ASSUMED decisions fit: one swimlane map per process, each step on its wireframe, every question pinned to a step (project-bin/process-map.js validate, anchor, build) | +| `skills/learned-multilingual-app.md` | Adding a second language, a language picker or a translation script | | `project-bin/conformance-check.sh` | Running the ledger rung alone | | `project-bin/graph-sweep.sh` | Running the wiring rung alone | | `skills/fixture-seeding.md` | Seed data, test data, demo data or fixtures are needed, or pages come up empty | diff --git a/bin/brd-report.sh b/bin/brd-report.sh index 03e9040b..0ef928c9 100755 --- a/bin/brd-report.sh +++ b/bin/brd-report.sh @@ -40,7 +40,7 @@ # this instrument never judges whether the CONTENT is right, only whether it is there. # # Usage: -# brd-report.sh [--html PATH] [--json] [--quiet] +# brd-report.sh [--html PATH] [--stage N] [--json] [--quiet] # # exit 0 rendered (faults included — a fault is information, and the gate is gate-check's job) # 2 usage 3 no BRDs found (never a pass) 4 a BRD is not valid JSON @@ -60,12 +60,13 @@ HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" usage() { cat >&2 <<'USAGE' -usage: brd-report.sh [--html PATH] [--json] [--quiet] +usage: brd-report.sh [--html PATH] [--stage N] [--json] [--quiet] Renders the Stage 2 BRD surface from every BRD in the project, whatever produced them — a pipeline extractor, kb-generation.md, an SME interview, or a text editor. --html PATH where to write (default: /analysis/brd-report.html) + --stage N open questions up to stage N (the gate passes its own stage) --json machine-readable summary on stdout instead of prose --quiet render, say nothing @@ -80,11 +81,13 @@ shift 2>/dev/null || true case "$PROJECT_DIR" in -h|--help|'') usage ;; esac HTML_OUT="" +STAGE_ARG="" AS_JSON=0 QUIET=0 while [ $# -gt 0 ]; do case "$1" in --html) HTML_OUT="${2:-}"; shift 2 ;; + --stage) STAGE_ARG="${2:-}"; shift 2 ;; --json) AS_JSON=1; shift ;; --quiet) QUIET=1; shift ;; -h|--help) usage ;; @@ -93,6 +96,9 @@ while [ $# -gt 0 ]; do done [ -d "$PROJECT_DIR" ] || { echo "brd-report: not a directory: $PROJECT_DIR" >&2; exit 2; } +# Git Bash hands the interpreter /c/work/proj, which native Windows Python cannot open. +PROJECT_DIR="$(native_path "$PROJECT_DIR")" +[ -z "$HTML_OUT" ] || HTML_OUT="$(native_path "$HTML_OUT")" # Was: `command -v python3` in the scripts this one is modelled on — which succeeds on the # Windows Store alias stub. require_py probes by running the interpreter. @@ -101,7 +107,17 @@ require_py BRD_FILES=$(discover_brds "$PROJECT_DIR") BRD_COUNT=$(count_lines "$BRD_FILES") -if [ "$BRD_COUNT" = "0" ]; then +# Stage 0/1 has no BRDs yet, but the gate can still block on sme-questions.md questions and must +# hand the user a page. With no BRDs and no questions found, nothing is rendered (exit 3). +SME_ONLY=0 +if [ "$BRD_COUNT" = "0" ] && [ -x "$HERE/open-questions.sh" ]; then + _n=$(bash "$HERE/open-questions.sh" "$PROJECT_DIR" ${STAGE_ARG:+--stage "$STAGE_ARG"} --json 2>/dev/null | "$PY" -c 'import json,sys +try: print(json.load(sys.stdin)["counts"]["total"]) +except Exception: print(0)' 2>/dev/null) + [ "${_n:-0}" -gt 0 ] 2>/dev/null && SME_ONLY=1 +fi + +if [ "$BRD_COUNT" = "0" ] && [ "$SME_ONLY" = "0" ]; then # An empty result set is never a pass (report-schema.md). Distinct exit code so no caller # can read "found nothing" as "nothing to find". echo "brd-report: no BRDs found under $PROJECT_DIR — nothing rendered." >&2 @@ -114,7 +130,7 @@ fi mkdir -p "$(dirname "$HTML_OUT")" 2>/dev/null || true BRD_FILES="$BRD_FILES" BRD_COUNT="$BRD_COUNT" HTML_OUT="$HTML_OUT" AS_JSON="$AS_JSON" \ -QUIET="$QUIET" PROJECT_DIR="$PROJECT_DIR" "$PY" <<'PY' +QUIET="$QUIET" PROJECT_DIR="$PROJECT_DIR" TOOLKIT_BIN="$HERE" STAGE_ARG="$STAGE_ARG" "$PY" <<'PY' import json, os, sys, html, datetime files = [f for f in os.environ['BRD_FILES'].splitlines() if f.strip()] @@ -749,12 +765,8 @@ for a in analysed: note='

Rules found inside extracted logic that the domain model does ' 'not show. Derived from microflows[].hiddenRules.

'), section_block(a, 'openQuestions', 'Open questions', - '
    ' - + ''.join('
  • ' + esc(q.get('question') if isinstance(q, dict) else q) - + (f' {esc(q.get("status"))}' - if isinstance(q, dict) and q.get('status') else '') - + '
    • ' for q in a['raw']['openQuestions']) - + '
'), + '

Listed with their state, drafting position and process step in ' + 'Requirements and open questions at the top of this page.

'), ] chips = ''.join(f'{esc(m)}' for m in a['modules']) \ @@ -768,6 +780,331 @@ for a in analysed: f'

{esc(a["path"])}

' '') +# --- process map + open questions, fused (2026-10-16) ----------------------------------------- +# +# The page opens with the process map (pass 1: no screens needed) and lists, under it, every +# requirement (use case) and every open question or assumption. A click on a step filters that +# list to the rows about the step. Rows tied to no step stay listed under "whole app". +# +# The map is NOT drawn here: `project-bin/process-map.js build --embed` renders it (the one +# renderer, also used for the Stage 3 map beside the wireframes) and this page frames it. The +# open questions are NOT collected here either: `bin/open-questions.sh --json` is the one +# collector. This block only joins the two onto the BRDs. It replaces docs/open-questions.html. +import subprocess, shutil, tempfile, re + +toolkit_bin = os.environ['TOOLKIT_BIN'] +stage_arg = os.environ.get('STAGE_ARG', '') +proj_abs = os.path.abspath(project) + +def _find_model(): + cands = [os.path.join(proj_abs, 'design', 'process-model.json')] + adir = os.path.join(proj_abs, 'analysis') + if os.path.isdir(adir): + for d in sorted(os.listdir(adir)): + cands.append(os.path.join(adir, d, 'design', 'process-model.json')) + for c in cands: + if os.path.isfile(c): + return c + return None + +def _find_pm(): + for c in (os.path.join(toolkit_bin, '..', 'project-bin', 'process-map.js'), + os.path.join(proj_abs, 'bin', 'process-map.js')): + if os.path.isfile(c): + return os.path.normpath(c) + return None + +pm_data, pm_frame, pm_note = None, '', '' +_model, _pm, _node = _find_model(), _find_pm(), shutil.which('node') +if not _model: + pm_note = ('No process map yet. Draw it at Stage 2 close (skills/process-map.md, pass 1); ' + 'until then every requirement and question is listed under "whole app".') +elif not (_pm and _node): + pm_note = ('A process model exists (' + os.path.relpath(_model, proj_abs) + ') but the map ' + 'could not be drawn: ' + ('node is not on PATH' if not _node else 'process-map.js not found') + '.') +else: + _tmp = tempfile.mkdtemp(prefix='brdmap.') + try: + _r = subprocess.run([_node, _pm, 'build', '--embed', '--project', proj_abs, '--model', _model, + '--out', os.path.join(_tmp, 'embed.html'), + '--data-out', os.path.join(_tmp, 'data.json')], + capture_output=True, text=True, timeout=120) + if _r.returncode == 0 and os.path.isfile(os.path.join(_tmp, 'data.json')): + with open(os.path.join(_tmp, 'data.json')) as fh: + pm_data = json.load(fh) + with open(os.path.join(_tmp, 'embed.html')) as fh: + pm_frame = fh.read() + else: + _errs = [l for l in (_r.stdout + _r.stderr).splitlines() if l.startswith(('ERROR', 'process-map:'))] + pm_note = ('The process model (' + os.path.relpath(_model, proj_abs) + ') did not build: ' + + ' | '.join(_errs[:4]) + '. Fix the model (skills/process-map.md) and re-run.') + except Exception as e: + pm_note = 'The process map could not be drawn: ' + str(e) + finally: + shutil.rmtree(_tmp, ignore_errors=True) + +# open questions: the one collector +oq, oq_note = None, '' +try: + _c = subprocess.run(['bash', os.path.join(toolkit_bin, 'open-questions.sh'), project] + + (['--stage', stage_arg] if stage_arg else []) + ['--json'], + capture_output=True, text=True, timeout=120) + oq = json.loads(_c.stdout) +except Exception as e: + oq_note = ('The open-questions collector did not run (' + str(e)[:120] + '), so states are unknown. ' + 'The questions below are read straight from the BRDs.') + oq = None + +brd_of_base = {os.path.basename(a['path']): a['id'] for a in analysed} +BIG = 1 << 30 +step_ord, step_meta = {}, {} +if pm_data: + for _p in pm_data['processes']: + for _s in _p['steps']: + step_ord[_s['id']] = len(step_ord) + step_meta[_s['id']] = {'label': f"{_p['id']} step {_s['n']}: {_s['title']}", 'proc': _p['id'], 'n': _s['n']} +steps_of_ref = {} # "F001:Q01" -> {step ids}, from the map's items +if pm_data: + for _it in pm_data['items']: + for _ref in _it['refs']: + steps_of_ref.setdefault(_ref, set()).update(_it['steps']) + +def _order(steps): + return sorted(steps, key=lambda s: step_ord.get(s, BIG)) + +def _first(steps): + return min([step_ord.get(s, BIG) for s in steps] or [BIG]) + +def step_chips(steps): + if not steps: + return 'whole app' + return ''.join(f'{esc(step_meta[s]["proc"])} · {step_meta[s]["n"]}' + for s in _order(steps)) + +def _attr_steps(steps): + return esc(' '.join(_order(steps))) + +# rows: (..., steps, html). The second-last element is always the step list. +req_rows, q_rows = [], [] +for a in analysed: + for u in a['raw']['useCases']: + if not isinstance(u, dict) or not u.get('id'): + continue + ref = f"{a['id']}:{u['id']}" + info = (pm_data or {}).get('useCases', {}).get(ref) + steps = info['steps'] if info else [] + actors, _ = alist(u, 'actors', 'actor') + why = '' + if info and info.get('standalone'): + why = f'
Not part of a process: {esc(info["standalone"])}
' + actor_html = f'
{esc("; ".join(str(x) for x in actors))}
' if actors else '' + req_rows.append(('req', 'req', (_first(steps), ref), steps, + f'
' + f'
{esc(ref)} {esc(u.get("title"))}
' + + actor_html + why + f'
{step_chips(steps)} BRD detail
')) + +STATE_NOTE = { + 'UNRAISED': 'Never put to you. These block the gate.', + 'UNRECOGNISED': 'Status text no rule could map. Treated as unraised.', + 'RAISED': 'Put to you, awaiting your reply.', + 'ASSUMED': 'A position was taken and you consented to it. Recorded, not blocking.', + 'ANSWERED': 'Settled with a recorded answer.', + 'MOOT': 'Descoped, not silently dropped.', +} +STATE_ORDER = ['UNRECOGNISED', 'UNRAISED', 'RAISED', 'ASSUMED', 'ANSWERED', 'MOOT'] +BLOCKING = ('UNRAISED', 'UNRECOGNISED') + +def q_row(state, qid, origin, text, prop, steps, meta, rid): + cls = 'block' if state in BLOCKING else 'ok' + if prop: + prophtml = ('
Drafting position: confirm or overturn' + f'
{esc(prop)}
') + elif state in BLOCKING: + prophtml = ('
No proposed answer' + '
Handed over bare. The stage that raised it should propose two named options ' + 'and a recommendation before it goes to the user.
') + else: + prophtml = '' + meta_html = f'
{meta}
' if meta else '' + return ('q', state, (_first(steps), str(qid)), steps, + f'
' + f'
{esc(qid)} {esc(state)} ' + f'{esc(origin)}
{esc(text)}
{prophtml}{meta_html}' + f'
{step_chips(steps)}
') + +if oq is not None: + for q in oq.get('questions', []): + base = os.path.basename(str(q.get('origin', ''))) + bid = brd_of_base.get(base) + steps = sorted(steps_of_ref.get(f"{bid}:{q.get('id')}", [])) if bid else [] + meta = ' · '.join(x for x in [ + (f'recorded as {esc(str(q.get("rawStatus"))[:120])}' if q.get('rawStatus') else ''), + esc(q.get('reason') or ''), (f'stage {esc(q.get("stage"))}' if q.get('stage') is not None else '')] if x) + rid = f"q-{bid}-{q.get('id')}" if bid else 'q-' + re.sub('[^A-Za-z0-9_-]', '-', base) + '-' + str(q.get('id')) + q_rows.append(q_row(q.get('state', 'UNRECOGNISED'), q.get('id', '?'), q.get('origin', ''), q.get('question', ''), + (q.get('assumption') or '').strip(), steps, meta, rid)) +else: + for a in analysed: + for q in a['raw']['openQuestions']: + if not isinstance(q, dict): + continue + steps = sorted(steps_of_ref.get(f"{a['id']}:{q.get('id')}", [])) + q_rows.append(q_row('NOT COLLECTED', q.get('id', '?'), os.path.basename(a['path']), q.get('question', ''), + (q.get('assumption') or '').strip(), steps, '', f"q-{a['id']}-{q.get('id')}")) +for it in (pm_data or {}).get('items', []): + if it['kind'] != 'decision': + continue + steps = it['steps'] + q_rows.append(('q', 'ASSUMED', (_first(steps), it['id']), steps, + f'
' + f'
{esc(it["id"])} ASSUMPTION ' + f'decision register
{esc(it["text"])}
' + f'
{step_chips(steps)}
')) + +def _srank(r): + return STATE_ORDER.index(r[1]) if r[1] in STATE_ORDER else 9 + +def _pick(rows, whole): + sel = [r for r in rows if (not r[-2]) == whole] + if rows is q_rows: + sel.sort(key=(lambda r: (_srank(r), r[2])) if whole else (lambda r: (r[2], _srank(r)))) + else: + sel.sort(key=lambda r: r[2]) + return sel + +def _group(title, rows): + if not rows: + return '' + return (f'

{esc(title)} {len(rows)}

' + + ''.join(r[-1] for r in rows) + '
') + +n_blocking = (oq or {}).get('blocking', 0) +if oq is None: + q_banner = f'' +else: + _bare = [q for q in oq.get('questions', []) if q.get('state') in BLOCKING and not (q.get('assumption') or '').strip()] + q_banner = ((f'') + if n_blocking else '') + if _bare: + q_banner += (f'') + _cnt = oq.get('counts', {}) + _rows_t = ''.join(f'{s}{_cnt.get(s, 0)}' + f'{esc(STATE_NOTE[s])}' for s in STATE_ORDER if _cnt.get(s, 0)) + if _rows_t: + q_banner += ('
What was found, by state
' + '' + _rows_t + '
StateCountMeaning
') + +n_by_step = len([r for r in req_rows + q_rows if r[-2]]) +n_whole = len(req_rows) + len(q_rows) - n_by_step +by_step_html = '' +if n_by_step: + by_step_html = (f'

About the steps of the map ({n_by_step})

' + + _group('Requirements', _pick(req_rows, False)) + _group('Open questions and assumptions', _pick(q_rows, False))) +whole_html = _group('Requirements', _pick(req_rows, True)) + _group('Open questions and assumptions', _pick(q_rows, True)) + +# First paint height: the frame reports its real height once it has a width, but a screenshot or a slow +# frame must not show a clipped map in the meantime. About 105 px per lane row, plus legend and title. +pm_h0 = 70 + sum(120 + 105 * len({st.get('lane') for st in pr.get('steps', [])}) for pr in (pm_data or {}).get('processes', [])) if pm_frame else 120 +if pm_frame: + map_html = ('

Process map

' + '

One map per business process. Rows are the people and the system, cards are the steps in order. ' + 'Click a step to see the requirements and open questions about it below.

' + f'
') +else: + map_html = f'

Process map

' + +list_html = ('
' + '

Requirements and open questions

' + q_banner + by_step_html + + f'

Whole app ({n_whole})

' + '

Not tied to one step: they apply to the whole app or to one kind of data, or no map is drawn yet.

' + + (f'
Show the {n_whole} whole-app items{whole_html}
' if n_whole > 8 else whole_html) + + '
') + +PM_JS = """ +(function(){ +var META = %s; +var list = document.getElementById('pm-list'); if (!list) return; +var frame = document.getElementById('pm-frame'), bar = document.getElementById('pm-bar'), + txt = document.getElementById('pm-bar-text'), whole = document.getElementById('pm-whole'); +var current = ''; +function post(){ if (frame && frame.contentWindow) frame.contentWindow.postMessage({pm:'select', step:current}, '*'); } +function apply(id){ + current = id || ''; + var rows = list.querySelectorAll('.pmrow'), n = {req:0, q:0}; + Array.prototype.forEach.call(rows, function(r){ + var on = !current || (' ' + r.getAttribute('data-steps') + ' ').indexOf(' ' + current + ' ') >= 0; + r.hidden = !on; if (on && current) n[r.getAttribute('data-kind') === 'req' ? 'req' : 'q']++; + }); + Array.prototype.forEach.call(list.querySelectorAll('.pmgroup'), function(g){ + var v = g.querySelectorAll('.pmrow:not([hidden])').length; g.hidden = !!current && !v; + var c = g.querySelector('.cnt'); if (c) c.textContent = String(v); }); + Array.prototype.forEach.call(list.querySelectorAll('.pmh'), function(e){ e.hidden = !!current; }); + if (whole) whole.hidden = !!current; + if (bar){ bar.hidden = !current; if (current && META[current]) txt.textContent = META[current] + ': ' + n.req + ' requirement(s), ' + n.q + ' question(s) or assumption(s)'; } + post(); +} +function pick(id, scroll){ + apply(id); + try{ history.replaceState(null, '', id ? '#step=' + id : location.pathname + location.search); }catch(e){} + if (scroll) { var t = document.getElementById('pm-title'); if (t) t.scrollIntoView({behavior:'smooth', block:'start'}); } +} +window.addEventListener('message', function(e){ + if (!frame || e.source !== frame.contentWindow) return; + var d = e.data || {}; + if (d.pm === 'height') { if (d.h > 80) frame.style.height = d.h + 'px'; } + else if (d.pm === 'step') pick(d.step, true); + else if (d.pm === 'ready') post(); +}); +list.addEventListener('click', function(e){ + var a = e.target.closest ? e.target.closest('a.stepchip') : null; + if (a){ e.preventDefault(); pick(a.getAttribute('data-step'), false); var m = document.getElementById('pm-map'); if (m) m.scrollIntoView({behavior:'smooth', block:'start'}); } +}); +document.getElementById('pm-clear').addEventListener('click', function(){ pick('', false); }); +var m = /#step=([^&]+)/.exec(location.hash); if (m && META[decodeURIComponent(m[1])]) apply(decodeURIComponent(m[1])); +})(); +""" % json.dumps({k: v['label'] for k, v in step_meta.items()}) + +PM_CSS = """ + .pmmap iframe { width: 100%; border: 1px solid var(--line); border-radius: var(--radius); background: var(--surface); min-height: 120px; display: block; } + .banner { border-radius: var(--radius); padding: 11px 15px; margin: 12px 0; font-size: 14px; } + .banner.pass { background: var(--pass-bg); color: var(--pass-ink); } + .banner.fail { background: var(--fail-bg); color: var(--fail-ink); } + .banner.warn { background: var(--warn-bg); color: var(--warn-ink); } + .pmh { font-size: 15px; margin: 26px 0 4px; } + .pmgroup h4 { margin: 18px 0 8px; } + .pmrow { background: var(--surface); border: 1px solid var(--line); border-left: 3px solid var(--ink-faint); + border-radius: var(--radius); padding: 11px 14px; margin: 8px 0; overflow-wrap: anywhere; } + .pmrow[hidden], .pmgroup[hidden], .pmh[hidden], #pm-whole[hidden], .pmbar[hidden] { display: none; } + .pmrow.q.block { border-left-color: #dc2626; } + .pmrow.req { border-left-color: var(--accent); } + .pmhead { font-size: 14px; } + .pmsub, .pmrow .src { font-size: 12.5px; color: var(--ink-soft); margin-top: 3px; } + .qtext { font-size: 14.5px; margin-top: 4px; } + .prop, .gap { margin-top: 8px; padding: 8px 11px; border-radius: 8px; font-size: 13.5px; } + .prop { background: var(--warn-bg); color: var(--warn-ink); } + .gap { background: #f1f5f9; color: var(--ink-soft); } + .lbl { display: block; font-size: 11px; text-transform: uppercase; letter-spacing: .06em; font-weight: 700; margin-bottom: 2px; } + .st { font-size: 10.5px; font-weight: 700; letter-spacing: .05em; padding: 1px 8px; border-radius: 999px; } + .st-block { background: var(--fail-bg); color: var(--fail-ink); } + .st-ok { background: #eef2f7; color: var(--ink-soft); } + .pmchips { margin-top: 8px; display: flex; flex-wrap: wrap; gap: 6px; align-items: center; } + .stepchip { font-size: 11.5px; text-decoration: none; color: var(--accent); background: #eef4fb; border: 1px solid #cfe0f3; + border-radius: 999px; padding: 2px 10px; } + .stepchip:hover { background: #dde9f7; } + .whole { font-size: 11.5px; color: var(--ink-faint); border: 1px dashed var(--line); border-radius: 999px; padding: 2px 10px; } + .pmdetail { font-size: 11.5px; color: var(--ink-faint); margin-left: auto; } + .pmbar { position: sticky; top: 0; z-index: 5; display: flex; gap: 12px; align-items: center; justify-content: space-between; + background: var(--accent); color: #fff; border-radius: var(--radius); padding: 9px 14px; margin: 0 0 10px; font-size: 14px; } + .pmbar button { font: inherit; color: var(--accent); background: #fff; border: 0; border-radius: 6px; padding: 4px 12px; cursor: pointer; flex: none; } + details.more { margin: 8px 0; font-size: 13.5px; } + details.more summary { cursor: pointer; color: var(--ink-soft); } + @media (max-width: 560px) { body { padding: 20px 16px 60px; } .brd { padding: 16px 14px; } h1 { font-size: 23px; } } +""" + stat = lambda val, lbl: (f'
{esc(val)}
' f'
{esc(lbl)}
') @@ -882,15 +1219,17 @@ page = f""" .risk-medium {{ color: var(--warn-ink); font-weight: 700; }} .footer {{ margin-top: 56px; font-size: 12px; color: var(--ink-faint); border-top: 1px solid var(--line); padding-top: 14px; }} - +{PM_CSS}
-

Stage 2 — what the BRDs actually say

-

Rendered {esc(datetime.datetime.now().strftime('%Y-%m-%d'))} from - {len(analysed)} BRD(s) in {len(kb_dirs)} knowledge-base director(ies) — whatever produced - them. Every section carries a verdict, so a section that is empty because this kind of BRD - has nothing to put there reads differently from one that is empty because something broke.

+

{'Open questions, stage ' + stage_arg + ' view' if (not analysed and stage_arg) else ('Open questions' if not analysed else 'Stage 2 — what the BRDs actually say')}

+

{('Rendered ' + esc(datetime.datetime.now().strftime('%Y-%m-%d')) + '. No BRDs exist yet: these are the questions collected from analysis/sme-questions.md.') if not analysed else ('Rendered ' + esc(datetime.datetime.now().strftime('%Y-%m-%d')) + ' from ' + str(len(analysed)) + ' BRD(s) in ' + str(len(kb_dirs)) + ' knowledge-base director(ies), whatever produced them. Every section carries a verdict, so a section that is empty because this kind of BRD has nothing to put there reads differently from one that is empty because something broke.')} + {esc('Questions shown: up to stage ' + stage_arg + '.') if stage_arg else ''}

+ + {map_html} + + {list_html}
{stat(len(analysed), 'BRDs')} @@ -940,6 +1279,7 @@ page = f""" directory, not the first one.
+ """ diff --git a/bin/doctor.sh b/bin/doctor.sh index 2d0c8f7c..f97feb8c 100755 --- a/bin/doctor.sh +++ b/bin/doctor.sh @@ -57,7 +57,7 @@ # # --install never downloads silently: it prints the plan first — what, from where, how big, # why, and the detected OS/arch — then asks [y/N] at a terminal, or requires --yes when there -# is no terminal to ask on. $MXCLI_VERSION=vX.Y.Z pins the mxcli release; default is latest. +# is no terminal to ask on (agents and CI: always pass --yes). $MXCLI_VERSION=vX.Y.Z pins the mxcli release; default is latest. # # --install is the one deliberate exception to the no-dependency rule: it needs network access # (and curl or wget), runs only when asked, and a failed or declined download degrades to the @@ -533,6 +533,45 @@ fi fi +# --- installed Claude hooks: copies that call a toolkit script which is gone ------------------ +# +# install-claude-hooks.sh writes COPIES into ~/.claude/hooks with the toolkit path filled in. +# When a toolkit update removes a script (bin/questions-report.sh was retired in v1.0), an old +# copy keeps calling it: the hook still runs but silently loses its output. A warning only, +# never a failure: the hooks are user-global and optional. +if [ -d "$HOME/.claude/hooks" ]; then + head_ "Installed Claude hooks (~/.claude/hooks)" + STALE_HOOKS=0 + HOOKS_SEEN=0 + for hf in "$HOME"/.claude/hooks/*.sh; do + [ -f "$hf" ] || continue + HOOKS_SEEN=$((HOOKS_SEEN + 1)) + # The toolkit root the installer wrote, e.g. TOOLKIT="/path/to/toolkit". + HOOK_ROOT="$(sed -n 's/^[A-Z_]*TOOLKIT[A-Z_]*="\(\/[^"]*\)".*/\1/p' "$hf" | head -1)" + HOOK_REFS="" + if [ -n "$HOOK_ROOT" ]; then + HOOK_REFS="$(grep -oE '[$][{]?[A-Z_]*TOOLKIT[A-Z_]*[}]?/[A-Za-z0-9_./-]+[.](sh|py|js|tsv)' "$hf" 2>/dev/null \ + | sed -e 's/^[$][{]*[A-Z_]*[}]*//' -e "s|^|$HOOK_ROOT|" | sort -u)" + fi + HOOK_GONE="" + while IFS= read -r ref; do + [ -n "$ref" ] || continue + [ -e "$ref" ] || HOOK_GONE="$HOOK_GONE ${ref##*/}" + done <> "$PROJECT_DIR/.gitignore" 2>/dev/null || true note "(added /.claude/toolkit.env to the project's .gitignore — it holds this machine's paths)" fi + # The progress report's session cache holds absolute session-log paths (user name, session ids). + # -e, not -d: in a git worktree .git is a file. --no-index: a tracked file is never "ignored" otherwise, + # and the line would be appended again on every run. + if [ -e "$PROJECT_DIR/.git" ] && ! git -C "$PROJECT_DIR" check-ignore -q --no-index .claude/progress-report-sessions.txt 2>/dev/null; then + printf '\n# Machine-local progress report session cache (absolute session-log paths)\n/.claude/progress-report-sessions.txt\n' \ + >> "$PROJECT_DIR/.gitignore" 2>/dev/null || true + note "(added /.claude/progress-report-sessions.txt to the project's .gitignore — the cache is machine-local)" + fi + if [ -e "$PROJECT_DIR/.git" ] && git -C "$PROJECT_DIR" ls-files --error-unmatch .claude/progress-report-sessions.txt >/dev/null 2>&1; then + note "(.claude/progress-report-sessions.txt is already tracked by git; untrack it with: git rm --cached .claude/progress-report-sessions.txt)" + fi fi if [ "$FAIL" -gt 0 ]; then diff --git a/bin/extraction-report.sh b/bin/extraction-report.sh index 62ea64a0..3ea41215 100755 --- a/bin/extraction-report.sh +++ b/bin/extraction-report.sh @@ -107,6 +107,11 @@ done [ -d "$PROJECT_DIR" ] || { echo "extraction-report: not a directory: $PROJECT_DIR" >&2; exit 2; } +# Git Bash hands the interpreter /c/work/proj, which native Windows Python cannot open; the +# newline-joined KB list below is passed in an env var, where MSYS converts only the first line. +PROJECT_DIR="$(native_path "$PROJECT_DIR")" +[ -z "$HTML_OUT" ] || HTML_OUT="$(native_path "$HTML_OUT")" + # Was: `command -v python3` in the scripts this one is modelled on — which succeeds on the # Windows Store alias stub. require_py probes by running the interpreter. require_py diff --git a/bin/facts-lock.sh b/bin/facts-lock.sh index 434d8703..2f303451 100755 --- a/bin/facts-lock.sh +++ b/bin/facts-lock.sh @@ -59,6 +59,7 @@ case "$CMD" in build|check|show) ;; *) # shellcheck disable=SC1091 . "$(cd "$(dirname "$0")" && pwd)/lib/discover-brds.sh" +PROJECT_DIR="$(native_path "$PROJECT_DIR")" # newline-joined KBS goes to Python via env; MSYS converts only line 1 KBS=$(discover_brds "$PROJECT_DIR" | sed 's|/brd/[^/]*$||' | sort -u) [ -n "$KBS" ] || { echo "facts-lock: no /brd directory found under $PROJECT_DIR" >&2; exit 2; } diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 78a62a9a..9ec15e29 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -1732,6 +1732,9 @@ stage_surface_status() { # test-report.html is answered by the same function check_stage_6 uses (#233), so the # harness report pair also discharges it. if [ "$pat" = "test-report.html" ] && test_surface_hit; then continue; fi + # progress-report.html replaced build-plan.html; a project that still has the old page keeps + # its Surface green (the file is never deleted, so it must not read as missing). + if [ "$pat" = "progress-report.html" ] && [ -e "$PROJECT_DIR/architecture/build-plan.html" ]; then continue; fi # SURFACE_BASES, not just the project root. §2's Surface cells are inconsistent about # paths: some carry one ("architecture/blueprint.html", "docs/report.json") and some name # the file bare ("design-system.html", "build-plan.html", "test-report.html"). A bare cell @@ -1787,10 +1790,10 @@ stage_protocol_paths() { 0) echo "skills/interview-protocol.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-scope.md skills/source-triage.md skills/image-transcription.md skills/small-project-tier.md skills/existing-app-change.md skills/decompose-app.md skills/app-analysis.md skills/layering-review.md skills/module-dependency-review.md skills/microflow-loop-antipatterns.md skills/assess-migration.md skills/migration-pipeline.md skills/rehost-app.md skills/migrate-general.md skills/migrate-outsystems.md skills/source-os11.md skills/os-xml-schema.md skills/source-node-express-react.md skills/document-discovery.md skills/extractor-quality-loop.md skills/qa-loop-goal-pattern.md skills/mendix-epics-api.md skills/corpus-extraction-integrity.md skills/gate-check-file-locations.md" ;; 1) echo "skills/interview-protocol.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-extraction.md skills/dispatching.md skills/image-transcription.md skills/small-project-tier.md skills/migration-pipeline.md skills/rehost-app.md skills/source-os11.md skills/os-xml-schema.md skills/source-node-express-react.md skills/document-discovery.md skills/extractor-quality-loop.md skills/kb-generation.md skills/corpus-extraction-integrity.md" ;; 2) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-brd.md skills/checkpoints/checkpoint-architecture.md skills/dispatching.md skills/image-transcription.md skills/small-project-tier.md skills/kb-generation.md skills/brd-generation.md skills/brd-validation.md" ;; - 3) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-design.md skills/dispatching.md skills/small-project-tier.md skills/mendix-best-practices-index.md skills/decompose-app.md skills/mendix-service-seam.md skills/layering-review.md skills/architecture-blueprint.md skills/modularize-domain.md skills/design-artifacts.md skills/process-map.md skills/interaction-model.md skills/brd-to-build-plan.md skills/workflow-structure-rules.md skills/rest-between-mendix-apps.md skills/learned-mdl-cannot-express.md" ;; + 3) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-design.md skills/dispatching.md skills/small-project-tier.md skills/mendix-best-practices-index.md skills/decompose-app.md skills/mendix-service-seam.md skills/layering-review.md skills/architecture-blueprint.md skills/modularize-domain.md skills/design-artifacts.md skills/process-map.md skills/interaction-model.md skills/brd-to-build-plan.md skills/learned-multilingual-app.md skills/workflow-structure-rules.md skills/rest-between-mendix-apps.md skills/learned-mdl-cannot-express.md" ;; 4) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-build.md skills/agent-roles.md skills/dispatching.md skills/small-project-tier.md skills/module-brief.md skills/module-folder-convention.md skills/brd-to-build-plan.md skills/coverage-ledger.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/rest-between-mendix-apps.md skills/learned-constants-and-secrets.md" ;; - 5|build-ready) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/agent-roles.md skills/dispatching.md skills/module-brief.md skills/learned-mdl-preflight.md skills/module-folder-convention.md skills/learned-microflow-patterns.md skills/microflow-preflight.md skills/mendix-best-practices-index.md skills/ui-preflight-pages.md skills/design-spacing.md skills/learned-stylegallery.md skills/ui-loop.md skills/module-review.md skills/testing-shape.md skills/decompose-app.md skills/mendix-service-seam.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/iterative-build-loop.md skills/mdl-cookbook-microflows.md skills/build/mdl/oneshot-mdl-method.md skills/learned-page-patterns.md skills/oneshot-page-structure-patterns.md skills/mendix-agents.md skills/mendix-agent-ui.md skills/mendix-agent-setup.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/load-test.md skills/unit-test.md skills/e2e-locators-that-lie.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/learned-workflow-patterns.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/learned-sidebar-collapse-icons.md skills/learned-popup-navigation.md skills/learned-datagrid-customcontent-binding.md skills/learned-popup-feedback-pattern.md skills/learned-mdl-cannot-express.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-dg2-patterns.md skills/learned-file-upload-widget.md skills/security-is-not-a-later-script.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/preview-over-hub-tunnel.md skills/walking-skeleton.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; - 6) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/dispatching.md skills/mendix-best-practices-index.md skills/module-review.md skills/testing-shape.md skills/existing-app-assurance.md skills/decompose-app.md skills/app-analysis.md skills/module-dependency-review.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/qa-loop-goal-pattern.md skills/mendix-agent-setup.md skills/e2e-harness-base.md skills/learned-db-assertions.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/load-test.md skills/unit-test.md skills/e2e-locators-that-lie.md skills/learned-skill-ux-audit.md skills/learned-skill-scope-delta.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/e2e-evidence-report.md skills/record-demo-video.md skills/demo-recording-polish.md skills/share-demo-package.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/workflow-structure-rules.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/anonymize-client-app-for-demo.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/handoff-to-studio-pro.md skills/preview-over-hub-tunnel.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; + 5|build-ready) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/agent-roles.md skills/dispatching.md skills/module-brief.md skills/learned-mdl-preflight.md skills/module-folder-convention.md skills/learned-microflow-patterns.md skills/microflow-preflight.md skills/mendix-best-practices-index.md skills/ui-preflight-pages.md skills/design-spacing.md skills/learned-stylegallery.md skills/ui-loop.md skills/module-review.md skills/testing-shape.md skills/decompose-app.md skills/mendix-service-seam.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/iterative-build-loop.md skills/mdl-cookbook-microflows.md skills/build/mdl/oneshot-mdl-method.md skills/learned-page-patterns.md skills/learned-multilingual-app.md skills/oneshot-page-structure-patterns.md skills/mendix-agents.md skills/mendix-agent-ui.md skills/mendix-agent-setup.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/load-test.md skills/unit-test.md skills/e2e-locators-that-lie.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/learned-workflow-patterns.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/learned-sidebar-collapse-icons.md skills/learned-popup-navigation.md skills/learned-datagrid-customcontent-binding.md skills/learned-popup-feedback-pattern.md skills/learned-mdl-cannot-express.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-dg2-patterns.md skills/learned-file-upload-widget.md skills/security-is-not-a-later-script.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/preview-over-hub-tunnel.md skills/walking-skeleton.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; + 6) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/dispatching.md skills/mendix-best-practices-index.md skills/module-review.md skills/testing-shape.md skills/existing-app-assurance.md skills/decompose-app.md skills/app-analysis.md skills/module-dependency-review.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/qa-loop-goal-pattern.md skills/learned-multilingual-app.md skills/mendix-agent-setup.md skills/e2e-harness-base.md skills/learned-db-assertions.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/load-test.md skills/unit-test.md skills/e2e-locators-that-lie.md skills/learned-skill-ux-audit.md skills/learned-skill-scope-delta.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/e2e-evidence-report.md skills/record-demo-video.md skills/demo-recording-polish.md skills/share-demo-package.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/workflow-structure-rules.md skills/rest-between-mendix-apps.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/anonymize-client-app-for-demo.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/handoff-to-studio-pro.md skills/preview-over-hub-tunnel.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; 7) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/close-the-loop.md skills/share-demo-package.md skills/handoff-to-studio-pro.md skills/platform-link.md skills/teamserver-alignment.md skills/deploy-to-sandbox.md" ;; *) echo "" ;; # @@ -3318,14 +3321,15 @@ if [ -n "$REQUESTED_STAGE" ]; then echo "" >&2 echo "Gate BLOCKED: $blk question(s) at stage $st have never been raised with the user." >&2 echo " $counts" >&2 - # Write the triage report the user reads, alongside the batch the agent reads. - # A terminal dump scrolls away and only the agent ever sees it; the HTML is the artefact - # a human can open, sort by state, and hand to someone else. --no-html suppresses it for - # the same reason it suppresses the board: a read-only query must not dirty a tree. - QR_SCRIPT="$TOOLKIT_DIR/bin/questions-report.sh" - QR_OUT="$PROJECT_DIR/docs/open-questions.html" + # Write the report the user reads, alongside the batch the agent reads. A terminal dump + # scrolls away and only the agent ever sees it; the HTML is the artefact a human can open and + # hand to someone else. Since 2026-10-16 the questions live in the Stage 2 BRD report, under + # the process map (bin/brd-report.sh), not in docs/open-questions.html. --no-html suppresses + # it for the same reason it suppresses the board: a read-only query must not dirty a tree. + QR_SCRIPT="$TOOLKIT_DIR/bin/brd-report.sh" + QR_OUT="$PROJECT_DIR/analysis/brd-report.html" if [ "$HTML_MODE" != "never" ] && [ -x "$QR_SCRIPT" ]; then - "$QR_SCRIPT" "$PROJECT_DIR" $arg -o "$QR_OUT" --quiet >/dev/null 2>&1 + "$QR_SCRIPT" "$PROJECT_DIR" $arg --html "$QR_OUT" --quiet >/dev/null 2>&1 [ -f "$QR_OUT" ] || QR_OUT="" else QR_OUT="" @@ -3334,7 +3338,9 @@ if [ -n "$REQUESTED_STAGE" ]; then echo "" >&2 echo " See them, as a chat-ready batch:" >&2 echo " $OQ_SCRIPT $PROJECT_DIR${arg:+ $arg}" >&2 - [ -n "$QR_OUT" ] && echo " Triage report (open this, and give the user the path): $QR_OUT" >&2 + [ -n "$QR_OUT" ] && echo " BRD report, questions under the process map (open this, and give the user the path): $QR_OUT" >&2 + # Back-compat: the retired page is never read and never fails a gate; it only gets pointed away from. + [ -f "$PROJECT_DIR/docs/open-questions.html" ] && echo " (docs/open-questions.html is the retired page: its content is in analysis/brd-report.html now. Safe to delete.)" >&2 echo "" >&2 echo " Raising is mandatory; assuming is not forbidden. Anything the user hands back" >&2 echo " ('you decide') is recorded ASSUMED with consentBy/consentAt and stops blocking." >&2 diff --git a/bin/init-project.sh b/bin/init-project.sh index cabed48e..aed1111e 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -472,7 +472,7 @@ fi # analysis/ — THE OUTPUT SIDE, and it was never scaffolded (fixed 2026-08-20). # # Every rendered surface in the pipeline lands here: triage.html (bin/triage-report.sh), -# source-sufficiency.html, brd-report.html (bin/brd-report.sh), open-questions.html. gate-check.sh +# source-sufficiency.html, brd-report.html (bin/brd-report.sh; the open questions live in it). gate-check.sh # resolves $ANALYSIS_BASE against it. sources/ was created here from day one; analysis/ was not, # on the unstated assumption that an extraction pipeline would `mkdir -p` it on its way past. # @@ -504,8 +504,7 @@ and the honest way to change one is to re-run its producer. | `source-sufficiency.html` | `bin/source-sufficiency.sh report` | 0 | | `triage.html` | `bin/triage-report.sh ` | 0 | | `extraction-report.html` | the extraction pipeline, or `kb-generation.md` for a document corpus | 1 | -| `brd-report.html` | `bin/brd-report.sh ` | 2 | -| `open-questions.html` | `bin/questions-report.sh --stage N` | any gate | +| `brd-report.html` | `bin/brd-report.sh [--stage N]`: process map, requirements and open questions | 2, any gate | **This folder exists in every entry mode.** A requirements-driven or greenfield project runs no extraction pipeline, but it still produces a sufficiency grade, a scope decision and a BRD report @@ -614,6 +613,18 @@ else echo "Created/updated: .gitignore (.claude/settings.local.json ignored)" fi +# The progress report caches which session logs belong to this project: absolute paths with a +# user name and session ids. Machine-local, never tracked. +if [ -f "$GITIGNORE" ] && grep -qE '^/?\.claude/progress-report-sessions\.txt$' "$GITIGNORE"; then + echo "Skip: .claude/progress-report-sessions.txt already listed in .gitignore." +else + if [ -f "$GITIGNORE" ] && [ -s "$GITIGNORE" ] && [ -n "$(tail -c 1 "$GITIGNORE")" ]; then + printf '\n' >> "$GITIGNORE" + fi + printf '# Machine-local progress report session cache (absolute session-log paths) — never shared\n/.claude/progress-report-sessions.txt\n' >> "$GITIGNORE" + echo "Created/updated: .gitignore (.claude/progress-report-sessions.txt ignored)" +fi + # --------------------------------------------------------------------------- # Register this project's name with the toolkit's leak guard. # diff --git a/bin/install-claude-hooks.sh b/bin/install-claude-hooks.sh index 1dff2e00..758c7eac 100755 --- a/bin/install-claude-hooks.sh +++ b/bin/install-claude-hooks.sh @@ -68,7 +68,7 @@ TIER 2 --full Changes your working rhythm. Useful after a bad bill; precompact-guard Warns once before a compaction discards unfiled work. interview-gate BLOCKS the end of a turn while questions the pipeline generated have never been put to you, and writes - docs/open-questions.html for you to review. Only fires on + analysis/brd-report.html for you to review. Only fires on projects that actually have BRDs; never blocks the same question set twice, so it cannot trap a session. diff --git a/bin/lib/artifact-manifest.tsv b/bin/lib/artifact-manifest.tsv index 6808cadd..d4ab8b25 100644 --- a/bin/lib/artifact-manifest.tsv +++ b/bin/lib/artifact-manifest.tsv @@ -81,7 +81,7 @@ extraction-report 1 analysis/*/knowledge-base/extraction-report.html|analysis/kn source-ledger 1 analysis/source-ledger.html bin/source-ledger.sh report — every inventoried source file and what consumed it (Stage 1 owes a disposition per row; gate-check blocks 1–2 on the check) bin/gate-check.sh (LEDGER block — the check; the html is the human-facing table of the same rows) migration,requirements,existing-app-change report brds 2 analysis/*/knowledge-base/brd/*.brd.json|analysis/knowledge-base/brd/*.brd.json|knowledge-base/brd/*.brd.json pipeline generators (migration), skills/kb-generation.md (documents), skills/brd-generation.md (by hand) — the reader must not care which bin/lib/discover-brds.sh:43 (the one path authority), bin/gate-check.sh:773 (check_stage_2), bin/brd-report.sh, bin/open-questions.sh migration,requirements,existing-app-change gate brd-validation 2 analysis/*/knowledge-base/reports/validation-report.md|analysis/knowledge-base/reports/validation-report.md|knowledge-base/reports/validation-report.md skills/brd-validation.md loop, run to Clean bin/gate-check.sh:777-807 (check_stage_2 Stop-condition test) migration,requirements,existing-app-change gate -brd-surface 2 analysis/brd-report.html bin/brd-report.sh (default output, bin/brd-report.sh:113) conversion-runbook.md §2 Stage-2 Surface row, parsed at bin/gate-check.sh:1284 (stage_surface_cells) migration,requirements,existing-app-change report +brd-surface 2 analysis/brd-report.html bin/brd-report.sh (default output, bin/brd-report.sh:113); also holds the pass-1 process map and the open questions (replaced docs/open-questions.html, 2026-10-16) conversion-runbook.md §2 Stage-2 Surface row, parsed at bin/gate-check.sh:1284 (stage_surface_cells) migration,requirements,existing-app-change report module-design 3 architecture/module-design.html|module-design.html skills/modularize-domain.md:88 (the human-facing sign-off artifact) skills/modularize-domain.md:109 (gate checklist), conversion-runbook.md:478 Stage-3 Surface row (parsed at bin/gate-check.sh:1284) migration,requirements,existing-app-change report fit-gap 3 architecture/fit-gap.md skills/architecture-blueprint.md Step 4 bin/gate-check.sh:912 (check_stage_3), bin/gate-check.sh:939 (blueprint.html staleness input) migration,requirements,existing-app-change gate blueprint-md 3 architecture/blueprint.md skills/architecture-blueprint.md Steps 2-6 — the markdown is the source of truth bin/gate-check.sh:939 (staleness input), skills/architecture-blueprint.md:54 (render source for blueprint.html) migration,requirements,existing-app-change report @@ -91,8 +91,8 @@ ds-css 3 design/ds.css skills/design-artifacts.md:75 (the physical token/compone wireframes 3 design/wireframes/*.html skills/design-artifacts.md Step 3, one per screen bin/gate-check.sh:923 (check_stage_3), bin/gate-check.sh:1129 (build-ready), skills/ui-preflight-pages.md (mdl-agent pre-flight input) migration,requirements,existing-app-change gate module-definition 3 architecture/modules/*/definition.md skills/architecture-blueprint.md Step 1 (:60 — the directory shape, one doc per module) THIS ROW IS ITS FIRST MECHANICAL CONSUMER — as of 2026-08-22 nothing executable read definition.md (improvement-plan P7 residue, the newest instance of the pattern this table exists to stop); prose consumers: skills/architecture-blueprint.md:43, skills/coverage-ledger.md:27 migration,requirements,existing-app-change report build-plan 4 architecture/build-plan.md skills/brd-to-build-plan.md bin/gate-check.sh:955 (check_stage_4), project-bin/build-plan-status.sh (progress board), project-bin/coverage-preflight.sh:238 (the claims join) migration,requirements,existing-app-change gate -build-plan-html 4 architecture/build-plan.html project-bin/build-plan-status.sh --html (skills/brd-to-build-plan.md:47 — derived view, never hand-edited; refreshed by exec.sh and full gate-check.sh runs via --refresh) conversion-runbook.md §2 Stage-4 Surface row, parsed at bin/gate-check.sh:1284 migration,requirements,existing-app-change report -build-plan-json 4 architecture/build-plan.json project-bin/build-plan-status.sh --json (derived from build-plan.md's Step-5 Phase headings, row tables and claims: blocks, never hand-edited; not written when the markdown has no Phase headings) MXTK Studio Plan room (a view built from data, not from the HTML); nothing in the toolkit reads it and no verdict may depend on it migration,requirements,existing-app-change report +build-plan-html 4 architecture/progress-report.html project-bin/build-plan-status.sh --html (skills/brd-to-build-plan.md:47 — derived view, never hand-edited; refreshed by exec.sh and full gate-check.sh runs via --refresh) conversion-runbook.md §2 Stage-4 Surface row, parsed at bin/gate-check.sh:1284 migration,requirements,existing-app-change report +build-plan-json 4 architecture/build-plan.json project-bin/build-plan-status.sh --json (derived from build-plan.md's Step-5 Phase headings, row tables and claims: blocks, never hand-edited; not written when the markdown has no Phase headings; --refresh keeps an existing file current but never creates it) MXTK Studio Plan room (a view built from data, not from the HTML); nothing in the toolkit reads it and no verdict may depend on it migration,requirements,existing-app-change report module-brief 4 architecture/modules/*/module-brief.md|architecture/modules/*-brief.md skills/module-brief.md — ba-agent writes the first at Stage 4, the rest just-in-time bin/gate-check.sh:1121 (build-ready fails without one), project-bin/build-plan-status.sh:109 migration,requirements,existing-app-change gate coverage-ledger 4 architecture/modules/*/coverage-ledger.md|architecture/modules/*-coverage-ledger.md|architecture/coverage-ledger.md skills/coverage-ledger.md procedure at Stage 4 (BRD leaves × build-plan claims); the three shapes are §"Where the ledger lives", first canonical project-bin/conformance-check.sh:85-93, project-bin/coverage-preflight.sh:134-137, project-bin/verify-module.sh:299-300, project-bin/review-module.sh:289, conversion-runbook.md:487 (Stage-4 produces row) — the eleven-consumers-no-producer artifact of improvement-plan Finding 12 migration,requirements report ui-reviews 5 design/ui-reviews/ui-review-*.html skills/module-review.md pass (review-agent), one report per module close-out bin/gate-check.sh:984 (check_stage_6), bin/lib/obligations.tsv:47 (the look row — per-module coverage and denominators live THERE; this row only asks whether the stage produced any report at all) migration,requirements,greenfield,existing-app-change gate diff --git a/bin/lib/closeout.sh b/bin/lib/closeout.sh index f16e8110..5243c2a1 100755 --- a/bin/lib/closeout.sh +++ b/bin/lib/closeout.sh @@ -166,7 +166,7 @@ _co_plain_artifact() { wireframes) echo "one wireframe per screen" ;; module-definition) echo "one definition document per module" ;; build-plan) echo "the numbered build plan" ;; - build-plan-html) echo "the build progress board" ;; + build-plan-html) echo "the progress report" ;; module-brief) echo "the first module's brief (what the builder gets)" ;; coverage-ledger) echo "the ledger proving every requirement is claimed by a build step" ;; ui-reviews) echo "the page review report" ;; diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index 56e46064..5eb0928f 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -119,7 +119,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # the generator reads the template from its own directory. The project runs it at Stage 3 # (skills/process-map.md Step 4) after every change to design/process-model.json, the # anchors, a wireframe or a BRD, and gate-check.sh check_stage_3 fails a stale render. -MXTK_PROJECT_BIN="process-map.js process-map.template.html app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh" +MXTK_PROJECT_BIN="process-map.js process-map.template.html app-restart.sh test-scratch.sh assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py mxpatch.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh affected-tests.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh metrics-ledger.sh metrics-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh progress-report.py done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh mx-metadata-note.py session-check.sh constants-audit.sh context-pack.sh check-staged-secrets.sh" # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either diff --git a/bin/lib/portable.sh b/bin/lib/portable.sh index 3af07bf7..31d995b3 100644 --- a/bin/lib/portable.sh +++ b/bin/lib/portable.sh @@ -60,6 +60,20 @@ require_py() { export PY } +# --- native paths for a Windows interpreter ----------------------------------------------- +# +# Git Bash spells C:\work\proj as /c/work/proj. A native Windows Python cannot open that, and +# MSYS converts only the leading path of an env var, so a newline-joined path list (or a path read +# back from a pipe) is the risk: later entries stay /c/.... native_path +# echoes the mixed form (C:/work/proj, which bash and Windows both open) on MINGW/MSYS/Cygwin and +# the path unchanged everywhere else, or when cygpath is missing. +native_path() { + case "$(uname -s 2>/dev/null)" in + MINGW*|MSYS*|CYGWIN*) cygpath -m -- "$1" 2>/dev/null || printf '%s\n' "$1" ;; + *) printf '%s\n' "$1" ;; + esac +} + # --- stat(1) ------------------------------------------------------------------------------- # # `stat -f` is BSD, `stat -c` is GNU coreutils. Neither is a Windows/Linux/macOS common subset, diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index 27d8707a..6f2b3096 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -161,6 +161,7 @@ iterative-build-loop skills/iterative-build-loop.md An exec.sh guard refused a w mdl-cookbook-microflows skills/mdl-cookbook-microflows.md Writing MDL microflow scripts — worked recipes mdl 5 ondemand build/mdl oneshot-mdl-method skills/build/mdl/oneshot-mdl-method.md Writing a single MDL script that takes a project from nothing to a working vertical slice — execution order, why it is deliberately non-idempotent, the instrument hierarchy, and the silent failures that pass every check mdl 5 ondemand build/mdl learned-page-patterns skills/learned-page-patterns.md Building and auditing Mendix pages, or starting a UI fix round across many pages — widget patterns, datasource shapes; for a round: SCSS first, four regressions to check mdl 5 ondemand build/pages +learned-multilingual-app skills/learned-multilingual-app.md Adding a second language, a language picker or a translation script — and the moment a language picker errors, shows a blank label or changes nothing: source language, enabled languages, the user's starting language, a switch a non-admin role can use architect,mdl,review 3,5,6 ondemand build oneshot-page-structure-patterns skills/oneshot-page-structure-patterns.md Generating a whole page tree in one script — the structure patterns that survive it mdl 5 ondemand build/pages mendix-agents skills/mendix-agents.md Building a Mendix AI agent — the agent is runtime data not a model document, so JSON import, tool microflows, knowledge base chunk loading and the runtime wiring all sit outside MDL, and mxbuild stays green when they are wrong mdl,architect 5 ondemand build/agents mendix-agent-ui skills/mendix-agent-ui.md Embedding a copilot chat panel — the frame is yours, ConversationalUI owns the conversation; wireframe to tokens to snippet to page placement mdl 5 ondemand build/agents @@ -188,7 +189,7 @@ tool-output-is-not-ground-truth skills/tool-output-is-not-ground-truth.md Any ti agent-permission-friction skills/agent-permission-friction.md Any refused, denied or blocked command — BEFORE rewriting a permission rule and before telling the user a tool is blocked. A rule matches the START of the command line, so an allowlisted tool prefixed with cd matches nothing all - ondemand diagnose process-coherence-pass skills/process-coherence-pass.md Checking whether the whole journey hangs together rather than each piece — finds correctly-built components nothing reaches, which per-element conformance and UI tests both miss review 5,6 ondemand verify coherence-cadence project-bin/coherence-cadence.sh After every module's CONFIRM stage — counts proven modules since the last cluster/full coherence pass and exits DUE once the threshold is reached, so the cadence isn't left to memory mdl,gate,review 5,6 ondemand verify -build-plan-status project-bin/build-plan-status.sh After marking a module done, or any time "how much is built vs proven" is asked — renders build-plan.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file) architect,gate,review 4,5,6 ondemand build +build-plan-status project-bin/build-plan-status.sh After marking a module done, or any time "how much is built vs proven" is asked — renders progress-report.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file); --refresh keeps an existing build-plan.json current but never creates one architect,gate,review 4,5,6 ondemand build e2e-evidence-report skills/e2e-evidence-report.md Turning an already-rigorous run into a narrated proof a stakeholder can trust without running anything test,review 6 ondemand verify record-demo-video skills/record-demo-video.md Recording a narrated screen-capture demo of a running app for a human to watch — opening on the app instead of a blank frame, and keeping captions synced to the pixels test 6 ondemand verify demo-recording-polish skills/demo-recording-polish.md A demo film someone will watch looks captured, not produced — 25 fps smear, scroll jumps, no visible pointer, a dense page nobody can read: 2x CDP screencast at 60 fps, drawn cursor and click ripple, eased scroll, and a zoom rendered afterwards from the click log (never a CSS transform in the page) test 6 ondemand verify diff --git a/bin/questions-report.sh b/bin/questions-report.sh deleted file mode 100755 index 15cee9c2..00000000 --- a/bin/questions-report.sh +++ /dev/null @@ -1,299 +0,0 @@ -#!/bin/bash -# questions-report.sh — render a project's open questions as a reviewable HTML page. -# -# WHY THIS EXISTS -# --------------- -# bin/open-questions.sh already produces a chat-ready batch. That solves "the agent has the -# questions". It does not solve the two things the user actually reported missing: -# -# 1. There is no surface showing WHAT WAS FOUND — what got examined, what is still open, -# and which open items are blocking — that a human can read away from the terminal. -# 2. Most questions arrive with no proposed answer. A question with no proposed answer is -# work handed back to the user. A question with two named options and a recommendation -# is a decision they can make in five seconds. -# -# This script renders (1), and marks every question that lacks (2) as a defect in the -# generating stage rather than quietly presenting a bare question. -# -# Usage: -# bin/questions-report.sh [--stage N] [-o ] [--quiet] -# -# Default output: /docs/open-questions.html -# -# Exit codes: 0 = report written, nothing blocking. 1 = report written, blocking questions -# exist. 2 = usage/environment error. 3 = nothing was examined (NOT a clean -# result — same meaning as open-questions.sh). - -set -uo pipefail - -# Portability: resolve a real Python 3 by EXECUTING candidates, not by looking one up on PATH. -# On Windows `python3` is usually the Microsoft Store alias stub, which `command -v` finds and -# which then opens the Store instead of running. See bin/lib/portable.sh. -# shellcheck disable=SC1091 -. "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/portable.sh" - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -COLLECTOR="$SCRIPT_DIR/open-questions.sh" - -PROJECT_DIR="" -STAGE_ARGS=() -OUT="" -QUIET=0 - -while [ $# -gt 0 ]; do - case "$1" in - --stage) STAGE_ARGS+=(--stage "${2:-}"); shift 2 ;; - -o|--output) OUT="${2:-}"; shift 2 ;; - --quiet) QUIET=1; shift ;; - -h|--help) sed -n '2,28p' "${BASH_SOURCE[0]}"; exit 0 ;; - -*) echo "questions-report.sh: unknown option $1" >&2; exit 2 ;; - *) if [ -z "$PROJECT_DIR" ]; then PROJECT_DIR="$1"; else STAGE_ARGS+=("$1"); fi; shift ;; - esac -done - -[ -n "$PROJECT_DIR" ] || { echo "usage: questions-report.sh [--stage N] [-o out.html]" >&2; exit 2; } -[ -d "$PROJECT_DIR" ] || { echo "questions-report.sh: not a directory: $PROJECT_DIR" >&2; exit 2; } -[ -x "$COLLECTOR" ] || [ -f "$COLLECTOR" ] || { echo "questions-report.sh: collector not found: $COLLECTOR" >&2; exit 2; } - -PROJECT_DIR="$(cd "$PROJECT_DIR" && pwd)" -[ -n "$OUT" ] || OUT="$PROJECT_DIR/docs/open-questions.html" - -JSON=$(bash "$COLLECTOR" "$PROJECT_DIR" "${STAGE_ARGS[@]+"${STAGE_ARGS[@]}"}" --json 2>/dev/null) -COLLECT_RC=$? - -# rc 3 means no BRDs and no sme-questions.md were found. That is not "no open questions"; -# it means the instrument found nothing to read, and reporting it as clean is the exact -# class of false-green this toolkit keeps getting bitten by. -# Belt and braces: the flag is read from the payload as well as the exit code. rc=3 was -# unreachable in --json mode until 2026-08-12, and this script trusting it alone is how that -# went unnoticed — it rendered a clean-looking report over an empty directory. -NOTHING=$(printf '%s' "$JSON" | grep -c '"nothingExamined": *true' | head -1) -if [ "$COLLECT_RC" -eq 3 ] || [ "${NOTHING:-0}" -gt 0 ] || [ -z "$JSON" ]; then - echo "questions-report.sh: nothing was examined (no BRDs, no sme-questions.md) — refusing to" >&2 - echo " write a report that would read as 'no open questions'. Exit 3." >&2 - exit 3 -fi - -mkdir -p "$(dirname "$OUT")" || { echo "questions-report.sh: cannot create $(dirname "$OUT")" >&2; exit 2; } - -require_py - -PROJECT_NAME="$(basename "$PROJECT_DIR")" -GENERATED="$(date '+%Y-%m-%d %H:%M')" - -# The collector's JSON goes through a temp FILE, not a pipe. `"$PY" - <<'PYEOF'` takes its -# program from stdin, so a pipe into the same command is swallowed by the heredoc and json.load -# reads an empty string. Costing a temp file is cheaper than an inline-quoted program. -JSON_TMP=$(mktemp "${TMPDIR:-/tmp}/qreport.XXXXXX.json") || exit 2 -trap 'rm -f "$JSON_TMP"' EXIT -printf '%s' "$JSON" > "$JSON_TMP" - -PROJECT_NAME="$PROJECT_NAME" GENERATED="$GENERATED" JSON_TMP="$JSON_TMP" \ - PROJECT_DIR="$PROJECT_DIR" OUT="$OUT" "$PY" - <<'PYEOF' -import html, json, os, sys - -with open(os.environ["JSON_TMP"], encoding="utf-8") as fh: - d = json.load(fh) -qs = d.get("questions", []) -counts = d.get("counts", {}) -ex = d.get("examined", {}) -blocking = d.get("blocking", 0) -project = os.environ["PROJECT_NAME"] -generated = os.environ["GENERATED"] -out_path = os.environ["OUT"] - -BLOCKING_STATES = {"UNRAISED", "UNRECOGNISED"} - -# Order: what needs the user first, then what is merely on record. -ORDER = ["UNRECOGNISED", "UNRAISED", "RAISED", "ASSUMED", "ANSWERED", "MOOT"] -STATE_NOTE = { - "UNRAISED": "Never put to you. These block the gate.", - "UNRECOGNISED": "Status text no rule could map. Treated as unraised — a status nobody can parse is not an answer.", - "RAISED": "Put to you, awaiting your reply.", - "ASSUMED": "A position was taken AND you consented to it. Recorded, not blocking.", - "ANSWERED": "Settled with a recorded answer.", - "MOOT": "Descoped — genuinely dead, not silently dropped.", -} - -def esc(s): - return html.escape(str(s or "")) - -# A question the user can act on needs a proposed answer. One without is not "a question", -# it is unfinished work by the stage that produced it — so it gets called out as such. -def has_proposal(q): - return bool((q.get("assumption") or "").strip()) - -groups = {} -for q in qs: - groups.setdefault(q.get("state", "UNRECOGNISED"), []).append(q) - -actionable = [q for q in qs if q.get("state") in BLOCKING_STATES] -no_proposal = [q for q in actionable if not has_proposal(q)] - -rows = [] -for st in ORDER: - n = counts.get(st, 0) - if n: - cls = "b" if st in BLOCKING_STATES else "" - rows.append(f'{st}{n}{esc(STATE_NOTE.get(st,""))}') -tbl = "\n".join(rows) - -stage_filter = ex.get("stageFilter") -scope = f"stage ≤ {esc(stage_filter)}" if stage_filter else "all stages" - -cards = [] -for st in ORDER: - items = groups.get(st, []) - if not items: - continue - badge = "block" if st in BLOCKING_STATES else "ok" - cards.append(f'

{esc(st)} {len(items)}

') - cards.append(f'

{esc(STATE_NOTE.get(st,""))}

') - for q in items: - prop = (q.get("assumption") or "").strip() - raw = (q.get("rawStatus") or "").strip() - reason = (q.get("reason") or "").strip() - bits = [f'
'] - bits.append(f'
{esc(q.get("id","?"))} {esc(q.get("origin",""))}
') - bits.append(f'
{esc(q.get("question",""))}
') - if prop: - bits.append( - '
Drafting position — confirm or overturn' - f'
{esc(prop)}
') - elif st in BLOCKING_STATES: - bits.append( - '
No proposed answer' - '
This question was handed over bare. Before putting it to the user, the owning ' - 'stage should propose at least two named options and a recommendation — otherwise the ' - 'user is being asked to do the analysis the stage was meant to do.
') - meta = [] - if raw: - meta.append(f'recorded as: {esc(raw[:160])}') - if reason: - meta.append(esc(reason)) - if q.get("stage") is not None: - meta.append(f'stage {esc(q.get("stage"))}') - if meta: - bits.append('
' + " · ".join(meta) + "
") - bits.append("
") - cards.append("".join(bits)) - -banner_cls = "fail" if blocking else "pass" -banner_txt = (f"{blocking} question(s) have never been put to the user — this gate is blocked." - if blocking else - "No unraised questions. Every question on record has been seen by the user.") - -gapline = "" -if no_proposal: - gapline = (f'') - -doc = f""" - - -Open questions — {esc(project)} - -
-

Open questions — {esc(project)}

-

Generated {esc(generated)} · scope: {scope} · - examined {esc(ex.get("brdFiles",0))} BRD file(s) and {esc(ex.get("smeQuestionFiles",0))} SME question file(s), - {esc(ex.get("duplicatesCollapsed",0))} duplicate(s) collapsed

- - - {gapline} - -

What was found

- - {tbl} -
StateCountMeaning
- - {"".join(cards)} - -
- mxcli-project-toolkit · generated by bin/questions-report.sh from - bin/open-questions.sh --json. Regenerate after answering: rerun the same command. - An ASSUMED state without recorded consent is normalised to UNRAISED and - blocks — a position the user never saw is not an assumption they made. -
-
-""" - -with open(out_path, "w") as f: - f.write(doc) -PYEOF - -PY_RC=$? -[ "$PY_RC" -eq 0 ] || { echo "questions-report.sh: render failed (python exit $PY_RC)" >&2; exit 2; } - -if [ "$QUIET" -eq 0 ]; then - echo "Wrote: $OUT" - # Reads the temp file, not a pipe, and no 2>/dev/null: a summary that silently prints - # nothing when it breaks is worse than one that says why. - JSON_TMP="$JSON_TMP" "$PY" - <<'PYSUM' -import json, os -with open(os.environ["JSON_TMP"], encoding="utf-8") as fh: - d = json.load(fh) -c = d.get("counts", {}) -qs = d.get("questions", []) -bare = [q for q in qs - if q.get("state") in ("UNRAISED", "UNRECOGNISED") - and not (q.get("assumption") or "").strip()] -print(" blocking: %s total: %s without a proposed answer: %s" - % (d.get("blocking", 0), c.get("total", 0), len(bare))) -PYSUM -fi - -[ "$COLLECT_RC" -eq 1 ] && exit 1 -exit 0 diff --git a/bin/status.sh b/bin/status.sh index cd648493..1b26ceea 100755 --- a/bin/status.sh +++ b/bin/status.sh @@ -88,21 +88,22 @@ fi N_STALL="$(printf '%s' "$STALL_LIST" | grep -c .)"; N_STALL="${N_STALL:-0}" NOT_DONE_PASS="$(printf '%s\n' "$STALL_LIST" | sed '/^$/d' | head -3 | tr '\n' ' ')" # done- is gated on the coverage checklist (iterative-build-loop.md). Field case 2026-10-06: 47 -# scripts built, 0 done-, build-plan.html empty — the checklist could never pass and nothing said +# scripts built, 0 done-, progress report empty — the checklist could never pass and nothing said # so. Several gate-passes and ZERO done- is a stall, not a backlog. DONE_STALL="" [ "$N_DONE" -eq 0 ] && [ "$N_STALL" -ge 3 ] && DONE_STALL="done- stalled: $N_STALL gate-pass, 0 done-" -# build-plan.html older than the newest mdlsource/ change reads as a plan nobody is building. +# the progress report (or an older build-plan.html) older than the newest mdlsource/ change reads as a plan nobody is building. BP_STALE="" -BP_HTML="$PROJECT_DIR/architecture/build-plan.html" +BP_HTML="$PROJECT_DIR/architecture/progress-report.html" +[ -f "$BP_HTML" ] || BP_HTML="$PROJECT_DIR/architecture/build-plan.html" # pre-progress-report projects if [ -f "$BP_HTML" ] && [ -d "$MDL_DIR" ] && git -C "$PROJECT_DIR" rev-parse --git-dir >/dev/null 2>&1; then _mdl_ts="$(git -C "$PROJECT_DIR" log -1 --format=%ct -- mdlsource 2>/dev/null)" # Commit time when the page is tracked (a fresh clone resets every mtime); mtime only for an # untracked, locally generated page. - _bp_ts="$(git -C "$PROJECT_DIR" log -1 --format=%ct -- architecture/build-plan.html 2>/dev/null)" + _bp_ts="$(git -C "$PROJECT_DIR" log -1 --format=%ct -- "${BP_HTML#"$PROJECT_DIR"/}" 2>/dev/null)" [ -n "$_bp_ts" ] || _bp_ts="$(stat -c %Y "$BP_HTML" 2>/dev/null || stat -f %m "$BP_HTML" 2>/dev/null)" case "$_mdl_ts$_bp_ts" in ''|*[!0-9]*) : ;; *) - [ "$_bp_ts" -lt "$_mdl_ts" ] && BP_STALE="build-plan.html STALE (older than the newest mdlsource/ commit — bin/build-plan-status.sh --html)" ;; + [ "$_bp_ts" -lt "$_mdl_ts" ] && BP_STALE="$(basename "$BP_HTML") STALE (older than the newest mdlsource/ commit — bin/build-plan-status.sh --html)" ;; esac fi # Agent slots still unfilled (#211) — by design until each agent's stage starts, never invisible. diff --git a/bin/sync-project.sh b/bin/sync-project.sh index 8453f253..ba56ae24 100755 --- a/bin/sync-project.sh +++ b/bin/sync-project.sh @@ -1610,6 +1610,26 @@ if [ "$WIRED" -eq 1 ] && [ -x "$SCRIPT_DIR/install-harness-permissions.sh" ]; th fi fi +# ── Progress report session cache: machine-local, keep it out of git ───────────────────── +# It holds absolute session-log paths (user name, session ids). Same shape as doctor.sh's +# receipt and toolkit.env lines; a project synced from an older toolkit has no such line. +# -e, not -d: in a git worktree .git is a file. --no-index: a tracked file is never "ignored" otherwise. +if [ "$WIRED" -eq 1 ] && [ -e "$PROJECT_DIR/.git" ] \ + && ! git -C "$PROJECT_DIR" check-ignore -q --no-index .claude/progress-report-sessions.txt 2>/dev/null; then + if [ "$DRY_RUN" -eq 1 ]; then + echo "Would add: /.claude/progress-report-sessions.txt to .gitignore (machine-local cache)" + else + if [ -s "$PROJECT_DIR/.gitignore" ] && [ -n "$(tail -c 1 "$PROJECT_DIR/.gitignore")" ]; then printf '\n' >> "$PROJECT_DIR/.gitignore"; fi + printf '\n# Machine-local progress report session cache (absolute session-log paths)\n/.claude/progress-report-sessions.txt\n' >> "$PROJECT_DIR/.gitignore" + echo "Added: /.claude/progress-report-sessions.txt to .gitignore (machine-local cache)" + fi + CHANGES=$((CHANGES + 1)) +fi +if [ "$WIRED" -eq 1 ] && [ -e "$PROJECT_DIR/.git" ] \ + && git -C "$PROJECT_DIR" ls-files --error-unmatch .claude/progress-report-sessions.txt >/dev/null 2>&1; then + echo "Note: .claude/progress-report-sessions.txt is already tracked by git; untrack it with: git rm --cached .claude/progress-report-sessions.txt" +fi + # ── Starlark lint rules ────────────────────────────────────────────────────────────────── # Gated on WIRED for the same reason as the crash net: a mistyped path must not scatter # .star files into an unrelated directory. Unlike the crash net this is also a REPAIR path, diff --git a/claude-hooks/hooks/interview-gate.sh b/claude-hooks/hooks/interview-gate.sh index bf7a3f1f..b36c88df 100755 --- a/claude-hooks/hooks/interview-gate.sh +++ b/claude-hooks/hooks/interview-gate.sh @@ -50,7 +50,7 @@ except Exception: print("True") PROJECT_DIR="${CLAUDE_PROJECT_DIR:-$PWD}" TOOLKIT="__TOOLKIT_ROOT__" OQ="$TOOLKIT/bin/open-questions.sh" -QR="$TOOLKIT/bin/questions-report.sh" +QR="$TOOLKIT/bin/brd-report.sh" IM="$TOOLKIT/bin/interview-mode.sh" [ -x "$OQ" ] || exit 0 @@ -92,8 +92,8 @@ printf '%s' "$FP" > "$STATE" 2>/dev/null || true REPORT="" if [ -x "$QR" ]; then - "$QR" "$PROJECT_DIR" -o "$PROJECT_DIR/docs/open-questions.html" --quiet >/dev/null 2>&1 - [ -f "$PROJECT_DIR/docs/open-questions.html" ] && REPORT="$PROJECT_DIR/docs/open-questions.html" + "$QR" "$PROJECT_DIR" --html "$PROJECT_DIR/analysis/brd-report.html" --quiet >/dev/null 2>&1 + [ -f "$PROJECT_DIR/analysis/brd-report.html" ] && REPORT="$PROJECT_DIR/analysis/brd-report.html" fi # In auto mode — and when the user is remote — the report is the whole job. Refreshing it @@ -113,7 +113,7 @@ REASON="$BLOCKING question(s) in this project have never been put to the user, a Before you stop: put them in the chat. Read $TOOLKIT/skills/interview-protocol.md — every question carries two named options and your recommendation; a bare question hands your analysis back to the user. Batch them, then end the turn and wait for answers. " [ -n "$REPORT" ] && REASON="${REASON} -Triage report written for the user to review: $REPORT — give them this path. +BRD report (open questions under the process map) written for the user to review: $REPORT — give them this path. " REASON="${REASON} This project is in interview mode '$MODE'. If the user wants speed over involvement here, diff --git a/commands/mobile-auto-test-prompt.md b/commands/mobile-auto-test-prompt.md index 9834910b..cb006e25 100644 --- a/commands/mobile-auto-test-prompt.md +++ b/commands/mobile-auto-test-prompt.md @@ -89,7 +89,7 @@ For each module: 8. After every module, run project-bin/coherence-cadence.sh — when it reports DUE, run the full skills/process-coherence-pass.md sweep before moving to the next module, then run coherence-cadence.sh --record. -9. Run project-bin/build-plan-status.sh --html periodically to keep architecture/build-plan.html +9. Run project-bin/build-plan-status.sh --html periodically to keep architecture/progress-report.html current — it shows built (done- prefixes) vs proven (verify-module.sh + improvement-register.md) as two separate views on purpose. diff --git a/contrib/inbox/2026-10-13-durable-rebuild-kit-legacy-widgets.md b/contrib/inbox/2026-10-13-durable-rebuild-kit-legacy-widgets.md new file mode 100644 index 00000000..94dd1cf7 --- /dev/null +++ b/contrib/inbox/2026-10-13-durable-rebuild-kit-legacy-widgets.md @@ -0,0 +1,83 @@ +# Durable rebuild of legacy-widget pages: a four-part kit, and "restate from DESCRIBE" as a STOP item + +**From:** a field project (anonymised) — an existing Mendix app (old widgets throughout) changed with mxcli + this toolkit +**Date:** 2026-10-13 +**Kind:** process +**Field evidence:** 79 non-writable widgets on 40 live documents (DESCRIBE `-- NOT re-executable` count) +and 97 role-locked widgets in 16 documents (BSON scan; DESCRIBE showed none). 34 pages and snippets +were restated whole; the read-back afterwards found 0 non-writable widgets and 0 role locks on them. +Two drafted scripts would have silently dropped role or row visibility had they run. Mendix 11.14.0 +Beta (Classic client, Production security), mxcli v0.25.0, toolkit c1efcb3. +**Proposed target:** `skills/existing-app-change.md` (a "legacy-widget pages" section), a STOP row in +`skills/learned-mdl-preflight.md`, possibly a `project-bin/` scan once field-proven as a file + +**Currency check:** 2026-10-16 against master d1cbd41: partly stale. STOP row 21 in `skills/learned-mdl-preflight.md` now covers restating a document another script also edits. Still open: the legacy-widget section in `skills/existing-app-change.md` and the scan for role locks that DESCRIBE does not show. +--- + +## What is already covered + +- `bug-logs/mxcli-bugs.md` BUG-115 addendum: "Grep a generated script for `NOT re-executable` before + executing it" (for the File Manager widget). +- `bug-logs/mxcli-bugs.md` (the `hasRole()` entry): no MDL syntax writes `ConditionalVisibilitySettings.ModuleRoles`. +- `skills/learned-mdl-cannot-express.md` (master bda5bed, #277): "A widget visible to one role and not + another — nothing", with the execute-grant recipe for action buttons. + +## What is not covered (findings) + +1. **DESCRIBE does not print module-role visibility.** One portal page had 19 role-locked widgets; DESCRIBE + printed none of them (nor a hidden "show page" action behind 9 image widgets). A page restated from that + DESCRIBE builds clean and shows every widget to every role. +2. **A drafted restatement said "the snippet has no visibility rules".** It restated a side-menu snippet + from DESCRIBE (navigation-list items have no names, so one item cannot be dropped alone). A BSON audit + found 13 role-locked items. Running the draft would have exposed all of them, with a clean build. +3. **DESCRIBE does not print layout-grid row visibility.** Two form pages had nine detail sections shown + by type through row visibility; a snippet had 56 row-level "show only when it has a value" rules. None + appear in DESCRIBE. The drafters re-wrote them as container `Visible:` (which DESCRIBE does print and + mxcli does write); an audit confirmed every rule was carried over. +4. **Tab-page `Visible:` and `CaptionParams` are accepted and not written** (see the learned-entries note). + Found only by reading the unit back after exec. + +So "copy the page from DESCRIBE, edit, `create or replace`" is silently lossy on any app built in Studio Pro. + +## Proposed STOP row (learned-mdl-preflight.md) + +| # | Operation | Why STOP | Do instead | +|---|---|---|---| +| new | Restate an existing page, snippet or layout from `describe` output (`create or replace`, or `alter … replace` of a container) | DESCRIBE omits module-role visibility, layout-grid row/column visibility and legacy-widget internals; check, exec and mxbuild all pass while the rules vanish | Run the visibility scan (below) on the unit first. Zero role locks and zero row rules: go. Otherwise rewrite each rule as container `Visible:` or flow access, or leave the page to `alter page` / Studio Pro | + +Probe that falsifies it: in Studio Pro, set one widget visible to one module role and one layout row +visible on an attribute; `describe page`; grep the output for the role name and the attribute. Today +(v0.25.0): 0 hits for both. + +## The kit (each part was done by hand in the field; none is a toolkit file yet) + +1. **BSON visibility scan, per document.** Decode each page/snippet/layout unit under `mprcontents/` + (the toolkit's `project-bin/mxunit_bson.py` codec is enough) and count (a) any + `ConditionalVisibilitySettings` whose `ModuleRoles` list is non-empty, (b) any + `ConditionalVisibilitySettings` on a `Forms$LayoutGridRow` or `Forms$LayoutGridColumn`. Report per + document: widget name, kind, roles or attribute. The field scans were inline one-offs, not a saved + script; a shipped version needs its own fixture. +2. **Non-writable count, per document.** `describe` every page, snippet and layout in the app's own + modules (Marketplace modules excluded) and count `-- NOT re-executable` lines, grouped by the BSON + type in the comment. Field result: `Forms$DropDown` 29, `Forms$ReferenceSelector` 27, + `Forms$DataGrid` 16, sign-in widgets 4, `Forms$TemplateGrid` 1, `Forms$ImageUploader` 1, + `Forms$SidebarToggleButton` 1. +3. **Rebuild brief per document.** Inventory first (DESCRIBE plus the scan), then each widget marked + KEEP / CUT / MOVED / NEW against the wireframe, then the restatement. Replacements used: + drop-down → `combobox` on the enumeration; reference selector → `combobox` on the association (a + microflow data source when the filter needs the context object); old data grid → Data grid 2; + template grid → `listview`; image uploader → File Uploader in files mode plus a `dynamicimage`; + role-only visibility → a per-viewer flag from a data-source microflow, or flow access (Mendix hides a + button whose flow the user cannot run). The restating script becomes the document's single owner; + older `alter page` blocks on it are marked superseded (compare #269). +4. **Post-exec read-back.** After each exec: DESCRIBE again (0 `NOT re-executable`), rerun the scan + (0 role locks unless intended), and grep the unit for every `Visible:` / `CaptionParams` the script + wrote. Item 4 above was caught only this way. + +Cost, for scale: six drafting agents in parallel; one gated exec per document (about 5–10 minutes each +with mxbuild and lint), serial because there is one writer to the `.mpr`. + +Upstream asks (mxcli), for the maintainer to file or link: print module-role and row/column visibility in +DESCRIBE (or at least a `-- role-based visibility not shown (N roles)` comment), accept it on write, give +navigation-list items stable names, and a legacy-widget migration (`DataGrid` → Data grid 2, +`ReferenceSelector`/`DropDown` → combobox) behind a dry run. diff --git a/contrib/inbox/2026-10-13-exec-keeps-write-when-mxbuild-cannot-run.md b/contrib/inbox/2026-10-13-exec-keeps-write-when-mxbuild-cannot-run.md new file mode 100644 index 00000000..3ebeb7f3 --- /dev/null +++ b/contrib/inbox/2026-10-13-exec-keeps-write-when-mxbuild-cannot-run.md @@ -0,0 +1,75 @@ +# exec.sh keeps an MDL write when mxbuild crashes after it, and the next exec runs on an unloadable model + +**From:** a field project (anonymised) — an existing Mendix app being changed with mxcli + this toolkit +**Date:** 2026-10-13 +**Kind:** bug +**Field evidence:** the project's BUILD-LOG shows two "gate could not run" rows 46 s apart: the +script that made the model unloadable, then the next script, which exec.sh accepted and ran on top +of it. Mendix 11.14.0 Beta (Classic client), mxcli v0.25.0, toolkit commit c1efcb3 (`project-bin/exec.sh` +is byte-identical on master 0c62f6f, blob a03adec). +**Proposed target:** `project-bin/exec.sh` (gate branch "mxbuild failed to run" and the pre-flight +"Baseline NOT measured" branch); one line in `skills/iterative-build-loop.md` + +--- + +## What happened (finding, reproduced once in the field) + +A page script set a data view's data source to a two-hop association path in its full form +(`$Parent/MyModule.A_B/MyModule.B/MyModule.B_C/MyModule.C`). `mxcli check` passed. After the +exec, the mxbuild loader itself crashed instead of reporting CE errors: + +``` +An error occurred when trying to set the 'DestinationEntity' property of a Entity ref step in a Page. +``` + +exec.sh printed: + +``` + ✗ mxbuild failed to run (exit 1) — gate could not verify the model. + → Snapshot preserved. Open in SP to verify manually before proceeding. +``` + +and kept the write. It did exit 1 and cleared the verification stamp (good), but the agent driving +the build ran the next script. exec.sh's pre-flight printed `⚠ Baseline NOT measured: mxbuild exited 1 +without checking the model.`, carried on, applied the next script, and hit the same crash. + +BUILD-LOG rows, verbatim except the script names: + +``` +| 2026-10-13T12:26:18 | `25g-PageA-rebuild.mdl` | fail | ❌ gate failed | 5 error(s): CE1613 — script rolled back · approval: auto (PROJECT.md) | +| 2026-10-13T12:28:46 | `25g-PageA-rebuild.mdl` | fail | ❌ gate could not run | mxbuild exit 1 · approval: auto (PROJECT.md) | +| 2026-10-13T12:29:32 | `25h-SnippetB-rebuild.mdl` | fail | ❌ gate could not run | mxbuild exit 1 · approval: auto (PROJECT.md) | +``` + +Recovery: `./bin/restore-mpr.sh ` then +`./bin/verify-model.sh` (0 errors). The 25h row had to be re-run afterwards. + +## Why it matters + +- The header comment of exec.sh says an unverified gate is rolled back **for `--patch` only**. That is + true and deliberate, but for MDL the "kept" case was designed for "mxbuild could not start" + (wrong version, no JDK). Here mxbuild started, ran on the baseline cleanly a minute earlier, and + crashed only on the written model. That is a property of the write, not of the machine. +- The next exec has no way to tell the two apart: "Baseline NOT measured" is a warning, not a refusal. + So in an unattended chain the damage compounds: every following script is written onto a model + nothing can load, and each one gets its own snapshot of the broken state. + +## Proposed fix (hypothesis, not built) + +1. **Restore when mxbuild ran clean on the baseline but cannot run after the write.** exec.sh already + has both facts: the pre-flight measured a baseline (`BASE_KNOWN=1`) and the post-exec run exits + non-zero with no errors file. In that case, call `restore_snapshot` exactly as the `--patch` path + does, and log `↩ rolled back: mxbuild crashed on the written model`. Keep today's behaviour when the + baseline itself was not measured (that is the real "machine cannot build" case). +2. **Refuse a write when the baseline cannot be measured** and the last BUILD-LOG row for this model + is `gate could not run`, with one STOP line naming `bin/restore-mpr.sh` and the snapshot to restore. + `ALLOW_UNVERIFIED=1` stays the explicit override. +3. Reword the kept-write message: "Open in SP to verify" is not something an agent can do. Something + like `STOP: the model may not load. Restore with ./bin/restore-mpr.sh ` is actionable. + +Probe that would falsify the claim (for the retest fixture): a fake mxbuild that exits 0 on the +baseline and exits 1 with no errors file after the exec; today exec.sh exits 1 and leaves the +written model in place; after the fix the model on disk equals the snapshot. + +Related: `--patch` already restores on an unverified gate (header lines 25–29); PR #304 touches the +same file (per-project lock) but not this branch. diff --git a/contrib/inbox/2026-10-13-learned-widgets-and-data-sources.md b/contrib/inbox/2026-10-13-learned-widgets-and-data-sources.md new file mode 100644 index 00000000..e5d7d710 --- /dev/null +++ b/contrib/inbox/2026-10-13-learned-widgets-and-data-sources.md @@ -0,0 +1,135 @@ +# Learned entries from a legacy-widget rebuild: File Uploader image mode, native-mapped keywords, tab pages, XPath comparands, multi-hop data sources + +**From:** a field project (anonymised) +**Date:** 2026-10-13 +**Kind:** learning +**Field evidence:** each entry below was hit while restating 34 pages of an existing app, through +`bin/exec.sh` (mxbuild delta gate) on mxcli v0.25.0, Mendix 11.14.0 Beta, Classic (Dojo) client, +toolkit c1efcb3. Error texts are verbatim; module/entity names are genericised. +**Proposed target:** per entry below (`skills/learned-file-upload-widget.md`, +`skills/learned-mdl-cannot-express.md`, `skills/learned-page-patterns.md`, `bug-logs/mxcli-bugs.md`) + +--- + +## 1. File Uploader: image mode fails the build; files mode against a `System.Image` subtype works + +Target: `skills/learned-file-upload-widget.md` §7 ("Image mode … was not tested"). +Status: files mode is ALREADY covered there (PR #139); this adds image mode and the image-preview pattern. + +```sql +fileuploader upPictures ( + uploadMode: 'images', + associatedImages: association $currentObject/MyModule.Picture_Asset, + readOnlyMode: true, + enableCustomButtons: false +) +``` + +`mxcli check` passed. mxbuild: `CE0463 The definition of this widget has changed.` on +`File uploader upPictures`; the gate rolled back (rollback verified by DESCRIBE). The same widget with +`uploadMode: 'files'` builds clean (finding). + +What built clean instead (finding, mxbuild 0 errors): uploads in **files** mode against an entity that +extends `System.Image` (which extends `FileDocument`), with a separate `dynamicimage` preview beside it; +read-only picture lists as a `listview` whose template holds a `dynamicimage`. + +Also stale: `skills/learned-mdl-cannot-express.md` row 1 still says "A file upload control — nothing … +⛔ not buildable", which contradicts `learned-file-upload-widget.md`. Suggest pointing that row at the +File Uploader skill. + +Related (finding): the installed FileDropper widget's definition lists `dataFileEntity` and +`dataContextAssociation` as known properties with no mapping; `check` accepts them, exec does not write +them, and mxbuild then fails. Upstream ask: refuse unmapped definition properties with MDL-WIDGET07. + +## 2. `radiobuttons` and `image` can resolve to NATIVE mobile widgets on a web page + +Target: `skills/learned-page-patterns.md` (or the widgets section of `learned-mdl-cannot-express.md`). +Status: NEW (no toolkit mention). + +In this project `DESCRIBE WIDGET TYPE radiobuttons` resolves to `com.mendix.widget.native.radiobuttons`, +and `check` warns MDL-WIDGET10 that the label is dropped. The project's `image.def.json` maps `image` to +`com.mendix.widget.native.image.Image`; a probe wrote `image` with no properties. DESCRIBE prints existing +**web** radio groups with the same `radiobuttons` keyword, so re-running DESCRIBE output writes a native +widget into a web page (finding for the resolution and the warning; the radiobuttons case was not taken +through mxbuild). + +Rule used: Boolean → `checkbox`, enumeration → `combobox`, images → `dynamicimage`. Before relying on a +keyword in an existing app, run `DESCRIBE WIDGET TYPE ` and check the id is a `web` widget. + +## 3. Tab pages: `CaptionParams` and `Visible:` pass every gate and are not written + +Target: `bug-logs/mxcli-bugs.md` as `## BUG-DRAFT-tabpage-captionparams-visible-dropped:`, plus a row in +`learned-mdl-cannot-express.md` next to the existing "count badge on a tab" row. +Status: NEW. (The existing row covers `Badge:`, which `check` refuses; these two are accepted.) + +```sql +tabpage tpDue (Caption: 'Due today or earlier · {1}', CaptionParams: ({1} = DueCount)) { … } +tabpage tpOther (Caption: 'Unassigned · {1}', Visible: $currentObject/ShowOtherTab) { … } +``` + +`check`, exec and mxbuild all passed, with no MDL-WIDGET07. The stored caption is literal text (the tab +reads `· {1}`), and the tab page has no visibility setting: `ShowOtherTab` occurs 0 times in the unit. +Found by a BSON read-back after exec (finding). + +Rule: no dynamic captions and no visibility on tab pages; put the count in a line inside the tab, and +put anything conditional in a container with `Visible:` (container visibility is written correctly). + +## 4. DESCRIBE omits module-role visibility and layout-grid row visibility + +Target and detail: see the inbox note "Durable rebuild of legacy-widget pages" (same date). Listed +here so triage sees it among the learned entries. Status: PARTLY (role visibility is known to be +unwritable; that DESCRIBE also hides it is not recorded). + +## 5. Page XPath with a `$currentObject/` comparand → CE0161 (BUG-39 on widgets) + +Target: `bug-logs/mxcli-bugs.md` BUG-39 (add a confirmation), `learned-page-patterns.md`. +Status: PARTLY. BUG-39 records it for microflow retrieves only. + +```sql +combobox cbLevel2 ( + Attribute: Helper_Level2, + DataSource: database from "MyModule"."Level2" + where [MyModule.Level2_Level1 = $currentObject/MyModule.Helper_Level1] +) +``` + +`check --references` passed; mxbuild: 3 × `CE0161 Error(s) in XPath constraint`, one per cascading combo +box; rolled back (finding). Studio Pro accepts the same constraint. + +Workaround that built: each combo box takes its options from a microflow data source that retrieves the +parent into a variable and filters on it. + +## 6. Two-hop association data source: neither path form is writable → use a microflow data source + +Target: `learned-page-patterns.md` (next to the BUG-04 multi-hop ContentParams section), and +`bug-logs/mxcli-bugs.md` as `## BUG-DRAFT-two-hop-datasource-path:`. +Status: NEW. BUG-20 (cross-module traversal writing a null `DestinationEntityId`) was fixed in v0.17.0; +this is a same-module, two-hop path on v0.25.0. + +Short form, exactly as DESCRIBE prints it: + +```sql +DataSource: $Parent/MyModule.A_B/MyModule.B_C +``` + +mxcli stored the last segment as an entity: +`CE1613 The selected entity 'MyModule.B_C' no longer exists.` (5 errors); rolled back. + +Full form: + +```sql +DataSource: $Parent/MyModule.A_B/MyModule.B/MyModule.B_C/MyModule.C +``` + +`check` passed; the mxbuild loader crashed: +`An error occurred when trying to set the 'DestinationEntity' property of a Entity ref step in a Page.` +(This crash is what exposed the exec.sh gap in the separate inbox note.) + +So DESCRIBE output containing such a path is not re-executable. Rule: no multi-hop association path as a +data-view source; use a microflow data source that retrieves over each association in turn (built clean). + +## Not included here + +Other cases from the same run (Data grid 2 custom-content column losing its Attribute; XPath through a +reverse association needing read access mxbuild does not check; names `check --references` does not +validate) are upstream mxcli material and are left for a separate note. diff --git a/contrib/inbox/2026-10-13-patch-unit-row-delete-orphan.md b/contrib/inbox/2026-10-13-patch-unit-row-delete-orphan.md new file mode 100644 index 00000000..9355c51b --- /dev/null +++ b/contrib/inbox/2026-10-13-patch-unit-row-delete-orphan.md @@ -0,0 +1,43 @@ +# exec.sh --patch: deleting a Unit row is refused by `mxcli fix hashes --repair` (ORPHAN file); rename-then-drop is the route that works + +**From:** a field project (anonymised) +**Date:** 2026-10-12 +**Kind:** learning +**Field evidence:** two pages with the same qualified name (a live one and an excluded, archived copy in +another folder). mxcli v0.25.0, Mendix 11.14.0 Beta, MPR v2, toolkit c1efcb3. The failed patch was +rolled back by exec.sh; the second route built at 0 errors. +**Proposed target:** `skills/learned-mdl-preflight.md` (a workaround row) or `skills/learned-mdl-cannot-express.md`; +an upstream mxcli ask + +--- + +## Situation + +`MyModule.Detail` existed twice: the live page and an archived copy (excluded from the project) with +the same name in `Archive/Pages`. mxcli addresses documents by qualified name only, and different +commands resolved the name to different copies: `DESCRIBE PAGE` found the live one, `SHOW ACCESS ON PAGE` +the archived one, so a `grant view` by name landed on the archived copy. There is no `drop page … in +folder '…'` and no drop by unit id. + +## Route 1, refused (finding) + +A gated patch through `./bin/exec.sh --patch` deleted the archived page's row from the `.mpr` `Unit` +table and left its `.mxunit` file on disk (file removal being left to the owner). The patch's +`mxcli fix hashes --repair` step then reported the file as **ORPHAN** (present on disk, not indexed) +and failed; exec.sh restored the snapshot. Afterwards both pages were present and 3161 units verified. + +## Route 2, worked (finding) + +1. A gated patch renamed only the archived unit's top-level `Name` to `Detail_Archived`, after a + byte-identical decode/encode round-trip check of that unit. mxbuild stayed at 0 errors. +2. Then plain MDL, through exec.sh: `drop page if exists "MyModule"."Detail_Archived";` + +## Rule proposed + +To remove one of two same-named documents: rename it by patch, then drop it by MDL. Never delete a +`Unit` row by hand; `fix hashes --repair` treats the leftover file as an orphan and the gate rolls back. + +Ask for upstream mxcli: warn when a qualified name resolves to more than one document, make every +command resolve it the same way, and add `drop page … in folder '…'` or a drop by unit id. + +Related, not the same: #294 (a shared patch helper) and #295 (adopt `mxcli fix hashes` in `--patch`). diff --git a/contrib/inbox/2026-10-13-test-endpoint-after-startup-left-after-signal.md b/contrib/inbox/2026-10-13-test-endpoint-after-startup-left-after-signal.md new file mode 100644 index 00000000..a2e11133 --- /dev/null +++ b/contrib/inbox/2026-10-13-test-endpoint-after-startup-left-after-signal.md @@ -0,0 +1,56 @@ +# `mxcli run --local --test-endpoint` stopped by a signal leaves AfterStartupMicroflow on the injected flow (second field case for #244, and the toolkit-side half) + +**From:** a field project (anonymised) +**Date:** 2026-10-13 +**Kind:** bug +**Field evidence:** an agent stopped a running `mxcli run --local --test-endpoint` (a background-task +stop, so a signal, not a clean exit) to restart the app. Later a clean-up removed the injected +`MxTest` module but not the setting; exec.sh's pre-flight then reported the model "ALREADY has 1 error" +(CE1613), and the next app start recursed with a StackOverflowError in `MxTest.RegisterEndpoint`. +mxcli v0.25.0, Mendix 11.14.0 Beta, toolkit c1efcb3. +**Proposed target:** upstream mxcli for the root cause (already logged here as #244, closed 2026-10-14 by #322 (toolkit refusal); the upstream root cause remains); in the +toolkit, `skills/testing-shape.md` (the `--test-endpoint` paragraph) and an exec.sh pre-flight hint + +**Currency check:** 2026-10-16 against master d1cbd41: partly stale. exec.sh's pre-flight now refuses on a leftover injected setting and prints the `ALTER SETTINGS` repair (`_common.sh`), so the exec.sh hint half is done. Still open: the upstream root cause behind #244 closed 2026-10-14 by #322 (toolkit refusal); the upstream root cause remains, and `skills/testing-shape.md` does not say that a stop by signal skips the restore. +--- + +## What is already covered + +- #244 (closed 2026-10-14 by #322 (toolkit refusal); the upstream root cause remains) describes the same family on mxcli v0.25.0 / Mendix 11.15.0: the wrapper chains the + "current" after-startup flow, which is already `MxTest.RegisterEndpoint`, so it calls itself. It + asks upstream to restore on interrupt and to refuse a self-wrap. +- `skills/testing-shape.md` says `--test-endpoint` writes into the working model while the app runs and + that "a clean stop prints 'test endpoint removed; project restored'". It does not say that a stop by + signal skips that restore. + +## What this case adds + +1. **Trigger is a signal stop of `run --test-endpoint` itself**, not an interrupted `mxcli test` run. + Agents routinely stop background tasks this way to restart an app. +2. **A partial clean-up makes it worse.** Removing the `MxTest` module afterwards (by hand or by a later + cleanup) leaves `AfterStartupMicroflow` pointing at a flow that no longer exists: CE1613 at build, + which exec.sh's delta gate then carries as a "pre-existing" error on every later script. Re-creating + it by starting with `--test-endpoint` again gives the self-recursion from #244. +3. **The repair that worked**, run through `bin/exec.sh` (an `ALTER SETTINGS`, said in chat): + + ```sql + mdl 1; + alter settings runtime (AfterStartupMicroflow: 'MyModule.ASU_Startup'); + ``` + + mxbuild went back to 0 errors; the app then started normally. + +## Proposed toolkit changes (hypothesis, not built) + +- `testing-shape.md`: add "Stop `run --test-endpoint` with Ctrl+C in its own terminal or let it exit; + a kill or task stop skips the restore. Afterwards `git status` on the model must be empty; if the + Settings unit is still modified, restore it from git (or with the `alter settings runtime` line + above) before any exec." Run build-time apps without `--test-endpoint` (the skill already says so). +- `exec.sh` pre-flight: when the baseline holds CE1613 on the project Settings, or `describe settings` + shows `AfterStartupMicroflow` starting with `MxTest.`, print one line naming the cause and the repair, + instead of only "Model ALREADY has 1 error(s)". +- Upstream (#244): restore the original after-startup on SIGTERM/SIGINT too, and refuse to wrap an + after-startup that is already `MxTest.RegisterEndpoint`. + +Probe: start `mxcli run --local --test-endpoint` on a scratch copy, stop it with `kill -TERM `, +then `describe settings`. Today (v0.25.0): after-startup is still `MxTest.RegisterEndpoint`. diff --git a/docs/pipeline-walks.html b/docs/pipeline-walks.html index dbd31540..ad08b7bb 100644 --- a/docs/pipeline-walks.html +++ b/docs/pipeline-walks.html @@ -169,7 +169,7 @@

Migration Legacy source code existsP KickoffScaffold the project. Answer every intake question. Propose the entry mode with evidence and record it CONFIRMED. Gate: every intake question answered. Owner ba-agent.bin/init-project.sh <project>
bin/init-agents.sh <session-root>
bin/gate-check.sh <project> P 0 Triage ✋Inventory every source file. Grade sufficiency (SPECIFICATION / OUTLINE / SKETCH). Decide extract-or-not per structure and fill the coverage matrix. Scan documents. Run the CAC-1 scope brainstorm, order the slices, get sign-off in triage.md. Owner ba-agent.bin/source-sufficiency.sh init <project>
bin/source-sufficiency.sh report <project>
bin/interview-mode.sh --set … (only after the user agrees)
bin/triage-report.sh <project>
bin/source-sufficiency.sh --refresh for late files
bin/gate-check.sh <project> 0 1 AnalysisPath A: run the platform pipeline (extract, knowledge base, BRD scaffolds, reports). Path B: convert documents to Markdown and build the KB. Path C: interview the SME. Every source file must name the artifact that consumed it. Gate: source ledger clean, four extraction quality checks. CAC-1b.cd pipelines/<platform>/pipeline && npm install
node run.js · node run.js 2 · node run.js 3 · npm run reports
bin/html-to-md.sh <project> · bin/images-to-md.sh
bin/source-ledger.sh report <project>
bin/source-ledger.sh check <project>
bin/extraction-report.sh <project>
bin/gate-check.sh <project> 1 - 2 RequirementsTurn scaffolds into validated BRDs, one per feature. Drive blocking open questions to zero. Lock facts. CAC-2 BRD sign-off, CAC-3 architecture entry.bin/brd-report.sh <project>
bin/open-questions.sh <project> --stage 2
bin/questions-report.sh · bin/facts-lock.sh · bin/question-kinds.sh
bin/gate-check.sh <project> 2 + 2 RequirementsTurn scaffolds into validated BRDs, one per feature. Drive blocking open questions to zero. Lock facts. CAC-2 BRD sign-off, CAC-3 architecture entry.bin/brd-report.sh <project> [--stage N]
bin/open-questions.sh <project> --stage 2
bin/facts-lock.sh · bin/question-kinds.sh
bin/gate-check.sh <project> 2 3 Design ✋Module boundaries, blueprint, fit-gap, security model, integrations. Design system and one wireframe per screen. CAC-4. Owner architect-agent, interviews by ba-agent.skills: modularize-domain.md, architecture-blueprint.md, design-artifacts.md
outputs: architecture/blueprint.html, design/design-system.html, design/wireframes/*.html
bin/gate-check.sh <project> 3 4 Build plan ✋Dependency-ordered script sequence, coverage ledger, one brief per module. CAC-5, then the build-ready wiring check.bin/coverage-check.sh
bin/gate-check.sh <project> 4
bin/gate-check.sh <project> build-ready 5 BuildWalking skeleton, then the five-stage module loop per module (see below). Coherence pass every two or three modules../mxcli check s.mdl -p app.mpr --references
bin/exec.sh s.mdl · ./mxcli lint -p app.mpr
project-bin/verify-module.sh <Module>
./mxcli graph-report -p app.mpr
bin/gate-check.sh <project> 5 diff --git a/docs/releases/v1.0.md b/docs/releases/v1.0.md index 46b7f75d..7debe18e 100644 --- a/docs/releases/v1.0.md +++ b/docs/releases/v1.0.md @@ -5,179 +5,519 @@ agent and [mxcli](https://github.com/mendixlabs/mxcli). It gives the agent a sta with recorded decisions (`PROJECT.md`), skills it loads only when a task calls for them, and mechanical gates (`bin/gate-check.sh`, lint, mxbuild, end-to-end journeys) that report what is actually in your project. It runs for migrations, requirements-driven builds, greenfield builds -and changes to a live app, or à la carte to audit and test an app you already have. - -1.0 is the first named milestone on top of the release train. It is not a rewrite: these notes -pick out what changed in the two weeks before it (2026-09-30 to 2026-10-14). Every line traces to -an entry in [`CHANGELOG.md`](../../CHANGELOG.md), which has the full detail and field evidence. -How milestones relate to the `vYYYY.MM.DD` tags: [`docs/RELEASE-PROCESS.md`](../RELEASE-PROCESS.md) -§8. - -1.0 is the first release-train tag published to GitHub, cut with `bin/cut-release.sh` when this -release is announced. Earlier `vYYYY.MM.DD` headings in CHANGELOG were never pushed as tags. - -Most PRs linked below landed together through -[#329](https://github.com/MendixMau/mxcli-project-toolkit/pull/329) and show as closed on GitHub; their -changes are on master. - -## Highlights - -### Every merge is reviewed by someone who did not write it - -A PR now gets an independent review (MERGE / MERGE-AFTER-FIX / HOLD, on value, regression risk -after sync, correctness and context cost) before it lands; green CI and the author's own report no -longer count as one. Nine PRs had merged on 2026-10-13 without one -([#316](https://github.com/MendixMau/mxcli-project-toolkit/pull/316)). Several fixes in this -release came out of those reviews (the exec lock race, the final-LOOK timing, the -`test-stack-up` exit codes). - -**For you:** what reaches your project on `git pull` has been read by a second reviewer, not just -tested by its author. - -### Less text in every session - -The always-loaded skill tier had reached 79,997 words against an 80,000-word cap. Incident stories -moved behind pointers into `process/baseline-incident-history.md` and the tier dropped to 76,259 -words with no rule removed ([#315](https://github.com/MendixMau/mxcli-project-toolkit/pull/315)). -New rule: room is made by moving text, never by deleting rules. `change-intake.md` became a -331-word core with an on-demand detail file -([#327](https://github.com/MendixMau/mxcli-project-toolkit/pull/327)), dispatch guidance moved to -its own small file ([#320](https://github.com/MendixMau/mxcli-project-toolkit/pull/320)), a -scaffolded `CLAUDE.local.md` is 17% shorter -([#292](https://github.com/MendixMau/mxcli-project-toolkit/pull/292)), routing in -`CLAUDE.local.md` carries triggers only -([#226](https://github.com/MendixMau/mxcli-project-toolkit/pull/226)), helper stubs read about 18% -less per dispatch ([#223](https://github.com/MendixMau/mxcli-project-toolkit/pull/223)), and a -build helper can get one context-pack file per build step instead of a reading list -([#198](https://github.com/MendixMau/mxcli-project-toolkit/pull/198)). - -**For you:** run `bin/sync-project.sh` and existing projects pick up the shorter wiring; sessions -start with less to read. - -### `exec.sh`: per-project lock, restart verdicts, safer restarts - -In a 20-task field run, 57 of 148 `exec.sh` runs applied nothing, and a machine-wide guard blocked -writes because a *different* project was writing. `exec.sh` now locks per project with a stale-lock -takeover, offers an opt-in `--commit-first`, and ends every applied script with a verdict: restart -needed, watch can apply this, or unknown -([#304](https://github.com/MendixMau/mxcli-project-toolkit/pull/304), -[#324](https://github.com/MendixMau/mxcli-project-toolkit/pull/324)). The review of #304 found the -takeover could let up to three contenders hold the lock; it is now serialised, and a fixture runs -10 rounds of 12 contenders and asserts at most one holder. New `bin/app-restart.sh` (#304) restarts only -this project's own `mxcli run --local`; since -[#322](https://github.com/MendixMau/mxcli-project-toolkit/pull/322) it also refuses over a leftover `mxcli test` injection. - -**For you:** two projects on one machine no longer block each other, and the agent is told when a -change needs an app restart instead of guessing. - -### Lint you can trust and tune - -UX001/002 plus Batches 1–4 landed: UX001/UX002, then ERR001/LOOP001, PERF001/002/004, -TXT001/TXT002/UI001, and Batch 4 with **SEC014** (an external role that can write status, audit or ownership members) and a -CONV009 that reads its `maxObjects` option -([#238](https://github.com/MendixMau/mxcli-project-toolkit/pull/238), -[#248](https://github.com/MendixMau/mxcli-project-toolkit/pull/248), -[#270](https://github.com/MendixMau/mxcli-project-toolkit/pull/270), -[#305](https://github.com/MendixMau/mxcli-project-toolkit/pull/305), -[#312](https://github.com/MendixMau/mxcli-project-toolkit/pull/312)). When `mxcli init` reverts -toolkit rules to stock, the lint gate now stops as **BLIND** instead of blaming the next script -for the jump, and sync restores them -([#308](https://github.com/MendixMau/mxcli-project-toolkit/pull/308)). Turning a rule off has one -supported route (`enabled: false` in `lint-config.yaml`, recorded in `PROJECT.md`). A metrics -ledger records outcome, speed, lint total and `mxcli report` score for every exec -([#265](https://github.com/MendixMau/mxcli-project-toolkit/pull/265)). - -**For you:** SEC014 asks one question (which roles are external) and prints a NOTE until you -answer it. - -### End-to-end proof that fails for the right reason - -The journey runner preflights the suite (a refused admin credential aborts before the first -journey) and journeys can declare `requires` / `leaves` / `consumes` -([#302](https://github.com/MendixMau/mxcli-project-toolkit/pull/302)). OQL rows are read by SELECT -alias, an app that dies mid-suite reads APP-DOWN rather than FAIL, and negative-only claims get a -liveness reference ([#310](https://github.com/MendixMau/mxcli-project-toolkit/pull/310)). -`test-stack-up.sh` identifies this project's own `run --local` by its command line -([#319](https://github.com/MendixMau/mxcli-project-toolkit/pull/319)). Re-runs are scoped to what -a change touched, UI-only changes get a light lane, and unit tests are off by default -([#290](https://github.com/MendixMau/mxcli-project-toolkit/pull/290)). The page-fidelity scorer -matches what a page displays, on whole words -([#318](https://github.com/MendixMau/mxcli-project-toolkit/pull/318)). - -**For you:** fewer false reds from the harness, and less time re-proving changes nobody touched. -**Check after upgrading:** `oqlScalar` now returns the first SELECT alias. - -### Stage 3 designs how users reach each page - -Stage 3 now writes `design/interaction-model.md` before the first wireframe: a home per role, an -inbox per user task, and for each page how it is reached and what it receives. Every wireframe -opens with a contract block ([#317](https://github.com/MendixMau/mxcli-project-toolkit/pull/317), -part 1 of [#291](https://github.com/MendixMau/mxcli-project-toolkit/issues/291)). Three apps had -passed every gate while one rooted 8 pages on a parameterless retrieve and gave all 6 roles one -home page. - -**For you:** navigation and page-parameter defects surface at design time, when they are cheap. - -### Changing an existing app: delivery rounds and mid-build changes - -In change-an-existing-app mode, `bin/status.sh` reports the delivery round from a `## Rounds` -table in `PROJECT.md`, drops noise that does not apply there, and counts targeted manual Studio -Pro steps written from `templates/manual-steps.md` -([#326](https://github.com/MendixMau/mxcli-project-toolkit/pull/326)). In every mode, feedback -typed mid-build is logged as a change-log row in `PROJECT.md` before any edit, and the -requirement or design is changed before the MDL -([#306](https://github.com/MendixMau/mxcli-project-toolkit/pull/306)). A final whole-app LOOK is -now owed at Stage 6 ([#303](https://github.com/MendixMau/mxcli-project-toolkit/pull/303), -[#321](https://github.com/MendixMau/mxcli-project-toolkit/pull/321)). - -**For you:** "where are we" has an answer in rounds, not a stage number that never moves. - -## Upgrading +and changes to a live app, or a la carte to audit and test an app you already have. + +1.0 is the first named milestone on top of the release train. Its cut is tagged `v2026.10.19`, +the first tag published to GitHub, and the GitHub Release on that tag is titled "1.0". Earlier +dated headings in the CHANGELOG were never pushed as tags. All 1.0 work merged into the +`release/v1.0` branch while master stayed frozen; that branch then merged to master, where the +cut and its tag were made. Every PR named below has merged, some through the integration batch +[#329](https://github.com/MendixMau/mxcli-project-toolkit/pull/329). Every line traces to an entry in +[`CHANGELOG.md`](../../CHANGELOG.md) or to a linked PR. How milestones relate to tags: +[`docs/RELEASE-PROCESS.md`](../RELEASE-PROCESS.md) §8. + +## 1. Performance improvements + +### Does the toolkit help? + +`tests/behaviour/` gives the mdl helper the same drafting tasks with the toolkit's rules and +without them ([#389](https://github.com/MendixMau/mxcli-project-toolkit/pull/389), +`tests/behaviour/results/2026-10-15-*.tsv`). 5 runs per arm is a small sample. Without the +toolkit, no run refused a planted bad binding (a page task that asks to bind a system attribute, +AutoChangedBy): 0 of 5, against 5 of 5 with it. No page draft passed `mxcli check` (0 of 5, +against 2 to 3 of 5), 1 of 5 microflow drafts passed (against 5 of 5), and 2 of 5 committed +inside a loop (against 0 of 5). Tool calls were not compared. The runs without the toolkit still +inherited the host project's own CLAUDE.md (a generic mxcli guide), so this compares the +toolkit's rules against an ordinary mxcli setup, not against no guidance. Every PR that changes +always-loaded text or an agent stub is to run the suite base against head before merge, and the +without-toolkit arm reruns at each release cut. + +### Testing at key moments + +A twenty-task UX-fix retro measured what testing every change costs +([#274](https://github.com/MendixMau/mxcli-project-toolkit/issues/274)): UI-only changes were +re-proven 5 to 10 times (their own journey, the regression journeys of 4 to 5 other modules, +reviewer re-runs), a full `mxcli test` took about 6 minutes at every batch close, and 143 minutes +of unit tests found stale strings and credential problems, not logic bugs. +The toolkit now runs tests at fixed moments and keeps a change small. Per change, the build gate +plus what the change touched. The full run happens once when the build is complete and once +before every later deploy. Unit tests are off by default and switched on only where logic needs them +([#290](https://github.com/MendixMau/mxcli-project-toolkit/pull/290), +[#417](https://github.com/MendixMau/mxcli-project-toolkit/pull/417)). They are chosen per flow: a module brief lists the +`Unit flows:` worth a test, one line each with a reason, and `affected-tests.sh` runs only the +listed flows a change touched. That follows the maintainer's policy that unit-testing everything +is overkill: logic and integrations get unit tests, and UI journeys catch the rest +([#435](https://github.com/MendixMau/mxcli-project-toolkit/pull/435), [#436](https://github.com/MendixMau/mxcli-project-toolkit/pull/436)). + +| Moment | What runs | +|---|---| +| Every change (build and feedback) | The gate, plus what the change touched | +| Module done | That module's journeys, plus its listed unit flows | +| Build complete | Everything, once | +| Before every later deploy | Everything, once again | +| You ask | Everything | + +"What the change touched" comes from the script itself: `./bin/affected-tests.sh ` lists +the unit tests and journeys a change owes, from the elements the script writes and the flows +that call them +([#416](https://github.com/MendixMau/mxcli-project-toolkit/pull/416)). On three real change +scripts in a five-app project it selected 0 to 1 of 20 journey files and 0 to 1 unit test files +per change. It is advisory and exits 0. It also reports `journeys unmapped N of M` for journey +files that name-based selection cannot reach (12 of the 20 held no qualified name), so an empty +selection is never read as "nothing owed". `exec.sh` prints the `affected tests:` hint on every +MDL run. The first call takes about 19 seconds (it builds the mxcli catalog), later calls about 3 +seconds. + +### Other speed items + +- **A pasted error finds its bug-ledger entry.** `bin/bug-lookup.sh` now ranks ledger entries + against the whole pasted error message, offline and with no embeddings. On a held-out set + written before the ranker ran, the right entry is in the top 3 for 100% of queries (12 + positives), up from 0% for the old keyword match; on the development set, which was also used + for tuning, 94%, up from 9%. It can still give a confident wrong answer: 3 of 18 held-out + queries, all errors with no CE code and no ledger entry, and 3 of 40 development queries. A + better no-match test is a follow-up + ([#395](https://github.com/MendixMau/mxcli-project-toolkit/pull/395)). +- **Where an exec spends its time.** The run summary and `docs/BUILD-LOG.md` now carry per-step + seconds (`steps: guards Ns check Ns snapshot Ns baseline Ns exec Ns gate Ns lint Ns score Ns`), + and a run whose gate fails keeps its gate seconds. Those failed or rolled-back runs were 47 of + 238 rows in the measured project and its costliest 20%. The timing costs about 0.2 seconds per + exec, 0.5 to 0.7% of a 28 to 41 second exec + ([#413](https://github.com/MendixMau/mxcli-project-toolkit/pull/413)). +- **Fewer build rows.** A `BUILD` row is now one use case, and a script is the rollback unit. + Two Stage 4 runs on the same requirements had produced 137 and 246 rows (2.5 rows per use + case); build cost followed the row count, 11 versus 25 model calls per row and about 11 versus + about 40 hours. A Stage 4 A/B on two fixture modules (22 use cases) gave 1.05 rows per use case + with the new text and 2.27 to 2.45 with the old. +- **Stage 5 drafts in parallel.** Every build-plan row whose dependencies are applied, minus rows + that write the same document, is drafted at once, one helper per script. Applies stay one at a + time through `bin/exec.sh`, and the ready set is recomputed after each + ([#343](https://github.com/MendixMau/mxcli-project-toolkit/pull/343)). +- **Less to read at the start.** The always-loaded skill tier went from 79,997 to 76,259 words + with no rule removed + ([#315](https://github.com/MendixMau/mxcli-project-toolkit/pull/315)). A helper reads only the + baseline rows for its own role. The mdl helper at Stage 5 went from 56,236 to 43,599 words + (the runbook slice for its stage) and then to 30,647 (a routing row on demand for one role) + ([#385](https://github.com/MendixMau/mxcli-project-toolkit/pull/385)). + Routing tables are trigger lists, so a session no longer re-reads the whole runbook (about 24k + tokens) on every later call + ([#183](https://github.com/MendixMau/mxcli-project-toolkit/pull/183)). At release the + always-loaded tier stands at 73,682 words (`bin/render-routing.sh --check`). +- **The app map refreshes only what changed.** On a 17,500-unit app the loop sweep re-described + every loop-containing microflow: 1,917 describes in 2 hours 45 minutes at 6 workers, while about + 80 units had changed between rounds. It now describes only changed units. On a synthetic + 2,000-unit app with 1% changed that is 20 describes instead of 2,000, and `bin/status.sh` shows + how stale the map is. No field run of the incremental sweep yet. +- **The UI loop skips a redundant build and waits on the page.** `exec.sh` ran two mxbuilds per + exec. It now skips the baseline mxbuild when the model is byte-identical to its last clean pass. + Journeys and the page audit wait on the page's own state instead of a fixed sleep: a fast page + settles in 775 ms instead of 1,200, and a page with a 1.8 second progress bar is no longer + captured half-loaded. In the field run that prompted it, `exec.sh` took 22% of tool time + ([#220](https://github.com/MendixMau/mxcli-project-toolkit/pull/220)). +- **Helpers are dispatched on purpose.** In a measured 20-task run, 15 of 21 top-tier helpers + were on that tier only because they inherited the lead's model: 7.2% of all tokens. + `agent-roles.md` now says to name the model tier on every call, start scans as fresh agents + instead of forks, give a fix round to a fresh fixer, and batch notices + ([#292](https://github.com/MendixMau/mxcli-project-toolkit/pull/292)). +- **Nine routing triggers that never fired are fixed.** An audit ranked 55 labelled phrasings + against each routing row's trigger. The right row was in the top 3 for 46 of 55. Nine + triggers now start with the words people type, and the right row is in the top 3 for 55 of 55 + ([#391](https://github.com/MendixMau/mxcli-project-toolkit/pull/391), audit in + [#394](https://github.com/MendixMau/mxcli-project-toolkit/pull/394)). +- **Context packs** cut helper tool calls and time; see section 3. + +## 2. Settings you control + +These are the settings a project owner controls: what runs and when, how much the agent +asks, and what it is allowed to touch. They survive `git pull` and `bin/sync-project.sh`. + +| What you want | The line or command | Where it lives | +|---|---|---| +| Unit tests across the project | `Unit tests default: no` (the default) or `yes` | `PROJECT.md`, intake Q13 | +| Unit tests for one module | Set the Unit row of the module brief's Test plan to `yes` and name why: `\| Unit (.test.mdl) \| yes \| \|` | `architecture/modules//module-brief.md`, `skills/module-brief.md` | +| What runs at each moment | `## Test strategy` table, offered at CAC-5 (architecture and offer, before the build). Status `CONFIRMED`, or `ASSUMED` in auto mode. Change any cell in your answer to the offer | `PROJECT.md`, `checkpoint-build.md` | +| Everything, now | Say "run everything". The full suite runs once on request | in chat | +| Which flows get a unit test | List them in the module brief as `Unit flows:`, one `- ACT_X: reason` line each. A list wins for its module even under `Unit tests default: yes` | `architecture/modules//module-brief.md` | +| Per-change test selection depth | `./bin/affected-tests.sh \n\n'; diff --git a/project-bin/build-plan-status.sh b/project-bin/build-plan-status.sh index 779ae491..b9530361 100755 --- a/project-bin/build-plan-status.sh +++ b/project-bin/build-plan-status.sh @@ -28,13 +28,22 @@ # --html and --json outputs and never touches the .mpr. # # Usage: -# build-plan-status.sh [project-dir] [--html] [--json] [--quiet] [--refresh] (flags and path in any order) -# --html also write architecture/build-plan.html (self-contained, no external deps): the +# build-plan-status.sh [project-dir] [--html] [--json] [--quiet] [--refresh] [--open] (flags and path in any order) +# --html also write architecture/progress-report.html (self-contained, no external deps), +# which replaced architecture/build-plan.html: stages P to 6, time and token data +# when a Claude Code session log exists (project-bin/progress-report.py), and the # plan itself (phases, rows, claims from build-plan.md, see "P." below) followed -# by views A and B +# by views A and B. An existing build-plan.html is left alone. +# --session-log PATH use this Claude Code session log (a .jsonl file or a folder of them; repeatable) instead of +# discovery, for a session started outside the project folder. Passed to progress-report.py. # --refresh the unattended form of --html, for exec.sh and gate-check.sh: quiet, writes only -# when build-plan.md exists, never over a build-plan.html this script did not -# write, and leaves the file untouched when nothing but the timestamp would change +# when build-plan.md exists, never over a progress-report.html this script did not +# write, and leaves the file untouched when nothing but the stamp lines would change +# It also keeps an EXISTING architecture/build-plan.json current (same parser as +# --json, same no-rewrite-for-a-timestamp rule) and never creates one +# --open --html, then show it: on macOS reload the Chrome tab whose URL contains +# progress-report.html, else open the file once; elsewhere a no-op +# PROJECT.md `Progress report: off | ask | auto` (default ask): off = never write the page # --json also write architecture/build-plan.json, parsed from architecture/build-plan.md's # Step-5 Phase headings, row tables and claims: blocks (see "C." below) # --quiet suppress the stdout table (useful when only --html/--json output is wanted) @@ -53,9 +62,15 @@ WRITE_HTML=0 WRITE_JSON=0 QUIET=0 REFRESH=0 +OPEN=0 +SESSION_LOGS=() +_want_log=0 for a in "$@"; do + if [ "$_want_log" -eq 1 ]; then SESSION_LOGS+=("$a"); _want_log=0; continue; fi case "$a" in + --session-log) _want_log=1 ;; --html) WRITE_HTML=1 ;; + --open) WRITE_HTML=1; OPEN=1 ;; --refresh) WRITE_HTML=1; REFRESH=1; QUIET=1 ;; --json) WRITE_JSON=1 ;; --quiet) QUIET=1 ;; @@ -63,9 +78,11 @@ for a in "$@"; do *) if [ -z "$ROOT" ]; then ROOT="$a"; else echo "unexpected second path: $a (first was $ROOT)" >&2; exit 2; fi ;; esac done +[ "$_want_log" -eq 0 ] || { echo "--session-log needs a path" >&2; exit 2; } [ -n "$ROOT" ] || ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)" cd "$ROOT" || exit 0 +ROOT="$(pwd)" # a relative project path must survive the cd (the page is written under $ROOT) MDLSOURCE="$ROOT/mdlsource" MODULES_DIR="$ROOT/architecture/modules" @@ -73,6 +90,24 @@ REGISTER="$ROOT/docs/improvement-register.md" BUILD_PLAN="$ROOT/architecture/build-plan.md" STAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)" +# PROJECT.md `Progress report: off | ask | auto` (default ask). off = this script never writes the +# page; ask and auto only change what the agent does at a stage gate or module close (the skills +# say: ask once, or open it), so the script treats them the same. +report_name() { # the page name lives in progress-report.py (REPORT_FILE); asked once, here + REPORT_FILE="$("${PY:-python3}" "$(dirname "${BASH_SOURCE[0]}")/progress-report.py" --print-name 2>/dev/null)" # portability-ok: PY is resolve_py output; python3 only if unset + [ -n "$REPORT_FILE" ] || REPORT_FILE="progress-report.html" +} +progress_report_setting() { + _v="$(grep -iE '^[[:space:]>*_-]*Progress report[[:space:]]*:' "$ROOT/PROJECT.md" 2>/dev/null | head -1 \ + | sed -E 's/^[^:]*:[[:space:]*_`]*//' | tr 'A-Z' 'a-z')" + case "$_v" in + *"|"*) echo ask ;; # the option list pasted verbatim is not a choice + off*) echo off ;; + auto*) echo auto ;; + *) echo ask ;; + esac +} + # ── A. Build-plan phase progress, from mdlsource//done- counts ─────── # Phase folders are read as they exist on disk, not matched against build-plan.md's prose Phase # headers — a fuzzy name-match would be a guess, and skills-over-scripts.md says code carries no @@ -265,18 +300,27 @@ def column_map(headers): m[key] = i return m if "number" in m and "state" in m else None +# Same prefixes as classify_state in progress-report.py, so the phase card, the plan header and +# the phase badge give one count. brd-to-build-plan.md lists `built` / `partial` / `not built` / +# `blocked`; the rest are the spellings the card already counts. `blocked` counts as not built. +# A "not needed" style row is out of the count and out of the rollup. None = cannot be read. def classify(state): - s = state.lower() - if s.startswith("not built"): return "not built" - if s.startswith("built"): return "built" + s = state.lower().replace("**", "").strip() if s.startswith("pending a person"): return "pending a person" + if s.startswith(("blocked",)): return "blocked" + if s.startswith(("not built", "not done", "not started", "planned", "to do", "todo", "open", "pending", "failed")): return "not built" + if s.startswith(("not needed", "n/a", "dropped", "skipped", "waived", "deferred")): return "skipped" + if s.startswith(("built", "done", "complete", "verified", "proven", "passed")): return "built" + if s.startswith(("partial", "in progress", "started", "wip")): return "partial" return None def rollup(steps): kinds = [classify(s["state"]) for s in steps] - if not kinds or None in kinds: return "unknown" + if None in kinds: return "unknown" + kinds = [k for k in kinds if k != "skipped"] + if not kinds: return "unknown" if all(k == "built" for k in kinds): return "built" - if all(k == "not built" for k in kinds): return "not built" + if all(k in ("not built", "blocked") for k in kinds): return "not built" if all(k in ("built", "pending a person") for k in kinds): return "pending a person" return "in progress" @@ -425,6 +469,20 @@ if [ "$WRITE_JSON" -eq 1 ] || [ "$WRITE_HTML" -eq 1 ]; then case $? in 0) PLAN_JSON="$PLAN_TARGET" JSON_MSG="wrote ${JSON_OUT#$ROOT/}" + if [ "$REFRESH" -eq 1 ] && [ "$WRITE_JSON" -eq 0 ] && [ -f "$JSON_OUT" ]; then + # --refresh keeps an EXISTING build-plan.json current (never creates one): exec.sh and + # gate-check.sh run --refresh after every build step, so a json written once at Stage 4 + # would otherwise read 0 built after a day of building. Same parser as --json; the + # write is skipped when only generatedAt would differ, like the html below. + if "$PY" -c 'import json,sys +def load(f): + d=json.load(open(f,encoding="utf-8")); d.pop("generatedAt",None); return d +sys.exit(0 if load(sys.argv[1])==load(sys.argv[2]) else 1)' "$JSON_OUT" "$PLAN_TARGET" >/dev/null 2>&1; then + JSON_MSG="${JSON_OUT#$ROOT/} already current" + else + cp -f "$PLAN_TARGET" "$JSON_OUT" 2>/dev/null && JSON_MSG="refreshed ${JSON_OUT#$ROOT/}" + fi + fi if [ "$WRITE_JSON" -eq 1 ]; then # A producer guarantees its own output is ignored (snapshot-mpr.sh's rule). The JSON # is a build product: regenerated on demand in under a second, derived entirely from @@ -507,19 +565,23 @@ def on_disk(pat): if any(fnmatch.fnmatchcase(f, pat) for f in files): return "written" return "missing" -def classify(state): # the parser's rollup reading, per row - s = (state or "").lower() - if s.startswith("not built"): return "not built" - if s.startswith("built"): return "built" +def classify(state): # the parser's rollup reading, per row (keep in step with classify in parse_plan) + s = (state or "").lower().replace("**", "").strip() if s.startswith("pending a person"): return "pending a person" + if s.startswith("blocked"): return "blocked" + if s.startswith(("not built", "not done", "not started", "planned", "to do", "todo", "open", "pending", "failed")): return "not built" + if s.startswith(("not needed", "n/a", "dropped", "skipped", "waived", "deferred")): return "skipped" + if s.startswith(("built", "done", "complete", "verified", "proven", "passed")): return "built" + if s.startswith(("partial", "in progress", "started", "wip")): return "partial" return None CLS = {"built": "st-built", "not built": "st-todo", "pending a person": "st-person", + "partial": "st-progress", "blocked": "st-todo", "skipped": "st-todo", "in progress": "st-progress", "unknown": "st-unknown", None: "st-unknown"} phases = d.get("phases", []) all_steps = [s for p in phases for s in p["steps"]] -tally = {"built": 0, "not built": 0, "pending a person": 0, None: 0} +tally = {"built": 0, "not built": 0, "pending a person": 0, "partial": 0, "blocked": 0, "skipped": 0, None: 0} for s in all_steps: tally[classify(s["state"])] += 1 named_total = named_done = drift_rows = 0 @@ -558,9 +620,12 @@ for p in phases: plural = lambda n, w: "%d %s%s" % (n, w, "" if n == 1 else "s") print("

The plan

") print("

From architecture/build-plan.md: %s, %s: " - "%d built, %d not built, %d pending a person%s.

" % ( + "%d built, %d partial, %d not built%s, %d pending a person%s%s.

" % ( plural(len(phases), "phase"), plural(len(all_steps), "row"), - tally["built"], tally["not built"], tally["pending a person"], + tally["built"], tally["partial"], tally["not built"] + tally["blocked"], + (" (%d of them blocked)" % tally["blocked"]) if tally["blocked"] else "", + tally["pending a person"], + (", %d not needed" % tally["skipped"]) if tally["skipped"] else "", (", %d with a State cell this page cannot read" % tally[None]) if tally[None] else "")) if named_total: print("

Scripts named in the plan: %d, of which %d are renamed done- under mdlsource/.%s

" % ( @@ -598,51 +663,38 @@ if w: PYEOF } +# ── Progress report page ────────────────────────────────────────────────── +# architecture/progress-report.html replaced build-plan.html. The page is rendered by +# progress-report.py (stages P to 6, session time and token data when a Claude Code log exists); +# this script contributes the part only it can read: the plan tables and views A and B, written +# to a scratch fragment above and embedded as the page's build detail. An older project's +# architecture/build-plan.html is left exactly where it is: not read, not rewritten, not deleted. if [ "$WRITE_HTML" -eq 1 ]; then - OUT="$ROOT/architecture/build-plan.html" SKIP_HTML="" - if [ "$REFRESH" -eq 1 ]; then + GEN="$(dirname "${BASH_SOURCE[0]}")/progress-report.py" + report_name + OUT="$ROOT/architecture/$REPORT_FILE" + SETTING="$(progress_report_setting)" + if [ "$SETTING" = "off" ]; then + SKIP_HTML="PROJECT.md says Progress report: off" + elif [ "$REFRESH" -eq 1 ] && [ ! -f "$BUILD_PLAN" ]; then # Unattended callers (exec.sh, gate-check.sh) get gate-check.sh's index.html rules: no plan, - # no page; and never over a page this script did not write (before 2026-08-19 agents wrote - # this file by hand, and a project may still carry one). Every render from this script names - # it in the stamp line, so older renders are recognised and adopted. - if [ ! -f "$BUILD_PLAN" ]; then - SKIP_HTML="no build-plan.md" - elif [ -f "$OUT" ] && ! grep -qF 'build-plan-status.sh' "$OUT"; then - SKIP_HTML="architecture/build-plan.html was not written by this script" - fi + # no page; and never over a page this script did not write. + SKIP_HTML="no build-plan.md" + elif [ -f "$OUT" ] && ! grep -qF 'build-plan-status.sh' "$OUT"; then + SKIP_HTML="architecture/$REPORT_FILE was not written by this script" + elif [ -z "${PY:-}" ]; then + SKIP_HTML="no working Python 3 found (tried python3, python, py), which the report needs" # portability-ok: names in a diagnostic + elif [ ! -f "$GEN" ]; then + SKIP_HTML="progress-report.py is not next to this script (run bin/sync-project.sh)" fi if [ -n "$SKIP_HTML" ]; then - [ "$QUIET" -eq 0 ] && echo "" && echo " build-plan.html not written: $SKIP_HTML" + [ "$QUIET" -eq 0 ] && echo "" && echo " $REPORT_FILE not written: $SKIP_HTML" else mkdir -p "$ROOT/architecture" OUT_TMP="$OUT.tmp.$$" + FRAG_TMP="$OUT.frag.$$" { - echo '' - echo '' - echo 'Build Plan' - echo '
' - echo "

Build Plan

Generated $STAMP by project-bin/build-plan-status.sh from architecture/build-plan.md, mdlsource/ and the module review files. exec.sh and gate-check.sh regenerate it; never hand-edit.

" - render_plan_section echo "

A. Script folders: how much is built

" @@ -672,17 +724,64 @@ details{margin:.5rem 0} summary{cursor:pointer;color:var(--muted);font-size:.85r echo "" fi echo "

The plan: what build-plan.md says. A: built, from mdlsource/ done- prefixes. B: proven, from verify-module.sh + docs/improvement-register.md. A phase at 100% with no reviewed row in B is built, not proven.

" - echo '
' - } > "$OUT_TMP" - # Two renders of the same state differ only in the stamp line. A refresh that changed nothing - # leaves the file alone, or every exec.sh run would dirty the working tree. - if [ "$REFRESH" -eq 1 ] && [ -f "$OUT" ] \ - && [ "$(grep -v 'id=stamp' "$OUT")" = "$(grep -v 'id=stamp' "$OUT_TMP")" ]; then + } > "$FRAG_TMP" + GEN_ARGS=(--out "$OUT_TMP" --fragment "$FRAG_TMP" --plan-msg "${PLAN_MSG:-}") + [ -n "$PLAN_JSON" ] && GEN_ARGS+=(--plan-json "$PLAN_JSON") + for _sl in ${SESSION_LOGS[@]+"${SESSION_LOGS[@]}"}; do GEN_ARGS+=(--session-log "$_sl"); done + if "$PY" "$GEN" "$ROOT" "${GEN_ARGS[@]}" >/dev/null 2>"$OUT.err.$$"; then + # Two renders of the same state differ only in the stamp lines. A refresh that changed + # nothing leaves the file alone, or every exec.sh run would dirty the working tree. + if [ "$REFRESH" -eq 1 ] && [ -f "$OUT" ] \ + && [ "$(grep -v 'id=stamp' "$OUT")" = "$(grep -v 'id=stamp' "$OUT_TMP")" ]; then + rm -f "${OUT_TMP:?}" + else + mv -f "$OUT_TMP" "$OUT" + fi + [ "$QUIET" -eq 0 ] && echo "" && echo " wrote ${OUT#$ROOT/}" + else rm -f "${OUT_TMP:?}" + _why="$(tail -n 1 "$OUT.err.$$" 2>/dev/null | head -c 200)" + if [ "$QUIET" -eq 0 ]; then echo "" && echo " $REPORT_FILE not written: the generator failed: $(head -c 300 "$OUT.err.$$" 2>/dev/null)" + else echo "WARN: $REPORT_FILE not updated, progress-report.py failed: ${_why:-no message}" >&2; fi # unattended callers stay quiet on success, never on a crash + fi + rm -f "${FRAG_TMP:?}" "${OUT}.err.$$" + fi +fi + +# ── --open: show the page ──────────────────────────────────────────────────── +# macOS with Chrome running: reload the tab showing this project's report (its exact file:// URL), never a new +# window when a tab exists. Otherwise open the file once. Anywhere else (Linux, Windows, no GUI) +# it does nothing and says so. The setting gates the write above; this only needs the file. +if [ "$OPEN" -eq 1 ]; then report_name; fi +if [ "$OPEN" -eq 1 ] && [ -f "$ROOT/architecture/$REPORT_FILE" ] && [ "$(progress_report_setting)" != "off" ]; then + if [ "$(uname -s)" = "Darwin" ]; then + _res="none" + if pgrep -x "Google Chrome" >/dev/null 2>&1; then # portability-ok: --open is macOS-only by design (guarded by uname) + # Match this project's file URL exactly (file:// + absolute path, percent-encoded), never any tab that + # merely contains the page name: another project's report must not be reloaded in its place. + _url="file://$("${PY:-python3}" -c 'import sys,urllib.parse;print(urllib.parse.quote(sys.argv[1]))' "$ROOT/architecture/$REPORT_FILE" 2>/dev/null)" # portability-ok: PY is resolve_py output + _as='tell application "Google Chrome" +repeat with w in windows +repeat with t in tabs of w +set u to URL of t +if u is "'"$_url"'" or u starts with "'"$_url"'#" or u starts with "'"$_url"'?" then +reload t +return "reloaded" +end if +end repeat +end repeat +end tell +return "none"' + _res="$(osascript -e "$_as" 2>/dev/null || echo none)" # portability-ok: --open is macOS-only by design (guarded by uname) + fi + if [ "$_res" = "reloaded" ]; then + [ "$QUIET" -eq 0 ] && echo " reloaded the open $REPORT_FILE tab" else - mv -f "$OUT_TMP" "$OUT" + open "$ROOT/architecture/$REPORT_FILE" >/dev/null 2>&1 || true # portability-ok: --open is macOS-only by design (guarded by uname) + [ "$QUIET" -eq 0 ] && echo " opened architecture/$REPORT_FILE" fi - [ "$QUIET" -eq 0 ] && echo "" && echo " wrote ${OUT#$ROOT/}" + else + [ "$QUIET" -eq 0 ] && echo " --open does nothing outside macOS; open architecture/$REPORT_FILE yourself" fi fi diff --git a/project-bin/exec.sh b/project-bin/exec.sh index 352f44ea..fa1390b7 100755 --- a/project-bin/exec.sh +++ b/project-bin/exec.sh @@ -437,6 +437,8 @@ BUILD_LOG="$PROJECT_ROOT/docs/BUILD-LOG.md" GATE_STATE="not-run" # not-run | skipped | unverified | pass | fail GATE_ERRORS="?" # mxbuild error count behind a pass (0 = clean); recorded in the stamp MXB_WHY="" # mxbuild's errors[] reason when it exited without checking the model +MXB_LOG_NOTE="" # "mxbuild log: " when the gate's full output was saved (BUILD-LOG detail) +GATE_NOTE="" # "retry after infra exit" when a gate mxbuild was run twice # ── Exec approval (auto records itself) ────────────────────────────────────── # bin/exec-approval.sh decides ask vs auto; the ASKING happens before exec.sh ever runs (in @@ -490,7 +492,7 @@ HDR2 # The gate cell is read from GATE_STATE rather than passed in, so no caller # can omit it — a blank cell in this table reads as "fine", which is the exact # false-green the gate exists to prevent. - _detail="$2${CHANGE_CLASS:+ · change: $CHANGE_CLASS}${CR_ID:+ · cr: $CR_ID}${SCAN_NOTE:+ · $SCAN_NOTE}${HV_NOTE:+ · $HV_NOTE}$_EXEC_APPROVAL_SUFFIX${STEP_TIMES:+ · steps: $STEP_TIMES}" + _detail="$2${CHANGE_CLASS:+ · change: $CHANGE_CLASS}${CR_ID:+ · cr: $CR_ID}${SCAN_NOTE:+ · $SCAN_NOTE}${GATE_NOTE:+ · $GATE_NOTE}${MXB_LOG_NOTE:+ · $MXB_LOG_NOTE}${HV_NOTE:+ · $HV_NOTE}$_EXEC_APPROVAL_SUFFIX${STEP_TIMES:+ · steps: $STEP_TIMES}" printf '| %s | `%s` | %s | %s | %s |\n' \ "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$LOG_NAME" \ "${GATE_STATE:-not-run}" "$1" "$_detail" >> "$BUILD_LOG" @@ -708,7 +710,8 @@ if [ "$LOCK_TAKEN" != 1 ]; then echo " → Wait for the other exec to finish, then re-run. Nothing was written." exit 1 fi -trap 'exec_lock_release' EXIT +trap 'mxtk_resume_watchers; exec_lock_release' EXIT +mxtk_resume_leftover "$PROJECT_ROOT" # a watcher an earlier, SIGKILLed exec.sh left stopped # 3. A raw `mxcli exec` elsewhere on the machine is NOT this project's business. The old guard # here (`pgrep -fl "mxcli exec"`) matched every exec on the machine, so a test run on a @@ -960,6 +963,30 @@ run_lint_gate() { # row at all (measured on the first wired run in a real project, 2026-09-24). Capture explicitly. LINT_RC=0 bash "$PROJECT_ROOT/bin/lint-gate.sh" >"$_lo" 2>&1 || LINT_RC=$? + # Self-heal a re-seed (#461). mxcli's SessionStart hook runs `mxcli init --sync-skills`, which + # rewrites its bundled lint rules from the binary on every compaction and subagent start, with + # no opt-out; the toolkit's fixed rules revert to stock and lint-gate reports BLIND. Failing + # blind there only sent the user to sync-project.sh. The installer's own classification decides + # what is safe (stock or receipt-matching copies are restored; a hand edit is never lost), + # so run it, then lint once more. A rule it will not restore stays BLIND, as before. + _restored="" + if [ "$LINT_RC" -eq 1 ] && [ "${MXTK_NO_LINT_RESTORE:-}" != "1" ] \ + && grep -q '^BLIND — lint rules reverted to stock' "$_lo"; then + _tk="" + type find_toolkit_root >/dev/null 2>&1 && _tk="$(find_toolkit_root 2>/dev/null || true)" + if [ -n "$_tk" ] && [ -f "$_tk/bin/lib/install-lint-rules.sh" ]; then + _restored=$(sed -n '/^!! LINT RULES REVERTED TO STOCK/,/^$/p' "$_lo" \ + | grep -oE '^ +[A-Za-z0-9_.-]+\.star:' | tr -d ' :' | paste -sd, -) + echo " ↻ lint rules reverted to stock (mxcli's session-start refresh?): restoring the toolkit versions from $_tk" + # Capture to a file first: piping the installer into head can stop it after 12 lines of output. + _io="$(mktemp)" + ( . "$_tk/bin/lib/install-lint-rules.sh"; mxtk_install_lint_rules "$_tk" "$PROJECT_ROOT" 0 "" ) >"$_io" 2>&1 || true + head -12 "$_io" | sed 's/^/ /' + rm -f "$_io" + LINT_RC=0 + bash "$PROJECT_ROOT/bin/lint-gate.sh" >"$_lo" 2>&1 || LINT_RC=$? + fi + fi # lint-gate's non-blocking SEC014 note (#312: rule installed, decision not made) is otherwise # visible only on a failing run, because the capture above is shown only then. Always echo it. [ -f "$_lo" ] && { grep '^note: SEC014' "$_lo" 2>/dev/null | sed 's/^/ /' || true; } @@ -1001,6 +1028,9 @@ run_lint_gate() { *) LINT_DETAIL="lint could not run (lint-gate exit $LINT_RC)"; echo " ⚠ $LINT_DETAIL" tail -5 "$_lo" | sed 's/^/ /' ;; esac + # Said in the row either way: a restored run is lint on the toolkit's rules, and the reader + # should know the files changed under them. + [ -n "$_restored" ] && [ "$LINT_RC" -ne 3 ] && LINT_DETAIL="$LINT_DETAIL; toolkit lint rules auto-restored after a stock re-seed (${_restored})" rm -f "$_lo" } # One place decides the applied-row wording, so both mxbuild-clean branches agree. @@ -1353,7 +1383,8 @@ if [ "${SKIP_BASELINE:-0}" != "1" ] && [ -x "$MXBUILD" ] && [ -x "$JAVA_EXE" ]; # can drift apart. Its own printed count is discarded here: this call site # already has err_set/err_count/err_codes on hand and needs BASELINE_SET # (the message set, not just a count) for the delta-gate comparison below. - mxtk_mxbuild_error_count "$MPR" "${MXTK_GATE_TIMEOUT:-0}" "$_BF" >/dev/null || true + mxtk_gate_mxbuild "$MPR" "${MXTK_GATE_TIMEOUT:-0}" "$_BF" >/dev/null || true + [ -n "$MXTK_GATE_RETRY_NOTE" ] && GATE_NOTE="$MXTK_GATE_RETRY_NOTE" # Studio Pro 11's mxbuild (its --help says so) writes --write-errors "only if the # project has errors": on a clean model the mktemp'd file stays EMPTY, json.load # fails, and this read "Model ALREADY has ? error(s) [?]" on every clean run @@ -1633,18 +1664,53 @@ if [ -x "$MXBUILD" ] && [ -x "$JAVA_EXE" ]; then # discarded here: this call site keeps its own file-presence/exit-code # branching below unchanged, computing CE_COUNT via err_count once it knows # which branch applies. - mxtk_mxbuild_error_count "$MPR" "${MXTK_GATE_TIMEOUT:-0}" "$ERRORS_FILE" >/dev/null || true + mxtk_gate_mxbuild "$MPR" "${MXTK_GATE_TIMEOUT:-0}" "$ERRORS_FILE" >/dev/null || true + [ -n "$MXTK_GATE_RETRY_NOTE" ] && GATE_NOTE="$MXTK_GATE_RETRY_NOTE" MXBUILD_EXIT="$MXTK_MXBUILD_EXIT" MXBUILD_OUT="$MXTK_MXBUILD_OUT" MXB_WHY="${MXTK_MXBUILD_WHY:-}" step_done gate + # Keep the gate's full mxbuild output when the gate did not pass. The screen shows only the + # top lines (the failure detail is often in the tail), and the temp file behind + # mxtk_mxbuild_error_count is gone by now, so without this the evidence of a failed gate was + # lost (a field migration project lost the detail of a CE1613 this way). mxbuild prints output + # on a clean build too, so the log is written from the failure branches below (mxb_log_hint) + # and never for a pass, a delta-gate pass over pre-existing errors included. The one exception + # is a retry: the first attempt's output is the evidence of the infra exit, so a retry always + # leaves a log, with both attempts in it. One file per run, under .mpr-snapshots/.mxbuild-logs/, + # the newest 30 kept. The DOT in the name is load-bearing: restore-mpr.sh (no argument picks + # the newest) and snapshot-mpr.sh (keeps 5) glob `.mpr-snapshots/*/`, which skips dot dirs. + # Its project-relative path goes on screen and in the row. + MXB_LOG_REL="" + mxb_save_log() { + [ -z "$MXB_LOG_REL" ] || return 0 + [ -n "$MXBUILD_OUT$MXTK_GATE_FIRST_OUT" ] || return 0 + _MXB_LOG_DIR="$PROJECT_ROOT/.mpr-snapshots/.mxbuild-logs" + MXB_LOG_REL=".mpr-snapshots/.mxbuild-logs/$(date +%Y%m%d-%H%M%S)-$$.log" + if mkdir -p "$_MXB_LOG_DIR" 2>/dev/null && { + if [ -n "$MXTK_GATE_RETRY_NOTE" ]; then + printf '=== attempt 1 (exit %s, infra failure, retried) ===\n%s\n=== attempt 2 (final, exit %s) ===\n%s\n' \ + "$MXTK_GATE_FIRST_EXIT" "$MXTK_GATE_FIRST_OUT" "$MXBUILD_EXIT" "$MXBUILD_OUT" + else + printf '%s\n' "$MXBUILD_OUT" + fi > "$PROJECT_ROOT/$MXB_LOG_REL"; } 2>/dev/null; then + MXB_LOG_NOTE="mxbuild log: $MXB_LOG_REL" + ls -t "$_MXB_LOG_DIR"/*.log 2>/dev/null | sed -n '31,$p' | while IFS= read -r _old; do rm -f "$_old"; done + else + MXB_LOG_REL="" + fi + } + mxb_log_hint() { mxb_save_log; [ -z "$MXB_LOG_REL" ] || echo " full mxbuild output: $MXB_LOG_REL"; } + if [ -n "$MXTK_GATE_RETRY_NOTE" ]; then mxb_save_log; fi + if [ -f "$ERRORS_FILE" ] && [ -s "$ERRORS_FILE" ]; then CE_COUNT=$(err_count "$ERRORS_FILE") if [ "$CE_COUNT" = "?" ]; then GATE_STATE="unverified" echo " ⚠ mxbuild errors file could not be parsed — check manually." echo "$MXBUILD_OUT" | grep -v "^$\|icon\|Microsoft\|Assembly\|__" | head -10 || true + mxb_log_hint elif [ "$CE_COUNT" = "0" ] && [ "$MXBUILD_EXIT" -ne 0 ]; then # Non-empty errors file, 0 Error problems, NON-ZERO exit: mxbuild never checked the # model. The reason sits in errors[], which err_count does not read, e.g. (verbatim, @@ -1658,6 +1724,7 @@ if [ -x "$MXBUILD" ] && [ -x "$JAVA_EXE" ]; then echo " ✗ mxbuild exited $MXBUILD_EXIT and listed 0 model problems — it did NOT check the model." echo " ${MXB_WHY:-no reason in errors[]; mxbuild output follows}" [ -z "$MXB_WHY" ] && { echo "$MXBUILD_OUT" | grep -v "^$\|icon\|Microsoft\|Assembly\|__" | tail -10 || true; } + mxb_log_hint echo " A version mismatch is fixed by the matching mxbuild: ./mxcli setup mxbuild -p $MPR_BASE" echo " (find_mxbuild prefers it once cached), or MXBUILD_PATH=." elif [ "$CE_COUNT" = "0" ]; then @@ -1707,6 +1774,7 @@ if [ -x "$MXBUILD" ] && [ -x "$JAVA_EXE" ]; then rm -f "$ERRORS_FILE" else GATE_STATE="fail" + mxb_save_log CE_CODES=$(err_codes "$ERRORS_FILE") read -r NEW_COUNT NEW_CODES </dev/null || true + mxtk_gate_mxbuild "$MPR" "${MXTK_GATE_TIMEOUT:-0}" "$BASE_ERRS" >/dev/null || true # Same empty-file rule as the pre-flight baseline: SP 11 mxbuild writes no # errors file on a clean build, and that plus exit 0 is a verified 0. if [ ! -s "$BASE_ERRS" ] && [ "$MXTK_MXBUILD_EXIT" -eq 0 ]; then @@ -1797,6 +1865,7 @@ EOF echo " ✗ mxbuild failed to run (exit $MXBUILD_EXIT) — gate could not verify the model." [ -n "$MXB_WHY" ] && echo " mxbuild says: $MXB_WHY" echo "$MXBUILD_OUT" | grep -v "^$\|icon\|Microsoft\|Assembly\|__" | head -20 || true + mxb_log_hint # mxbuild CRASHED ON THE WRITTEN MODEL (field 2026-10-13, Mendix 11.14.0, mxcli v0.25.0): a # two-hop data source passed `check`, then mxbuild's loader died with no errors file. This # branch kept the write, the next exec printed "Baseline NOT measured" and wrote on top of a @@ -1951,12 +2020,13 @@ if [ "$PATCH_MODE" = 0 ] && [ -f "$(dirname "$0")/look-ledger.sh" ]; then fi # ── Build plan page ────────────────────────────────────────────────────────── -# Refresh architecture/build-plan.html so it tracks the build without anyone remembering to +# Refresh architecture/progress-report.html so it tracks the build without anyone remembering to # (its only trigger used to be iterative-build-loop.md step 16, so a page written at Stage 4 # stayed empty for the whole build). --refresh is quiet, writes only over a page that script # generated, and leaves the file alone when nothing but the timestamp would change. if [ -f "$(dirname "$0")/build-plan-status.sh" ]; then - bash "$(dirname "$0")/build-plan-status.sh" "$PROJECT_ROOT" --refresh >/dev/null 2>&1 || true + # A crashed generator leaves one visible WARN line (the script prints it on stderr); never a failed step. + { bash "$(dirname "$0")/build-plan-status.sh" "$PROJECT_ROOT" --refresh 2>&1 >/dev/null || true; } | grep -m1 '^WARN: ' || true fi # ── Watch compatibility (advisory) ─────────────────────────────────────────── diff --git a/project-bin/lint-gate.sh b/project-bin/lint-gate.sh index e342b0bb..0aace996 100755 --- a/project-bin/lint-gate.sh +++ b/project-bin/lint-gate.sh @@ -169,7 +169,7 @@ fi MXCLI="" case "$(uname -s 2>/dev/null)" in MINGW*|MSYS*|CYGWIN*) [ -x ./mxcli.exe ] && MXCLI=./mxcli.exe ;; esac [ -n "$MXCLI" ] || { [ -x ./mxcli ] && MXCLI=./mxcli; } -[ -n "$MXCLI" ] || { echo "lint-gate: no runnable mxcli in $ROOT (Windows: mxcli.exe) — bin/doctor.sh --install" >&2; exit 2; } +[ -n "$MXCLI" ] || { echo "lint-gate: no runnable mxcli in $ROOT (Windows: mxcli.exe) — bin/doctor.sh --install --yes " >&2; exit 2; } [ -e "$MPR" ] || { echo "lint-gate: $MPR not found" >&2; exit 2; } # Build the --exclude list from the vendor file (comments and blanks stripped). diff --git a/project-bin/process-map.js b/project-bin/process-map.js index 9957a6a1..413f11b9 100644 --- a/project-bin/process-map.js +++ b/project-bin/process-map.js @@ -23,8 +23,17 @@ // process-map.js validate [--strict] check the model (and anchors, when present) // process-map.js anchor derive anchors mechanically, keep judged ones, write // the anchors file, list what is still unanchored +// process-map.js migrate a model whose "anchors" key points at an old path (say the +// prototype folder): move the file to design/question-anchors.json, +// keep the old one as *.bak, rewrite the key, print what carried over // process-map.js build validate, then write design/process-map.html +// process-map.js build --embed the same maps as a bare page for an