From 6aed21f3a843151c6fc16f62ba0b0be80241f1a0 Mon Sep 17 00:00:00 2001 From: Cristiano Calcagno Date: Sat, 17 Oct 2026 02:21:12 +0200 Subject: [PATCH] Draw commit records as semantic diffs of the compiler's objects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Why: A record states its change as prose sections and a JavaScript Before/After pair. The objects most compiler changes touch, a typing rule, a parsetree node, a match compiler's clause matrix, a Lambda rewrite, an error message, have no form of their own in it: the reviewer reconstructs them from the code diff of typecore.ml or matching.ml. Code added (scripts/recordRender.js, the drawing of a record): A pure module, with no file system and no git: it parses a record (sections, change blocks, git trailers) and draws it as HTML. Each kind of object has its drawing: a typing rule as an inference rule, an AST or typed tree fragment as a tree, a clause matrix as a matrix, Lambda and JavaScript as highlighted code with Before and After side by side, a runtime representation as `ReScript ↦ JavaScript` lines, a diagnostic as its text. The kinds are a typed table, so a kind without a drawing does not type-check. A modified block's two versions of code, a tree or text are drawn as aligned rows: a line both have faces itself, the changed lines between are padded with gaps, and on a narrow screen the rows read as a unified diff. An added or removed block's one version spans the block. A long line wraps, its continuation indented past the line's own indentation or tree connectors. It checks each record against its diff: changelog fragments, the required trailers, the Tests trailer against the test files both ways (every listed path is in the diff; every test file in the diff other than a generated snapshot is listed, the missing ones named), generated JavaScript against Output, magic numbers, the v0 PPX bridge, super_errors fixtures and tests/ERROR_VARIANTS.md. Code modified (scripts/commitRecords.js, the Semantic Diff page): Reads each commit's record, files, magic-number diff and changes/ fragments from git and hands them to recordRender.js, which parses, checks and draws them. Its options are --out, --pr, --title and --fragment. Code added (scripts/recordFacts.js, the facts a record abstracts): Prints, for one example and one compiler build, the parser's tree, the tree after the built-in PPX, the JSX transform's output, the formatter's output, the typed tree, the inferred interface, Lambda as translated and after each pass that changes it, the generated ES module and the diagnostic, each as the compiler prints it, under a line naming the compiler and the commit its checkout is at. Run with --bsc on a build of the parent commit, it gives a modified block's Before. Code added (scripts/commitRecordHook.sh, a commit-msg hook, not installed): Refuses a message without the Area, Output, Tests and Signed-off-by trailers, with a trailer after Signed-off-by, or with a malformed Output or Tests; for a commit that changes compiler, runtime, build system or editor tooling sources, also one without Why: or a change block. Docs modified (.claude/skills/records/SKILL.md, the record format): The skill defines the kinds of object, each with the code that owns it, how to extract its exact form, its drawing and what a change of it brings; and the trailers, each with the check the card makes. A change that replaces one shape by another is one modified block on one example, Before extracted from a build of the parent commit and After from this commit's, the Source line naming both builds; added and removed blocks are for objects with no counterpart. Before (text): Example (rescript), Before (js), After (js), Source, Breaking, Review Area, Output, Formatting, Formats, PPX, Editor, Errors, Tests, Measured, PR, Signed-off-by After (text): Why (), with Before and After extracted from the parent commit's build and this commit's Example, Effect, Source, Breaking, Review Area, Output, Tests, Measured, PR, Signed-off-by Command (sh): node scripts/recordFacts.js -e 'let f = (c, a, b) => c ? a : b' parse lambda js node scripts/commitRecords.js --out review.html --pr 1234 --title "Title" master..my-branch ln -sf "$PWD/scripts/commitRecordHook.sh" "$(git rev-parse --git-path hooks)/commit-msg" Review: Do records in the earlier format still render? Yes: a top-level `Before (js):` and `After (js):` pair draws side by side, and trailers outside the format (Formats, Errors, ...) draw as facts. Review: Where do Formatting, Formats, PPX and Editor go? A formatter or editor change is a Format or Editor change block; the card reads the magic numbers, the v0 bridge fixtures and the error catalogue from the diff. Area: scripts docs tests Output: unchanged Tests: ~scripts/commitRecords.test.js PR: #8882 Co-Authored-By: Claude Opus 5.5 Signed-off-by: Cristiano Calcagno --- .claude/skills/records/SKILL.md | 411 ++++-- changes/record-object-kinds.internal.md | 1 + scripts/commitRecordHook.sh | 74 + scripts/commitRecords.js | 962 +----------- scripts/commitRecords.test.js | 247 +++- scripts/recordFacts.js | 207 +++ scripts/recordRender.js | 1799 +++++++++++++++++++++++ 7 files changed, 2671 insertions(+), 1030 deletions(-) create mode 100644 changes/record-object-kinds.internal.md create mode 100755 scripts/commitRecordHook.sh create mode 100755 scripts/recordFacts.js create mode 100644 scripts/recordRender.js diff --git a/.claude/skills/records/SKILL.md b/.claude/skills/records/SKILL.md index 3bf9820df9..9f65c05f85 100644 --- a/.claude/skills/records/SKILL.md +++ b/.claude/skills/records/SKILL.md @@ -1,139 +1,344 @@ --- name: records -description: Commit records and review pages for the ReScript compiler repository. Use when committing a change, when an agent reports its work, and when someone asks to review a commit, a pull request or a branch. Every commit message is a structured record that answers the questions ReScript reviewers ask; review pages are rendered from records with scripts/commitRecords.js, never written by hand. +description: Commit records and Semantic Diff pages for the ReScript compiler repository. Use when committing a change, when an agent reports its work, and when someone asks to review a commit, a pull request or a branch. Every commit message is a record, a semantic diff with its reason; review pages are drawn from records with scripts/commitRecords.js, never written by hand. --- -# Commit records and review pages +# Commit records and Semantic Diff pages -A **record** is a commit message whose body has named prose sections and -whose last block is git trailers. Its fields are the questions reviewers of -this repository ask on pull requests, answered before they are asked. A -review page is rendered from records: one card per commit, and for a range a -summary, a changelog draft and the files more than one commit changes. +## Principles + +1. **A record is a semantic diff.** A code diff is a delta on files; a + record is the delta on the objects the code implements: a typing rule, + the shape of an AST node, a Lambda rewrite, the JavaScript a program + compiles to, an error message. Each change names its object and says + whether it is added, removed or modified, with `Before:` and `After:` + for a modification. +2. **Every change has its reason.** The record opens with `Why:`: the + concrete problem (the program that fails, the crash, the error message or + the wrong output) and why the old design cannot handle it. A reader + learns what was wrong before reading what changed. +3. **Each kind of object has its visual language.** A typing rule is an + inference rule; an AST fragment a tree; Lambda and JavaScript highlighted + code; an error message its text. The kinds below fix the language of each. +4. **Discretion, guided by tools.** What to show, what to hide, where a + schematic form explains better than the exact one is the author's + judgement. The exact forms come from the compiler + (`scripts/recordFacts.js`, below), and every abstraction in a record is + one those forms support. +5. **Plain words.** Write for someone who knows ReScript and compilers but + not this codebase. Use the field's standard terms, define notation before + the formula that uses it, and prefer spelling a set out to introducing a + symbol. Mathematics is Unicode, never TeX. Lists, not tables. +6. **A record is the commit's history.** What was wrong and what changed + belong in the commit. Each sentence states only what the evidence + supports; a number says what produced it. +7. **The commit is plain text.** The message reads well in `git log`; it is + not Markdown. Drawing happens only where records are rendered. ## The record ``` - + - +Why: + -Example (rescript): - + (): + Before: + + After: + -Before (js): - +Example: + -After (js): - +Effect: + → -Source: +Source: -Breaking: +Breaking: -Review: +Review: Area: syntax printer frontend typechecker lambda codegen runtime rewatch analysis reanalyze gentype tools playground scripts ci docs tests -Output: unchanged | changed , -Formatting: unchanged | changed -Formats: none | -PPX: unchanged | -Editor: unchanged | -Errors: -Tests: +added/path ~updated/path -removed/path | none, because +Output: unchanged | changed +Tests: +added/path ~changed/path -removed/path | none, because Measured: after, and what produced it>; ... PR: # Signed-off-by: ``` -- **Prose sections** are a line `Name:` or `Name (lang):` followed by text or - a block indented by 4 spaces. `lang` is `rescript`, `js`, `ts`, `sh`, - `rust`, `ocaml` or `text`; the renderer highlights ReScript and JavaScript. - `Before` and `After` sections of one language render side by side with - changed lines marked. Use `Before (rescript)` / `After (rescript)` for a - formatter change and `Diagnostic (text)` for compiler output. Any other - name is allowed and renders as prose or code. -- **The final block is git trailers**: `Key: value` lines, continuation lines - indented by one space, nothing after them. `git interpret-trailers --parse` - must read every key. `Signed-off-by` (DCO) is required on every commit. -- **No changelog in the record**: the `changes/..md` - fragment the commit adds is the single source of truth for its changelog - category and entry (see `changes/README.md`). The record repeats neither; - the review page reads the fragment from the diff. -- **Required keys**: `Area`, `Output`, `Tests`. `Formatting`, `Formats`, - `PPX` and `Editor` are required whenever the change touches the syntax - library, the parsetree, the typed tree or Lambda. Omit the others when they - do not apply. -- **Wording** follows the writing rules: present facts, positive statements, - claims as strong as their evidence, every number labelled by what produced - it. A JavaScript `Before`/`After` pair says where it came from in a - `Source:` paragraph (compiler run, or a snapshot file in the diff). -- **Lists in trailers**: `Measured` separates items with `; ` - (outside parentheses); each item renders on its own line. `Output` and the - other state trailers start with their state word (`unchanged`, `changed`, - `none`), followed by the detail. - -## What each field states - -| Field | What the record states | -| --- | --- | -| `Area` | The layers the change touches. They decide who reviews it and which suites apply; a parsetree or typed-tree change also reaches editor analysis. | -| `Output` + `Before`/`After (js)` | The generated JavaScript for the example before and after, and which existing programs compile differently. `unchanged` holds when no committed generated JS changes, or when a rebuild of the test suites is byte-identical. | -| `Formatting` + `Before`/`After (rescript)` | Whether `rescript format` rewrites existing code, and whether formatted output parses back to the same tree with its comments. | -| `Formats` | The magic numbers bumped in `compiler/ext/config.ml`. Any change to the marshaled parsetree, typed tree (`.cmi`, `.cmt`) or `.cmj` layout, constructor order included, bumps the matching number. | -| `PPX` | The effect on the frozen v0 PPX wire: `parsetree0.ml` is unchanged, and a change to the v0 mapping comes with an ast-mapping fixture. | -| `Editor` | Completion, hover and inlay hints on new or changed nodes, and the analysis fixtures that cover them. | -| `Errors` | The `tests/ERROR_VARIANTS.md` rows added, removed or relinked for changed error variants or fixtures. | -| `Tests` | The tests added, changed or removed, each observing the effect the record claims, so that it fails without the change. `none, because …` gives the reason when no test changes. | -| `Measured` | Performance before and after, with the input and the command that produced each number. | -| `Breaking` | What existing code or tooling stops working, and the migration. | -| `Review` | Questions a reviewer raised or will raise, with their answers. | -| `PR` | The pull request; the renderer links it. | -| `Signed-off-by` | The DCO sign-off, with the commit author's name and email, as the last trailer. | - -## Rendering +- **Sections** start with `Name:` at the start of a line; the content is the + rest of that line or a block indented by 4 spaces. In a change block, + `Before:` and `After:` are indented by 2 spaces and their content by 4; + `Before (lang):` and `After (lang):` draw the versions as that language. + `Before (lang):` followed by `After (lang):` (`js`, `rescript`, `ts`, + `ocaml`, `lambda`, `text`) as sections of their own is a pair of code + blocks, drawn side by side. +- **A replacement is one `modified` block.** When the change replaces one + shape by another (a node, a typed tree node, a Lambda term, the + JavaScript a program compiles to), the block shows one example as the + parent commit's compiler gives it, `Before:`, and as this commit's gives + it, `After:`, both extracted (below). `added` and `removed` are for an + object with no counterpart on the other side: a new error message, a + deleted flag. An added object shows its form under `After:`; a removed one + may show its form under `Before:`. +- **The final block is git trailers**: `Key: value`, continuation lines + indented by one space, nothing after them; `git interpret-trailers --parse` + reads every key. `Co-Authored-By` lines go before `Signed-off-by`, which is + the last trailer. +- **Required trailers**: `Area`, `Output`, `Tests` and `Signed-off-by` on + every commit. `Why:` and at least one change block on every commit that + changes compiler, runtime, build system or editor tooling sources. +- **The changelog is the `changes/..md` fragments** the + commit adds or changes (`changes/README.md`). The record repeats neither + the category nor the entry; the card reads each fragment from the commit's + tree. A commit that makes more than one user-facing change adds one + fragment per change. +- **The diff states the rest.** The card reads the bumped magic numbers, the + committed generated JavaScript, the error catalogue and the fixtures from + the commit itself, and checks them against the record and the + repository's rules (below). + +## The trailers + +| Trailer | What it states | What the card checks | +| --- | --- | --- | +| `Area` | The layers the change touches, from the list in the template. They decide who reviews it and which suites apply. | The trailer is present. | +| `Output` | `unchanged` when no existing program compiles to different JavaScript; `changed ` otherwise, with a `JS` or `Lambda` change block showing the difference. | Under `unchanged`, no committed generated JavaScript (`.mjs`, `.cjs`, `.res.js` under `tests/` and `rewatch/testrepo/`, the runtime's `lib/`) is modified; new files from new tests are counted. | +| `Tests` | The tests added (`+`), changed (`~`) and removed (`-`), by path from the repository root (a directory stands for the files under it), each failing without the change; or `none, because `. Every test file the diff adds, changes or removes (`tests/`, `rewatch/tests/`, `*.test.js`) is listed, except the snapshots tests generate: committed generated JavaScript, files in `expected/` directories, `.expected` files and `rewatch/tests/snapshots/`, which may be listed. | Every listed path is in the diff; every test file in the diff other than a snapshot is listed, the missing ones named by path; `none` gives its reason, and the diff then changes no test file. | +| `Measured` | Performance before and after, `before -> after`, with the input and the command that produced each number; items separated by `; `. | Drawn as a list, each delta marked. | +| `PR` | The pull request; the card links it. | | +| `Signed-off-by` | The DCO sign-off, with the commit author's name and email, as the last trailer. | The DCO app checks it on the pull request. | + +## Kinds of object + +Each kind gives the object, the code that owns it, how to extract its exact +form, how it is drawn, and what a change of it brings with it. + +- **`Typing`**, a typing rule: when the type checker accepts an expression, + pattern or declaration, and at which type. Owned by `compiler/ml/typecore.ml`, + `typetexp.ml`, `ctype.ml`, `typedecl.ml`, `typemod.ml`, `includemod.ml`. + Extract: `recordFacts.js … types` (the inferred interface), `typed`, and + `diagnostic` for a refusal. Drawn as an inference rule: premises side by + side, separated by three or more spaces; a line of `─` ending with the + rule's name in parentheses; the conclusion. Side conditions are written as + words (`τ is an @unboxed record`). Brings a test that a program the rule + newly accepts compiles, and a `super_errors` fixture for one it refuses. +- **`Syntax`**, the surface grammar: what the parser accepts and how a form + is spelled. Owned by `compiler/syntax/src/res_core.ml`, `res_scanner.ml`, + `res_grammar.ml`. Extract: `recordFacts.js … parse`. Drawn as grammar + productions (`expr ::= expr ? expr : expr`) in mathematics. Brings a + fixture in `tests/syntax_tests/data/parsing`. +- **`AST`**, the shape of a parsetree node, or of its v0 encoding for + external PPXs. Owned by `compiler/ml/parsetree.ml`; the v0 wire by + `parsetree0.ml`, `ast_mapper_to0.ml`, `ast_mapper_from0.ml`. Extract: + `recordFacts.js … parse` (`bsc -bs-no-builtin-ppx -only-parse + -dparsetree`, the parser's output), and `expanded` for the tree after the + built-in PPX. Drawn as a tree: one node a line, two spaces a level, + `label: Node` for a field; the compiler's printout pastes as it is, its + `[` and `]` lines not drawn. A change to `parsetree.ml` bumps the ast + magic numbers in `compiler/ext/config.ml`; `parsetree0.ml` is frozen; a + change to the v0 mapping brings a fixture in + `tests/syntax_tests/data/ast-mapping` or a bridge unit test. +- **`Typedtree`**, the shape of a typed tree node. Owned by + `compiler/ml/typedtree.mli`. Extract: `recordFacts.js … typed` + (`bsc -dtypedtree`). Drawn as a tree. A change to `typedtree`, `types` or + `cmt_format` bumps the cmi or cmt magic number. +- **`Transform`**, what a built-in PPX produces from source: the JSX + transform, `@deriving`, externals. Owned by `compiler/syntax/src/jsx_*.ml` + and `compiler/frontend/`. Extract: `recordFacts.js --jsx … transform` + (`bsc -reprint-source`). Drawn as ReScript code; `source ⇝ result` draws + the rewrite. Brings a fixture in `tests/syntax_tests/data/ppx`. +- **`Format`**, what `rescript format` prints. Owned by + `compiler/syntax/src/res_printer.ml`, `res_parens.ml`, + `res_comments_table.ml`. Extract: `recordFacts.js … format`. Drawn as + ReScript code, Before and After. Brings a fixture in + `tests/syntax_tests/data/printer` or `idempotency`; formatted output parses + back to the same tree with its comments. +- **`Diagnostic`**, the text of an error or warning. Owned by each layer's + `report_error`, `compiler/ml/printtyp.ml`, `error_message_utils.ml`, + `compiler/ext/warnings.ml`. Extract: `recordFacts.js … diagnostic`, or the + fixture's `.expected` file. Drawn as the message text, without the code + frame unless the frame is the point. A `super_errors` fixture added or + removed comes with its row in `tests/ERROR_VARIANTS.md`. +- **`Matching`**, how the match compiler splits a `switch`: its clause + matrix, exhaustiveness and unused cases. Owned by `compiler/ml/matching.ml`, + `parmatch.ml`, `switch.ml`. Extract: `recordFacts.js … lambda` (the + decision tree), and the matrix by reading the patterns. Drawn as a matrix: + one row a line, columns three or more spaces apart, a `note: …` line below. +- **`Lambda`**, the intermediate language: what a construct translates to + (`compiler/ml/translcore.ml`, `translmod.ml`) or what a pass rewrites + (`compiler/core/lam_*.ml`). Extract: `recordFacts.js … lambda` (as + translated) and `passes` (after each pass that changes it). Drawn as + highlighted Lambda; a rewrite is `lhs`, a line `⇝`, `rhs`, then `when: …` + conditions. One-letter capitals (`T`, `A`) stand for terms. +- **`IR`**, an internal representation: an OCaml type the compiler's layers + share (`Lambda.lambda`, `Types.type_desc`, `J.expression`, a record of + `Parsetree` that is not a node shape). Owned by the `.mli` defining it. + Extract: the type in the diff (`git show -- `). Drawn as + OCaml. +- **`Representation`**, how a ReScript value is represented at run time: + `ReScript ↦ JavaScript`. Owned by `compiler/ml/variant_*.ml`, + `record_runtime.ml`, `ast_untagged_variants.ml` and the code generator. + Extract: `recordFacts.js … js` on a value. Drawn as one `↦` line a value. + A changed representation is `Output: changed`. +- **`JS`**, the JavaScript a program compiles to. Owned by + `compiler/core/lam_compile*.ml`, `js_*.ml`. Pinned by the committed + `tests/tests/src/*.mjs` and the other generated files. Extract: + `recordFacts.js … js`, or `git show :` for a committed + snapshot. Drawn as highlighted JavaScript, Before and After side by side. + The snapshots the change modifies are the programs `Output: changed` + names. +- **`API`**, a signature of the runtime or standard library. Owned by + `packages/@rescript/runtime/*.resi`, `packages/@rescript/belt`. Extract: + the `.resi`, or `recordFacts.js … types`. Drawn as ReScript. Brings tests + in `tests/tests/src/stdlib` or `tests/docstring_tests`. +- **`TypeScript`**, what genType emits. Owned by `compiler/gentype`. Drawn + as TypeScript. Brings a snapshot in `tests/gentype_tests`. +- **`Editor`**, what completion, hover, inlay hints or code actions return. + Owned by `analysis/src`. Extract: the analysis test's expected output. + Drawn as that text. Brings a fixture in `tests/analysis_tests`. +- **`Build`**, what `rescript build` or `rescript watch` does: which files it + compiles, when it rebuilds, its configuration. Owned by `rewatch/src` and + `rewatch-ocaml`. Drawn as prose; a `Command (sh):` section shows a + session. Brings a test in `rewatch/tests`. +- **`Semantics`**, the meaning of a construct at run time, and + **`Notation`**, the symbols a later formula uses. Drawn as mathematics. +- **`Code`**, an internal module, script or function added, moved or + removed, with no other object changing. **`Docs`** and **`CI`** for + documentation and workflows. Drawn as prose. +- **`Example`** and **`Counterexample`**: ReScript programs, drawn as code. + **`Effect`**: ` → ` lines. + +A section `Name (lang):` that is none of these draws its block as `lang` +code (`rescript`, `js`, `ts`, `ocaml`, `lambda`, `text`, `sh`). + +## Writing the forms + +- **An inference rule** is its premises on one line, separated by three or + more spaces, a line of `─` with the rule's name in parentheses, then the + conclusion; a blank line separates it from other formulas. One-letter names + are variables (italic); a name applied to arguments is an operator; a + capitalised name a constructor; a run of English words a side condition; + `or`, `and`, `not` are words; `null`, `undefined`, `true`, `false` and + backticked `code` are drawn as code. `_x`, `_{…}` and `^{…}` are sub- and + superscripts; an indented line continues the formula above. +- **A tree** is indented by two spaces a level; `label: Node "name"` shows a + field's label, `@attr` an attribute. Show only the nodes the change + concerns; `…` stands for the rest. +- **A rewrite** (Lambda, Transform, JS) is the left side, a line holding + `⇝`, the right side, then `when: …` lines. +- **Before and After** are drawn side by side, the lines each lacks from the + other shaded, so keep unchanged lines identical. Code, trees and text are + drawn as aligned rows: a line both versions have faces itself, and the + changed lines between are padded so that the next common line faces + itself again. A long line wraps, its continuation indented. + +## Extracting the facts + +`scripts/recordFacts.js` prints the exact forms a compiler build produces for +one example, the compiler's own output with colour codes removed: + +```sh +node scripts/recordFacts.js [view...] +node scripts/recordFacts.js -e 'let f = (c, a, b) => c ? a : b' parse typed lambda js +node scripts/recordFacts.js --jsx transform js +node scripts/recordFacts.js --bsc /bsc.exe all +``` + +Views: `parse`, `expanded`, `transform`, `format`, `typed`, `types`, +`lambda`, `passes`, `js`, `diagnostic`, `all` (default `types lambda js`). +The default compiler is the checkout's +`packages/@rescript//bin/bsc.exe`, built by `make`; the first line +of the output names the compiler and the commit its checkout is at. + +A modified block's two versions come from two builds, on the same example: +`After:` from this commit's build, `Before:` from a build of the parent +commit, in a second worktree: + +```sh +parent=$(mktemp -d)/parent +git worktree add --detach "$parent" ^ && (cd "$parent" && yarn install && make) +node scripts/recordFacts.js --bsc "$parent"/packages/@rescript//bin/bsc.exe example.res parse +node scripts/recordFacts.js example.res parse +``` + +For a generated file a commit changes, the two versions are +`git show ^:tests/tests/src/.mjs` and +`git show :tests/tests/src/.mjs`. The `Source:` line names both: +the command and view, the build of the parent commit for `Before:` and of +this commit for `After:`, or the revisions of the snapshot. + +## The Semantic Diff page ```sh node scripts/commitRecords.js --out review.html ... -node scripts/commitRecords.js --out review.html ..... +node scripts/commitRecords.js --out review.html ..... +node scripts/commitRecords.js --out review.html --pr 1234 --title "" .. node scripts/commitRecords.js --fragment --out part.html ... -node scripts/commitRecords.js --out review.html --pr 1234 --title "" .. +make semantic-diff ``` - `--pr` and `--title` head the page with the pull request's number (linked) and title; without them the page is headed "Semantic Diff". + `make semantic-diff` renders `master..HEAD` to `_build/semantic-diff.html` + and opens it. +- Each record is a card: the subject, the changelog categories and areas, + the change blocks in their drawings (a modified block's versions side by + side, an added or removed block's one version across the card), the + changelog entries of its fragments, the trailers as facts, and the checks + of the record against its diff: + - each `changes/` fragment the commit adds or changes, validated as + `yarn changelog:check` validates it (the placeholder PR link fails), and + an edit of `CHANGELOG.md`, which belongs to release pull requests; + - the required trailers `Area`, `Output` and `Tests`; + - `Tests` against the diff, as the trailer table states; + - committed generated JavaScript modified and added, against `Output`; + - magic numbers bumped in `compiler/ext/config.ml`, and a change to a + marshaled type (`parsetree`, `typedtree`, `types`, `cmt_format`) with + none; + - a change to the frozen `parsetree0.ml`, and a change to the v0 bridge + without an `ast-mapping` fixture or bridge unit test; + - `super_errors` fixtures added or removed without + `tests/ERROR_VARIANTS.md`. +- A range adds a summary (commits, kinds changed, areas, changelog + categories, failed checks), a changelog draft from the fragments the range + leaves in their last version, and the files more than one commit changes. + Use commit hashes as range bases. +- The page is one HTML file with no script, light and dark themes, phone + width. The drawing is `scripts/recordRender.js`, a pure module (no file + system, no git), so any other reader of commit messages draws records the + same way. Its kinds are a typed table: a kind without a drawing does not + type-check. +- A note on `refs/notes/records` replaces a commit's message as its record: + use it for commits that exist already, such as squash merges. +- On GitHub, `.github/workflows/semantic_diff.yml` renders each pull + request's range when it is opened or pushed, and comments the page's link + (`Semantic diff: https://cdn.rescript-lang.org/semantic-diff/pr-/…`). -- A revision renders one card. A range renders a summary row, a changelog - draft grouped by category, its commits as cards, and the files more than - one commit (or more than one range) changes. The changelog category, the - entries and the draft come from the `changes/` fragments in the diff: a - card shows the fragments its commit adds or changes, and a range's draft - shows the fragments the range leaves, in their last version. -- Each card checks the record against its diff: a `changes/` fragment that - `yarn changelog:check` rejects (such as one still carrying the placeholder - PR link), an edit to `CHANGELOG.md`, modified generated `.mjs`, `.cjs` and `.res.js` files under - `Output: unchanged` (new ones from new tests are counted, not flagged), a - magic number bumped in `compiler/ext/config.ml` that - `Formats` does not name, a parsetree change with no ast-mapping fixture, a - new `super_errors` fixture with no `ERROR_VARIANTS.md` change, a missing - `Tests` trailer, `Tests: none` on a diff that changes tests. A failed - check is shown on the card. The DCO app checks `Signed-off-by`. -- The page is one self-contained HTML file: no script, no external data - (fonts fall back to system faces), light and dark themes, phone width. - `--fragment` omits the document skeleton for embedding. -- A note on `refs/notes/records` replaces the commit message's record. Use it - for commits that exist already (history, a squash merge) and for review - outcomes added later; a new commit carries its record in its message. +## Enforcement + +`scripts/commitRecordHook.sh` is a `commit-msg` hook. It refuses a message +without the required trailers, with a trailer after `Signed-off-by`, with an +`Output` that starts with neither `unchanged` nor `changed`, or with a +`Tests` that is neither a list of `+~-` paths nor `none, because `; +for a commit that changes compiler, runtime, build system or editor tooling +sources, it also requires `Why:` and a change block. Merge, `fixup!` and +`squash!` commits pass. A developer enables it in a checkout with + +```sh +ln -sf "$PWD/scripts/commitRecordHook.sh" "$(git rev-parse --git-path hooks)/commit-msg" +``` ## When to use -- Every commit an agent makes carries a record in its message. -- An agent's report lists its commits and the rendered page; the records are - the description of the work. A pull request description is a rendering of - its range. -- Before review or merge: render `master..` and hand over the page. -- On GitHub, `.github/workflows/semantic_diff.yml` renders each pull - request's range when it is opened or pushed, and comments the page's link - (`Semantic diff: https://cdn.rescript-lang.org/semantic-diff/pr-/…`). +- Every commit carries a record in its message. +- An agent's report lists its commits; their records are the description of + the work. +- Before review: render `..` and hand over the page. diff --git a/changes/record-object-kinds.internal.md b/changes/record-object-kinds.internal.md new file mode 100644 index 0000000000..20c827c38c --- /dev/null +++ b/changes/record-object-kinds.internal.md @@ -0,0 +1 @@ +- Draw commit records as semantic diffs of the compiler's objects, each kind of object in its own drawing, and extract the exact forms a record abstracts from the compiler with `scripts/recordFacts.js`. https://github.com/rescript-lang/rescript/pull/8882 diff --git a/scripts/commitRecordHook.sh b/scripts/commitRecordHook.sh new file mode 100755 index 0000000000..d3b4dfaf76 --- /dev/null +++ b/scripts/commitRecordHook.sh @@ -0,0 +1,74 @@ +#!/bin/bash + +# commit-msg hook: the commit message is a commit record +# (.claude/skills/records/SKILL.md). Every commit carries the required +# trailers Area, Output, Tests and Signed-off-by, with Signed-off-by last; a +# commit that changes the compiler, the runtime, the build system or the +# editor tooling also has a Why: section and at least one change block naming +# the object it changes. Enable it in a checkout with +# ln -sf "$PWD/scripts/commitRecordHook.sh" "$(git rev-parse --git-path hooks)/commit-msg" + +exec 1>&2 + +message_file=$1 +git_dir=$(git rev-parse --git-dir) + +# a merge commit has no record of its own +if [[ -f "$git_dir/MERGE_HEAD" ]]; then + exit 0 +fi + +body=$(grep -v '^#' "$message_file") +# fixup! and squash! commits are folded into a recorded commit +case "$body" in + fixup!*|squash!*|amend!*) exit 0 ;; +esac + +trailers=$(printf '%s\n' "$body" | git interpret-trailers --parse) +value() { + printf '%s\n' "$trailers" | sed -n "s/^$1: //p" | head -n 1 +} + +problems=() +for key in Area Output Tests Signed-off-by; do + if ! printf '%s\n' "$trailers" | grep -q "^$key: "; then + problems+=("no '$key:' trailer") + fi +done +last=$(printf '%s\n' "$trailers" | tail -n 1) +if [[ -n "$last" && "$last" != Signed-off-by:* ]]; then + problems+=("the last trailer is '${last%%:*}'; Signed-off-by is the last trailer") +fi + +output=$(value Output) +case "$output" in + unchanged*|changed*|"") ;; + *) problems+=("Output '$output' starts with neither 'unchanged' nor 'changed'") ;; +esac +tests=$(value Tests) +case "$tests" in + "none, because "?*|[+~-]?*|"") ;; + none*) problems+=("Tests: none gives its reason as 'none, because '") ;; + *) problems+=("Tests lists tests as +added ~changed -removed paths, or is 'none, because '") ;; +esac + +if git diff --cached --name-only | + grep -Eq '^(compiler|rewatch/src|analysis|tools/src|packages/@rescript/(runtime|belt))/'; then + kinds='Syntax|Format|AST|Transform|Typing|Typedtree|Diagnostic|Matching|Lambda|IR|Representation|JS|API|TypeScript|Editor|Build|Semantics|Code' + if ! printf '%s\n' "$body" | grep -q '^Why:'; then + problems+=("no 'Why:' section (the problem this commit addresses)") + fi + if ! printf '%s\n' "$body" | grep -Eq "^($kinds) (added|modified|removed)( \(.*\))?:"; then + problems+=("no change block such as 'Typing modified (rule name):' or 'Code added (module):'") + fi +fi + +if (( ${#problems[@]} )); then + echo "The commit message is not a commit record:" + for p in "${problems[@]}"; do + echo " - $p" + done + echo "Format: .claude/skills/records/SKILL.md. The trailers are the last block of" + echo "the message, one 'Key: value' per line; git interpret-trailers --parse reads them." + exit 1 +fi diff --git a/scripts/commitRecords.js b/scripts/commitRecords.js index 0522fd5f5a..519bfa4914 100644 --- a/scripts/commitRecords.js +++ b/scripts/commitRecords.js @@ -2,15 +2,16 @@ // @ts-check -/* Renders commit records as a review page. A record is a commit message (or a - * note on refs/notes/records, which takes precedence) in the format of - * .claude/skills/records/SKILL.md: a subject, prose sections, code sections - * such as "Before (js):" and git trailers. The changelog comes from the - * changes/..md fragments the commits add, not from the - * record. +/* Draws commit records as a Semantic Diff page: one card per commit, and for + * a range a summary, a changelog draft and the files more than one commit + * changes. A record is a commit message, or a note on refs/notes/records, + * which takes precedence, in the format of the `records` skill + * (.claude/skills/records/SKILL.md). The changelog is the + * changes/..md fragments the commits add or change. The + * drawing is scripts/recordRender.js; this file reads git. * * node scripts/commitRecords.js --out review.html ... - * node scripts/commitRecords.js --out review.html ..... + * node scripts/commitRecords.js --out review.html ..... * --pr and --title head the page with the pull request. * --fragment omits the document skeleton. */ @@ -18,189 +19,27 @@ import { execFileSync } from "node:child_process"; import { writeFileSync } from "node:fs"; import { parseFragment } from "./changelog.js"; +import { CSS, esc, pageBody, pageTitle, parseRecord } from "./recordRender.js"; -const REPO_URL = "https://github.com/rescript-lang/rescript"; +/** @import { CommitRecord, FileChange, Fragment, Group } from "./recordRender.js" */ -/** - * @param {string[]} args - * @param {string} [input] - */ -const git = (args, input) => +/** @param {string[]} args */ +const git = args => execFileSync("git", args, { - input, encoding: "utf8", - maxBuffer: 64 << 20, - stdio: ["pipe", "pipe", "ignore"], + maxBuffer: 1 << 28, + stdio: ["ignore", "pipe", "ignore"], }); -/** @param {string} s */ -const esc = s => - s - .replace(/&/g, "&") - .replace(//g, ">") - .replace(/"/g, """); - -/** @param {string} s */ -/** Short shas of the cards on the page, for linking mentions. */ -const pageShas = new Set(); - -/** @param {string} s */ -const inline = s => - esc(s) - .replace(/`([^`]+)`/g, "$1") - .replace( - /https:\/\/github\.com\/rescript-lang\/rescript\/(pull|issues)\/(\d+)/g, - `#$2`, - ) - .replace(/\b([0-9a-f]{9})\b(\^?)/g, (m, sha, caret) => - pageShas.has(sha) && !caret - ? `${sha}` - : m, - ) - .replace( - /(^|[\s(])#(\d{3,5})\b/g, - `$1#$2`, - ) - .replace( - /(\d[\d.,]*\s*(?:ms|s|%|KB|MB)?)\s*->\s*(\d[\d.,]*\s*(?:ms|s|%|KB|MB)?)/g, - '$1 → $2', - ); - -// --------------------------------------------------------------------------- -// Syntax highlighting - -const RES_KEYWORDS = new Set( - "let rec and switch if else type module open include mutable async await for in to downto while external exception try catch when of as private constraint assert lazy dict list true false".split( - " ", - ), -); -const JS_KEYWORDS = new Set( - "let const var function return if else for while do new import export from as default this true false null undefined typeof instanceof class async await throw try catch finally of in switch case break continue void delete yield".split( - " ", - ), -); - -/** @type {Record} */ -const TOKENS = { - rescript: - /(\/\/[^\n]*|\/\*[\s\S]*?\*\/)|("(?:\\.|[^"\\\n])*"|`(?:\\.|[^`\\])*`)|(@{1,2}[A-Za-z_][\w.]*|%{1,2}[A-Za-z_][\w.]*)|(#[A-Za-z_]\w*)|(\b0x[0-9a-fA-F_]+n?\b|\b\d[\d_]*(?:\.\d+)?(?:[eE][+-]?\d+)?n?\b)|([A-Za-z_][\w']*)|(=>|->|\+\+|:=|\.\.\.|\|>|[=<>!|&+\-*/:?]+)/g, - js: /(\/\/[^\n]*|\/\*[\s\S]*?\*\/)|("(?:\\.|[^"\\\n])*"|'(?:\\.|[^'\\\n])*'|`(?:\\.|[^`\\])*`)|((?!x)x)|((?!x)x)|(\b0x[0-9a-fA-F]+n?\b|\b\d+(?:\.\d+)?(?:[eE][+-]?\d+)?n?\b)|([A-Za-z_$][\w$]*)|(=>|===|!==|\*\*|[=<>!|&+\-*/:?%^]+)/g, -}; - -/** - * @param {string} code - * @param {string} lang - */ -function highlight(code, lang) { - const re = TOKENS[lang]; - if (!re) return esc(code); - const keywords = lang === "js" ? JS_KEYWORDS : RES_KEYWORDS; - let out = ""; - let last = 0; - re.lastIndex = 0; - for (let m = re.exec(code); m; m = re.exec(code)) { - out += esc(code.slice(last, m.index)); - last = m.index + m[0].length; - const [text, comment, str, attr, poly, num, ident, op] = m; - const cls = comment - ? "c" - : str - ? "s" - : attr - ? "a" - : poly - ? "p" - : num - ? "n" - : ident - ? keywords.has(ident) - ? "k" - : /^[A-Z]/.test(ident) - ? "t" - : "" - : op - ? "o" - : ""; - // A span never crosses a line break: lines are split after highlighting. - out += cls - ? text - .split("\n") - .map(part => `${esc(part)}`) - .join("\n") - : esc(text); - } - return out + esc(code.slice(last)); -} - -/** - * Lines of a code block, each highlighted, marked when the paired block of a - * Before/After pair lacks it. - * @param {string[]} lines - * @param {string} lang - * @param {Set | undefined} other - * @param {string} mark - */ -function codeLines(lines, lang, other, mark) { - const text = lines.join("\n").trimEnd(); - const shown = highlight(text, lang).split("\n"); - return text - .split("\n") - .map((line, i) => { - const changed = other && line.trim() && !other.has(line.trim()); - return `${shown[i] || " "}`; - }) - .join(""); -} - -// --------------------------------------------------------------------------- -// Records - -/** - * @typedef {{ name: string, lang: string, text: string[], block: string[] }} Section - * @typedef {{ status: string, path: string, added: number, removed: number }} FileChange - * @typedef {{ path: string, category: string, text: string, error: string }} Fragment - * @typedef {{ - * sha: string, author: string, date: string, subject: string, - * sections: Section[], trailers: Map, message: string, - * coauthors: string[], files: FileChange[], fragments: Fragment[], - * patch: string, noted: boolean - * }} CommitRecord - */ - -/** @param {string} text */ -function parseTrailers(text) { - /** @type {Map} */ - const trailers = new Map(); - for (const line of git(["interpret-trailers", "--parse"], text).split("\n")) { - const i = line.indexOf(":"); - if (i <= 0) continue; - const key = line.slice(0, i).trim(); - const value = line.slice(i + 1).trim(); - trailers.set( - key, - trailers.has(key) ? `${trailers.get(key)}\n${value}` : value, - ); - } - return trailers; -} - -/** @param {Section[]} sections */ -function pushProse(sections) { - /** @type {Section} */ - const s = { name: "", lang: "", text: [], block: [] }; - sections.push(s); - return s; -} - /** @param {string} path */ const isFragment = path => /^changes\/[^/]+\.md$/.test(path) && path !== "changes/README.md"; /** - * The changelog fragments a commit adds or changes, as they stand in its tree. - * A fragment that fails validation (such as one with the placeholder PR link) - * keeps the category from its name and records the error. + * The changelog fragments a commit adds or changes, as they stand in its + * tree. A fragment that `yarn changelog:check` rejects (such as one with the + * placeholder PR link) keeps the category from its name and carries the + * reason. * @param {string} rev * @param {FileChange[]} files * @returns {Fragment[]} @@ -226,99 +65,37 @@ function fragmentsOf(rev, files) { } /** - * The bullets of a fragment, without their "- " and with continuation lines - * joined. - * @param {string} text + * The record of one commit: its note if it has one, else its message, with + * the files it changes and the diff of the files the checks read. + * @param {string} rev + * @returns {CommitRecord} */ -const bullets = text => - text - .split(/\n(?=- )/) - .map(b => b.replace(/^-\s+/, "").replace(/\n\s+/g, " ")) - .filter(Boolean); - -/** @param {string} rev */ function readRecord(rev) { - let text = ""; + let note = ""; try { - text = git(["notes", "--ref=records", "show", rev]); + note = git(["notes", "--ref=records", "show", rev]); } catch { // no note } const message = git(["show", "-s", "--format=%B", rev]); - const noted = text.trim() !== ""; - if (!noted) text = message; - const lines = text.trimEnd().split("\n"); - let end = lines.length; - while ( - end > 1 && - (/^[A-Za-z-]+: /.test(lines[end - 1]) || /^ \S/.test(lines[end - 1])) - ) - end--; - /** @type {Section[]} */ - const sections = []; - /** @type {Section | undefined} */ - let current; - for (const line of lines.slice(1, end)) { - const head = - /^([A-Z][A-Za-z]*(?: [A-Za-z]+){0,2})(?: \(([a-z]+)\))?:\s*(.*)$/.exec( - line, - ); - if (head) { - current = { - name: head[1], - lang: head[2] ?? "", - text: head[3] ? [head[3]] : [], - block: [], - }; - sections.push(current); - } else if (line.startsWith(" ")) { - if (!current) current = pushProse(sections); - current.block.push(line.slice(4)); - } else if (line.trim() === "" && current?.block.length) { - current.block.push(""); - } else if (line.trim()) { - if (!current || current.block.length) current = pushProse(sections); - current.text.push(line.trim()); - } else if (current?.text.length) current = undefined; - } /** @type {Map} */ const files = new Map(); - for (const line of git([ - "diff-tree", - "--no-commit-id", - "-r", - "-M", - "--name-status", - rev, - ]).split("\n")) { + const tree = ["diff-tree", "--no-commit-id", "-r", "--no-renames", "--root"]; + for (const line of git([...tree, "--name-status", rev]).split("\n")) { const [status, ...paths] = line.split("\t"); - if (!status) continue; const path = paths[paths.length - 1]; - files.set(path, { status: status[0], path, added: 0, removed: 0 }); + if (status && path) + files.set(path, { status: status[0], path, added: 0, removed: 0 }); } - for (const line of git([ - "diff-tree", - "--no-commit-id", - "-r", - "-M", - "--numstat", - rev, - ]).split("\n")) { - const [a, r, ...rest] = line.split("\t"); - if (!rest.length) continue; - const path = rest - .join("\t") - .replace(/\{[^{}]* => ([^{}]*)\}/, "$1") - .replace(/^.* => /, "") - .replace(/\/\//g, "/"); + for (const line of git([...tree, "--numstat", rev]).split("\n")) { + const [added, removed, path] = line.split("\t"); const f = files.get(path); if (f) { - f.added = Number(a) || 0; - f.removed = Number(r) || 0; + f.added = Number(added) || 0; + f.removed = Number(removed) || 0; } } - /** @type {CommitRecord} */ - const record = { + return parseRecord(note.trim() ? note : message, { sha: git(["rev-parse", "--short=9", rev]).trim(), author: git(["show", "-s", "--format=%an", rev]).trim(), date: git([ @@ -328,658 +105,35 @@ function readRecord(rev) { "--date=format:%b %d, %Y", rev, ]).trim(), - subject: lines[0], - sections, - trailers: parseTrailers(text), message, - noted, - coauthors: [ - ...new Set( - message - .split("\n") - .filter(l => /^Co-Authored-By:/i.test(l)) - .map(l => - l.replace(/^Co-Authored-By:\s*/i, "").replace(/\s*<[^>]*>/, ""), - ), - ), - ], + noted: note.trim() !== "", files: [...files.values()], fragments: fragmentsOf(rev, [...files.values()]), - patch: git([ - "show", - "--format=", - "-U0", - rev, - "--", - "compiler/ext/config.ml", - ]), - }; - return record; -} - -/** - * @param {CommitRecord} r - * @param {string} key - */ -const words = (r, key) => - (r.trailers.get(key) ?? "").split(/\s+/).filter(Boolean); - -// --------------------------------------------------------------------------- -// What the diff says - -/** @type {[RegExp, string][]} */ -const AREAS = [ - [/^compiler\/syntax\//, "syntax"], - [/^compiler\/core\/lam_/, "lambda"], - [/^compiler\/core\//, "codegen"], - [/^compiler\/ml\//, "typechecker"], - [/^compiler\/frontend\//, "frontend"], - [/^compiler\/gentype\//, "gentype"], - [/^compiler\//, "compiler"], - [/^analysis\/reanalyze\//, "reanalyze"], - [/^analysis\//, "analysis"], - [/^rewatch\/tests\//, "tests"], - [/^rewatch\//, "rewatch"], - [/^packages\/@rescript\/runtime\//, "runtime"], - [/^packages\/dev-playground\//, "playground"], - [/^tools\//, "tools"], - [/^tests\//, "tests"], - [/^scripts\//, "scripts"], - [/^\.github\/|^Makefile$/, "ci"], - [/^changes\/|^CHANGELOG\.md$/, "changelog"], - [/\.md$|^docs\//, "docs"], -]; - -/** @param {string} path */ -const areaOf = path => AREAS.find(([re]) => re.test(path))?.[1] ?? "other"; - -/** @param {string} path */ -const isGeneratedJs = path => - /\.(mjs|cjs)$|\.res\.js$/.test(path) && - /^(tests\/|packages\/@rescript\/runtime\/lib\/|rewatch\/testrepo\/)/.test( - path, - ); - -/** - * Checks of the record against its own diff. Each is [ok, text]. - * @param {CommitRecord} r - * @returns {[boolean, string][]} - */ -function checks(r) { - /** @type {[boolean, string][]} */ - const out = []; - const has = (/** @type {RegExp} */ re) => r.files.some(f => re.test(f.path)); - for (const f of r.fragments) - out.push( - f.error - ? [false, `${f.path}: ${f.error.replace(/^[^:]*: /, "")}`] - : [true, `Changelog fragment ${f.path}`], - ); - if (has(/^CHANGELOG\.md$/)) - out.push([ - false, - "Edits CHANGELOG.md; a changes/ fragment carries the entry outside release PRs", - ]); - - const testsValue = r.trailers.get("Tests"); - const testFiles = r.files.filter(f => - /^tests\/|^rewatch\/tests\/|\.test\.js$/.test(f.path), - ); - if (testsValue === undefined) out.push([false, "No Tests trailer"]); - else if (/^none\b/.test(testsValue)) - out.push( - testFiles.length - ? [ - false, - `Tests: none, but the diff changes ${testFiles.length} test file${testFiles.length === 1 ? "" : "s"}`, - ] - : [true, "No test changes, as recorded"], - ); - - // Output of existing programs: modified generated JS. A new test's generated - // JS is output of a new program, counted separately. - const changedJs = r.files.filter( - f => isGeneratedJs(f.path) && f.status !== "A", - ); - const newJs = r.files.filter(f => isGeneratedJs(f.path) && f.status === "A"); - const jsFiles = (/** @type {number} */ n) => - `${n} committed generated JS file${n === 1 ? "" : "s"}`; - const newNote = newJs.length ? `; ${newJs.length} new` : ""; - const output = r.trailers.get("Output"); - if (output !== undefined) { - if (/^unchanged/.test(output) && changedJs.length) - out.push([ - false, - `Output: unchanged, but ${jsFiles(changedJs.length)} change${changedJs.length === 1 ? "s" : ""}${newNote}`, - ]); - else if (/^unchanged/.test(output)) - out.push([true, `No committed generated JS changes${newNote}`]); - else - out.push([ - true, - `${jsFiles(changedJs.length)} change${changedJs.length === 1 ? "s" : ""}${newNote}`, - ]); - } - - const bumped = [ - ...r.patch.matchAll(/^\+(?:let|and) (\w+)_magic_number = "([^"]+)"/gm), - ].map(m => `${m[1]} ${m[2]}`); - const formats = r.trailers.get("Formats"); - if (bumped.length) - out.push([ - formats !== undefined && formats !== "none", - `Magic numbers bumped: ${bumped.join(", ")}`, - ]); - else if (has(/^compiler\/ml\/(parsetree|typedtree|types|lambda)\.mli?$/)) - out.push([ - formats !== undefined && formats !== "none", - "Marshaled AST, typed tree or Lambda type changes with no magic number bump", - ]); - if (has(/^compiler\/ml\/parsetree0\.ml$/)) - out.push([false, "Changes the frozen compiler/ml/parsetree0.ml"]); - if (has(/^compiler\/ml\/ast_mapper_(from|to)0\.ml$/)) - out.push([ - has( - /^tests\/syntax_tests\/data\/ast-mapping\/|^tests\/ounit_tests\/ounit_ast_mapper0_tests\.ml$/, - ), - "v0 PPX bridge changes; ast-mapping fixture or mapper unit test in the diff", - ]); - const newFixtures = r.files.filter( - f => - f.status === "A" && - /^tests\/build_tests\/super_errors[^/]*\/fixtures\//.test(f.path), - ); - if (newFixtures.length) - out.push([ - has(/^tests\/ERROR_VARIANTS\.md$/) || r.trailers.has("Errors"), - `${newFixtures.length} new super_errors fixture${newFixtures.length === 1 ? "" : "s"}; ERROR_VARIANTS.md ${has(/^tests\/ERROR_VARIANTS\.md$/) ? "updated" : r.trailers.has("Errors") ? "unchanged, as the Errors trailer states" : "unchanged"}`, - ]); - return out; -} - -// --------------------------------------------------------------------------- -// Cards - -/** @type {Record} */ -const CATEGORIES = { - breaking: ["💥", "Breaking change"], - feature: ["🚀", "New feature"], - fix: ["🐛", "Bug fix"], - docs: ["📝", "Documentation"], - polish: ["💅", "Polish"], - internal: ["🏠", "Internal"], - none: ["", "No changelog"], -}; - -/** @param {string} category */ -const changelogPill = category => { - const [emoji, label] = CATEGORIES[category] ?? ["", category]; - return `${emoji ? `${emoji} ` : ""}${esc(label)}`; -}; - -/** @param {string} value */ -function tests(value) { - /** @type {Map} */ - const groups = new Map(); - for (const item of value.split(/\s+(?=[+~-]\S)/)) { - const kind = - item[0] === "+" - ? "add" - : item[0] === "~" - ? "upd" - : item[0] === "-" - ? "del" - : ""; - const body = kind ? item.slice(1) : item; - const suite = /^tests\/syntax_tests\//.test(body) - ? "syntax" - : /^tests\/tests\//.test(body) - ? "integration" - : /^tests\/ounit_tests\//.test(body) - ? "ounit" - : /^tests\/build_tests\/super_errors/.test(body) - ? "super_errors" - : /^tests\/build_tests\//.test(body) - ? "build" - : /^tests\/analysis_tests\//.test(body) - ? "analysis" - : /^tests\/gentype_tests\//.test(body) - ? "gentype" - : /^rewatch\/tests\//.test(body) - ? "rewatch integration" - : /^rewatch\/src\//.test(body) - ? "cargo test" - : "other"; - // The suite's root directory heads the group; items show the rest. - const root = - /^(tests\/(syntax_tests\/data|tests\/src|ounit_tests|build_tests)\/|rewatch\/(tests|src)\/)/.exec( - body, - )?.[0] ?? ""; - const label = `${suite}${root ? ` ${esc(root)}` : ""}`; - /** @type {Record} */ - const MARKS = { add: "+", upd: "~", del: "−" }; - const mark = MARKS[kind] ?? ""; - groups.set(label, [ - ...(groups.get(label) ?? []), - `${mark ? `${mark} ` : ""}${esc(body.slice(root.length))}`, - ]); - } - return `
${[...groups] - .map( - ([label, items]) => - `
${label}${items.join("")}
`, - ) - .join("")}
`; -} - -/** - * A state trailer: its leading word as a chip, the rest as prose. - * @param {string} v - */ -const stateChip = v => { - const [, word, rest] = /^([^\s,]+),?\s*(.*)$/s.exec(v) ?? ["", v, ""]; - const cls = /^(unchanged|none)$/.test(word) ? "same" : "diff"; - return `
${esc(word)}${rest ? `${inline(rest)}` : ""}
`; -}; - -/** - * Splits on "; " outside parentheses: the items of a Measured trailer. - * @param {string} v - */ -function items(v) { - const out = []; - let depth = 0; - let start = 0; - for (let i = 0; i < v.length; i++) { - if (v[i] === "(") depth++; - else if (v[i] === ")") depth = Math.max(0, depth - 1); - else if (v[i] === ";" && depth === 0) { - out.push(v.slice(start, i).trim()); - start = i + 1; - } - } - out.push(v.slice(start).trim()); - return out.filter(Boolean); -} - -const FACTS = [ - "Output", - "Formatting", - "Formats", - "PPX", - "Editor", - "Errors", - "Tests", - "Measured", -]; - -/** @param {Section[]} sections */ -function renderSections(sections) { - const out = []; - for (let i = 0; i < sections.length; i++) { - const s = sections[i]; - if (["Breaking", "Review"].includes(s.name)) continue; - const next = sections[i + 1]; - if ( - s.name === "Before" && - next?.name === "After" && - next.lang === s.lang && - s.block.length && - next.block.length - ) { - const before = new Set(s.block.map(l => l.trim())); - const after = new Set(next.block.map(l => l.trim())); - out.push( - `
${[ - [s, after, "del", "Before"], - [next, before, "add", "After"], - ] - .map( - ([sec, other, mark, label]) => - `
${label} ${esc(langName(/** @type {Section} */ (sec).lang))}${ - /** @type {Section} */ (sec).text.length - ? ` · ${inline(/** @type {Section} */ (sec).text.join(" "))}` - : "" - }
${codeLines(
-                /** @type {Section} */ (sec).block,
-                /** @type {Section} */ (sec).lang,
-                /** @type {Set} */ (other),
-                /** @type {string} */ (mark),
-              )}
`, - ) - .join("")}
`, - ); - i++; - continue; - } - if (s.name === "Source" && !s.block.length) { - out.push( - `

Source ${inline(s.text.join(" "))}

`, - ); - continue; - } - if (s.name) - out.push( - `

${esc(s.name)}${s.lang ? ` ${esc(langName(s.lang))}` : ""}

`, - ); - if (s.text.length) out.push(`

${inline(s.text.join(" "))}

`); - if (s.block.length) { - const lang = s.lang || (s.name === "Example" ? "rescript" : ""); - const body = - lang === "text" - ? esc(s.block.join("\n").trimEnd()) - : codeLines(s.block, lang, undefined, ""); - out.push( - `
${body}
`, - ); - } - } - return out.join("\n"); -} - -/** @param {string} lang */ -const langName = lang => - ({ - rescript: "ReScript", - js: "JavaScript", - ts: "TypeScript", - sh: "shell", - rust: "Rust", - ocaml: "OCaml", - text: "output", - })[lang] ?? lang; - -/** @param {CommitRecord} r */ -function card(r) { - const categories = [...new Set(r.fragments.map(f => f.category))]; - const pr = ( - r.trailers.get("PR") ?? - /\(#(\d+)\)\s*$/.exec(r.subject)?.[0] ?? - "" - ).replace(/\D/g, ""); - const subject = r.subject.replace(/\s*\(#\d+\)\s*$/, ""); - const out = [ - `
`, - ]; - out.push( - `
${r.sha} · ${esc(r.author)} · ${esc(r.date)}${ - pr ? ` · #${pr} discussion ↗` : "" - }${r.noted ? ' · note' : ""}
` + - `

${inline(subject)}

` + - `
${categories.map(changelogPill).join("")}${words( - r, - "Area", - ) - .map(a => `${esc(a)}`) - .join("")}
`, - ); - const breaking = r.sections.find(s => s.name === "Breaking"); - if (breaking) - out.push( - ``, - ); - out.push(renderSections(r.sections)); - for (const f of r.fragments) - out.push( - `
${changelogPill(f.category)}${bullets(f.text).map(inline).join("
")}
`, - ); - const reviews = r.sections.filter(s => s.name === "Review"); - if (reviews.length) - out.push( - `

Review

    ${reviews - .map(s => `
  • ${inline([...s.text, ...s.block].join(" "))}
    • `) - .join("")}
`, - ); - const STATES = ["Output", "Formatting", "Formats", "PPX", "Editor"]; - const same = STATES.filter(k => - /^(unchanged|none)$/.test(r.trailers.get(k) ?? ""), - ); - const facts = FACTS.flatMap(k => { - const v = r.trailers.get(k); - if (v === undefined || same.includes(k)) return []; - const one = v.replace(/\n/g, " "); - const listed = items(one); - const shown = - k === "Tests" - ? /^none\b/.test(one) - ? stateChip(one) - : tests(one) - : STATES.includes(k) - ? stateChip(one) - : k === "Measured" && listed.length > 1 - ? `
    ${listed.map(l => `
  • ${inline(l)}
    • `).join("")}
` - : v - .split("\n") - .map(line => `${inline(line)}`) - .join(""); - return [`
${k}
${shown}
`]; + patch: git(["show", "--format=", rev, "--", "compiler/ext/config.ml"]), }); - if (same.length) - facts.unshift( - `
No effect
${same.map(k => `${k}: ${esc(r.trailers.get(k) ?? "")}`).join("")}
`, - ); - if (facts.length) out.push(`
${facts.join("")}
`); - out.push( - `
    ${checks(r) - .map( - ([ok, text]) => `
  • ${esc(text)}
    • `, - ) - .join("")}
`, - ); - /** @type {Map} */ - const byArea = new Map(); - for (const f of r.files) { - const a = areaOf(f.path); - const e = byArea.get(a) ?? { n: 0, a: 0, d: 0 }; - byArea.set(a, { n: e.n + 1, a: e.a + f.added, d: e.d + f.removed }); - } - const added = r.files.reduce((n, f) => n + f.added, 0); - const removed = r.files.reduce((n, f) => n + f.removed, 0); - out.push( - `
${r.files.length} files · +${added} −${removed} · ${[ - ...byArea, - ] - .map(([a, e]) => `${esc(a)} ${e.n}`) - .join(" · ")}${r.files - .map( - f => - ``, - ) - .join("")}
${f.status}${esc(f.path)}+${f.added}−${f.removed}
`, - ); - if (r.coauthors.length) - out.push(`
with ${esc(r.coauthors.join(" · "))}
`); - out.push("
"); - return out.join("\n"); } -// --------------------------------------------------------------------------- -// Ranges - /** - * @typedef {{ name: string, range?: string, records: CommitRecord[] }} Group + * A revision is one card; a range `base..tip` is its commits, oldest first. + * @param {string} arg + * @returns {Group} */ - -/** @param {string} arg */ function group(arg) { if (!arg.includes("..")) return { name: arg, records: [readRecord(arg)] }; const revs = git(["rev-list", "--reverse", "--no-merges", arg]) .split("\n") .filter(Boolean); return { - name: arg.split("..")[1], + name: arg.split("..")[1] || arg, range: arg, records: revs.map(readRecord), }; } -/** - * The fragments a range leaves in changes/: the last version of each one its - * commits add or change, minus those a later commit deletes. - * @param {Group} g - */ -function rangeFragments(g) { - /** @type {Map} */ - const byPath = new Map(); - for (const r of g.records) { - for (const f of r.files) if (f.status === "D") byPath.delete(f.path); - for (const f of r.fragments) byPath.set(f.path, f); - } - return [...byPath.values()]; -} - -/** @param {Group[]} groups */ -function summary(groups) { - const ranges = groups.filter(g => g.range); - if (!ranges.length) return ""; - const rows = ranges.map(g => { - const cats = new Map(); - for (const f of rangeFragments(g)) - cats.set(f.category, (cats.get(f.category) ?? 0) + 1); - if (!cats.size) cats.set("none", 1); - const areas = [...new Set(g.records.flatMap(r => words(r, "Area")))]; - const js = g.records.filter(r => - /^changed/.test(r.trailers.get("Output") ?? ""), - ).length; - const formats = [ - ...new Set( - g.records - .map(r => r.trailers.get("Formats") ?? "none") - .filter(v => v !== "none"), - ), - ]; - const warnings = g.records.reduce( - (n, r) => n + checks(r).filter(([ok]) => !ok).length, - 0, - ); - return ( - `${esc(g.range ?? g.name)}${g.records.length}` + - `${[...cats].map(([c, n]) => `${changelogPill(c)}${n > 1 ? ` ×${n}` : ""}`).join(" ")}` + - `${areas.map(a => `${esc(a)}`).join("")}` + - `${js}${formats.map(inline).join("
") || 'none'}` + - `${warnings ? `${warnings}` : "0"}` - ); - }); - /** @type {Map} */ - const owners = new Map(); - for (const g of ranges) - for (const r of g.records) - for (const f of r.files) - owners.set(f.path, [ - ...(owners.get(f.path) ?? []), - ranges.length > 1 ? g.name : r.sha, - ]); - const shared = [...owners] - .map(([f, os]) => /** @type {[string, string[]]} */ ([f, [...new Set(os)]])) - .filter(([, os]) => os.length > 1) - .sort(); - /** @type {Map} */ - const draft = new Map(); - for (const g of ranges) - for (const f of rangeFragments(g)) - draft.set(f.category, [ - ...(draft.get(f.category) ?? []), - ...bullets(f.text), - ]); - const order = Object.keys(CATEGORIES); - return ( - `

Summary

${rows.join("")}
RangeCommitsChangelogAreasJS output changesFormatsFailed checks
` + - (draft.size - ? `

Changelog draft

${[...draft] - .sort((a, b) => order.indexOf(a[0]) - order.indexOf(b[0])) - .map( - ([c, es]) => - `
${changelogPill(c)}
    ${es.map(e => `
  • ${inline(e)}
    • `).join("")}
`, - ) - .join("")}
` - : "") + - (shared.length - ? `

Files changed by more than one ${ranges.length > 1 ? "range" : "commit"}

${shared - .map( - ([f, os]) => - ``, - ) - .join("")}
${esc(f)}${os.map(o => (ranges.length > 1 ? esc(o) : `${o}`)).join(", ")}
` - : "") + - `
` - ); -} - -// --------------------------------------------------------------------------- -// Page - -const LIGHT = `--bg:#f7f6f4; --surface:#ffffff; --ink:#1f1d1c; --muted:#6b6560; --rule:#e2ded9; --accent:#d43a3f; --accent-soft:#fbe9e9; - --code-bg:#f4f2ef; --code-ink:#2a2725; --k:#a3238e; --s:#1f7a3a; --n:#b35c00; --c:#8a8580; --t:#1d5fa8; --a:#8a5a00; --o:#6b6560; --p:#7a3db8; - --add:#e3f4e6; --del:#fbe5e3; --ok:#23784a; --ok-bg:#e2f3e8; --warn:#9a5b00; --warn-bg:#fdf0d9; - --same:#5a6470; --same-bg:#eceff2; --diff:#1d5fa8; --diff-bg:#e3edf9; --plus:#23784a; --minus:#b3261e;`; -const DARK = `--bg:#151413; --surface:#1e1c1b; --ink:#ece8e4; --muted:#a39d97; --rule:#36322f; --accent:#ff6b6f; --accent-soft:#3a1d1e; - --code-bg:#252220; --code-ink:#e6e1dc; --k:#e48ad6; --s:#8fd6a0; --n:#f2b36b; --c:#8a847e; --t:#8ab8f0; --a:#e0bc6a; --o:#b5aea7; --p:#c6a2f0; - --add:#1c3324; --del:#3a1f1d; --ok:#8fd6a0; --ok-bg:#1c3324; --warn:#f2c36b; --warn-bg:#3a2d16; - --same:#b0b8c2; --same-bg:#2a2e33; --diff:#8ab8f0; --diff-bg:#1d2b3d; --plus:#8fd6a0; --minus:#f19a93; color-scheme:dark;`; - -const CSS = ` -:root { ${LIGHT} - --display:"Fraunces",Georgia,serif; --body:"Inter","Helvetica Neue",Arial,sans-serif; --mono:"JetBrains Mono",Menlo,Consolas,monospace; } -@media (prefers-color-scheme: dark) { :root:not([data-theme="light"]) { ${DARK} } } -:root[data-theme="dark"] { ${DARK} } -* { box-sizing:border-box } body { margin:0; background:var(--bg); color:var(--ink); font:15px/1.6 var(--body) } -main { max-width:1120px; margin:0 auto; padding:32px 16px 64px; display:flex; flex-direction:column; gap:22px } -h1 { font:600 30px/1.2 var(--display); margin:0; text-wrap:balance } h1.brand { font-size:34px } h1 span, .kicker span { color:var(--accent) } h1 a { text-decoration:none } -.kicker { font:600 13px/1 var(--display); letter-spacing:.04em; margin:0 0 8px; color:var(--muted) } -h2 { font:600 22px/1.2 var(--display); margin:16px 0 8px; padding-top:14px; border-top:2px solid var(--ink) } -h3 { font:600 20px/1.3 var(--display); margin:4px 0 8px; text-wrap:balance } -h4 { font:600 11px/1 var(--body); letter-spacing:.08em; text-transform:uppercase; color:var(--muted); margin:14px 0 6px } -p { margin:2px 0; max-width:75ch } .lede, .muted { color:var(--muted) } -a { color:var(--accent) } :focus-visible { outline:2px solid var(--accent); outline-offset:2px } -code { font:12.5px/1.5 var(--mono); background:var(--code-bg); padding:1px 4px; border-radius:3px; overflow-wrap:anywhere } -.card { background:var(--surface); border:1px solid var(--rule); border-radius:10px; padding:20px 22px; display:flex; flex-direction:column; gap:6px; min-width:0 } -.card.breaking { border-top:4px solid var(--accent) } -.meta { font:12px/1.5 var(--mono); color:var(--muted) } .meta code { background:none; padding:0 } -.chips { display:flex; flex-wrap:wrap; gap:6px; align-items:center } -.chip { font:500 12px/1 var(--body); border:1px solid var(--rule); border-radius:99px; padding:5px 9px; color:var(--muted) } -.pill { font:600 12px/1 var(--body); padding:6px 10px; border-radius:99px; white-space:nowrap; background:var(--same-bg); color:var(--same) } -.pill.cat-breaking { background:var(--accent); color:#fff } .pill.cat-fix { background:var(--warn-bg); color:var(--warn) } -.pill.cat-feature { background:var(--ok-bg); color:var(--ok) } .pill.cat-polish { background:var(--diff-bg); color:var(--diff) } -.banner { background:var(--accent-soft); border-left:4px solid var(--accent); padding:10px 14px; border-radius:6px; margin:4px 0 } -.banner strong { color:var(--accent); margin-right:6px; text-transform:uppercase; font-size:12px; letter-spacing:.06em } -pre { margin:4px 0; padding:12px 0; border-radius:8px; overflow-x:auto; background:var(--code-bg); color:var(--code-ink); font:12.5px/1.6 var(--mono) } -pre .ln { display:block; padding:0 14px; white-space:pre; min-height:1.6em } pre .ln.add { background:var(--add) } pre .ln.del { background:var(--del) } -pre.term { padding:12px 14px; white-space:pre } -.k { color:var(--k) } .s { color:var(--s) } .n { color:var(--n) } .c { color:var(--c); font-style:italic } .t { color:var(--t) } .a { color:var(--a) } .o { color:var(--o) } .p { color:var(--p) } -.pair { display:grid; grid-template-columns:minmax(0,1fr) minmax(0,1fr); grid-template-rows:auto auto; column-gap:10px; margin:8px 0 } -figure { margin:0; min-width:0; display:grid; grid-row:span 2; grid-template-rows:subgrid; row-gap:0 } figure pre { margin:0 } figcaption { font:12px/1.4 var(--body); color:var(--muted); margin-bottom:4px } figcaption b { font-weight:600; font-size:11px; letter-spacing:.06em; text-transform:uppercase } -.lang { font:500 10px/1 var(--mono); letter-spacing:0; text-transform:none; border:1px solid var(--rule); border-radius:4px; padding:2px 5px; color:var(--muted) } -.entry { display:flex; gap:10px; align-items:baseline; background:var(--code-bg); padding:10px 12px; border-radius:8px; margin-top:8px; font-size:14px; flex-wrap:wrap } -.entry span:last-child { flex:1 1 300px; min-width:0 } -ul.review { margin:0; padding-left:18px } ul.review li { margin:2px 0 } -dl.facts { display:grid; grid-template-columns:max-content minmax(0,1fr); gap:8px 16px; margin:12px 0 0; padding-top:12px; border-top:1px solid var(--rule); font-size:13.5px } -dt { font:600 11px/2 var(--body); letter-spacing:.06em; text-transform:uppercase; color:var(--muted) } -dd { margin:0; min-width:0; display:flex; flex-direction:column; gap:3px } dd.row, .row { display:flex; flex-direction:row; flex-wrap:wrap; gap:6px; align-items:baseline } -.source { font-size:12.5px; color:var(--muted) } .source b { font-weight:600; font-size:11px; letter-spacing:.06em; text-transform:uppercase; margin-right:4px } -.state { align-self:flex-start; padding:3px 8px; border-radius:5px } .state.same { background:var(--same-bg); color:var(--same) } .state.diff { background:var(--diff-bg); color:var(--diff) } -.delta { font-variant-numeric:tabular-nums; white-space:nowrap } .delta .from { color:var(--muted); text-decoration:line-through } .delta .to { font-weight:600; color:var(--ok) } -.tests { display:flex; flex-direction:column; gap:4px } .tests > div { display:flex; flex-wrap:wrap; gap:4px; align-items:baseline } -.suite { font:600 11px/1 var(--body); color:var(--muted); min-width:9em } -.test { font:12px/1.3 var(--mono); padding:3px 7px; border-radius:4px; background:var(--code-bg); overflow-wrap:anywhere } -.test.add { background:var(--ok-bg); color:var(--ok) } .test.del { background:var(--del); color:var(--minus) } .test b { font-weight:600 } -.suite code { font-size:11px; font-weight:400; background:none; padding:0 } .run { margin:0; padding-left:18px } .run li { margin:1px 0 } -ul.checks { list-style:none; margin:10px 0 0; padding:0; display:flex; flex-wrap:wrap; gap:6px; font-size:12.5px } -ul.checks li { padding:3px 9px; border-radius:5px } ul.checks li.ok { background:var(--ok-bg); color:var(--ok) } ul.checks li.ok::before { content:"✓ " } -ul.checks li.warn { background:var(--warn-bg); color:var(--warn) } ul.checks li.warn::before { content:"! " } -details { margin-top:8px; font-size:13px; color:var(--muted) } summary { cursor:pointer } -table { border-collapse:collapse; width:100%; font-size:13px } th, td { text-align:left; vertical-align:top; padding:7px 10px; border-bottom:1px solid var(--rule) } -th { font:600 11px/1.2 var(--body); letter-spacing:.06em; text-transform:uppercase; color:var(--muted); white-space:nowrap } -td.num { font-variant-numeric:tabular-nums; text-align:right; white-space:nowrap } .plus { color:var(--plus) } .minus { color:var(--minus) } -table.files td { padding:3px 8px } td.st { font:600 11px var(--mono) } .st-A { color:var(--plus) } .st-D { color:var(--minus) } -.table-wrap { overflow-x:auto; border:1px solid var(--rule); border-radius:10px; background:var(--surface) } -.draft { display:flex; flex-direction:column; gap:10px; background:var(--surface); border:1px solid var(--rule); border-radius:10px; padding:14px 16px } -td.nowrap code { white-space:nowrap } .draft ul { margin:6px 0 0; padding-left:18px } .warn-n { color:var(--warn); font-weight:600 } -pre, code, .test { font-variant-ligatures:none; font-feature-settings:"calt" 0, "liga" 0 } -@media (max-width:720px) { .pair { grid-template-columns:minmax(0,1fr); grid-template-rows:none; row-gap:10px } figure { grid-row:auto; grid-template-rows:auto auto } dl.facts { grid-template-columns:minmax(0,1fr) } dt { line-height:1.2; margin-top:6px } .card { padding:16px } }`; - const args = process.argv.slice(2); /** @type {Record} */ const options = {}; +/** @type {string[]} */ const targets = []; let fragment = false; for (let i = 0; i < args.length; i++) { @@ -991,37 +145,17 @@ for (let i = 0; i < args.length; i++) { const outFile = options.out; if (!outFile) { console.error( - "usage: commitRecords.js --out [--pr ] [--title ] [--fragment] ...", + "usage: commitRecords.js --out [--pr ] [--title ] [--fragment] ...", ); process.exit(1); } -const pr = (options.pr ?? "").replace(/\D/g, ""); -const title = options.title ?? ""; -const groups = targets.map(group); -for (const g of groups) for (const r of g.records) pageShas.add(r.sha); -const commits = groups.reduce((n, g) => n + g.records.length, 0); -const commitCount = `${commits} commit${commits === 1 ? "" : "s"}, rendered from ${commits === 1 ? "its record" : "their records"}.`; -const heading = - pr || title - ? `

Semantic Diff

${pr ? `#${pr} ` : ""}${inline(title)}

` - : `

Semantic Diff

`; -const body = `
${heading}

${commitCount}

-${summary(groups)} -${groups - .map( - g => - (g.range - ? `

${esc(g.range)}

` - : "") + g.records.map(card).join("\n"), - ) - .join("\n")} -
`; -const pageTitle = - pr || title - ? `${pr ? `#${pr} ` : ""}${title} · Semantic Diff` - : "Semantic Diff"; -const head = `${esc(pageTitle)} - +const heading = { + pr: (options.pr ?? "").replace(/\D/g, ""), + title: options.title ?? "", +}; +const body = pageBody(targets.map(group), heading); +const head = `${esc(pageTitle(heading))} + `; writeFileSync( outFile, diff --git a/scripts/commitRecords.test.js b/scripts/commitRecords.test.js index 510a458d72..fb3bb35e41 100644 --- a/scripts/commitRecords.test.js +++ b/scripts/commitRecords.test.js @@ -12,6 +12,19 @@ const record = `Compile x + 0 to x Integer addition of the literal 0 compiles to the operand. +Why: + \`x + 0\` on an int compiles to \`x + 0 | 0\`, which is \`x\`. + +AST modified (Pexp_apply of +): + Before: + Pexp_apply + funct: Pexp_ident "+" + args: + Pexp_ident "x" + Pexp_constant 0 + After: + Pexp_ident "x" + Example (rescript): let f = (x: int) => x + 0 @@ -23,9 +36,10 @@ After (js): Source: bsc before and after this commit. +Review: Does it hold for floats? No: \`x +. 0.\` keeps the addition. + Area: codegen tests Output: changed integer + 0; one snapshot -Formats: none Tests: +tests/tests/src/a_test.mjs ~tests/tests/src/b_test.mjs Measured: build 10 s -> 8 s (make, 1 run); bundle 5 KB -> 4 KB (esbuild) Signed-off-by: A Person @@ -87,21 +101,55 @@ test("renders a record and checks it against the diff", async () => { ); has(html, "

Compile x + 0 to x

"); has(html, '

Semantic Diff

'); - has(html, ''); - has(html, ''); + has(html, "Semantic Diff"); + // the JavaScript Before/After pair, side by side in one row, changed + // lines marked + has( + html, + '
Before js', + ); + has( + html, + 'return x + 0', + ); + has( + html, + 'return x;', + ); has(html, '

Source'); + // the AST change block, drawn as two trees in aligned rows + has( + html, + '

modifiedAST Pexp_apply of +

', + ); + has(html, '
'); + has( + html, + '├─\u00a0funct: Pexp_ident', + ); + has( + html, + // the operand moves to the root, so it is a changed line; After's + // shorter side is padded with gaps + 'Pexp_ident "x"', + ); + has(html, '
'); + has(html, '
  • Does it hold for floats?'); has(html, "Changelog fragment changes/identity.polish.md"); has( html, - '
    💅 PolishCompile x + 0', + '
    💅 PolishCompile x + 0', ); - has(html, "1 committed generated JS file changes; 1 new"); - has(html, "Formats: none"); + has(html, "1 committed generated JS file modified, 1 added"); + has(html, "The tests listed are in the diff"); + has(html, "Every test file in the diff is listed"); has(html, "tests/tests/src/"); + has(html, ''); has(html, "
  • bundle "); - has(html, "changelog 1"); - has(html, '0'); - has(html, "
  • Compile x + 0"); + has(html, '10 s'); + has(html, 'codegen'); + has(html, "no failed checks"); + has(html, "

    Changelog draft

    "); }); test("flags a record that disagrees with its diff", async () => { @@ -126,8 +174,9 @@ test("flags a record that disagrees with its diff", async () => { '

    #42 Fix <output> & more

    ', ); has(html, "#42 Fix <output> & more · Semantic Diff"); - has(html, "Output: unchanged, but 1 committed generated JS file changes"); + has(html, "Output: unchanged, but 1 committed generated JS file modified"); has(html, "No Tests trailer"); + has(html, 'No changelog'); }); test("accepts Tests: none with a reason, and flags it against test files", async () => { @@ -164,6 +213,139 @@ test("accepts Tests: none with a reason, and flags it against test files", async has(html, 'No changelog'); }); +test("checks the tests a Tests trailer lists against the diff", async () => { + const html = await render( + async (git, dir) => { + await write(dir, { "README.md": "a\n" }); + git("add", "."); + git("commit", "-q", "-m", "base"); + await write(dir, { "tests/tests/src/d_test.res": "let x = 1\n" }); + git("add", "."); + git( + "commit", + "-q", + "-m", + "Add a test\n\nArea: tests\nOutput: unchanged\nTests: +tests/tests/src/d_test.res ~tests/tests/src/e_test.res", + ); + }, + ["HEAD"], + ); + has( + html, + "Tests lists tests/tests/src/e_test.res, which the diff does not change", + ); +}); + +test("flags test files the diff changes that Tests does not list", async () => { + const html = await render( + async (git, dir) => { + await write(dir, { + "tests/tests/src/f_test.res": "let x = 1\n", + "scripts/a.test.js": "a\n", + }); + git("add", "."); + git("commit", "-q", "-m", "base"); + await write(dir, { + "tests/tests/src/f_test.res": "let x = 2\n", + "tests/tests/src/f_test.mjs": "let x = 2;\n", + "tests/syntax_tests/data/printer/expected/g.res.txt": "g\n", + "tests/syntax_tests/data/printer/g.res": "g\n", + "tests/ounit_tests/h_tests.ml": "let () = ()\n", + }); + git("rm", "-q", "scripts/a.test.js"); + git("add", "."); + git( + "commit", + "-q", + "-m", + "Change tests\n\nArea: tests\nOutput: unchanged\nTests: ~tests/tests/src/f_test.res +tests/syntax_tests/data/printer/g.res", + ); + }, + ["HEAD"], + ); + // the generated JavaScript and the expected output are snapshots, which + // the trailer may leave out + has( + html, + "Tests omits test files the diff changes: -scripts/a.test.js +tests/ounit_tests/h_tests.ml", + ); + has(html, "The tests listed are in the diff"); +}); + +test("draws an added or removed object across the block", async () => { + const long = `and raw_source = Raw_string of string loc (* ${"the JavaScript source, ".repeat(6)}*)`; + const html = await render( + async (git, dir) => { + await write(dir, { "README.md": "a\n" }); + git("add", "."); + git("commit", "-q", "-m", "base"); + await write(dir, { "README.md": "b\n" }); + git("add", "."); + git( + "commit", + "-q", + "-m", + `Add raw nodes\n\nIR added (raw_source):\n After:\n ${long}\n\nJS removed (braced output):\n Before (js):\n var x = 1;\n\nArea: docs\nOutput: unchanged\nTests: none, because only prose changes`, + ); + }, + ["HEAD"], + ); + has( + html, + '

    addedIR raw_source

    and raw_source',
+  );
+  has(
+    html,
+    '
    removedJS braced output
    var x',
+  );
+  assert.ok(!html.includes("none
    "), "an empty version is drawn");
+  assert.ok(!html.includes('class="versions"'), "a one-sided pair is drawn");
+  // a long line wraps, its continuation indented
+  has(html, "white-space:pre-wrap");
+  has(html, "text-indent:calc((var(--i, 0) + 2) * -1ch)");
+});
+
+test("shows and checks each of a commit's changelog fragments", async () => {
+  const link = "https://github.com/rescript-lang/rescript/pull";
+  const html = await render(
+    async (git, dir) => {
+      await write(dir, { "README.md": "a\n" });
+      git("add", ".");
+      git("commit", "-q", "-m", "base");
+      await write(dir, {
+        "changes/printer-crash.fix.md": `- Fix a printer crash. ${link}/9\n`,
+        "changes/printer-option.feature.md": `- Add a printer option. ${link}/XXXX\n`,
+      });
+      git("add", ".");
+      git(
+        "commit",
+        "-q",
+        "-m",
+        "Fix and extend the printer\n\nArea: printer\nOutput: unchanged\nTests: none, because the fragments are the change",
+      );
+    },
+    ["HEAD~1..HEAD"],
+  );
+  has(html, "Changelog fragment changes/printer-crash.fix.md");
+  has(
+    html,
+    "changes/printer-option.feature.md: replace the placeholder https://github.com/rescript-lang/rescript/pull/XXXX with the PR link",
+  );
+  has(
+    html,
+    '
    🐛 Bug fixFix a printer crash. #9
    ',
+  );
+  has(
+    html,
+    '
    🚀 New featureAdd a printer option.',
+  );
+  has(
+    html,
+    'changelog: 🐛 Bug fix 🚀 New feature',
+  );
+  has(html, "1 failed check");
+});
+
 test("takes the changelog from the fragments a range leaves", async () => {
   const link = "https://github.com/rescript-lang/rescript/pull";
   const html = await render(
@@ -176,14 +358,19 @@ test("takes the changelog from the fragments a range leaves", async () => {
         "changes/gone.feature.md": `- Add a thing. ${link}/7\n`,
       });
       git("add", ".");
-      git("commit", "-q", "-m", "Draft\n\nArea: compiler");
+      git("commit", "-q", "-m", "Draft\n\nArea: compiler\nOutput: unchanged");
       await write(dir, {
         "changes/crash.fix.md": `- Fix a crash\n  in the printer. ${link}/8\n- Fix another. ${link}/8\n`,
         "CHANGELOG.md": "# Changelog\n\n- Edited.\n",
       });
       git("rm", "-q", "changes/gone.feature.md");
       git("add", ".");
-      git("commit", "-q", "-m", "Link the PR\n\nArea: compiler");
+      git(
+        "commit",
+        "-q",
+        "-m",
+        "Link the PR\n\nArea: compiler\nOutput: unchanged",
+      );
     },
     ["HEAD~2..HEAD"],
   );
@@ -196,10 +383,44 @@ test("takes the changelog from the fragments a range leaves", async () => {
     html,
     "Edits CHANGELOG.md; a changes/ fragment carries the entry outside release PRs",
   );
-  has(html, '🐛 Bug fix');
+  has(html, 'changelog: 🐛 Bug fix
    • '); has( html, '
    🐛 Bug fix
    • Fix a crash in the printer. #8
    • Fix another.', ); assert.ok(!html.includes("
    • Add a thing."), "deleted fragment in draft"); }); + +test("the commit-msg hook requires the record's trailers", async () => { + const hook = fileURLToPath(new URL("./commitRecordHook.sh", import.meta.url)); + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "commit-record-hook-")); + execFileSync("git", ["init", "-q"], { cwd: dir }); + await write(dir, { "compiler/a.ml": "let x = 1\n" }); + execFileSync("git", ["add", "."], { cwd: dir }); + const run = async (/** @type {string} */ message) => { + await write(dir, { msg: message }); + try { + execFileSync("bash", [hook, "msg"], { cwd: dir, stdio: "pipe" }); + return ""; + } catch (e) { + return String(/** @type {{ stderr: Buffer }} */ (e).stderr); + } + }; + assert.equal( + await run( + "Add a\n\nWhy:\n a is missing\n\nCode added (a):\n let x = 1\n\nArea: compiler\nOutput: unchanged\nTests: none, because a is unused\nSigned-off-by: A Person \n", + ), + "", + ); + const refused = await run( + "Add a\n\nArea: compiler\nOutput: same\nSigned-off-by: A Person \nCo-Authored-By: B \n", + ); + for (const problem of [ + "no 'Tests:' trailer", + "the last trailer is 'Co-Authored-By'", + "Output 'same' starts with neither", + "no 'Why:' section", + "no change block", + ]) + assert.ok(refused.includes(problem), `hook output lacks ${problem}`); +}); diff --git a/scripts/recordFacts.js b/scripts/recordFacts.js new file mode 100755 index 0000000000..a4fa77d136 --- /dev/null +++ b/scripts/recordFacts.js @@ -0,0 +1,207 @@ +#!/usr/bin/env node + +// @ts-check + +/* Prints the exact forms a commit record abstracts, for one ReScript example, + * as a given compiler build produces them: the facts an author starts from + * when writing a change block (.claude/skills/records/SKILL.md). Each view is + * the compiler's own output; nothing is reformatted except that colour codes + * are removed. + * + * node scripts/recordFacts.js [--bsc ] [--jsx] [view...] + * node scripts/recordFacts.js [--bsc ] -e '' [view...] + * + * Views (default: types lambda js): + * parse the parser's tree (bsc -bs-no-builtin-ppx -only-parse -dparsetree) AST, Syntax + * expanded the tree after the built-in PPX (bsc -only-parse -dparsetree) Transform + * transform the source after the JSX transform (bsc -reprint-source) Transform + * format the formatter's output (bsc -format) Format + * typed the typed tree (bsc -dtypedtree) Typedtree + * types the inferred interface (rescript-editor-analysis createInterface) Typing, API + * lambda Lambda as translated (bsc -drawlambda) Lambda, Matching + * passes Lambda after each pass that changes it (bsc -debug-ir) Lambda + * js the generated ES module (bsc -bs-package-output esmodule) JS + * diagnostic what the compiler reports, when it reports anything Diagnostic + * all every view + * + * `--bsc` selects the build: by default the checkout's own + * packages/@rescript//bin/bsc.exe; rescript-editor-analysis.exe is + * taken from the same directory. `--jsx` passes `-bs-jsx 4`. The example is + * compiled in a fresh temporary directory, as module `Example` for `-e`. The + * first line names the build: the compiler's path and the commit its + * checkout is at, for the record's `Source:` line. + */ + +import { spawnSync } from "node:child_process"; +import { + copyFileSync, + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { basename, dirname, join, resolve } from "node:path"; + +const VIEWS = [ + "parse", + "expanded", + "transform", + "format", + "typed", + "types", + "lambda", + "passes", + "js", + "diagnostic", +]; + +const args = process.argv.slice(2); +/** @param {string} flag */ +const option = flag => { + const at = args.indexOf(flag); + if (at < 0) return undefined; + const [, value] = args.splice(at, 2); + return value; +}; +const root = resolve(import.meta.dirname, ".."); +const bsc = resolve( + option("--bsc") ?? + join( + root, + "packages", + "@rescript", + `${process.platform}-${process.arch}`, + "bin", + "bsc.exe", + ), +); +const source = option("-e"); +const jsxAt = args.indexOf("--jsx"); +if (jsxAt >= 0) args.splice(jsxAt, 1); +const jsx = jsxAt >= 0 ? ["-bs-jsx", "4"] : []; +const file = source === undefined ? args.shift() : undefined; +const wanted = args.includes("all") + ? VIEWS + : args.length + ? args + : ["types", "lambda", "js"]; +if ((source === undefined && !file) || wanted.some(v => !VIEWS.includes(v))) { + console.error( + "usage: recordFacts.js [--bsc ] [--jsx] | -e '' [parse|expanded|transform|format|typed|types|lambda|passes|js|diagnostic|all]...", + ); + process.exit(1); +} +if (!existsSync(bsc)) { + console.error(`no compiler at ${bsc}: build one (make) or pass --bsc`); + process.exit(1); +} + +const dir = mkdtempSync(join(tmpdir(), "record-facts-")); +const name = file ? basename(file) : "Example.res"; +if (file) copyFileSync(resolve(file), join(dir, name)); +else writeFileSync(join(dir, name), `${source}\n`); + +/** Terminal colour codes, which the compiler writes into diagnostics. */ +const COLOUR = new RegExp(`${String.fromCharCode(27)}\\[[0-9;]*m`, "g"); + +/** + * Runs a compiler binary in the example's directory. + * @param {string} exe + * @param {string[]} argv + */ +const run = (exe, argv) => { + const r = spawnSync(exe, argv, { cwd: dir, encoding: "utf8" }); + const plain = (/** @type {string} */ s) => (s ?? "").replace(COLOUR, ""); + return { out: plain(r.stdout), err: plain(r.stderr), ok: r.status === 0 }; +}; +const base = ["-bs-no-version-header", ...jsx]; + +/** @type {Record string>} */ +const show = { + parse: () => + run(bsc, [ + ...base, + "-bs-no-builtin-ppx", + "-only-parse", + "-dparsetree", + name, + ]).err, + expanded: () => run(bsc, [...base, "-only-parse", "-dparsetree", name]).err, + transform: () => { + const r = run(bsc, [...jsx, "-reprint-source", name]); + return r.out || r.err; + }, + format: () => run(bsc, ["-format", name]).out, + typed: () => run(bsc, [...base, "-bs-cmi-only", "-dtypedtree", name]).err, + types: () => { + const r = run(bsc, [...base, "-bs-cmi-only", name]); + if (!r.ok) return r.err; + const analysis = join(dirname(bsc), "rescript-editor-analysis.exe"); + return run(analysis, [ + "createInterface", + name, + name.replace(/\.res$/, ".cmi"), + ]).out; + }, + lambda: () => { + const r = run(bsc, [...base, "-drawlambda", name]); + // -drawlambda prints Lambda on stderr, before the JavaScript on stdout. + return r.err; + }, + passes: () => { + const r = run(bsc, [...base, "-debug-ir", name]); + if (!r.ok) return r.err; + const ir = join(dir, name.replace(/\.res$/, ".debug-ir")); + // A pass is printed when it changes the term; the passes that leave it + // unchanged are listed by name. + let previous = ""; + return readdirSync(ir) + .filter(f => f.endsWith(".lam")) + .sort() + .map(f => { + const term = readFileSync(join(ir, f), "utf8").trimEnd(); + const pass = f.replace(/\.lam$/, ""); + const same = term === previous; + previous = term; + return same ? `-- ${pass}: unchanged` : `-- ${pass}\n${term}`; + }) + .join("\n"); + }, + js: () => { + mkdirSync(join(dir, "out"), { recursive: true }); + const r = run(bsc, [ + ...base, + "-bs-project-root", + dir, + "-bs-package-name", + "example", + "-bs-package-output", + "esmodule:out:.mjs", + name, + ]); + const out = join(dir, "out", name.replace(/\.res$/, ".mjs")); + return r.ok && existsSync(out) ? readFileSync(out, "utf8") : r.err; + }, + diagnostic: () => { + const r = run(bsc, [...base, name]); + return r.err.trim() ? r.err : "(no diagnostic)"; + }, +}; + +/** The commit the build's checkout is at: packages/@rescript//bin. */ +const checkout = resolve(dirname(bsc), "..", "..", "..", ".."); +const commit = spawnSync( + "git", + ["-C", checkout, "rev-parse", "--short=9", "HEAD"], + { + encoding: "utf8", + }, +); +console.log( + `# ${bsc}${commit.status === 0 ? ` (checkout at ${commit.stdout.trim()})` : ""}\n`, +); +for (const view of wanted) + console.log(`== ${view}\n${show[view]().trimEnd()}\n`); diff --git a/scripts/recordRender.js b/scripts/recordRender.js new file mode 100644 index 0000000000..e455c4ceb0 --- /dev/null +++ b/scripts/recordRender.js @@ -0,0 +1,1799 @@ +// @ts-check + +/* Commit records, parsed from a commit message and drawn as HTML. Pure: no + * file system and no git, so the review page (commitRecords.js) and any later + * reader of commit messages, such as a pull request userscript, draw the same + * records the same way. The format is the `records` skill's + * (.claude/skills/records/SKILL.md): a subject, a `Why:` section, change + * blocks such as `Typing modified (rule name):`, examples, and git trailers. + * The changelog is the commit's changes/..md fragments, which + * the caller reads and validates. + */ + +const REPO_URL = "https://github.com/rescript-lang/rescript"; + +/** @param {string} s */ +export const esc = s => + s + .replace(/&/g, "&") + .replace(//g, ">") + .replace(/"/g, """); + +// --------------------------------------------------------------------------- +// The kinds of object a record changes, and how each is drawn + +/** + * How a block's content is drawn. + * @typedef {"math" | "tree" | "matrix" | "mapping" | "rescript" | "js" | "ts" | "ocaml" | "lambda" | "text" | "effect" | "prose"} Drawing + */ + +/** + * The kinds of object a change block names (the skill states what each is, + * which layer owns it and how to extract it). Each kind has one drawing and + * every drawing has a function in DRAW: a kind added here without a drawing, + * or a drawing without a function, does not type-check. + * @typedef {"Why" | "Syntax" | "Format" | "AST" | "Transform" | "Typing" | "Typedtree" | "Diagnostic" | "Matching" | "Lambda" | "IR" | "Representation" | "JS" | "API" | "TypeScript" | "Editor" | "Build" | "Semantics" | "Notation" | "Example" | "Counterexample" | "Effect" | "Code" | "Docs" | "CI"} Kind + */ + +/** @type {{ [K in Kind]: Drawing }} */ +export const KINDS = { + Why: "prose", + Syntax: "math", + Format: "rescript", + AST: "tree", + Transform: "rescript", + Typing: "math", + Typedtree: "tree", + Diagnostic: "text", + Matching: "matrix", + Lambda: "lambda", + IR: "ocaml", + Representation: "mapping", + JS: "js", + API: "rescript", + TypeScript: "ts", + Editor: "text", + Build: "prose", + Semantics: "math", + Notation: "math", + Example: "rescript", + Counterexample: "rescript", + Effect: "effect", + Code: "prose", + Docs: "prose", + CI: "prose", +}; + +/** @param {string} name @returns {name is Kind} */ +const isKind = name => Object.hasOwn(KINDS, name); + +/** Kinds whose parenthesis names a program or a version, not an object. */ +const UNNAMED = new Set(["Why", "Example", "Counterexample", "Effect"]); + +/** + * A parenthesis naming a language draws an unknown section as that code: + * `Command (sh):`, `Output (text):`. + * @type {Record} + */ +const LANGS = { + rescript: "rescript", + js: "js", + ts: "ts", + ocaml: "ocaml", + lambda: "lambda", + text: "text", + sh: "text", + json: "js", +}; + +// --------------------------------------------------------------------------- +// Records + +/** + * @typedef {{ name: string, text: string[], block: string[] }} Section + * @typedef {{ status: string, path: string, added: number, removed: number }} FileChange + * @typedef {{ path: string, category: string, text: string, error: string }} Fragment + * A changelog fragment as it stands in the commit's tree; `error` is the + * reason `yarn changelog:check` rejects it, or "". + * @typedef {{ + * sha: string, author: string, date: string, message: string, + * noted: boolean, files: FileChange[], fragments: Fragment[], patch: string + * }} CommitInfo + * `fragments` are the changelog fragments the commit adds or changes; + * `patch` is the diff of compiler/ext/config.ml, where the magic numbers are. + * @typedef {CommitInfo & { + * subject: string, sections: Section[], trailers: Map, + * coauthors: string[] + * }} CommitRecord + * @typedef {{ name: string, range?: string, records: CommitRecord[] }} Group + */ + +/** + * The trailers of a message: its last paragraph, when every line of it is + * `Key: value` or a continuation line starting with a space, as + * `git interpret-trailers --parse` reads it. A repeated key's values join + * with a line break. + * @param {string} text + */ +export function parseTrailers(text) { + /** @type {Map} */ + const trailers = new Map(); + const paragraphs = text.trimEnd().split(/\n\s*\n/); + if (paragraphs.length < 2) return trailers; + const last = paragraphs[paragraphs.length - 1].split("\n"); + const isTrailer = (/** @type {string} */ l) => /^[A-Za-z0-9-]+: /.test(l); + if (!isTrailer(last[0]) || !last.every(l => isTrailer(l) || /^\s/.test(l))) + return trailers; + /** @type {[string, string][]} */ + const entries = []; + for (const l of last) { + if (/^\s/.test(l)) entries[entries.length - 1][1] += ` ${l.trim()}`; + else { + const i = l.indexOf(":"); + entries.push([l.slice(0, i).trim(), l.slice(i + 1).trim()]); + } + } + for (const [key, value] of entries) { + const old = trailers.get(key); + trailers.set(key, old === undefined ? value : `${old}\n${value}`); + } + return trailers; +} + +/** + * A commit's record from its text (its note, else its message), with what + * git says of the commit. + * @param {string} text + * @param {CommitInfo} commit + * @returns {CommitRecord} + */ +export function parseRecord(text, commit) { + const trailers = parseTrailers(text); + const lines = text.trimEnd().split("\n"); + let end = lines.length; + if (trailers.size) while (end > 1 && lines[end - 1].trim()) end--; + /** @type {Section[]} */ + const sections = []; + /** @type {Section | undefined} */ + let current; + const prose = () => { + /** @type {Section} */ + const s = { name: "", text: [], block: [] }; + sections.push(s); + return s; + }; + for (const line of lines.slice(1, end)) { + const head = + /^([A-Z][A-Za-z]*(?: [a-z][A-Za-z]*){0,3}(?: \((?:[^()]|\([^()]*\))*\))?):\s*(.*)$/.exec( + line, + ); + if (head) { + current = { name: head[1], text: head[2] ? [head[2]] : [], block: [] }; + sections.push(current); + } else if (/^ {2,}\S/.test(line)) { + current ??= prose(); + current.block.push(line); + } else if (line.trim()) { + if (!current || current.block.length) current = prose(); + current.text.push(line.trim()); + } else if (current?.block.length) current.block.push(""); + else if (current && !current.name) current = undefined; + } + const coauthors = commit.message + .split("\n") + .filter(l => /^Co-Authored-By:/i.test(l)) + .map(l => l.replace(/^Co-Authored-By:\s*/i, "").replace(/\s*<[^>]*>/, "")); + return { + ...commit, + subject: lines[0], + sections, + trailers, + coauthors: [...new Set(coauthors)], + }; +} + +// --------------------------------------------------------------------------- +// Code, highlighted by token + +/** + * A language's tokens: a regular expression whose named groups are the + * token classes, and its keywords among the identifiers. + * @typedef {{ token: RegExp, keywords: Set }} Lang + */ + +/** @param {string} s */ +const words = s => new Set(s.split(" ")); + +/** @type {{ [L in "rescript" | "js" | "ts" | "ocaml" | "lambda"]: Lang }} */ +const LANG = { + rescript: { + token: + /(?\/\/[^\n]*|\/\*[\s\S]*?\*\/)|(?"(?:\\.|[^"\\\n])*"|`(?:\\.|[^`\\])*`|'(?:\\.|[^'\\\n])')|(?@{1,2}[A-Za-z_][\w.]*|%{1,2}[A-Za-z_][\w.]*)|(?

      #[A-Za-z_]\w*)|(?\b\d[\d_]*(?:\.\d+)?(?:[eE][+-]?\d+)?n?\b|\b0[xob][0-9a-fA-F_]+\b)|(?'[a-z]\w*)|(?[A-Za-z_][\w']*)/g, + keywords: words( + "let rec and switch if else type module open include mutable async await for in to downto while external exception try catch when as private constraint assert dict list true false break continue of", + ), + }, + js: { + token: + /(?\/\/[^\n]*|\/\*[\s\S]*?\*\/)|(?"(?:\\.|[^"\\\n])*"|'(?:\\.|[^'\\\n])*'|`(?:\\.|[^`\\])*`)|(?\b\d+(?:\.\d+)?(?:[eE][+-]?\d+)?n?\b|\b0x[0-9a-fA-F]+\b)|(?[A-Za-z_$][\w$]*)/g, + keywords: words( + "let const var function return if else for while do new import export from as default this true false null undefined typeof instanceof class async await throw try catch finally of in switch case break continue void delete yield", + ), + }, + ts: { + token: + /(?\/\/[^\n]*|\/\*[\s\S]*?\*\/)|(?"(?:\\.|[^"\\\n])*"|'(?:\\.|[^'\\\n])*'|`(?:\\.|[^`\\])*`)|(?\b\d+(?:\.\d+)?\b)|(?[A-Za-z_$][\w$]*)/g, + keywords: words( + "let const var function return if else import export from as default type interface readonly extends keyof typeof null undefined true false string number boolean bigint unknown never void any", + ), + }, + ocaml: { + token: + /(?\(\*[\s\S]*?\*\))|(?"(?:\\.|[^"\\\n])*")|(?'[a-z]\w*)|(?\b\d+(?:\.\d+)?\b)|(?[A-Za-z_][\w']*)/g, + keywords: words( + "type and of let rec in match with function fun if then else mutable private module struct sig end open include val external exception when as option list", + ), + }, + lambda: { + token: + /(?"(?:\\.|[^"\\\n])*")|(?

      #[A-Za-z_][\w.]*)|(?module\/exports)|(?[A-Za-z_][\w$.']*)(?\/\d+)?|(?-?\b\d+(?:\.\d+)?[nl]?\b)|(?[()])/g, + keywords: words( + "let letrec function if switch stringswitch staticcatch staticraise catch exit try with while for seq apply makeblock field setfield default case", + ), + }, +}; + +/** + * Code highlighted by token: comments, strings, attributes and extensions, + * polymorphic variants, numbers, keywords, capitalised names (modules, + * constructors), type variables; in Lambda, primitives (`#is_not_none`), + * identifier stamps (`x/1002`), parentheses, and one-letter capitals as + * metavariables standing for a term. + * @param {string} code + * @param {keyof typeof LANG} lang + */ +export function highlight(code, lang) { + const { token, keywords } = LANG[lang]; + let out = ""; + let last = 0; + for (const m of code.matchAll(token)) { + out += esc(code.slice(last, m.index)); + last = (m.index ?? 0) + m[0].length; + const g = m.groups ?? {}; + const span = (/** @type {string} */ cls, /** @type {string} */ t) => + t + .split("\n") + .map(part => `${esc(part)}`) + .join("\n"); + if (g.i !== undefined) { + const id = g.i; + out += keywords.has(id) + ? span("k", id) + : lang === "lambda" && /^[A-Z]$/.test(id) + ? `${esc(id)}` + : /^[A-Z]/.test(id) + ? span("t", id) + : esc(id); + if (g.stamp) out += span("stamp", g.stamp); + } else { + const cls = ["c", "s", "a", "p", "n", "v", "k", "paren"].find( + k => g[k] !== undefined, + ); + out += cls ? span(cls, m[0]) : esc(m[0]); + } + } + return out + esc(code.slice(last)); +} + +/** + * The pairs of line numbers a longest common subsequence of `a` and `b` + * matches, in order, lines compared by `key`. + * @param {string[]} a + * @param {string[]} b + * @param {(line: string) => string} key + * @returns {[number, number][]} + */ +function matchLines(a, b, key) { + const x = a.map(key); + const y = b.map(key); + /** @type {number[][]} */ + const t = Array.from({ length: x.length + 1 }, () => + new Array(y.length + 1).fill(0), + ); + for (let i = x.length - 1; i >= 0; i--) + for (let j = y.length - 1; j >= 0; j--) + t[i][j] = + x[i] === y[j] + ? t[i + 1][j + 1] + 1 + : Math.max(t[i + 1][j], t[i][j + 1]); + /** @type {[number, number][]} */ + const pairs = []; + for (let i = 0, j = 0; i < x.length && j < y.length; ) { + if (x[i] === y[j]) pairs.push([i++, j++]); + else if (t[i + 1][j] >= t[i][j + 1]) i++; + else j++; + } + return pairs; +} + +/** + * The lines of `a` and of `b` outside a longest common subsequence: the + * lines a Before/After pair changes. Lines compare without surrounding + * spaces. + * @param {string[]} a + * @param {string[]} b + * @returns {[Set, Set]} + */ +export function lineDiff(a, b) { + const pairs = matchLines(a, b, l => l.trim()); + /** @param {string[]} ls @param {Set} keep */ + const changed = (ls, keep) => + new Set(ls.flatMap((l, n) => (keep.has(n) || !l.trim() ? [] : [n]))); + return [ + changed(a, new Set(pairs.map(p => p[0]))), + changed(b, new Set(pairs.map(p => p[1]))), + ]; +} + +/** + * The rows of a Before/After pair drawn side by side: a line common to both + * versions is one row; between two common lines, the lines only Before has + * and the lines only After has share rows, the shorter side padded with + * gaps (`null`). Lines compare without trailing spaces, so a line that moves + * to another depth is a change. + * @param {string[]} a + * @param {string[]} b + * @returns {{ a: number | null, b: number | null, same: boolean }[]} + */ +export function alignLines(a, b) { + /** @type {{ a: number | null, b: number | null, same: boolean }[]} */ + const rows = []; + let i = 0; + let j = 0; + const ends = [...matchLines(a, b, l => l.trimEnd()), [a.length, b.length]]; + for (const [x, y] of ends) { + for (let k = 0; k < Math.max(x - i, y - j); k++) + rows.push({ + a: i + k < x ? i + k : null, + b: j + k < y ? j + k : null, + same: false, + }); + if (x < a.length) rows.push({ a: x, b: y, same: true }); + i = x + 1; + j = y + 1; + } + return rows; +} + +/** + * @typedef {{ marked?: Set, mark?: "add" | "del" }} Marks + * The lines of one version of a Before/After pair that the other lacks. + */ + +/** + * One drawn line. A line too long for its column wraps, its continuation + * indented two columns past `indent`, the column its content starts at. + * @param {string} html + * @param {number} indent + * @param {string} [cls] + * @param {string} [style] + */ +const line = (html, indent, cls = "", style = "") => + `${html || " "}`; + +/** The column a line's content starts at. @param {string} l */ +const indentOf = l => /^ */.exec(l)?.[0].length ?? 0; + +/** + * Lines of code, each drawn by `draw`, as HTML, one string a line. + * @param {string[]} lines + * @param {(code: string) => string} draw + */ +const codeHtml = (lines, draw) => draw(lines.join("\n")).split("\n"); + +/** + * Lines of code, each drawn by `draw`, the marked ones shaded. + * @param {string[]} lines + * @param {(code: string) => string} draw + * @param {Marks} [marks] + */ +function codeLines(lines, draw, marks = {}) { + return `

      ${codeHtml(lines, draw)
+    .map((l, i) =>
+      line(l, indentOf(lines[i] ?? ""), marks.marked?.has(i) ? marks.mark : ""),
+    )
+    .join("")}
      `; +} + +/** + * Code, or a rewrite of code: a line holding `⇝` separates the code before + * it from the code after it (`lhs ⇝ rhs` on one line, or `⇝` alone on a + * line); `label: text` lines after the right side are side conditions, + * drawn as mathematics. + * @param {string[]} lines + * @param {keyof typeof LANG} lang + * @param {Marks} [marks] + */ +function code(lines, lang, marks) { + const draw = (/** @type {string} */ c) => highlight(c, lang); + if (!lines.some(l => l.includes("⇝"))) return codeLines(lines, draw, marks); + return `
      ${groups(lines) + .map(g => { + const at = g.findIndex(l => l.includes("⇝")); + if (at < 0) return codeLines(g, draw); + const [left, right] = g[at].split("⇝"); + const rest = g.slice(at + 1); + const cond = rest.findIndex(l => /^\s*[a-z][\w ]{0,20}:\s/.test(l)); + const lhs = dedent([...g.slice(0, at), ...(left.trim() ? [left] : [])]); + const rhs = dedent([ + ...(right.trim() ? [right.trim()] : []), + ...(cond < 0 ? rest : rest.slice(0, cond)), + ]); + return ( + `
      ${codeLines(lhs, draw)}⇝${codeLines(rhs, draw)}
      ` + + (cond < 0 ? [] : rest.slice(cond)).map(condition).join("") + ); + }) + .join("")}
      `; +} + +/** A side condition: `label: formula`, or a formula. @param {string} l */ +function condition(l) { + const m = /^\s*([a-z][\w ]{0,20}):\s+(.*)$/.exec(l); + return `
      ${m ? `${esc(m[1])}` : ""}${math(m ? m[2] : l.trim())}
      `; +} + +// --------------------------------------------------------------------------- +// Inline text + +/** Short shas of the cards on the page, for linking mentions. */ +const pageShas = new Set(); + +/** + * Prose: `code` as highlighted ReScript, pull request and issue links, + * mentions of commits on the page. + * @param {string} s + */ +export const inline = s => + s + .split(/(`[^`]+`)/) + .map((part, i) => + i % 2 + ? `${highlight(part.slice(1, -1), "rescript")}` + : esc(part) + .replace( + /https:\/\/github\.com\/rescript-lang\/rescript\/(pull|issues)\/(\d+)/g, + `      #$2`, + ) + .replace( + /(^|[\s(])#(\d{3,5})\b/g, + `$1#$2`, + ) + .replace(/\b([0-9a-f]{9})\b(\^?)/g, (m, sha, caret) => + pageShas.has(sha) && !caret + ? `${sha}` + : m, + ) + .replace( + /(\d[\d.,]*\s*(?:ms|s|%|KB|MB)?)\s*->\s*(\d[\d.,]*\s*(?:ms|s|%|KB|MB)?)/g, + '$1 → $2', + ), + ) + .join(""); + +// --------------------------------------------------------------------------- +// Mathematics: typing rules, grammar, semantics + +/* Mathematics set as type theory sets it: one-letter names (variables, + * metavariables) italic; a name applied to arguments, `fields(τ)`, an upright + * operator; a capitalised name an upright constructor; binders and relations + * coloured; `code` in backticks highlighted ReScript; a run of two or more + * words, `τ is an @unboxed record`, a side condition in text. `_x`, `_{…}` + * and `^{…}` are sub- and superscripts. The input is the record's; nothing + * is added to it. */ +const MATH_KEYWORDS = words("if then else let in switch"); +const CONNECTIVES = words("or and not"); +/** Literal values, drawn as code wherever they appear. */ +const LITERALS = words("null undefined true false NaN"); +const BINDERS = new Set("∀∃λΛμ"); +const RELATIONS = new Set("⊢⊨≤≥⊑⊒∈∉⊆⊂=≠≡:⇝↦≜≔→⟶"); +const TEXT_WORD = + /^(?:a|an|A|An|[A-Z]{2,}|[A-Za-z][A-Za-z'-]*[a-z][A-Za-z'-]*)$/; +/** English words that never name a type, even after a relation. */ +const STOP_WORDS = words( + "the with of for is are to from where when that which its their this each every", +); + +/** + * @typedef {"code" | "str" | "script" | "word" | "num" | "space" | "sym"} TokenKind + * @typedef {{ kind: TokenKind, text: string }} MathToken + * @typedef {"text" | "code" | "str" | "lit" | "script" | "kw" | "op" | "var" | "con" | "bind" | "rel" | "plain"} MathRole + */ + +/** @param {string} text @returns {MathToken[]} */ +function mathTokens(text) { + const token = + /(?`[^`]+`)|(?"[^"\n]*")|(?