From 648f37de5280f21972e20f1cdfb0c078931cc907 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 17:28:45 +0000 Subject: [PATCH 01/19] Adopt the validator's CommonMark block model (legaldown-validator#49) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The validator now keeps HTML blocks, table alignment, every list marker, indented code, and the content after a Signature Block heading. The renderer follows its model: - An HTML block renders nothing; one that holds anything besides comments counts for the raw-html Warning (§8.6, §8.7) - The comment-across-blocks exception is removed: a comment block is now the validator's, and an unclosed ` shows as text. In an **HTML block**, every tag is dropped and reported (`raw-html`), but the - text between the tags renders as paragraph text ([#23](https://github.com/ForLegalAI/legaldown-validator/issues/23)). - - **A comment across blank lines** is stripped up to its `-->` (the one place the renderer - reads past the model), but the model still holds its content as blocks. So a `{#id}` inside - it is still a valid target for the validator, and a reference to it renders - `[BROKEN REF: id]` with no Error. When the comment ends inside a quote, code block, or table, - the rest of that block renders as plain paragraph text ([#23](https://github.com/ForLegalAI/legaldown-validator/issues/23)). - - **After an empty comment** (``, ``), a directive before a later `-->` on the same - line is shown as its source text, unresolved and unchecked ([#28](https://github.com/ForLegalAI/legaldown-validator/issues/28)). - - **Everything after a `# Signature Block {#signature-block}` heading** is dropped - ([#24](https://github.com/ForLegalAI/legaldown-validator/issues/24)). - - **Indented code blocks** are read as paragraphs ([#9](https://github.com/ForLegalAI/legaldown-validator/issues/9)). + - **Indented code inside a quote or a list item** is read as paragraph text + ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)). At the top level it is + code. + - **Empty list items** are left out ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)), + and a table row needs a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)). ## Beyond this level (Full, §17.4) diff --git a/docs/architecture.md b/docs/architecture.md index d8c26f7..41075ef 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -82,28 +82,17 @@ that renders nothing, led by a punctuation character so that an emphasis closer still closes. A defined term in a link title or alt text shows its term but is not the definition's anchor. -HTML comments are dropped (§8.6), and where one stood between two spaces, one space stays. Every -HTML tag is dropped, and each text that had one (a paragraph, a title, a table cell) counts once -for the `raw-html` Warning (§8.7). - -The builder never works around the validator's model, with **one exception: comments**. The -specification requires every comment to be stripped (§8.6), and the validator's model holds a -comment that spans a blank line as ordinary blocks. So a ``, and the blocks and items in between render nothing. The -text before the `` renders. An opener counts only when it is not -escaped, not in a code span or a directive, and not the empty comment `` or ``. The comment never runs past the end of -the section, or of the quote or list item, it opened in: a heading inside a comment is still a -section ([#23](https://github.com/ForLegalAI/legaldown-validator/issues/23)). Two gaps remain until -the model drops such comments itself, listed in `CONFORMANCE.md`: a `{#id}` inside one is still a -target, and a quote, code block, or table the comment ends in renders the rest as plain text. - -Everywhere else, where the model loses or misreads something, the output follows the model, and -the gap is listed in `CONFORMANCE.md` and filed on the validator: +An HTML block in the model (`kind="html"`), a comment block included, renders nothing (§8.6). +One that holds anything besides comments counts once for the `raw-html` Warning (§8.7). Within a +block's text, inline comments and tags are dropped; where one stood between two spaces, one space +stays. Each text that had a tag (a paragraph, a title, a table cell) counts once for the Warning. + +The builder never works around the validator's model. Where the model loses or misreads +something, the output follows the model, and the gap is listed in `CONFORMANCE.md` and filed on +the validator: - lists have one level of items -- table column alignment is not kept - a paragraph's line breaks are joined -- the text between an HTML block's tags stays -- content after a `Signature Block` heading is dropped +- indented code inside a quote or a list item is paragraph text Guessing at lost structure was tried, and it traded each gap for new bugs. diff --git a/docs/decisions/0007-one-parser-validator-model.md b/docs/decisions/0007-one-parser-validator-model.md index d135fca..a45ffa8 100644 --- a/docs/decisions/0007-one-parser-validator-model.md +++ b/docs/decisions/0007-one-parser-validator-model.md @@ -53,9 +53,11 @@ module, `validator_bridge.py`. That module is the list for roadmap item U3. - a paragraph's line breaks are joined - constructs the validator reads unusually, such as a heading inside an HTML comment, render as the validator reads them -- One deliberate exception: a comment left open in a paragraph runs on to its `-->` across the - following blocks, because §8.6 requires every comment to be stripped. It never crosses a - section boundary (docs/architecture.md, stage 3). +- One deliberate exception was added later: a comment left open in a paragraph ran on to its + `-->` across the following blocks, because §8.6 requires every comment to be stripped. It was + removed once the validator's model kept HTML blocks (validator #23), which is the path this + decision expects: the fix lands in the validator, and the renderer drops its workaround. + Table alignment came with the same change. - When the validator keeps nested lists (#14), the renderer gains them with no parser of its own. The resolver and writers already handle nested lists. - The build stage is smaller: about 100 lines of position rules and special cases are gone. diff --git a/docs/roadmap.md b/docs/roadmap.md index 9e29bb6..f8dcbd7 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -13,9 +13,9 @@ here. | U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Open. Until then lists render with one level of items (ADR 0007); the resolver and writers already handle nesting | | U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | | U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, frontmatter splitting, value checks, conditions, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`, the frontmatter fields it reads. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Open. Until then the dependency is pinned to one minor version | -| U1b | validator | **Parser gaps** the renderer inherits: list markers ([#21](https://github.com/ForLegalAI/legaldown-validator/issues/21)), tables ([#22](https://github.com/ForLegalAI/legaldown-validator/issues/22)), headings in comments ([#23](https://github.com/ForLegalAI/legaldown-validator/issues/23)), signature-block cutoff ([#24](https://github.com/ForLegalAI/legaldown-validator/issues/24)), hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), empty comments ([#28](https://github.com/ForLegalAI/legaldown-validator/issues/28)), indented code ([#9](https://github.com/ForLegalAI/legaldown-validator/issues/9)) | Open. Listed in `CONFORMANCE.md` | +| U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | -| U4b | validator | Template **assembly** with an answers set (§15.7) | Open. It blocks rendering templates with answers | +| U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Open. It blocks rendering templates with answers | | U5 | specification | A **rendering fixtures corpus**: source + style settings → expected plain-text output | Not started. The plain-text writer is designed to be its oracle | ## Milestones diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index f64154e..68c6de7 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -57,7 +57,7 @@ Table, Text, ) -from .validator_bridge import block_fragments, block_quotes, is_escaped, lex, placed_markers +from .validator_bridge import block_fragments, block_quotes, lex, placed_markers _OPEN, _CLOSE = "\ue000", "\ue001" # Leads the sentinel of source that renders nothing (a {{def:}}). It is @@ -66,9 +66,6 @@ _HIDDEN_LEAD = "\u2e31" # A fenced code block's opening line (validator's model keeps the fences). _FENCE_RE = re.compile(r"^ {0,3}(`{3,}|~{3,})(.*)$") -# The validator's block kinds that render as a paragraph, and as a list. -_PARAGRAPH_KINDS = frozenset({"paragraph", "definition", "ref", "term"}) -_LIST_KINDS = frozenset({"unordered_list", "ordered_list"}) # Stand where inline HTML or a comment was dropped, until _merge_text joins # the text around them. A dropped line break (
) still parts two words. _DROPPED = Text("") @@ -330,74 +327,25 @@ def _split_sentinels(self, text: str) -> list[Inline]: def blocks(self, blocks: list[ModelBlock], section: int | None, *, markers: bool = True) -> tuple[Block, ...]: """Render-tree blocks for the validator's *blocks* of section index *section* (None for the preamble). With *markers* False — text the - validator reads inside a quote or an item — no marker is placed. - - The one place the builder reads past the validator's model: a - comment (§8.6) left open in a paragraph or a list item runs on to - the first ``-->`` in the blocks that follow, which render nothing - (docs/architecture.md, stage 3; ForLegalAI/legaldown-validator#23). - It never runs past the end of *blocks*.""" + validator reads inside a quote or an item — no marker is placed.""" out: list[Block] = [] - in_comment = False top_level = section is not None and markers for index, block in enumerate(blocks): - built: Block | None - if in_comment and block.kind not in _PARAGRAPH_KINDS | _LIST_KINDS: - # A quote, code, table, or rule the comment runs into: what - # follows its "-->" is kept as text, its structure is lost. - built, in_comment = self.paragraph(render_block(block), "", "", top_level=top_level, in_comment=True) - if built is not None: - out.append(built) + if block.kind == "html": + self._html_block(block) continue placed = self._placed(block, section, index) if markers else {} - if block.kind in _PARAGRAPH_KINDS: - block, identifier, condition = _strip_markers(block, placed) - built, in_comment = self.paragraph( - _paragraph_source(block), identifier, condition, top_level=top_level, in_comment=in_comment) - elif block.kind in _LIST_KINDS: - built, in_comment = self.list(block, placed, in_comment=in_comment) - else: - built = self.block(block, placed) + built = self.block(block, placed, top_level=top_level) if built is not None: out.append(built) return tuple(out) - def paragraph( - self, source: str, identifier: str, condition: str, *, top_level: bool, in_comment: bool, - ) -> tuple[Paragraph | None, bool]: - """A paragraph for *source*, and whether a comment is left open at - its end. With *in_comment*, the text up to the first ``-->`` is - inside a comment opened earlier; without one, the whole text is.""" - if in_comment: - end = source.find("-->") - if end < 0: - return None, True - source = source[end + 3:] - opening = self._unclosed_comment(source) - if opening >= 0: - source = source[:opening] - inlines = self.text(source) - if not inlines and not identifier: - return None, opening >= 0 # only a comment - return Paragraph(inlines, anchor_id=identifier, condition=condition, top_level=top_level), opening >= 0 - - def _unclosed_comment(self, source: str) -> int: - """Offset of a ```` closes, or -1. - - The lexer's view has code spans and closed comments blanked; what - it leaves is literal to it. Of that, an escaped ``\\`` and ```` - (complete in CommonMark) open nothing.""" - lexed = self.markers.lex(source) - view = lexed.view - at = view.find("") - if end < 0: - continue - item, in_comment = item[end + 3:].lstrip(" "), False - if not item.strip() and not identifier: - continue # nothing after the comment's end first, _, rest = item.partition("\n") if first.lstrip().startswith(">") or _FENCE_RE.match(first): # The item opens with a quote or code, whose lines the # validator keeps as written (a drafting note included). blocks = self.fragment(item) else: - opening = self._unclosed_comment(first) - if opening >= 0: - first = first[:opening] - end = rest.find("-->") - rest, in_comment = ("", True) if end < 0 else (rest[end + 3:], False) - blocks = (Paragraph(self.text(first)),) + (self.fragment(rest) if rest.strip() else ()) + blocks = (Paragraph(self.text(first)),) + (self.fragment(rest) if rest else ()) items.append(ListItem(blocks=blocks, anchor_id=identifier, condition=condition)) - return (List(ordered=block.kind == "ordered_list", items=tuple(items)) if items else None), in_comment + return List(ordered=block.kind == "ordered_list", items=tuple(items)) def quote(self, block: ModelBlock) -> Block: """A block quote, its content read by the validator's parser. A @@ -556,11 +494,12 @@ def _paragraph_source(block: ModelBlock) -> str: def _code_block(text: str) -> CodeBlock: - """A fenced code block from the validator's model, fences included.""" + """A code block from the validator's model: fenced, fences included, + or indented, each line by four columns (CommonMark).""" lines = text.split("\n") opening = _FENCE_RE.match(lines[0]) if lines else None if opening is None: - return CodeBlock(text) + return CodeBlock("\n".join(_unindent(line) for line in lines) + "\n") fence = opening.group(1) body = lines[1:] if body and body[-1].strip().startswith(fence[0] * len(fence)) and not body[-1].strip().strip(fence[0]): @@ -568,6 +507,17 @@ def _code_block(text: str) -> CodeBlock: return CodeBlock("\n".join(body) + ("\n" if body else ""), opening.group(2).strip()) +def _unindent(line: str) -> str: + """*line* without an indented code block's four columns of + indentation, a tab counting to the next multiple of four.""" + column = 0 + for index, char in enumerate(line): + if column >= 4 or char not in " \t": + return line[index:] + column = column + 4 - column % 4 if char == "\t" else column + 1 + return "" + + def _trim(inlines: tuple[Inline, ...]) -> tuple[Inline, ...]: """*inlines* without spacing at either end (left where a comment or a marker was removed).""" diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index ab725a7..f49debe 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -17,12 +17,12 @@ from legaldown import Document from legaldown.definitions import block_fragments, text_fragments -from legaldown.directives import Lexed, is_escaped, lex +from legaldown.directives import Lexed, lex from legaldown.validator.core import _frontmatter_fields from legaldown.validator.templates import block_quotes from legaldown.validator.units import FoundMarker, find_markers -__all__ = ["PlacedMarkers", "block_fragments", "block_quotes", "is_escaped", "lex", "placed_markers"] +__all__ = ["PlacedMarkers", "block_fragments", "block_quotes", "lex", "placed_markers"] @dataclass(frozen=True, slots=True) diff --git a/tests/golden/features.default.html b/tests/golden/features.default.html index bb6b501..04f2d63 100644 --- a/tests/golden/features.default.html +++ b/tests/golden/features.default.html @@ -137,13 +137,13 @@

3. The Client shall pay $10,000.00 per month, and €1,234.50 for setup, within 30 days of July 1, 2026. The Provider's VAT number is DE123456789.

- - - + + + - - + +
ServiceFeeNotesServiceFeeNotes
Hosting$500.00per month
Support$250.75see 3
Hosting$500.00per month
Support$250.75see 3
diff --git a/tests/test_render.py b/tests/test_render.py index e2a413e..32ba578 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -406,7 +406,7 @@ def test_locale_option_and_language_hint() -> None: def test_unreadable_document() -> None: with pytest.raises(DocumentError): - render("---\n- a list\n---\n\n# A\n") + render("---\ntitle: [unclosed\n---\n\n# A\n") def test_options_object_and_keywords_are_exclusive() -> None: @@ -726,17 +726,30 @@ def _validator_rules(source: str) -> set[str]: return {d.rule for d in validate_document(parse_document(source)).diagnostics} -def test_a_heading_in_a_comment_renders_as_the_validator_reads_it() -> None: - # The validator reads the heading as a section; the renderer never - # crashes on a disagreement, because there is none to have. - result = render(FRONT + "# Zero\n\n\n", format="text") +def test_a_signature_block_heading_is_an_ordinary_section() -> None: + result = render(FRONT + "# One\n\nText.\n\n# Signature Block {#signature-block}\n\nSigned\n", format="text") assert [s.designation for s in result.tree.sections] == ["1", "2"] + assert "Signed" in result.output -def test_a_signature_block_heading_ends_the_body_as_in_the_validator() -> None: - result = render(FRONT + "# One\n\nText.\n\n# Signature Block {#signature-block}\n\nSigned\n", format="text") - assert [s.designation for s in result.tree.sections] == ["1"] - assert "Signed" not in result.output +def test_every_list_marker_makes_a_list() -> None: + output = text("# A\n\n* star\n* list\n\n1) one\n2) two\n") + assert "(a) star\n(b) list" in output + assert "1. one\n2. two" in output + + +def test_table_columns_keep_their_alignment() -> None: + html = render(FRONT + "# A\n\n| l | c | r | n |\n|:--|:-:|--:|---|\n| 1 | 2 | 3 | 4 |\n", standalone=False).output + assert 'l' in html + assert '2' in html + assert '3' in html + assert "4" in html + + +def test_indented_code_renders_as_code_without_its_indent() -> None: + html = render(FRONT + "# A\n\nPara.\n\n code {{ref: x}}\n deeper\n", standalone=False).output + assert "
code {{ref: x}}\n  deeper\n
" in html + assert "BROKEN REF" not in html def test_markers_and_references_agree_with_the_validators_diagnostics() -> None: @@ -794,17 +807,35 @@ def test_an_item_opening_with_a_drafting_note_or_code() -> None: assert "```" not in output -def test_html_tags_are_never_emitted() -> None: - # Every tag is dropped and reported. Text between the tags of an HTML - # block stays, as the validator's model holds it as paragraph text - # (ForLegalAI/legaldown-validator#23); it is escaped like any text. +def test_an_html_block_is_dropped_whole_and_reported() -> None: result = render(FRONT + "# A\n\n\n\nAfter.\n", standalone=False) - assert " None: + result = render(FRONT + "# Zero\n\n\n\nNext.\n", format="text") + assert [s.designation for s in result.tree.sections] == ["1"] + assert "hidden" not in result.output and "Old clause" not in result.output + assert "Next." in result.output + assert "raw-html" not in {d.rule for d in result.diagnostics} + + +def test_an_html_block_after_a_comment_is_still_reported() -> None: + result = render(FRONT + "# A\n\n tail\n", format="text") + assert "tail" not in result.output + assert "raw-html" in {d.rule for d in result.diagnostics} + + +def test_an_unclosed_comment_inside_a_paragraph_is_text() -> None: + # CommonMark: only a line that starts with after\n\nNext.\n") - assert "Before\n\nafter\n\nNext." in output - assert "hidden" not in output - assert "" not in output - - -def test_an_open_comment_ends_with_its_section() -> None: - output = text("# A\n\nShown shown\n- d\n\n- Item tail\n") - assert "(a) a\n(b) b\n(c) shown\n(d) d" in output - assert "hidden" not in output - assert "tail" in output - - -def test_a_comment_closing_in_a_list_keeps_its_items() -> None: - output = text("# A\n\nText \n- three {#three}\n- four\n\nSee {{ref: three}}.\n") - assert "(a) three\n(b) four" in output - assert "See 1(a)." in output - - def test_a_dropped_comment_or_tag_leaves_one_space() -> None: result = render(FRONT + "# A\n\na b
c d Line
two Wordboldword\n", format="text") From 1ae2c01e9040b655c7a473974161c55aa8620710 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 18:53:27 +0000 Subject: [PATCH 02/19] Take section numbers from the validator (legaldown-validator#50) The validator now keeps section numbers unique after a heading skip: a skipped level counts as 1, and numbers count from the shallowest heading (#, ###, ## -> 1, 1.1.1, 1.2). The renderer counted sections itself and disagreed. It now takes each section's number from ValidationResult.sections and only formats it, the n-th part with the n-th level format, so rendered numbers, references, and contents depth always agree with the validator's. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- docs/architecture.md | 9 +++-- docs/style-templates.md | 2 +- src/legaldown_render/resolve/resolver.py | 48 ++++++++---------------- src/legaldown_render/style/model.py | 5 ++- tests/test_render.py | 18 ++++++++- 5 files changed, 41 insertions(+), 41 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 41075ef..b62c56a 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -102,10 +102,11 @@ Guessing at lost structure was tried, and it traded each gap for new bugs. 1. **Survey.** It collects which definitions exist, whether template constructs are used, and which placeholder ids are used with conflicting types (§10.7). -2. **Structure.** It numbers sections under the style's level formats. **Alternatives** share a - number (§15.8). For sections, which ones are alternatives is taken from the validator's own - numbering, so rendered numbers always match `ValidationResult.sections`. For paragraphs and - list items, the validator's rule is applied the same way: an alternative directly follows its +2. **Structure.** Section numbers are the validator's own (`ValidationResult.sections`), so a + rendered number always matches the validator's: **alternatives** share a number (§15.8), and + a skipped heading level counts as 1. The style only formats them, the n-th part of a number + with the n-th level format. For paragraphs and list items, **alternatives** follow the + validator's rule, applied the same way: an alternative directly follows its sibling, has the same identifier and a valid condition, and its full presence (its own condition and every enclosing one) excludes that of every unit already holding the number (§15.4, the validator's `exclusive()`). It then labels diff --git a/docs/style-templates.md b/docs/style-templates.md index 75c1b91..88ee6f5 100644 --- a/docs/style-templates.md +++ b/docs/style-templates.md @@ -72,7 +72,7 @@ locale: cs-CZ # formatting locale (§10.1); unset: the document's numbering: # §13.1 scheme: decimal # decimal | legal-outline | mixed | none - levels: # optional; overrides the scheme level by level from level 1 + levels: # optional; overrides the scheme by number part: the first format for "2", the second for "2.1" - { counter: upper-roman, label: "Article {n}", ref: "{n}" } enumeration: # §13.2 diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index 4476984..33db9ec 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -255,54 +255,38 @@ def _register(self, identifier: str, target: _Target) -> None: def _number_sections(self) -> list[Section]: """Number the sections, and record each one's presence (§15.3). - Which sections are alternatives, sharing their previous sibling's - number (§15.8), is taken from the validator: it numbers them itself, - with the full presence of each, so the rendered numbers always - agree with ``ValidationResult.sections``. + The numbers are the validator's own (``ValidationResult.sections``), + so a rendered number and a reference to it always agree with the + validator: alternatives share a number (§15.8), and a skipped + heading level counts as 1. The style only formats them, the n-th + part of a number ("2.1" has two) with the n-th level format. """ levels = heading_levels(self.style.numbering) - counters = [0] * 6 # index 1-5: one counter per heading level - previous: dict[int, int] = {} # level -> index of the previous sibling presences: dict[int, Presence] = {} # level -> presence of the open section - indexed = self.result.sections out: list[Section] = [] - for index, section in enumerate(self.tree.sections): - # Numbered with its level clamped to 1-5, as the validator numbers it. + for section, indexed in zip(self.tree.sections, self.result.sections, strict=True): + # Stored clamped to 1-5 (§4.1), so every writer nests the section + # where the validator reads it. level = min(max(section.level, 1), 5) - sibling = previous.get(level) - alternative = ( - sibling is not None - and self.tree.sections[sibling].identifier == section.identifier - and indexed[sibling].number == indexed[index].number - ) - if not alternative: - counters[level] += 1 - counters[level + 1:] = [0] * (5 - level) - previous = {lvl: i for lvl, i in previous.items() if lvl < level} - previous[level] = index enclosing = max((lvl for lvl in presences if lvl < level), default=None) presences = {lvl: p for lvl, p in presences.items() if lvl < level} presences[level] = (presences[enclosing] if enclosing is not None else ALWAYS) | self._presence(section.condition) self.section_presences.append(presences[level]) + parts = [int(part) for part in indexed.number.split(".")] if self.textual: label, designation = None, self._title_text(section.title) else: designation = "" - for depth in range(1, level + 1): - if counters[depth]: - fmt = levels[depth - 1] - designation = extend(designation, fill(fmt.ref, n=format_counter(counters[depth], fmt.counter)), - textual=False) - fmt = levels[level - 1] - label = fill(fmt.label, n=format_counter(counters[level], fmt.counter), path=designation) + for position, counter in enumerate(parts): + fmt = levels[min(position, len(levels) - 1)] + designation = extend(designation, fill(fmt.ref, n=format_counter(counter, fmt.counter)), + textual=False) + fmt = levels[min(len(parts), len(levels)) - 1] + label = fill(fmt.label, n=format_counter(parts[-1], fmt.counter), path=designation) anchor = self._anchor(section.identifier) self._register(section.identifier, _Target(designation, anchor)) - # How deep the validator's own number goes ("2.1" is 2). - depth = indexed[index].number.count(".") + 1 - # The level is stored clamped too, so every writer nests the section - # where its number puts it. out.append(replace(section, level=level, label=label, designation=designation, anchor=anchor, - depth=depth)) + depth=len(parts))) return out def _presence(self, condition: str) -> Presence: diff --git a/src/legaldown_render/style/model.py b/src/legaldown_render/style/model.py index b7e7600..1a0f464 100644 --- a/src/legaldown_render/style/model.py +++ b/src/legaldown_render/style/model.py @@ -52,8 +52,9 @@ class Numbering: #: A built-in scheme. ``none`` shows no numbers, and references render #: the target's heading text instead (§13.3). scheme: Literal["decimal", "legal-outline", "mixed", "none"] = "decimal" - #: Overrides the scheme's formats, level by level from level 1 (at most - #: five, one per heading level, §4.1). Levels not listed keep the + #: Overrides the scheme's formats by part of a section number: the first + #: format for "2", the second for the ".1" of "2.1" (at most five, as + #: headings have five levels, §4.1). Levels not listed keep the #: scheme's format. levels: tuple[LevelFormat, ...] = () diff --git a/tests/test_render.py b/tests/test_render.py index 32ba578..f6dfde6 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -277,8 +277,22 @@ def test_contents_label_follows_the_language_and_marks_conditions() -> None: def test_contents_follow_the_numbering_depth_not_the_heading_level() -> None: - output = text("# A\n\n### B\n\n###### C\n", overrides={"contents.enabled": True}) - assert "Contents\n1. A\n 1.1 B\n\n" in output + body = "# A\n\n### B\n\n## C\n" + assert "Contents\n1. A\n 1.2 C\n\n" in text(body, overrides={"contents.enabled": True}) + assert "Contents\n1. A\n 1.1.1 B\n 1.2 C\n\n" in text( + body, overrides={"contents.enabled": True, "contents.depth": 3}) + + +@pytest.mark.parametrize("body", [ + "# A\n\n### B\n\n## C\n\n### D\n", + "## A\n\n### B\n\n## C\n", + "# A\n\n###### B\n\n# C\n", +]) +def test_section_numbers_are_the_validators(body: str) -> None: + from legaldown import parse_document, validate_document + result = render(FRONT + body, format="text") + expected = [entry.number for entry in validate_document(parse_document(FRONT + body)).sections] + assert [section.designation for section in result.tree.sections] == expected def test_contents_group_attachments_under_their_heading() -> None: From cc51d278cce3784a69235bd640fc2082f0c34ceb Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 19:29:36 +0000 Subject: [PATCH 03/19] Ask the validator's is_template instead of keeping a copy legaldown-validator#55 made the template decision one function, validator.core.is_template, used by validate_document and assembly alike. The renderer's copy of its formula, the frontmatter-field import it needed, and the test spy that checked the copy are gone; the tests now compare the renderer's markers-based answer with is_template alone. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- docs/architecture.md | 2 +- .../0007-one-parser-validator-model.md | 3 +- docs/roadmap.md | 2 +- src/legaldown_render/validator_bridge.py | 40 +++---------------- tests/conformance/test_spec_corpus.py | 8 ++-- tests/test_render.py | 4 +- tests/validator_spy.py | 29 -------------- 7 files changed, 15 insertions(+), 73 deletions(-) delete mode 100644 tests/validator_spy.py diff --git a/docs/architecture.md b/docs/architecture.md index b62c56a..1afa94b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -64,7 +64,7 @@ never decides a structural or LegalDown question itself: | Sections, headings, identifiers | `Document.sections`, `ValidationResult.sections` | | Blocks: paragraphs, lists and items, quotes, tables, code, rules | `Document` blocks; quote content and code inside items are read by the validator's parser too | | Where a marker (`{#id when=…}`) is placed, and what it means | The validator's `find_markers()`, with its own `placed(template)` | -| Whether the document is a template | The validator's own formula over those markers | +| Whether the document is a template | The validator's `is_template()`, over those markers | | Whether a quote is a drafting note | The validator's `block_quotes()` | | A lifted definition, `{{ref:}}` or `{{term:}}` block's source | The validator's `render_block()` | diff --git a/docs/decisions/0007-one-parser-validator-model.md b/docs/decisions/0007-one-parser-validator-model.md index a45ffa8..a43c968 100644 --- a/docs/decisions/0007-one-parser-validator-model.md +++ b/docs/decisions/0007-one-parser-validator-model.md @@ -30,7 +30,8 @@ The render tree is built **from the validator's `Document` only**: - **Block structure:** sections, blocks, list items, quotes, tables, and code all come from the validator's model. - **Markers:** placed exactly where the validator's own `find_markers()` places them. -- **Template decision:** the validator's own formula over those markers. +- **Template decision:** the validator's own formula over those markers. Since the validator + exposed it as `is_template()`, the renderer calls it instead of keeping a copy. - **Quote content, and code inside list items:** read with the validator's parser too. markdown-it-py stays, in **inline mode only**, for the Markdown inside one block's text: emphasis, diff --git a/docs/roadmap.md b/docs/roadmap.md index f8dcbd7..5aa3351 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -12,7 +12,7 @@ here. |---|---|---|---| | U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Open. Until then lists render with one level of items (ADR 0007); the resolver and writers already handle nesting | | U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | -| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, frontmatter splitting, value checks, conditions, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`, the frontmatter fields it reads. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Open. Until then the dependency is pinned to one minor version | +| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, frontmatter splitting, value checks, conditions, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`, the frontmatter fields it reads. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | | U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | | U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Open. It blocks rendering templates with answers | diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index f49debe..755eeb6 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -16,9 +16,9 @@ from functools import cache from legaldown import Document -from legaldown.definitions import block_fragments, text_fragments +from legaldown.definitions import block_fragments from legaldown.directives import Lexed, lex -from legaldown.validator.core import _frontmatter_fields +from legaldown.validator.core import is_template from legaldown.validator.templates import block_quotes from legaldown.validator.units import FoundMarker, find_markers @@ -29,8 +29,8 @@ class PlacedMarkers: """The validator's reading of a document's markers (§5.7, §15.3).""" - #: Whether the document is a template (§15.1), decided by the - #: validator's own formula. + #: Whether the document is a template (§15.1): the validator's own + #: decision (``is_template``), over the same markers. template: bool #: The placed markers, by (section index or None for the preamble, #: block index, fragment index in block_fragments(block)). @@ -45,40 +45,10 @@ def placed_markers(document: Document) -> PlacedMarkers: ``validate_document`` (legaldown.validator.core, §15.1).""" lexer = cache(lex) markers = find_markers(document, lexer) - template = _is_template(document, markers, lexer) + template = is_template(document, markers, lexer) placed = { (found.section, found.block, found.fragment): found for found in markers if found.marker is not None and found.placed(template) } return PlacedMarkers(template, placed, lexer) - - -def _is_template(document: Document, markers: list[FoundMarker], lexer: Callable[[str], Lexed]) -> bool: - """The validator's template test (legaldown.validator.core, - validate_document), over the validator's own findings. A copy, until the - validator exposes the decision (roadmap U3); the tests compare it with - the value validate_document itself computes, on every document.""" - metadata = document.metadata - - def names(texts: list[str]) -> set[str]: - return {directive.name for text in texts for directive in lexer(text or "").directives} - - return ( - metadata.questions is not None - or any(attachment.when for attachment in metadata.attachments) - or any(section.condition for section in document.sections) - or any(found.marker and found.marker.condition and not found.misplaced for found in markers) - or "choose" in names(_body_texts(document)) - or "choose" in names(_frontmatter_texts(document) + [section.title for section in document.sections]) - ) - - -def _body_texts(document: Document) -> list[str]: - return [text for _section, _index, block in document.iter_indexed_blocks() for text in text_fragments(block)] - - -def _frontmatter_texts(document: Document) -> list[str]: - structural, values = _frontmatter_fields(document.metadata) - return [text for _label, text in structural] + list(values) - diff --git a/tests/conformance/test_spec_corpus.py b/tests/conformance/test_spec_corpus.py index 430cf5e..38daccc 100644 --- a/tests/conformance/test_spec_corpus.py +++ b/tests/conformance/test_spec_corpus.py @@ -92,10 +92,10 @@ def test_raw_html_fixture_emits_no_html() -> None: @pytest.mark.parametrize("path", EXAMPLES + FIXTURES, ids=_name) def test_template_decision_is_the_validators(path: Path) -> None: - """The renderer's copy of the validator's template formula must give the - validator's own answer (validator_bridge._is_template, roadmap U3).""" + """The renderer asks the validator's is_template over the markers it + found itself; the answer must be the one the validator gives alone.""" from legaldown import parse_document - from validator_spy import validator_template + from legaldown.validator.core import is_template from legaldown_render.validator_bridge import placed_markers @@ -103,4 +103,4 @@ def test_template_decision_is_the_validators(path: Path) -> None: document = parse_document(path.read_text(encoding="utf-8")) except Exception: pytest.skip("not a readable document") - assert placed_markers(document).template == validator_template(document) + assert placed_markers(document).template == is_template(document) diff --git a/tests/test_render.py b/tests/test_render.py index f6dfde6..0071799 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -887,10 +887,10 @@ def test_table_rows_are_as_wide_as_the_header() -> None: def test_template_decision_matches_the_validator_on_the_test_documents() -> None: from legaldown import parse_document - from validator_spy import validator_template + from legaldown.validator.core import is_template from legaldown_render.validator_bridge import placed_markers for name in ("features", "template"): document = parse_document((DOCUMENTS / f"{name}.lgd").read_text(encoding="utf-8")) - assert placed_markers(document).template == validator_template(document) + assert placed_markers(document).template == is_template(document) diff --git a/tests/validator_spy.py b/tests/validator_spy.py deleted file mode 100644 index dca6903..0000000 --- a/tests/validator_spy.py +++ /dev/null @@ -1,29 +0,0 @@ -"""Test helper: the validator's own template decision, for checking the -renderer's copy of its formula (validator_bridge._is_template).""" -from __future__ import annotations - -import inspect -from unittest import mock - -from legaldown import Document, validate_document -from legaldown.validator import core - - -def validator_template(document: Document) -> bool: - """The ``template`` value validate_document computes and passes to its - Units — captured with a spy, since the validator does not expose it yet - (ForLegalAI/legaldown-validator#26).""" - seen: list[bool] = [] - original = core.Units - signature = inspect.signature(original) - assert "template" in signature.parameters, "Units no longer takes 'template'; update this spy" - - def spy(*args: object, **kwargs: object) -> object: - # However the validator passes it, by position or by name. - seen.append(bool(signature.bind(*args, **kwargs).arguments["template"])) - return original(*args, **kwargs) - - with mock.patch.object(core, "Units", spy): - validate_document(document) - assert seen, "validate_document no longer builds Units; update this spy" - return seen[-1] From c03e1e768751bc71a27dd18d1676f9997053938c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 19:32:02 +0000 Subject: [PATCH 04/19] =?UTF-8?q?Render=20templates=20with=20answers=20(?= =?UTF-8?q?=C2=A715.8),=20through=20the=20validator's=20assembly?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit With an answers set, a template is assembled by legaldown.assemble (§15.7, legaldown-validator#55) and the assembled document rendered; without one, it renders as its template view. - RenderOptions.answers / render(..., answers=...) and --answers FILE (YAML or JSON) on the CLI - Refused (RenderRefused, exit 1) when the template has Errors, which §15.7.2 defines no output for; when the answers do (answer-invalid, answer-missing); or when the template needs other files, which a renderer below Full does not read (§17.6) - Assembly Warnings (answer-unknown) are reported with the rest; an unanswered blank stays and --final reports it - The conformance run renders each of the specification's assembly fixtures with its answers and checks the output equals the expected document's, in text and HTML; the multi-file case is refused - Docs: pipeline stage 2, CONFORMANCE, README, roadmap v0.2 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 6 ++-- README.md | 20 +++++++----- docs/architecture.md | 21 ++++++++++-- docs/concepts.md | 3 +- docs/roadmap.md | 7 ++-- src/legaldown_render/api.py | 39 +++++++++++++++++++--- src/legaldown_render/cli.py | 30 ++++++++++++++++- src/legaldown_render/errors.py | 13 ++++---- tests/conformance/test_spec_corpus.py | 26 +++++++++++++++ tests/test_cli.py | 27 +++++++++++++++ tests/test_render.py | 47 +++++++++++++++++++++++++++ 11 files changed, 211 insertions(+), 28 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index abdff12..388ad24 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -21,6 +21,7 @@ It is checked against the specification's own examples and fixtures corpus (`tes | Party and side display rules (§3.6) | ✅ | | At least one output format (§13.6) | ✅ HTML (recommended) and plain text (optional) | | Comment stripping (§8.6) | ✅ Inline comments, and comment blocks across any number of lines. As in CommonMark, a comment block left unclosed runs to the end of the document | +| Templates with answers (§15.8) | ✅ Assembled first with `legaldown-validator`'s Assembly capability (§15.7), then rendered: `answers=` / `--answers answers.yaml`. A template that needs other files is refused (see below) | | Template view without answers (§15.8) | ✅ Conditions marked with style-defined labels; every `{{choose:}}` phrase shown as `[a / b]`; drafting notes styled; alternatives share a number | | `{{attach:}}` resolves to the declared title without reading the file | ✅ Linked to an attachment placeholder, or to the file when placeholders are omitted | @@ -42,8 +43,9 @@ It is checked against the specification's own examples and fixtures corpus (`tes ## Known limitations -- **Templates with answers.** Assembly (§15.7) is a Core capability that `legaldown-validator` - 0.2.0 does not provide yet, so a template always renders as its template view. +- **Assembly reads no other files.** A template with include fragments, LegalDown attachment + files, or `translations` is refused when rendered with answers, as §17.6 requires below Full. + A template with Errors is refused too, since §15.7.2 defines no output for one. - **Diagnostics have no line numbers**, because the validator's diagnostics do not carry them yet. - **Malformed directives** (a §11.2 grammar violation) render their type's marker without a value, for example `[INVALID AMOUNT]`. The specification defines no marker for this case. diff --git a/README.md b/README.md index fdd7d0e..e834a14 100644 --- a/README.md +++ b/README.md @@ -31,8 +31,9 @@ gives "1. června 2026", "10 000,00 Kč", and "12 měsíců", and amounts are ne 🎨 **Applies a style template**, so the same source renders as a US-style contract or a continental one without editing the text (§13.7). -📝 **Shows templates as templates.** Conditional clauses are marked, every `{{choose:}}` phrase is -shown, drafting notes are styled, and placeholders render as blanks (§15.8). +📝 **Shows templates as templates, or fills them in.** Without answers, conditional clauses are +marked, every `{{choose:}}` phrase is shown, drafting notes are styled, and placeholders render as +blanks. With an answers set, the template is assembled and the finished document rendered (§15.8). 🚩 **Never hides a problem.** Anything unresolved renders as a visible marker, such as `[BROKEN REF: id]`, and is reported with the specification's stable rule id. Nothing is dropped @@ -63,12 +64,14 @@ legaldown-render contract.lgd --set numbering.scheme=legal-outline --set enumera legaldown-render contract.lgd --set contents.enabled=true --set contents.depth=3 # table of contents legaldown-render contract.lgd --strict # refuse if the document has errors legaldown-render contract.lgd --final --strict # refuse if blanks or template constructs remain +legaldown-render template.lgd --answers answers.yaml -o nda.html # fill in a template, then render it legaldown-render --print-style --style continental # every setting, as YAML ``` Diagnostics go to stderr, one per line with its rule id. The exit code is 0 when the document -rendered, even if it has errors (they show as markers), 1 when rendering was refused or the file -cannot be read, and 2 for an invalid style or setting. +rendered, even if it has errors (they show as markers), 1 when rendering was refused (strict mode, +or a template that cannot be assembled with its answers) or a file cannot be read, and 2 for an +invalid style or setting. ## Python @@ -81,8 +84,9 @@ for diagnostic in result.diagnostics: open("contract.html", "w").write(result.output) ``` -`render()` also accepts a `RenderOptions`. Problems in the document never raise. Only an invalid -style (`StyleError`), an unreadable document (`DocumentError`), or strict mode +`render()` also accepts a `RenderOptions`. Pass `answers={...}` to render a template filled in. +Problems in the document never raise. Only an invalid style (`StyleError`), an unreadable document +(`DocumentError`), strict mode, or a template that cannot be assembled with its answers (`RenderRefused`) do. ## Styles and preferences @@ -101,8 +105,8 @@ definitions: { style: small-caps } typography: { font_family: "Georgia, serif", justify: true } ``` -**Render options** say what one job does: the output format, strict mode, the final check, a full -page or an HTML fragment. Values layer in this order: built-in defaults, then the `extends` chain, then your +**Render options** say what one job does: the output format, strict mode, the final check, the +answers to a template, a full page or an HTML fragment. Values layer in this order: built-in defaults, then the `extends` chain, then your style, then per-job `--set` overrides. Every value is validated before rendering starts. See [Styles and preferences](docs/style-templates.md). diff --git a/docs/architecture.md b/docs/architecture.md index 1afa94b..217d0dc 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -13,6 +13,10 @@ This page describes how the renderer is built. The decisions behind it are in │ normalize_source(): strip BOM, unify line endings ▼ ┌─────────────────────────┐ + │ 2. Assemble (answers) │ only with an answers set: legaldown-validator's assemble() turns + └───────────┬─────────────┘ the template into its assembled document, which the rest renders + ▼ + ┌─────────────────────────┐ │ 1. Parse & validate │ legaldown-validator: Document, ValidationResult (Core diagnostics, └───────────┬─────────────┘ resolved identifiers, party/side/definition/attachment lookups) ▼ @@ -31,9 +35,20 @@ This page describes how the renderer is built. The decisions behind it are in RenderResult(output, diagnostics, format, tree, style) ``` -Stage 2, assembly with an answers set (§15.7), is not implemented. It belongs in the core -package, and `legaldown-validator` 0.2.0 does not provide it yet ([roadmap](roadmap.md)). A -template is always rendered as its **template view** (§15.8). +Stage 2, **assembly** with an answers set (§15.7), runs only when the job has answers, before +the document is parsed for rendering. A template rendered with answers is assembled first and the +assembled document rendered; without answers, it renders as its **template view** (§15.8). +Assembly is the core package's (`legaldown.assemble`), exact to the byte as §15.7.2 requires; the +renderer only decides when to refuse: + +- the template has Errors: §15.7.2 defines assembly only for a template without them +- the answers do (`answer-invalid`, `answer-missing`) +- the template needs other files (include fragments, LegalDown attachment files, translations), + which the renderer does not read below the Full level (§17.6) + +A refusal raises `RenderRefused` with the findings. Assembly Warnings, such as `answer-unknown`, +are reported with the rest. Blanks left unanswered stay, and render as blanks: the result is a +draft (§15.1), which `--final` reports. `api.render()` runs the pipeline. Each stage returns frozen values. Only stage 4 reads the style's semantic settings, and only stage 5 knows about file formats. diff --git a/docs/concepts.md b/docs/concepts.md index 9582ba4..e3492ee 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -44,7 +44,8 @@ At Rendering level (§17.3) the renderer MUST: - apply the party and side display rules (§3.6) - produce at least one of PDF, DOCX, or HTML (§13.6) - strip comments (§8.6) -- render the template view when a template is rendered without answers (§15.8) +- render the template view when a template is rendered without answers, and assemble it first + when it is rendered with answers (§15.8) List enumeration (§13.2), style templates (§13.7), and signature blocks (§2.2) are SHOULD. We plan to support all three, because a legal renderer without them is not usable in practice. diff --git a/docs/roadmap.md b/docs/roadmap.md index 5aa3351..9e34568 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -15,7 +15,7 @@ here. | U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, frontmatter splitting, value checks, conditions, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`, the frontmatter fields it reads. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | | U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | -| U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Open. It blocks rendering templates with answers | +| U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Done in [validator#55](https://github.com/ForLegalAI/legaldown-validator/pull/55); the renderer uses it (v0.2) | | U5 | specification | A **rendering fixtures corpus**: source + style settings → expected plain-text output | Not started. The plain-text writer is designed to be its oracle | ## Milestones @@ -36,8 +36,9 @@ here. ### v0.2 — Templates with answers -- Rendering with an answers set, through the core package's assembly (U4b) -- `--answers answers.yaml` on the CLI +- Rendering with an answers set, through the core package's assembly (U4b): done +- `--answers answers.yaml` on the CLI: done +- Released together with the validator release that ships the CommonMark block model and assembly ### v0.3 — DOCX diff --git a/src/legaldown_render/api.py b/src/legaldown_render/api.py index ba3c542..2a53114 100644 --- a/src/legaldown_render/api.py +++ b/src/legaldown_render/api.py @@ -6,7 +6,8 @@ locale, labels, typography. It is a reusable YAML file (``--style``), with single values overridable per job (``--set key=value``, ``overrides``); * **render options** say what this one job does — the output format, strict - mode, the final check, a full HTML page or a fragment. They never change how a document + mode, the final check, the answers to a template, a full HTML page or a + fragment. They never change how a document looks, so they are not part of the style. """ from __future__ import annotations @@ -17,7 +18,7 @@ from typing import Any import yaml -from legaldown import Diagnostic, Document, ValidationResult, parse_document, validate_document +from legaldown import AssemblyError, Diagnostic, Document, ValidationResult, assemble, parse_document, validate_document from .build import build_tree, normalize_source from .errors import DocumentError, RenderRefused @@ -47,6 +48,11 @@ class RenderOptions: #: (``template-construct-present``) is an Error. With ``strict``, such a #: document is refused. final: bool = False + #: An answers set (§15.7.1): question id to answer, as YAML or JSON + #: would load it. With answers, a template is assembled first and the + #: assembled document is rendered (§15.8); without, a template renders + #: as its template view. + answers: Mapping[str, Any] | None = None #: HTML only: a complete page (True) or just the ``
`` (False). standalone: bool = True @@ -80,7 +86,7 @@ def render(source: str, options: RenderOptions | None = None, /, **settings: Any are listed in ``diagnostics``. Raises :class:`StyleError` for an invalid style or setting, :class:`DocumentError` for a document that cannot be read, and :class:`RenderRefused` in strict mode when the document has - errors. + errors, or when a template cannot be assembled with ``answers``. """ if options is None: options = RenderOptions(**settings) @@ -94,9 +100,12 @@ def render(source: str, options: RenderOptions | None = None, /, **settings: Any style = load_style(options.style, overrides=overrides) source = normalize_source(source) + assembled: list[Diagnostic] = [] + if options.answers is not None: + source, assembled = _assemble(source, options.answers) document = _parse(source) result = validate_document(document, final=options.final) - diagnostics = list(result.diagnostics) + diagnostics = assembled + list(result.diagnostics) if options.strict and any(d.level == "error" for d in diagnostics): raise RenderRefused(diagnostics) @@ -115,6 +124,28 @@ def render_file(path: str | Path, options: RenderOptions | None = None, /, **set return render(Path(path).read_text(encoding="utf-8"), options, **settings) +def _assemble(source: str, answers: Mapping[str, Any]) -> tuple[str, list[Diagnostic]]: + """*source* assembled with *answers* (§15.7), and the assembly's + Warnings (such as ``answer-unknown``). Raises RenderRefused when it + cannot be assembled: the template has Errors, for which §15.7.2 defines + no output, or the answers do (``answer-invalid``, ``answer-missing``), + or the template needs files a renderer below Full does not read + (includes, LegalDown attachments, translations; §17.6).""" + errors = [d for d in validate_document(_parse(source)).diagnostics if d.level == "error"] + if errors: + raise RenderRefused(errors, f"Assembly refused: the template has {len(errors)} error(s), " + "and only a template without errors can be assembled (§15.7.2).") + try: + result = assemble(source, answers) + except AssemblyError as error: + raise DocumentError(f"The template cannot be assembled: {error}") from error + if not result.ok: + errors = [d for d in result.diagnostics if d.level == "error"] + raise RenderRefused(result.diagnostics, f"Assembly refused: {len(errors)} error(s) in the answers or " + "the template (§15.7).") + return normalize_source(result.output), list(result.diagnostics) + + def _parse(source: str) -> Document: try: return parse_document(source) diff --git a/src/legaldown_render/cli.py b/src/legaldown_render/cli.py index a572acc..49cfc43 100644 --- a/src/legaldown_render/cli.py +++ b/src/legaldown_render/cli.py @@ -12,6 +12,8 @@ from pathlib import Path from typing import Any +import yaml + from . import __version__ from .api import RenderOptions, render from .errors import DocumentError, InternalError, RenderRefused @@ -47,6 +49,9 @@ def _parser() -> argparse.ArgumentParser: parser.add_argument("--locale", help="formatting locale, e.g. en-US or cs-CZ (default: the style's, else the document language)") parser.add_argument("--fragment", action="store_true", help="HTML: write only the
, without page and stylesheet") parser.add_argument("--strict", action="store_true", help="refuse to render a document that has errors") + parser.add_argument("--answers", metavar="FILE", + help="a YAML or JSON answers set (§15.7.1): the template is assembled with it and the " + "assembled document rendered") parser.add_argument("--final", action="store_true", help="the document is meant for signature: a remaining blank, questions key, condition, " "choice, or drafting note is an error (§15.9); with --strict, it is refused") @@ -86,8 +91,15 @@ def main(argv: list[str] | None = None) -> int: print(f"legaldown-render: cannot read {name}: {error.strerror or error}", file=sys.stderr) return 1 + answers = None + if args.answers: + answers, failure = _read_answers(args.answers) + if failure: + print(f"legaldown-render: {failure}", file=sys.stderr) + return 1 + options = RenderOptions(format=output_format, style=args.style, overrides=overrides, - strict=args.strict, final=args.final, standalone=not args.fragment) + strict=args.strict, final=args.final, answers=answers, standalone=not args.fragment) try: result = render(source, options) except StyleError as error: @@ -112,6 +124,22 @@ def main(argv: list[str] | None = None) -> int: return 0 +def _read_answers(path: str) -> tuple[dict | None, str | None]: + """The answers set in *path* (§15.7.1): a YAML mapping (JSON is YAML + too), or a message saying why it cannot be read.""" + try: + answers = yaml.safe_load(Path(path).read_text(encoding="utf-8")) + except OSError as error: + return None, f"cannot read {path}: {error.strerror or error}" + except (yaml.YAMLError, ValueError) as error: # ValueError: a date such as 2026-13-45 + return None, f"cannot read the answers in {path}: {error}" + if answers is None: + return {}, None + if not isinstance(answers, dict): + return None, f"the answers in {path} must be a mapping of question ids to answers" + return answers, None + + def _report(name: str, diagnostics: list, *, quiet: bool) -> None: if quiet: return diff --git a/src/legaldown_render/errors.py b/src/legaldown_render/errors.py index bb45ee7..5355828 100644 --- a/src/legaldown_render/errors.py +++ b/src/legaldown_render/errors.py @@ -11,18 +11,19 @@ class RenderError(Exception): class DocumentError(RenderError): - """The document cannot be read at all, e.g. its frontmatter is not a - YAML mapping.""" + """The document cannot be read at all, e.g. its frontmatter is not + valid YAML.""" class RenderRefused(RenderError): - """Rendering was refused because the document has errors and strict - mode is on. ``diagnostics`` holds every finding.""" + """Rendering was refused: the document has errors and strict mode is on, + or a template could not be assembled with its answers (§15.7). + ``diagnostics`` holds every finding.""" - def __init__(self, diagnostics: list[Diagnostic]) -> None: + def __init__(self, diagnostics: list[Diagnostic], reason: str | None = None) -> None: self.diagnostics = diagnostics errors = sum(1 for diagnostic in diagnostics if diagnostic.level == "error") - super().__init__(f"Rendering refused in strict mode: the document has {errors} error(s).") + super().__init__(reason or f"Rendering refused in strict mode: the document has {errors} error(s).") class InternalError(RenderError): diff --git a/tests/conformance/test_spec_corpus.py b/tests/conformance/test_spec_corpus.py index 38daccc..388f662 100644 --- a/tests/conformance/test_spec_corpus.py +++ b/tests/conformance/test_spec_corpus.py @@ -104,3 +104,29 @@ def test_template_decision_is_the_validators(path: Path) -> None: except Exception: pytest.skip("not a readable document") assert placed_markers(document).template == is_template(document) + + +ASSEMBLY = sorted(path for path in (SPEC / "fixtures" / "assembly").iterdir() if path.is_dir()) \ + if (SPEC / "fixtures" / "assembly").is_dir() else [] + + +@pytest.mark.parametrize("case", ASSEMBLY, ids=lambda case: case.name) +def test_rendering_with_answers_renders_the_assembled_document(case: Path) -> None: + """A template rendered with answers is its assembled document rendered + (§15.8); a case that needs Full, reading other files, is refused.""" + import yaml + + from legaldown_render import RenderRefused + + template = (case / "template.lgd").read_text(encoding="utf-8") + answers = yaml.safe_load((case / "answers.yaml").read_text(encoding="utf-8")) or {} + level = json.loads((case / "case.json").read_text(encoding="utf-8")).get("requires_level") \ + if (case / "case.json").exists() else "core" + if level == "full": + with pytest.raises(RenderRefused): + render(template, answers=answers) + return + expected = (case / "expected.lgd").read_text(encoding="utf-8") + for output_format in ("text", "html"): + assert render(template, answers=answers, format=output_format).output == \ + render(expected, format=output_format).output diff --git a/tests/test_cli.py b/tests/test_cli.py index 2d0ab3e..3385d32 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -42,6 +42,33 @@ def test_final_with_strict_exits_1(capsys: pytest.CaptureFixture[str], tmp_path: assert "template-construct-present" in capsys.readouterr().err +def test_answers_file(capsys: pytest.CaptureFixture[str], tmp_path: Path) -> None: + template = str(DOCUMENTS / "template.lgd") + answers = tmp_path / "answers.yaml" + answers.write_text("client-name: Beta Ltd\nfee: '5000.00'\nnon-solicit: false\nforum: courts\n", encoding="utf-8") + assert main([template, "--answers", str(answers), "-f", "text", "-q"]) == 0 + output = capsys.readouterr().out + assert "Disputes are resolved by the courts." in output and "arbitration" not in output + + answers.write_text("forum: mediation\n", encoding="utf-8") + assert main([template, "--answers", str(answers), "-f", "text"]) == 1 + assert "answer-invalid" in capsys.readouterr().err + + +@pytest.mark.parametrize(("content", "message"), [ + (None, "cannot read"), + ("- a list\n", "must be a mapping"), + ("date: 2026-13-45\n", "cannot read the answers"), +]) +def test_unreadable_answers_exit_1(capsys: pytest.CaptureFixture[str], tmp_path: Path, content: str | None, + message: str) -> None: + answers = tmp_path / "answers.yaml" + if content is not None: + answers.write_text(content, encoding="utf-8") + assert main([str(DOCUMENTS / "template.lgd"), "--answers", str(answers)]) == 1 + assert message in capsys.readouterr().err + + def test_print_and_list_styles(capsys: pytest.CaptureFixture[str]) -> None: assert main(["--print-style", "--style", "continental", "--set", "locale=de"]) == 0 printed = capsys.readouterr().out diff --git a/tests/test_render.py b/tests/test_render.py index 0071799..8a588df 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -399,6 +399,53 @@ def test_a_colon_ending_in_a_no_break_space_keeps_it_after_date_and_place() -> N assert "\nDate \u2013\xa0\n" in output and "\nPlace \u2013\xa0\n" in output +TEMPLATE = (DOCUMENTS / "template.lgd").read_text(encoding="utf-8") +ANSWERS = {"client-name": "Beta Ltd", "fee": "5000.00", "non-solicit": False, "forum": "arbitration"} + + +def test_a_template_with_answers_renders_its_assembled_document() -> None: + result = render(TEMPLATE, answers=ANSWERS, format="text") + assert not result.tree.is_template + assert "Consulting Agreement with Beta Ltd" in result.output + assert "Non-Solicitation" not in result.output + assert "Disputes are resolved by arbitration." in result.output + assert "courts" not in result.output + assert "a competent court or an emergency arbitrator" in result.output + assert "Drafting note" not in result.output and "partner in charge" not in result.output + assert "Only if" not in result.output + + +def test_assembly_warnings_are_reported_with_the_rest() -> None: + result = render(TEMPLATE, answers={**ANSWERS, "stray": "x"}, format="text") + assert "answer-unknown" in {d.rule for d in result.diagnostics} + assert result.ok + + +@pytest.mark.parametrize(("answers", "rule"), [ + ({key: value for key, value in ANSWERS.items() if key != "forum"}, "answer-missing"), + ({**ANSWERS, "forum": "mediation"}, "answer-invalid"), +]) +def test_assembly_errors_refuse_the_render(answers: dict, rule: str) -> None: + with pytest.raises(RenderRefused, match="Assembly refused") as caught: + render(TEMPLATE, answers=answers) + assert rule in {d.rule for d in caught.value.diagnostics} + + +def test_a_template_with_errors_is_not_assembled() -> None: + broken = TEMPLATE.replace("{{ref: disputes}}", "{{ref: nowhere}}") + with pytest.raises(RenderRefused, match="only a template without errors") as caught: + render(broken, answers=ANSWERS) + assert "ref-broken" in {d.rule for d in caught.value.diagnostics} + + +def test_unanswered_blanks_stay_and_the_final_check_catches_them() -> None: + answers = {key: value for key, value in ANSWERS.items() if key != "fee"} + result = render(TEMPLATE, answers=answers, format="text", final=True) + assert "[_____]" in result.output + assert "placeholder-unfilled" in {d.rule for d in result.diagnostics} + assert "template-construct-present" not in {d.rule for d in result.diagnostics} + + def test_final_check_reports_blanks_and_template_constructs() -> None: body = "# A\n\nPay {{placeholder: fee, type=text}}.\n\n> [!DRAFTING]\n> Check the fee.\n" assert not {"placeholder-unfilled", "template-construct-present"} & {d.rule for d in render(FRONT + body).diagnostics} From 06c47cf8284a3940621531563cb92a6044b5c7f7 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 19:42:07 +0000 Subject: [PATCH 05/19] Fix code review findings on PR #2 - Code blocks follow the validator's CommonMark helpers instead of a re-derivation: a fenced block's lines lose as much indentation as its opening fence had, and a fence indented four columns or more does not close it (legaldown.markdown: FENCE_OPEN_RE, closes_fence, dedent) - A section's rendered level is how many parts its number has, so its number format, heading tag, CSS class, and heading style agree when a document starts below level 1 or skips a level - The answers file is read by the validator's own reader, as `legaldown assemble` reads it - An assembly refused for template Errors carries every finding, the Warnings included; the redundant normalization of the output is gone - The template-decision tests compare with the value validate_document itself computes, not with is_template alone - pyproject.toml says the code needs the validator release after 0.2.0 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- docs/roadmap.md | 2 +- pyproject.toml | 3 ++ src/legaldown_render/api.py | 10 +++--- src/legaldown_render/build.py | 43 ++++++++++++------------ src/legaldown_render/cli.py | 22 ++---------- src/legaldown_render/resolve/resolver.py | 16 +++++---- src/legaldown_render/tree.py | 5 ++- src/legaldown_render/validator_bridge.py | 15 ++++++++- tests/conformance/test_spec_corpus.py | 6 ++-- tests/conftest.py | 22 ++++++++++++ tests/test_cli.py | 2 +- tests/test_render.py | 26 +++++++++++--- 12 files changed, 107 insertions(+), 65 deletions(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 9e34568..58cf38d 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -12,7 +12,7 @@ here. |---|---|---|---| | U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Open. Until then lists render with one level of items (ADR 0007); the resolver and writers already handle nesting | | U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | -| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, frontmatter splitting, value checks, conditions, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`, the frontmatter fields it reads. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | +| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, the CommonMark fence and indentation helpers, the answers-file reader, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | | U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | | U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Done in [validator#55](https://github.com/ForLegalAI/legaldown-validator/pull/55); the renderer uses it (v0.2) | diff --git a/pyproject.toml b/pyproject.toml index 6c9fc6e..e75dde9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -31,6 +31,9 @@ classifiers = [ dependencies = [ # The only LegalDown parser (docs/decisions/0002). A minor bump may change # the model, so the range is pinned to one minor version. + # NOT RELEASABLE YET: this code needs the validator release after 0.2.0 + # (the CommonMark block model, is_template, assemble). Move the range to + # that release when it is published. "legaldown-validator>=0.2.0,<0.3", # Inline Markdown (emphasis, links, code spans) within the validator's # blocks, parsed in inline mode only (docs/decisions/0007). diff --git a/src/legaldown_render/api.py b/src/legaldown_render/api.py index 2a53114..bcd053d 100644 --- a/src/legaldown_render/api.py +++ b/src/legaldown_render/api.py @@ -131,10 +131,11 @@ def _assemble(source: str, answers: Mapping[str, Any]) -> tuple[str, list[Diagno no output, or the answers do (``answer-invalid``, ``answer-missing``), or the template needs files a renderer below Full does not read (includes, LegalDown attachments, translations; §17.6).""" - errors = [d for d in validate_document(_parse(source)).diagnostics if d.level == "error"] + findings = validate_document(_parse(source)).diagnostics + errors = sum(1 for d in findings if d.level == "error") if errors: - raise RenderRefused(errors, f"Assembly refused: the template has {len(errors)} error(s), " - "and only a template without errors can be assembled (§15.7.2).") + raise RenderRefused(findings, f"Assembly refused: the template has {errors} error(s), " + "and only a template without errors can be assembled (§15.7.2).") try: result = assemble(source, answers) except AssemblyError as error: @@ -143,7 +144,8 @@ def _assemble(source: str, answers: Mapping[str, Any]) -> tuple[str, list[Diagno errors = [d for d in result.diagnostics if d.level == "error"] raise RenderRefused(result.diagnostics, f"Assembly refused: {len(errors)} error(s) in the answers or " "the template (§15.7).") - return normalize_source(result.output), list(result.diagnostics) + # The source was normalized, and assembly keeps its line breaks. + return result.output, list(result.diagnostics) def _parse(source: str) -> Document: diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index 68c6de7..a9eccc3 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -57,15 +57,22 @@ Table, Text, ) -from .validator_bridge import block_fragments, block_quotes, lex, placed_markers +from .validator_bridge import ( + FENCE_OPEN_RE, + block_fragments, + block_quotes, + closes_fence, + dedent, + indent_width, + lex, + placed_markers, +) _OPEN, _CLOSE = "\ue000", "\ue001" # Leads the sentinel of source that renders nothing (a {{def:}}). It is # Unicode punctuation, so an emphasis closer just before it still counts as # right-flanking and closes (CommonMark); a bare sentinel reads like a letter. _HIDDEN_LEAD = "\u2e31" -# A fenced code block's opening line (validator's model keeps the fences). -_FENCE_RE = re.compile(r"^ {0,3}(`{3,}|~{3,})(.*)$") # Stand where inline HTML or a comment was dropped, until _merge_text joins # the text around them. A dropped line break (
) still parts two words. _DROPPED = Text("") @@ -404,7 +411,7 @@ def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> Li offset, identifier, condition = placed[index] item = _cut_marker(item, offset) first, _, rest = item.partition("\n") - if first.lstrip().startswith(">") or _FENCE_RE.match(first): + if first.lstrip().startswith(">") or FENCE_OPEN_RE.match(first): # The item opens with a quote or code, whose lines the # validator keeps as written (a drafting note included). blocks = self.fragment(item) @@ -494,28 +501,20 @@ def _paragraph_source(block: ModelBlock) -> str: def _code_block(text: str) -> CodeBlock: - """A code block from the validator's model: fenced, fences included, - or indented, each line by four columns (CommonMark).""" + """A code block from the validator's model, read by the validator's + CommonMark rules: an indented block loses four columns from each line; + a fenced one loses its fences, and from each line as much indentation + as its opening fence had.""" lines = text.split("\n") - opening = _FENCE_RE.match(lines[0]) if lines else None + opening = FENCE_OPEN_RE.match(lines[0]) if lines else None if opening is None: - return CodeBlock("\n".join(_unindent(line) for line in lines) + "\n") - fence = opening.group(1) + return CodeBlock("\n".join(dedent(line, 4) for line in lines) + "\n") body = lines[1:] - if body and body[-1].strip().startswith(fence[0] * len(fence)) and not body[-1].strip().strip(fence[0]): + if body and closes_fence(body[-1], opening.group("fence")): body = body[:-1] - return CodeBlock("\n".join(body) + ("\n" if body else ""), opening.group(2).strip()) - - -def _unindent(line: str) -> str: - """*line* without an indented code block's four columns of - indentation, a tab counting to the next multiple of four.""" - column = 0 - for index, char in enumerate(line): - if column >= 4 or char not in " \t": - return line[index:] - column = column + 4 - column % 4 if char == "\t" else column + 1 - return "" + indent = indent_width(lines[0]) + body = [dedent(line, indent) for line in body] + return CodeBlock("\n".join(body) + ("\n" if body else ""), lines[0][opening.end():].strip()) def _trim(inlines: tuple[Inline, ...]) -> tuple[Inline, ...]: diff --git a/src/legaldown_render/cli.py b/src/legaldown_render/cli.py index 49cfc43..8a6dad3 100644 --- a/src/legaldown_render/cli.py +++ b/src/legaldown_render/cli.py @@ -12,12 +12,11 @@ from pathlib import Path from typing import Any -import yaml - from . import __version__ from .api import RenderOptions, render from .errors import DocumentError, InternalError, RenderRefused from .style import StyleError, builtin_styles, dump_style, load_style, parse_override +from .validator_bridge import read_answers from .writers import FORMATS, format_for_path @@ -93,7 +92,8 @@ def main(argv: list[str] | None = None) -> int: answers = None if args.answers: - answers, failure = _read_answers(args.answers) + # Read as legaldown-validator's `legaldown assemble` reads it (§15.7.1). + answers, failure = read_answers(Path(args.answers)) if failure: print(f"legaldown-render: {failure}", file=sys.stderr) return 1 @@ -124,22 +124,6 @@ def main(argv: list[str] | None = None) -> int: return 0 -def _read_answers(path: str) -> tuple[dict | None, str | None]: - """The answers set in *path* (§15.7.1): a YAML mapping (JSON is YAML - too), or a message saying why it cannot be read.""" - try: - answers = yaml.safe_load(Path(path).read_text(encoding="utf-8")) - except OSError as error: - return None, f"cannot read {path}: {error.strerror or error}" - except (yaml.YAMLError, ValueError) as error: # ValueError: a date such as 2026-13-45 - return None, f"cannot read the answers in {path}: {error}" - if answers is None: - return {}, None - if not isinstance(answers, dict): - return None, f"the answers in {path} must be a mapping of question ids to answers" - return answers, None - - def _report(name: str, diagnostics: list, *, quiet: bool) -> None: if quiet: return diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index 33db9ec..a0d2465 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -200,10 +200,10 @@ def _contents( if not settings.enabled: return () entries = [ - ContentsEntry(section.depth, section.label, unlinked(section.title), section.anchor, + ContentsEntry(section.level, section.label, unlinked(section.title), section.anchor, section.condition_label) for section in sections - if section.depth <= settings.depth + if section.level <= settings.depth ] if settings.attachments and attachments: # Under the attachments heading, unless a style blanks its label. @@ -259,14 +259,16 @@ def _number_sections(self) -> list[Section]: so a rendered number and a reference to it always agree with the validator: alternatives share a number (§15.8), and a skipped heading level counts as 1. The style only formats them, the n-th - part of a number ("2.1" has two) with the n-th level format. + part of a number ("2.1" has two) with the n-th level format. A + section's rendered level is how many parts its number has, so its + number format, heading, and heading style always go together. """ levels = heading_levels(self.style.numbering) presences: dict[int, Presence] = {} # level -> presence of the open section out: list[Section] = [] for section, indexed in zip(self.tree.sections, self.result.sections, strict=True): - # Stored clamped to 1-5 (§4.1), so every writer nests the section - # where the validator reads it. + # The heading's own level, clamped to 1-5 (§4.1), decides which + # sections enclose it, and so its presence (§15.3). level = min(max(section.level, 1), 5) enclosing = max((lvl for lvl in presences if lvl < level), default=None) presences = {lvl: p for lvl, p in presences.items() if lvl < level} @@ -285,8 +287,8 @@ def _number_sections(self) -> list[Section]: label = fill(fmt.label, n=format_counter(parts[-1], fmt.counter), path=designation) anchor = self._anchor(section.identifier) self._register(section.identifier, _Target(designation, anchor)) - out.append(replace(section, level=level, label=label, designation=designation, anchor=anchor, - depth=len(parts))) + out.append(replace(section, level=min(len(parts), 5), label=label, designation=designation, + anchor=anchor)) return out def _presence(self, condition: str) -> Presence: diff --git a/src/legaldown_render/tree.py b/src/legaldown_render/tree.py index 0dd74b7..6e15f78 100644 --- a/src/legaldown_render/tree.py +++ b/src/legaldown_render/tree.py @@ -250,6 +250,8 @@ class Section: """One heading and the blocks up to the next heading. Sections are kept flat, in document order; ``level`` gives the hierarchy.""" + #: The heading's level as written; after resolution, how many parts + #: its number has (1 for "2", 2 for "2.1"), clamped to 1-5. level: int title: tuple[Inline, ...] #: The identifier the validator resolved (explicit, or generated by §5.3). @@ -261,9 +263,6 @@ class Section: designation: str | None = None anchor: str | None = None condition_label: str | None = None - #: How deep its number goes: 1 for "2", 2 for "2.1". Less than - #: ``level`` when a heading level is skipped. - depth: int = 0 @dataclass(frozen=True, slots=True) diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index 755eeb6..b05f38f 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -16,13 +16,26 @@ from functools import cache from legaldown import Document +from legaldown.cli import _read_answers as read_answers from legaldown.definitions import block_fragments from legaldown.directives import Lexed, lex +from legaldown.markdown import FENCE_OPEN_RE, closes_fence, dedent, indent_width from legaldown.validator.core import is_template from legaldown.validator.templates import block_quotes from legaldown.validator.units import FoundMarker, find_markers -__all__ = ["PlacedMarkers", "block_fragments", "block_quotes", "lex", "placed_markers"] +__all__ = [ + "FENCE_OPEN_RE", + "PlacedMarkers", + "block_fragments", + "block_quotes", + "closes_fence", + "dedent", + "indent_width", + "lex", + "placed_markers", + "read_answers", +] @dataclass(frozen=True, slots=True) diff --git a/tests/conformance/test_spec_corpus.py b/tests/conformance/test_spec_corpus.py index 388f662..396481d 100644 --- a/tests/conformance/test_spec_corpus.py +++ b/tests/conformance/test_spec_corpus.py @@ -93,9 +93,9 @@ def test_raw_html_fixture_emits_no_html() -> None: @pytest.mark.parametrize("path", EXAMPLES + FIXTURES, ids=_name) def test_template_decision_is_the_validators(path: Path) -> None: """The renderer asks the validator's is_template over the markers it - found itself; the answer must be the one the validator gives alone.""" + found itself; the answer must be the one validate_document uses.""" + from conftest import validator_template from legaldown import parse_document - from legaldown.validator.core import is_template from legaldown_render.validator_bridge import placed_markers @@ -103,7 +103,7 @@ def test_template_decision_is_the_validators(path: Path) -> None: document = parse_document(path.read_text(encoding="utf-8")) except Exception: pytest.skip("not a readable document") - assert placed_markers(document).template == is_template(document) + assert placed_markers(document).template == validator_template(document) ASSEMBLY = sorted(path for path in (SPEC / "fixtures" / "assembly").iterdir() if path.is_dir()) \ diff --git a/tests/conftest.py b/tests/conftest.py index 61a55d5..24bcae9 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -28,3 +28,25 @@ def spec_root() -> Path | None: if candidate and (Path(candidate) / "fixtures").is_dir(): return Path(candidate) return None + + +def validator_template(document: object) -> bool: + """The template decision validate_document itself makes: what its own + call to is_template returns, captured. The renderer asks is_template + too; this checks that validate_document still decides the same way.""" + from unittest import mock + + from legaldown import validate_document + from legaldown.validator import core + + seen: list[bool] = [] + original = core.is_template + + def spy(*args: object, **kwargs: object) -> bool: + seen.append(original(*args, **kwargs)) + return seen[-1] + + with mock.patch.object(core, "is_template", spy): + validate_document(document) + assert seen, "validate_document no longer calls is_template; update this helper" + return seen[-1] diff --git a/tests/test_cli.py b/tests/test_cli.py index 3385d32..c735a81 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -57,7 +57,7 @@ def test_answers_file(capsys: pytest.CaptureFixture[str], tmp_path: Path) -> Non @pytest.mark.parametrize(("content", "message"), [ (None, "cannot read"), - ("- a list\n", "must be a mapping"), + ("- a list\n", "mapping"), ("date: 2026-13-45\n", "cannot read the answers"), ]) def test_unreadable_answers_exit_1(capsys: pytest.CaptureFixture[str], tmp_path: Path, content: str | None, diff --git a/tests/test_render.py b/tests/test_render.py index 8a588df..6cfd1d1 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -432,10 +432,12 @@ def test_assembly_errors_refuse_the_render(answers: dict, rule: str) -> None: def test_a_template_with_errors_is_not_assembled() -> None: - broken = TEMPLATE.replace("{{ref: disputes}}", "{{ref: nowhere}}") + broken = TEMPLATE.replace("{{ref: disputes}}", "{{ref: nowhere}}").replace("# Relief", "# Relief\n\n# Привет") with pytest.raises(RenderRefused, match="only a template without errors") as caught: render(broken, answers=ANSWERS) - assert "ref-broken" in {d.rule for d in caught.value.diagnostics} + levels = {d.rule: d.level for d in caught.value.diagnostics} + assert levels.get("ref-broken") == "error" + assert "warning" in levels.values() def test_unanswered_blanks_stay_and_the_final_check_catches_them() -> None: @@ -807,6 +809,22 @@ def test_table_columns_keep_their_alignment() -> None: assert "4" in html +def test_a_fence_s_indent_is_removed_from_its_lines() -> None: + html = render(FRONT + "# A\n\n ```py\n a\n b\n c\n ```\n", standalone=False).output + assert 'a\nb\n c\n' in html + + +def test_a_fence_indented_four_columns_does_not_close_code() -> None: + html = render(FRONT + "# A\n\n```\ncode\n ```\n", standalone=False).output + assert "code\n ```\n" in html + + +def test_a_section_s_level_is_its_number_s_depth() -> None: + html = render(FRONT + "## A\n\n### B\n", standalone=False, style="continental").output + assert '
\n

1.' in html + assert '
\n

1.1' in html + + def test_indented_code_renders_as_code_without_its_indent() -> None: html = render(FRONT + "# A\n\nPara.\n\n code {{ref: x}}\n deeper\n", standalone=False).output assert "
code {{ref: x}}\n  deeper\n
" in html @@ -933,11 +951,11 @@ def test_table_rows_are_as_wide_as_the_header() -> None: def test_template_decision_matches_the_validator_on_the_test_documents() -> None: + from conftest import validator_template from legaldown import parse_document - from legaldown.validator.core import is_template from legaldown_render.validator_bridge import placed_markers for name in ("features", "template"): document = parse_document((DOCUMENTS / f"{name}.lgd").read_text(encoding="utf-8")) - assert placed_markers(document).template == is_template(document) + assert placed_markers(document).template == validator_template(document) From c48f92a4e841e20fe251c6375c3c22806eadffa9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 04:50:10 +0000 Subject: [PATCH 06/19] Render nested lists from the validator's model (legaldown-validator#61) The validator now keeps nested lists: each listed item has its depth and the kind of the list it is in (models.listed_items). The builder nests them, an item of the other kind starting another list below the top level, as CommonMark does, and the resolver and writers, which already handled nesting, number and write them: an item nested under 2.1(b) is "2.1(b)(i)", no longer the flattened "2.1(c)". Also follows validator#60: a paragraph indented after a blank line is the list item's, so a marker there is misplaced and a reference to it broken, as the validator reports. Tests and golden files follow; CONFORMANCE.md, the roadmap (U1 done), and ADR 0007 drop the one-level limitation. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 9 +-- docs/architecture.md | 1 - .../0007-one-parser-validator-model.md | 4 +- docs/roadmap.md | 8 ++- src/legaldown_render/build.py | 57 ++++++++++++++----- src/legaldown_render/validator_bridge.py | 2 + tests/golden/features.continental.txt | 8 +-- tests/golden/features.default.html | 12 ++-- tests/golden/features.default.txt | 8 +-- tests/golden/features.outline.txt | 8 +-- tests/test_render.py | 34 ++++++----- 11 files changed, 94 insertions(+), 57 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 388ad24..e11c843 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -29,7 +29,7 @@ It is checked against the specification's own examples and fixtures corpus (`tes | Feature | Status | |---|---| -| List enumeration (§13.2) | ✅ Configurable per depth; can be disabled. Ordered lists are always renumbered. Lists have one level until the validator keeps nesting (see below) | +| List enumeration (§13.2) | ✅ Configurable per depth; can be disabled. Ordered lists are always renumbered. Nested lists keep their structure, so a nested item is referenced as "2.1(b)(i)" | | Paragraph numbering and section-qualified items (§13.2) | ✅ Style settings; the `continental` style uses both | | Style templates in a separate file (§13.7) | ✅ YAML, layered and validated; see [docs/style-templates.md](docs/style-templates.md) | | Signature blocks (§2.2) | ✅ One per party, with `legal_name` and each representative | @@ -52,11 +52,8 @@ It is checked against the specification's own examples and fixtures corpus (`tes - **One parser.** The renderer builds from the validator's document model only ([ADR 0007](docs/decisions/0007-one-parser-validator-model.md)). Anchors, markers, and the template view therefore always agree with the validator's diagnostics. The price is that some - structure renders as the validator's model holds it, until that model keeps more - ([legaldown-validator#14](https://github.com/ForLegalAI/legaldown-validator/issues/14), - roadmap U1–U2): - - **Nested lists** render as one level of items, so an item is referenced as "2.1(c)", never - "2.1(b)(i)" ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14), [#16](https://github.com/ForLegalAI/legaldown-validator/issues/16)). The resolver and writers already handle nesting. + structure renders as the validator's model holds it, until that model keeps more (roadmap + U1b–U2): - **Line breaks** within a paragraph are joined, and a backslash hard break shows as a literal backslash. Link reference definitions (`[label]: url`) are not resolved ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)). diff --git a/docs/architecture.md b/docs/architecture.md index 217d0dc..6cf9f66 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -105,7 +105,6 @@ stays. Each text that had a tag (a paragraph, a title, a table cell) counts once The builder never works around the validator's model. Where the model loses or misreads something, the output follows the model, and the gap is listed in `CONFORMANCE.md` and filed on the validator: -- lists have one level of items - a paragraph's line breaks are joined - indented code inside a quote or a list item is paragraph text diff --git a/docs/decisions/0007-one-parser-validator-model.md b/docs/decisions/0007-one-parser-validator-model.md index a43c968..817b5fa 100644 --- a/docs/decisions/0007-one-parser-validator-model.md +++ b/docs/decisions/0007-one-parser-validator-model.md @@ -60,5 +60,7 @@ module, `validator_bridge.py`. That module is the list for roadmap item U3. decision expects: the fix lands in the validator, and the renderer drops its workaround. Table alignment came with the same change. - When the validator keeps nested lists (#14), the renderer gains them with no parser of its - own. The resolver and writers already handle nested lists. + own. The resolver and writers already handle nested lists. This happened with validator #61: + the builder turns its listed items and their depths into nested lists, and "2.1(b)(i)" + replaced "2.1(c)". - The build stage is smaller: about 100 lines of position rules and special cases are gone. diff --git a/docs/roadmap.md b/docs/roadmap.md index 58cf38d..753bfcd 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -10,7 +10,7 @@ here. | # | Where | Change | Status | |---|---|---|---| -| U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Open. Until then lists render with one level of items (ADR 0007); the resolver and writers already handle nesting | +| U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Done in [validator#61](https://github.com/ForLegalAI/legaldown-validator/pull/61); the renderer nests lists from it (v0.2) | | U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | | U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, the CommonMark fence and indentation helpers, the answers-file reader, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | | U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | @@ -24,7 +24,7 @@ here. - The full pipeline: parse and validate, build, resolve, write - All four numbering schemes; list enumeration; paragraph numbering; item and paragraph anchors - with designations ("2.1(b)"), one list level until U1 + with designations ("2.1(b)") - Every directive, with every failure marker - Template view: conditions, `{{choose:}}`, drafting notes, alternatives sharing a number - Title block, attachment placeholders, signature blocks @@ -38,7 +38,9 @@ here. - Rendering with an answers set, through the core package's assembly (U4b): done - `--answers answers.yaml` on the CLI: done -- Released together with the validator release that ships the CommonMark block model and assembly +- Nested lists, with designations such as "2.1(b)(i)" (U1) +- Released together with the validator release that ships the CommonMark block model, nested + lists, and assembly ### v0.3 — DOCX diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index a9eccc3..d97a9a6 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -16,8 +16,8 @@ build, so no text in a document — written out, as an entity, or percent-encoded — can pass for a sentinel. -Until the validator keeps nested lists (ForLegalAI/legaldown-validator#14), -lists render as the validator holds them: one level of items. +Lists nest as the validator reads them: its listed items, each with a depth +and the kind of its list (ForLegalAI/legaldown-validator#61). """ from __future__ import annotations @@ -65,6 +65,7 @@ dedent, indent_width, lex, + listed_items, placed_markers, ) @@ -402,23 +403,29 @@ def block(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]], *, t raise InternalError(f"unexpected block kind '{block.kind}' in the validator's model") def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> List: - # Items are fragments after any text, prefix, and suffix, which a - # list block does not have. - items: list[ListItem] = [] - for index, item in enumerate(item for item in block.items if item): + """A list, nested as the validator reads it. Its items are the + validator's listed items, in order, each with its depth and the kind + of the list it is in; they are block_fragments' fragments too, as a + list block has no text, prefix, or suffix, so a marker's fragment + index is the item's index here.""" + entries: list[tuple[ListItem, int, str]] = [] + for index, (item, level, kind) in enumerate(listed_items(block)): identifier = condition = "" if index in placed: offset, identifier, condition = placed[index] item = _cut_marker(item, offset) - first, _, rest = item.partition("\n") - if first.lstrip().startswith(">") or FENCE_OPEN_RE.match(first): - # The item opens with a quote or code, whose lines the - # validator keeps as written (a drafting note included). - blocks = self.fragment(item) - else: - blocks = (Paragraph(self.text(first)),) + (self.fragment(rest) if rest else ()) - items.append(ListItem(blocks=blocks, anchor_id=identifier, condition=condition)) - return List(ordered=block.kind == "ordered_list", items=tuple(items)) + entries.append((ListItem(blocks=self._item_blocks(item), anchor_id=identifier, condition=condition), + level, kind)) + nested, _ = _nest(entries, 0, 0, block.kind) + return nested + + def _item_blocks(self, item: str) -> tuple[Block, ...]: + first, _, rest = item.partition("\n") + if first.lstrip().startswith(">") or FENCE_OPEN_RE.match(first): + # The item opens with a quote or code, whose lines the validator + # keeps as written (a drafting note included). + return self.fragment(item) + return (Paragraph(self.text(first)),) + (self.fragment(rest) if rest.strip() else ()) def quote(self, block: ModelBlock) -> Block: """A block quote, its content read by the validator's parser. A @@ -468,6 +475,26 @@ def tree(self) -> RenderTree: ) +def _nest(entries: list[tuple[ListItem, int, str]], start: int, level: int, kind: str) -> tuple[List, int]: + """The list of *kind* at depth *level* that starts at ``entries[start]``, + and where it ends. Deeper items form lists nested in the item before + them; at a depth below the top, an item of the other kind starts another + list (CommonMark).""" + items: list[ListItem] = [] + index = start + while index < len(entries): + item, depth, item_kind = entries[index] + if depth < level or (depth == level and items and level and item_kind != kind): + break + if depth == level: + items.append(item) + index += 1 + else: + nested, index = _nest(entries, index, depth, item_kind) + items[-1] = replace(items[-1], blocks=items[-1].blocks + (nested,)) + return List(ordered=kind == "ordered_list", items=tuple(items)), index + + def _strip_markers(block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> tuple[ModelBlock, str, str]: """*block* without its placed marker, and the marker's identifier and condition. Fragments are numbered as block_fragments numbers them.""" diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index b05f38f..02af662 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -20,6 +20,7 @@ from legaldown.definitions import block_fragments from legaldown.directives import Lexed, lex from legaldown.markdown import FENCE_OPEN_RE, closes_fence, dedent, indent_width +from legaldown.models import listed_items from legaldown.validator.core import is_template from legaldown.validator.templates import block_quotes from legaldown.validator.units import FoundMarker, find_markers @@ -33,6 +34,7 @@ "dedent", "indent_width", "lex", + "listed_items", "placed_markers", "read_answers", ] diff --git a/tests/golden/features.continental.txt b/tests/golden/features.continental.txt index 0e797a0..4f1ef56 100644 --- a/tests/golden/features.continental.txt +++ b/tests/golden/features.continental.txt @@ -29,9 +29,9 @@ This Services Agreement (this Agreement) is made between Acme and Beta Industrie 2.1.1 keep records of all work performed 2.1.2 notify the Client of any delay, including: -2.1.3 the cause of the delay -2.1.4 its expected duration -2.1.5 comply with the policies in the price list. + a) the cause of the delay + b) its expected duration +2.1.3 comply with the policies in the price list. (2) The steps of the escalation procedure are: @@ -40,7 +40,7 @@ This Services Agreement (this Agreement) is made between Acme and Beta Industrie 2.2 Changes -(1) A change under item 2.1.3 or step 2.1(2) is made in writing under Section 2.1. +(1) A change under item 2.1.2(a) or step 2.1(2) is made in writing under Section 2.1. 3. Fees diff --git a/tests/golden/features.default.html b/tests/golden/features.default.html index 04f2d63..e97dfaa 100644 --- a/tests/golden/features.default.html +++ b/tests/golden/features.default.html @@ -116,10 +116,12 @@

2.1 The Provider shall perform the Services on each business day, and shall:

  1. (a)

    keep records of all work performed

    2. -
  2. (b)

    notify the Client of any delay, including:

    3. -
  3. (c)

    the cause of the delay

    4. -
  4. (d)

    its expected duration

    5. -
  5. (e)

    comply with the policies in the price list.

    6. +
  6. (b)

    notify the Client of any delay, including:

    +
      +
    1. (i)

      the cause of the delay

      2. +
    2. (ii)

      its expected duration

      3. +
    7. +
  7. (c)

    comply with the policies in the price list.

The steps of the escalation procedure are:

    @@ -129,7 +131,7 @@

    2.1

    2.2 Changes

    -

    A change under item 2.1(c) or step 2.1(2) is made in writing under Section 2.1.

    +

    A change under item 2.1(b)(i) or step 2.1(2) is made in writing under Section 2.1.

diff --git a/tests/golden/features.default.txt b/tests/golden/features.default.txt index d2e36ef..b302f63 100644 --- a/tests/golden/features.default.txt +++ b/tests/golden/features.default.txt @@ -29,9 +29,9 @@ The Provider shall perform the Services on each business day, and shall: (a) keep records of all work performed (b) notify the Client of any delay, including: -(c) the cause of the delay -(d) its expected duration -(e) comply with the policies in the price list. + (i) the cause of the delay + (ii) its expected duration +(c) comply with the policies in the price list. The steps of the escalation procedure are: @@ -40,7 +40,7 @@ The steps of the escalation procedure are: 2.2 Changes -A change under item 2.1(c) or step 2.1(2) is made in writing under Section 2.1. +A change under item 2.1(b)(i) or step 2.1(2) is made in writing under Section 2.1. 3. Fees diff --git a/tests/golden/features.outline.txt b/tests/golden/features.outline.txt index 711a1d7..7f5255d 100644 --- a/tests/golden/features.outline.txt +++ b/tests/golden/features.outline.txt @@ -29,9 +29,9 @@ The Provider shall perform the Services on each business day, and shall: (a) keep records of all work performed (b) notify the Client of any delay, including: -(c) the cause of the delay -(d) its expected duration -(e) comply with the policies in the price list. + (i) the cause of the delay + (ii) its expected duration +(c) comply with the policies in the price list. The steps of the escalation procedure are: @@ -40,7 +40,7 @@ The steps of the escalation procedure are: B. Changes -A change under item II.A(c) or step II.A(2) is made in writing under Section II.A. +A change under item II.A(b)(i) or step II.A(2) is made in writing under Section II.A. III. Fees diff --git a/tests/test_render.py b/tests/test_render.py index 6cfd1d1..c579f28 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -70,14 +70,6 @@ def test_ordered_lists_are_renumbered() -> None: assert "1. first\n2. second" in output -def test_nested_lists_render_flat_until_the_validator_keeps_them() -> None: - # One parser: lists are the validator's, which keeps one level of items - # (ForLegalAI/legaldown-validator#14). - output = text("# A\n\n- one {#one}\n - nested {#nested}\n- two\n\nSee {{ref: nested}}.\n") - assert "(a) one\n(b) nested\n(c) two" in output - assert "See 1(b)." in output - - def test_ref_to_unenumerated_item_falls_back_with_warning() -> None: result = render(FRONT + NUMBERED, format="text", overrides={"enumeration.enabled": False}) assert "See 1.1, 1.1, and 1.1." in result.output @@ -765,11 +757,25 @@ def test_the_first_definition_in_document_order_keeps_the_anchor() -> None: assert 'id="def:fee"' not in sections -def test_a_later_paragraph_in_a_list_item_is_the_validators_paragraph() -> None: - # The validator ends the list at the blank line and reads "second" as a - # top-level paragraph, whose marker it places; so does the renderer. - output = text("# A\n\n- item one\n\n second {#sec}\n\nSee {{ref: sec}}.\n") - assert "(a) item one\n\nsecond\n\nSee 1." in output +def test_a_later_paragraph_in_a_list_item_is_the_item_s() -> None: + # Indented after a blank line, "second" is the item's (CommonMark), so + # its marker is misplaced (§5.7) and a reference to it is broken, as the + # validator reports. + result = render(FRONT + "# A\n\n- item one\n\n second {#sec}\n\nSee {{ref: sec}}.\n", format="text") + assert "(a) item one\n second" in result.output + assert "[BROKEN REF: sec]" in result.output + assert {"anchor-misplaced", "ref-broken"} <= {d.rule for d in result.diagnostics} + + +def test_nested_lists_keep_their_structure_and_designations() -> None: + body = ("# A\n\n- one\n 1. sub a {#suba}\n 2. sub b\n - deep {#deep}\n - bullet after ordered\n" + "- two\n\nSee {{ref: suba}} and {{ref: deep}}.\n") + output = text(body) + assert ("(a) one\n 1. sub a\n 2. sub b\n (A) deep\n (i) bullet after ordered\n(b) two" + in output) + assert "See 1(a)(1) and 1(a)(2)(A)." in output + html = render(FRONT + body, standalone=False).output + assert html.count(" None: @@ -835,7 +841,7 @@ def test_markers_and_references_agree_with_the_validators_diagnostics() -> None: cases = [ ("1. item\n ```\n code\n ```\n after {#x}\n\nSee {{ref: x}}.\n", True), ("1. # Heading {#x}\n\nSee {{ref: x}}.\n", False), - ("- item one\n\n second {#x}\n\nSee {{ref: x}}.\n", False), + ("- item one\n\n second {#x}\n\nSee {{ref: x}}.\n", True), ] for body, broken in cases: source = FRONT + "# One\n\n" + body From f0e64ccdee828bc5724408231411bf8dad4ba932 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 05:07:55 +0000 Subject: [PATCH 07/19] Fix code review findings on nested lists - Lists are grouped by the validator's list_runs, not a rule of the renderer's own, and the tree is built without recursion - A document nested so deeply that walking it exhausts the stack is a DocumentError, not an uncaught RecursionError - The text writer lines nested lists up with their item's text, as it already did the item's later paragraphs - Two gaps in the validator's model are filed and listed in CONFORMANCE.md: content after a nested list leaves its item (legaldown-validator#64), and a nested marker change stays one list (#65) - The roadmap's list of non-public validator imports names listed_items and list_runs; a stale test comment is rewritten Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 5 ++ docs/architecture.md | 2 + docs/roadmap.md | 4 +- src/legaldown_render/api.py | 18 ++++--- src/legaldown_render/build.py | 61 ++++++++++++++---------- src/legaldown_render/validator_bridge.py | 2 + src/legaldown_render/writers/text.py | 12 +++-- tests/golden/features.continental.txt | 4 +- tests/test_render.py | 21 ++++++-- 9 files changed, 85 insertions(+), 44 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index e11c843..c86fadc 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -57,6 +57,11 @@ It is checked against the specification's own examples and fixtures corpus (`tes - **Line breaks** within a paragraph are joined, and a backslash hard break shows as a literal backslash. Link reference definitions (`[label]: url`) are not resolved ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)). + - **Content after a nested list** in an item (a paragraph, a quote, more nested items) ends + the list; what follows starts a new top-level list, renumbered + ([#64](https://github.com/ForLegalAI/legaldown-validator/issues/64)). A nested list whose + bullet character or ordered delimiter changes is kept as one list + ([#65](https://github.com/ForLegalAI/legaldown-validator/issues/65)). - **Indented code inside a quote or a list item** is read as paragraph text ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)). At the top level it is code. diff --git a/docs/architecture.md b/docs/architecture.md index 6cf9f66..081f163 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -107,6 +107,8 @@ something, the output follows the model, and the gap is listed in `CONFORMANCE.m the validator: - a paragraph's line breaks are joined - indented code inside a quote or a list item is paragraph text +- content after a nested list leaves its item, and a nested list that changes its marker stays + one list Guessing at lost structure was tried, and it traded each gap for new bugs. diff --git a/docs/roadmap.md b/docs/roadmap.md index 753bfcd..5efb262 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -12,8 +12,8 @@ here. |---|---|---|---| | U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Done in [validator#61](https://github.com/ForLegalAI/legaldown-validator/pull/61); the renderer nests lists from it (v0.2) | | U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | -| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, the CommonMark fence and indentation helpers, the answers-file reader, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | -| U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | +| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, the CommonMark fence and indentation helpers, the answers-file reader, `listed_items` and `list_runs` for nested lists, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | +| U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)), content after a nested list ([#64](https://github.com/ForLegalAI/legaldown-validator/issues/64)), nested marker changes ([#65](https://github.com/ForLegalAI/legaldown-validator/issues/65)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | | U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Done in [validator#55](https://github.com/ForLegalAI/legaldown-validator/pull/55); the renderer uses it (v0.2) | | U5 | specification | A **rendering fixtures corpus**: source + style settings → expected plain-text output | Not started. The plain-text writer is designed to be its oracle | diff --git a/src/legaldown_render/api.py b/src/legaldown_render/api.py index bcd053d..52b9fdf 100644 --- a/src/legaldown_render/api.py +++ b/src/legaldown_render/api.py @@ -109,13 +109,17 @@ def render(source: str, options: RenderOptions | None = None, /, **settings: Any if options.strict and any(d.level == "error" for d in diagnostics): raise RenderRefused(diagnostics) - tree, extra = _resolve(document, result, style) - diagnostics += [d for d in extra if d not in diagnostics] - if options.strict and any(d.level == "error" for d in diagnostics): - raise RenderRefused(diagnostics) - - assert_resolved(tree) - output = _writer(options, style).write(tree) + try: + tree, extra = _resolve(document, result, style) + diagnostics += [d for d in extra if d not in diagnostics] + if options.strict and any(d.level == "error" for d in diagnostics): + raise RenderRefused(diagnostics) + assert_resolved(tree) + output = _writer(options, style).write(tree) + except RecursionError as error: + # Lists and quotes nested thousands deep: the tree is walked + # recursively, as deep as the document nests. + raise DocumentError("The document nests lists or quotes too deeply to render.") from error return RenderResult(output, diagnostics, options.format, tree, style) diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index d97a9a6..131f7f0 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -65,6 +65,7 @@ dedent, indent_width, lex, + list_runs, listed_items, placed_markers, ) @@ -408,16 +409,15 @@ def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> Li of the list it is in; they are block_fragments' fragments too, as a list block has no text, prefix, or suffix, so a marker's fragment index is the item's index here.""" - entries: list[tuple[ListItem, int, str]] = [] - for index, (item, level, kind) in enumerate(listed_items(block)): + listed = listed_items(block) + items: list[ListItem] = [] + for index, (item, _level, _kind) in enumerate(listed): identifier = condition = "" if index in placed: offset, identifier, condition = placed[index] item = _cut_marker(item, offset) - entries.append((ListItem(blocks=self._item_blocks(item), anchor_id=identifier, condition=condition), - level, kind)) - nested, _ = _nest(entries, 0, 0, block.kind) - return nested + items.append(ListItem(blocks=self._item_blocks(item), anchor_id=identifier, condition=condition)) + return _nest(items, [(level, kind) for _item, level, kind in listed]) def _item_blocks(self, item: str) -> tuple[Block, ...]: first, _, rest = item.partition("\n") @@ -475,24 +475,37 @@ def tree(self) -> RenderTree: ) -def _nest(entries: list[tuple[ListItem, int, str]], start: int, level: int, kind: str) -> tuple[List, int]: - """The list of *kind* at depth *level* that starts at ``entries[start]``, - and where it ends. Deeper items form lists nested in the item before - them; at a depth below the top, an item of the other kind starts another - list (CommonMark).""" - items: list[ListItem] = [] - index = start - while index < len(entries): - item, depth, item_kind = entries[index] - if depth < level or (depth == level and items and level and item_kind != kind): - break - if depth == level: - items.append(item) - index += 1 - else: - nested, index = _nest(entries, index, depth, item_kind) - items[-1] = replace(items[-1], blocks=items[-1].blocks + (nested,)) - return List(ordered=kind == "ordered_list", items=tuple(items)), index +def _nest(items: list[ListItem], nesting: list[tuple[int, str]]) -> List: + """*items*, with their depths and kinds (*nesting*), as the tree of lists + the validator's ``list_runs`` groups them into: each list nested in the + item before its first one, one level up. Built without recursion, so a + deep list costs no stack here.""" + runs = list_runs(nesting) + members: dict[int, list[int]] = {} # list -> its items, by index + parent: dict[int, int] = {} # nested list -> the item it is nested in + last: list[int] = [] # the last item at each depth + for index, ((level, _kind), run) in enumerate(zip(nesting, runs, strict=True)): + if run not in members: + members[run] = [] + if level: + parent[run] = last[level - 1] + members[run].append(index) + del last[level:] + last.append(index) + nested: dict[int, list[List]] = {} # item -> the lists nested in it, in order + built: dict[int, List] = {} + # A nested list starts after the item it is nested in, so its number is + # higher: building from the last list up, every nested list is ready + # before the item that holds it. + for run in sorted(members, reverse=True): + built[run] = List( + ordered=nesting[members[run][0]][1] == "ordered_list", + items=tuple(replace(items[i], blocks=items[i].blocks + tuple(reversed(nested.get(i, [])))) + for i in members[run]), + ) + if run in parent: + nested.setdefault(parent[run], []).append(built[run]) + return built[0] def _strip_markers(block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> tuple[ModelBlock, str, str]: diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index 02af662..33556b4 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -21,6 +21,7 @@ from legaldown.directives import Lexed, lex from legaldown.markdown import FENCE_OPEN_RE, closes_fence, dedent, indent_width from legaldown.models import listed_items +from legaldown.serializer import list_runs from legaldown.validator.core import is_template from legaldown.validator.templates import block_quotes from legaldown.validator.units import FoundMarker, find_markers @@ -34,6 +35,7 @@ "dedent", "indent_width", "lex", + "list_runs", "listed_items", "placed_markers", "read_answers", diff --git a/src/legaldown_render/writers/text.py b/src/legaldown_render/writers/text.py index 9a4e665..40167d4 100644 --- a/src/legaldown_render/writers/text.py +++ b/src/legaldown_render/writers/text.py @@ -124,7 +124,7 @@ def block(self, block: Block) -> str: text = f"{label} {text}" return f"[{condition}] {text}" if condition else text case List(): - return "\n".join(self.list(block, 0)) + return "\n".join(self.list(block, "")) case Quote(blocks=blocks): return _prefix("\n\n".join(self.block(child) for child in blocks), "> ") case DraftingNote(blocks=blocks, label=label): @@ -138,13 +138,15 @@ def block(self, block: Block) -> str: return "---" return "" - def list(self, block: List, depth: int) -> list[str]: + def list(self, block: List, indent: str) -> list[str]: + """The lines of *block*, each item's label at *indent*. Everything + else in an item, a nested list included, lines up with its text.""" lines: list[str] = [] - indent = _INDENT * depth for item in block.items: marker = item.label if item.label else "-" if item.condition_label: marker = f"{marker} [{item.condition_label}]" + content = indent + " " * (len(marker) + 1) first = True for child in item.blocks: if isinstance(child, List): @@ -153,14 +155,14 @@ def list(self, block: List, depth: int) -> list[str]: # still comes first, on a line of its own. lines.append(indent + marker) first = False - lines += self.list(child, depth + 1) + lines += self.list(child, content) continue text = self.block(child) if first: lines.append(_prefix(f"{marker} {text}", indent, hanging=" " * (len(marker) + 1))) first = False else: - lines.append(_prefix(text, indent + " " * (len(marker) + 1))) + lines.append(_prefix(text, content)) if first: lines.append(indent + marker) return lines diff --git a/tests/golden/features.continental.txt b/tests/golden/features.continental.txt index 4f1ef56..5c995c3 100644 --- a/tests/golden/features.continental.txt +++ b/tests/golden/features.continental.txt @@ -29,8 +29,8 @@ This Services Agreement (this Agreement) is made between Acme and Beta Industrie 2.1.1 keep records of all work performed 2.1.2 notify the Client of any delay, including: - a) the cause of the delay - b) its expected duration + a) the cause of the delay + b) its expected duration 2.1.3 comply with the policies in the price list. (2) The steps of the escalation procedure are: diff --git a/tests/test_render.py b/tests/test_render.py index c579f28..4d6d141 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -522,8 +522,10 @@ def test_source_holding_sentinel_characters_renders_them() -> None: def test_item_opening_with_a_nested_list_keeps_its_label_first() -> None: - # The text writer, on a tree with nesting (as the validator will give - # once it keeps nested lists): the item's label comes first. + # The writer's own case, on a hand-built tree: the validator never gives + # an item that opens with a nested list (an empty item is dropped and + # its items move up, models.listed_items), but a tree built in code can. + # The item's label comes first; nested lists line up with its text. from legaldown_render.tree import List, ListItem, Paragraph, RenderTree, Section, Text from legaldown_render.writers import TextWriter @@ -532,7 +534,7 @@ def test_item_opening_with_a_nested_list_keeps_its_label_first() -> None: outer = List(True, (ListItem((Paragraph((Text("first"),)),), label="1."), ListItem((middle,), label="2.")), enumerated=True) tree = RenderTree((), (), "en", (), (Section(1, (Text("A"),), "a", "", (outer,), label="1."),)) - assert "1. first\n2.\n (i)\n (A) deep" in TextWriter().write(tree) + assert "1. first\n2.\n (i)\n (A) deep" in TextWriter().write(tree) def test_an_unresolved_node_is_an_internal_error() -> None: @@ -757,6 +759,17 @@ def test_the_first_definition_in_document_order_keeps_the_anchor() -> None: assert 'id="def:fee"' not in sections +def test_nested_lists_line_up_with_their_item_s_text() -> None: + output = text("# A\n\n1. one\n\n more\n\n - child\n") + assert "1. one\n more\n (i) child" in output + + +def test_a_list_nested_thousands_deep_is_a_document_error_not_a_crash() -> None: + body = "# A\n\n" + "".join(" " * depth + f"- l{depth}\n" for depth in range(1500)) + with pytest.raises(DocumentError, match="too deeply"): + render(FRONT + body, format="html") + + def test_a_later_paragraph_in_a_list_item_is_the_item_s() -> None: # Indented after a blank line, "second" is the item's (CommonMark), so # its marker is misplaced (§5.7) and a reference to it is broken, as the @@ -771,7 +784,7 @@ def test_nested_lists_keep_their_structure_and_designations() -> None: body = ("# A\n\n- one\n 1. sub a {#suba}\n 2. sub b\n - deep {#deep}\n - bullet after ordered\n" "- two\n\nSee {{ref: suba}} and {{ref: deep}}.\n") output = text(body) - assert ("(a) one\n 1. sub a\n 2. sub b\n (A) deep\n (i) bullet after ordered\n(b) two" + assert ("(a) one\n 1. sub a\n 2. sub b\n (A) deep\n (i) bullet after ordered\n(b) two" in output) assert "See 1(a)(1) and 1(a)(2)(A)." in output html = render(FRONT + body, standalone=False).output From eb98b54d6c2722a13a6563e01052c61ca32cbf93 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 05:30:28 +0000 Subject: [PATCH 08/19] Fix second review findings on nested lists - A list of only empty items renders nothing, instead of raising a KeyError - Sibling lists nested in the same item and numbered in the same format share their counter, so no two of their items get the same designation - Nesting is limited to 100 levels of lists and quotes together, checked up front: a list by its deepest level, a quote by the quote markers that open its lines, before the validator's quote reading, whose cost grows with the depth. A deeper document is a DocumentError in well under a second, where it used to take a minute or more; the blanket RecursionError catch is gone, so a renderer bug is never blamed on the document - _nest copies only the items that hold nested lists and keeps no extra state; list() walks the listed items once - CONFORMANCE.md lists the nesting limit Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 2 + src/legaldown_render/api.py | 18 +++---- src/legaldown_render/build.py | 67 +++++++++++++++++------- src/legaldown_render/resolve/resolver.py | 11 ++-- tests/test_render.py | 42 +++++++++++++-- 5 files changed, 102 insertions(+), 38 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index c86fadc..15ca159 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -47,6 +47,8 @@ It is checked against the specification's own examples and fixtures corpus (`tes files, or `translations` is refused when rendered with answers, as §17.6 requires below Full. A template with Errors is refused too, since §15.7.2 defines no output for one. - **Diagnostics have no line numbers**, because the validator's diagnostics do not carry them yet. +- **Nesting limit.** Lists and quotes nested more than 100 levels deep, together, are refused + (`DocumentError`) before rendering starts, so that no document can exhaust the stack or the CPU. - **Malformed directives** (a §11.2 grammar violation) render their type's marker without a value, for example `[INVALID AMOUNT]`. The specification defines no marker for this case. - **One parser.** The renderer builds from the validator's document model only diff --git a/src/legaldown_render/api.py b/src/legaldown_render/api.py index 52b9fdf..bcd053d 100644 --- a/src/legaldown_render/api.py +++ b/src/legaldown_render/api.py @@ -109,17 +109,13 @@ def render(source: str, options: RenderOptions | None = None, /, **settings: Any if options.strict and any(d.level == "error" for d in diagnostics): raise RenderRefused(diagnostics) - try: - tree, extra = _resolve(document, result, style) - diagnostics += [d for d in extra if d not in diagnostics] - if options.strict and any(d.level == "error" for d in diagnostics): - raise RenderRefused(diagnostics) - assert_resolved(tree) - output = _writer(options, style).write(tree) - except RecursionError as error: - # Lists and quotes nested thousands deep: the tree is walked - # recursively, as deep as the document nests. - raise DocumentError("The document nests lists or quotes too deeply to render.") from error + tree, extra = _resolve(document, result, style) + diagnostics += [d for d in extra if d not in diagnostics] + if options.strict and any(d.level == "error" for d in diagnostics): + raise RenderRefused(diagnostics) + + assert_resolved(tree) + output = _writer(options, style).write(tree) return RenderResult(output, diagnostics, options.format, tree, style) diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index 131f7f0..9bb7d60 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -32,7 +32,7 @@ from markdown_it import MarkdownIt from markdown_it.tree import SyntaxTreeNode -from .errors import InternalError +from .errors import DocumentError, InternalError from .tree import ( Block, Code, @@ -80,6 +80,15 @@ _DROPPED = Text("") _DROPPED_BREAK = Text("") _BREAK_TAG_RE = re.compile(r"", re.IGNORECASE) +#: How deep lists and quotes may nest, together. Far beyond any real +#: document, and far enough below Python's recursion limit that resolving +#: and writing the tree never reach it. Deeper documents are refused up +#: front, before the work that grows with the depth. +MAX_NESTING = 100 +# The quote markers that open a line: an upper bound on how deep the quotes +# in a text nest, cheap to take before the validator reads them (its reading +# costs more the deeper they go). Only a safety limit, never a reading. +_QUOTE_MARKERS_RE = re.compile(r"^(?:[ \t]{0,3}>)+", re.MULTILINE) def normalize_source(source: str) -> str: @@ -166,6 +175,7 @@ def __init__(self, document: Document, result: ValidationResult) -> None: self.raw_html = 0 self._dropped_html = False self.markers = placed_markers(document) + self.depth = 0 # quotes and list items the builder is inside nonce = secrets.token_hex(8) # A hidden sentinel has its own form ("h"), so the lead character is # taken only with one, never from the source before another sentinel. @@ -403,13 +413,17 @@ def block(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]], *, t return Rule() raise InternalError(f"unexpected block kind '{block.kind}' in the validator's model") - def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> List: + def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> List | None: """A list, nested as the validator reads it. Its items are the validator's listed items, in order, each with its depth and the kind of the list it is in; they are block_fragments' fragments too, as a list block has no text, prefix, or suffix, so a marker's fragment - index is the item's index here.""" + index is the item's index here. None when every item is empty.""" listed = listed_items(block) + if not listed: + return None + nesting = [(level, kind) for _item, level, kind in listed] + self._check_depth(max(level for level, _kind in nesting) + 1) items: list[ListItem] = [] for index, (item, _level, _kind) in enumerate(listed): identifier = condition = "" @@ -417,7 +431,14 @@ def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> Li offset, identifier, condition = placed[index] item = _cut_marker(item, offset) items.append(ListItem(blocks=self._item_blocks(item), anchor_id=identifier, condition=condition)) - return _nest(items, [(level, kind) for _item, level, kind in listed]) + return _nest(items, nesting) + + def _check_depth(self, more: int) -> None: + """Refuse a document that nests lists and quotes deeper than + MAX_NESTING, *more* levels below where the builder is.""" + if self.depth + more > MAX_NESTING: + raise DocumentError(f"The document nests lists and quotes more than {MAX_NESTING} levels deep, " + "which is not rendered.") def _item_blocks(self, item: str) -> tuple[Block, ...]: first, _, rest = item.partition("\n") @@ -431,6 +452,8 @@ def quote(self, block: ModelBlock) -> Block: """A block quote, its content read by the validator's parser. A drafting note is decided by the validator's own test (§15.6).""" text = block.text + self._check_depth(1 + max((match.group().count(">") for match in _QUOTE_MARKERS_RE.finditer(text)), + default=0)) quotes = block_quotes(block) drafting = bool(quotes) and quotes[0].start == 0 and quotes[0].is_drafting_note if drafting: @@ -443,11 +466,16 @@ def fragment(self, text: str) -> tuple[Block, ...]: validator's parser — as a body, so that text starting with ``---`` is not frontmatter. No marker is placed there (§5.7), and a heading is not a section (§4.1).""" - inner = parse_document("\n" + text) - blocks = self.blocks(inner.preamble, None, markers=False) - for section in inner.sections: - blocks += (Paragraph((Strong(self.text(section.title)),)),) - blocks += self.blocks(section.blocks, None, markers=False) + self._check_depth(1) + self.depth += 1 + try: + inner = parse_document("\n" + text) + blocks = self.blocks(inner.preamble, None, markers=False) + for section in inner.sections: + blocks += (Paragraph((Strong(self.text(section.title)),)),) + blocks += self.blocks(section.blocks, None, markers=False) + finally: + self.depth -= 1 return blocks # -- document ------------------------------------------------------------- @@ -492,20 +520,19 @@ def _nest(items: list[ListItem], nesting: list[tuple[int, str]]) -> List: members[run].append(index) del last[level:] last.append(index) - nested: dict[int, list[List]] = {} # item -> the lists nested in it, in order - built: dict[int, List] = {} - # A nested list starts after the item it is nested in, so its number is - # higher: building from the last list up, every nested list is ready - # before the item that holds it. - for run in sorted(members, reverse=True): - built[run] = List( + nested: dict[int, list[List]] = {} # item -> the lists nested in it, last first + # list_runs numbers lists in order of their first item, and a nested list + # starts after the item holding it: from the last list back, every + # nested list is built before that item. The last one built is the top. + for run in reversed(members): + built = List( ordered=nesting[members[run][0]][1] == "ordered_list", - items=tuple(replace(items[i], blocks=items[i].blocks + tuple(reversed(nested.get(i, [])))) - for i in members[run]), + items=tuple(replace(items[i], blocks=items[i].blocks + tuple(reversed(nested[i]))) if i in nested + else items[i] for i in members[run]), ) if run in parent: - nested.setdefault(parent[run], []).append(built[run]) - return built[0] + nested.setdefault(parent[run], []).append(built) + return built def _strip_markers(block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> tuple[ModelBlock, str, str]: diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index a0d2465..3947f87 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -376,7 +376,10 @@ def _structure(self, block: Block, *, section: Section | None, depth: int, prese return block def _structure_list(self, block: List, *, section: Section | None, depth: int, parent: str, - presence: Presence) -> List: + presence: Presence, counters: dict[LevelFormat | None, _Counter] | None = None) -> List: + """*block* numbered under *parent*. Sibling lists nested in the same + item share *counters*: a second list in the same format goes on from + the first, so no two of their items get the same designation.""" enumeration = self.style.enumeration fmt: LevelFormat | None if block.ordered and enumeration.ordered == "renumber": @@ -387,7 +390,7 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p fmt = None base = section.designation if section else "" items: list[ListItem] = [] - counter = _Counter(self._exclusive) + counter = _Counter(self._exclusive) if counters is None else counters.setdefault(fmt, _Counter(self._exclusive)) for item in block.items: index = counter.next(item.anchor_id, self._own_presence(item.condition, presence)) # An item's nested blocks are present only when the item is. @@ -402,8 +405,10 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p anchor = self._anchor(item.anchor_id) if item.anchor_id and section is not None: self._register(item.anchor_id, replace(target, anchor=anchor)) + siblings: dict[LevelFormat | None, _Counter] = {} children = tuple( - self._structure_list(child, section=section, depth=depth + 1, parent=designation, presence=inner) + self._structure_list(child, section=section, depth=depth + 1, parent=designation, presence=inner, + counters=siblings) if isinstance(child, List) else self._structure(child, section=section, depth=depth + 1, presence=inner) for child in item.blocks diff --git a/tests/test_render.py b/tests/test_render.py index 4d6d141..efd6886 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -764,10 +764,44 @@ def test_nested_lists_line_up_with_their_item_s_text() -> None: assert "1. one\n more\n (i) child" in output -def test_a_list_nested_thousands_deep_is_a_document_error_not_a_crash() -> None: - body = "# A\n\n" + "".join(" " * depth + f"- l{depth}\n" for depth in range(1500)) - with pytest.raises(DocumentError, match="too deeply"): - render(FRONT + body, format="html") +def _nested_list(levels: int) -> str: + return "# A\n\n" + "".join(" " * depth + f"- l{depth}\n" for depth in range(levels)) + + +@pytest.mark.parametrize("output_format", ["html", "text"]) +def test_lists_and_quotes_nested_to_the_limit_render(output_format: str) -> None: + from legaldown_render.build import MAX_NESTING + assert "l99" in render(FRONT + _nested_list(MAX_NESTING), format=output_format).output + quotes = FRONT + "# A\n\n" + "> " * MAX_NESTING + "deepest\n" + assert "deepest" in render(quotes, format=output_format).output + with pytest.raises(DocumentError): + render(FRONT + "# A\n\n" + "> " * (MAX_NESTING + 1) + "deepest\n", format=output_format) + + +def test_nesting_past_the_limit_is_refused_up_front() -> None: + import time + + from legaldown_render.build import MAX_NESTING + with pytest.raises(DocumentError, match=f"more than {MAX_NESTING} levels"): + render(FRONT + _nested_list(MAX_NESTING + 1)) + start = time.monotonic() + with pytest.raises(DocumentError, match="levels deep"): + render(FRONT + "# A\n\n" + "> " * 2000 + "x\n") + assert time.monotonic() - start < 10 + + +def test_a_list_of_empty_items_renders_nothing() -> None: + output = text("# A\n\n- \n - \n\nAfter.\n") + assert "1. A\n\nAfter." in output + + +def test_sibling_lists_in_one_item_never_share_a_designation() -> None: + body = "# A\n\n- one\n 1. sub a {#x}\n - sub b {#y}\n\nSee {{ref: x}} and {{ref: y}}.\n" + output = text(body, overrides={"enumeration.ordered": "enumerate"}) + assert "(a) one\n (i) sub a\n (ii) sub b" in output + assert "See 1(a)(i) and 1(a)(ii)." in output + # Lists in different formats are told apart by their format already. + assert "See 1(a)(1) and 1(a)(i)." in text(body) def test_a_later_paragraph_in_a_list_item_is_the_item_s() -> None: From 53a50e8bffda7b8b10aa028c7f905e2dc39ffa0d Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 06:03:13 +0000 Subject: [PATCH 09/19] Fix third review findings: nesting limit, list designations Nesting limit (MAX_NESTING = 100, now lists, quotes, and inline formatting together): - A list item's content is built as deep as the item, so lists and quotes chained inside each other count together - Inline formatting counts too, from the depth markdown-it records on each token, before the tree is built: deep emphasis is a DocumentError, not a RecursionError - The quote bound counts markers however far apart, on the lexer's view, so code lines do not count and spaced markers do; a deep quote is refused before the validator's quote reading runs List designations: - Sibling lists in one item share a counter when their designations read alike (counter style and reference form), not only when their whole format is the same; a list reusing a counter starts its alternatives afresh - A {{ref:}} to an item whose designation another list's item shares (each list starts again at its first number) renders with a render-ref-ambiguous Warning; alternatives share a slot and never draw it. No document in the specification corpus draws it. Tests replace the wall-clock assertion with a check that the quote reading never runs; CONFORMANCE.md states the limit and the Warning. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 8 ++- src/legaldown_render/build.py | 40 +++++++++------ src/legaldown_render/resolve/resolver.py | 40 +++++++++++++-- tests/test_render.py | 62 ++++++++++++++++++++++-- 4 files changed, 123 insertions(+), 27 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 15ca159..d6a3a18 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -47,8 +47,12 @@ It is checked against the specification's own examples and fixtures corpus (`tes files, or `translations` is refused when rendered with answers, as §17.6 requires below Full. A template with Errors is refused too, since §15.7.2 defines no output for one. - **Diagnostics have no line numbers**, because the validator's diagnostics do not carry them yet. -- **Nesting limit.** Lists and quotes nested more than 100 levels deep, together, are refused - (`DocumentError`) before rendering starts, so that no document can exhaust the stack or the CPU. +- **Nesting limit.** Lists, quotes, and inline formatting (emphasis, links) nested more than 100 + levels deep, together, are refused (`DocumentError`). The check runs before the work that grows + with the depth, so a deeply nested document is refused quickly and cannot exhaust the stack. +- **Ambiguous references.** Each list starts again at its first number, as contracts are usually + drafted, so two lists in one section can both have an item "(a)". A `{{ref:}}` to such an item + renders with a `render-ref-ambiguous` Warning; numbered paragraphs tell them apart. - **Malformed directives** (a §11.2 grammar violation) render their type's marker without a value, for example `[INVALID AMOUNT]`. The specification defines no marker for this case. - **One parser.** The renderer builds from the validator's document model only diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index 9bb7d60..f1900ae 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -80,15 +80,17 @@ _DROPPED = Text("") _DROPPED_BREAK = Text("") _BREAK_TAG_RE = re.compile(r"", re.IGNORECASE) -#: How deep lists and quotes may nest, together. Far beyond any real -#: document, and far enough below Python's recursion limit that resolving -#: and writing the tree never reach it. Deeper documents are refused up -#: front, before the work that grows with the depth. +#: How deep lists, quotes, and inline formatting (emphasis, links) may nest, +#: together. Far beyond any real document, and far enough below Python's +#: recursion limit that building, resolving, and writing the tree never +#: reach it. Deeper documents are refused up front, before the work that +#: grows with the depth. MAX_NESTING = 100 -# The quote markers that open a line: an upper bound on how deep the quotes -# in a text nest, cheap to take before the validator reads them (its reading -# costs more the deeper they go). Only a safety limit, never a reading. -_QUOTE_MARKERS_RE = re.compile(r"^(?:[ \t]{0,3}>)+", re.MULTILINE) +# The quote markers that open a line, however far apart: an upper bound on +# how deep the quotes in a text nest, cheap to take before the validator +# reads them (its reading costs more the deeper they go). Taken on the +# lexer's view, so code lines do not count. A safety limit, never a reading. +_QUOTE_MARKERS_RE = re.compile(r"^(?:[ \t]*>)+", re.MULTILINE) def normalize_source(source: str) -> str: @@ -282,6 +284,7 @@ def inlines(self, content: str) -> tuple[Inline, ...]: tokens = self.md.parseInline(content, self.env) if not tokens or not tokens[0].children: return () + self._check_depth(max(token.level for token in tokens[0].children)) root = SyntaxTreeNode(tokens[0].children, create_root=True) return self._inline_children(root) @@ -425,20 +428,26 @@ def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> Li nesting = [(level, kind) for _item, level, kind in listed] self._check_depth(max(level for level, _kind in nesting) + 1) items: list[ListItem] = [] - for index, (item, _level, _kind) in enumerate(listed): + for index, (item, level, _kind) in enumerate(listed): identifier = condition = "" if index in placed: offset, identifier, condition = placed[index] item = _cut_marker(item, offset) - items.append(ListItem(blocks=self._item_blocks(item), anchor_id=identifier, condition=condition)) + # The item's content is as deep as the item, for the nesting limit. + self.depth += level + try: + blocks = self._item_blocks(item) + finally: + self.depth -= level + items.append(ListItem(blocks=blocks, anchor_id=identifier, condition=condition)) return _nest(items, nesting) def _check_depth(self, more: int) -> None: - """Refuse a document that nests lists and quotes deeper than - MAX_NESTING, *more* levels below where the builder is.""" + """Refuse a document that nests lists, quotes, and inline formatting + deeper than MAX_NESTING, *more* levels below where the builder is.""" if self.depth + more > MAX_NESTING: - raise DocumentError(f"The document nests lists and quotes more than {MAX_NESTING} levels deep, " - "which is not rendered.") + raise DocumentError(f"The document nests lists, quotes, and inline formatting more than " + f"{MAX_NESTING} levels deep, which is not rendered.") def _item_blocks(self, item: str) -> tuple[Block, ...]: first, _, rest = item.partition("\n") @@ -452,7 +461,8 @@ def quote(self, block: ModelBlock) -> Block: """A block quote, its content read by the validator's parser. A drafting note is decided by the validator's own test (§15.6).""" text = block.text - self._check_depth(1 + max((match.group().count(">") for match in _QUOTE_MARKERS_RE.finditer(text)), + view = self.markers.lex(text).view + self._check_depth(1 + max((match.group().count(">") for match in _QUOTE_MARKERS_RE.finditer(view)), default=0)) quotes = block_quotes(block) drafting = bool(quotes) and quotes[0].start == 0 and quotes[0].is_drafting_note diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index 3947f87..ed11c8c 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -20,6 +20,7 @@ """ from __future__ import annotations +import itertools from collections.abc import Callable from dataclasses import dataclass, replace from typing import Any @@ -130,6 +131,8 @@ def __init__( self.diagnostics: list[Diagnostic] = [] self.targets: dict[str, _Target] = {} self.used_anchors: set[str] = set() + # Item designation -> the numbered slots that read that way (render-ref-ambiguous). + self.item_slots: dict[str, set[tuple[int, int]]] = {} questions = self.metadata.questions self.questions: dict[str, Any] = questions if isinstance(questions, dict) else {} self.inconsistent_placeholders: set[str] = set() @@ -376,10 +379,11 @@ def _structure(self, block: Block, *, section: Section | None, depth: int, prese return block def _structure_list(self, block: List, *, section: Section | None, depth: int, parent: str, - presence: Presence, counters: dict[LevelFormat | None, _Counter] | None = None) -> List: + presence: Presence, counters: dict[tuple[str, str] | None, _Counter] | None = None) -> List: """*block* numbered under *parent*. Sibling lists nested in the same - item share *counters*: a second list in the same format goes on from - the first, so no two of their items get the same designation.""" + item share *counters*, one per way of writing a designation (counter + style and reference form): a second list whose designations would + read like the first's goes on from it, so none repeats.""" enumeration = self.style.enumeration fmt: LevelFormat | None if block.ordered and enumeration.ordered == "renumber": @@ -390,7 +394,12 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p fmt = None base = section.designation if section else "" items: list[ListItem] = [] - counter = _Counter(self._exclusive) if counters is None else counters.setdefault(fmt, _Counter(self._exclusive)) + if counters is None: + counter = _Counter(self._exclusive) + else: + key = (fmt.counter, fmt.ref) if fmt is not None else None + counter = counters.setdefault(key, _Counter(self._exclusive)) + counter.new_list() for item in block.items: index = counter.next(item.anchor_id, self._own_presence(item.condition, presence)) # An item's nested blocks are present only when the item is. @@ -400,12 +409,14 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p designation = extend(parent, fill(fmt.ref, n=n, section=base), textual=self.textual) label = fill(fmt.label, n=n, section=base, path=designation) target = _Target(designation, None) + # Which numbered slot reads this way; alternatives share one. + self.item_slots.setdefault(designation, set()).add((counter.key, index)) else: designation, label, target = parent, None, _Target(base, None, enumerated=False) anchor = self._anchor(item.anchor_id) if item.anchor_id and section is not None: self._register(item.anchor_id, replace(target, anchor=anchor)) - siblings: dict[LevelFormat | None, _Counter] = {} + siblings: dict[tuple[str, str] | None, _Counter] = {} children = tuple( self._structure_list(child, section=section, depth=depth + 1, parent=designation, presence=inner, counters=siblings) @@ -499,6 +510,13 @@ def _ref(self, directive: Directive, target_id: str) -> Inline: f"'{{{{ref: {target_id}}}}}' targets an item or paragraph that the style does not number; " f"it renders as its section's designation, '{target.designation}' (§6.3).", ) + elif len(self.item_slots.get(target.designation, ())) > 1: + self._warn( + "render-ref-ambiguous", + f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which more than one list item " + f"in the document reads as: each list starts again at its first number. Number the " + f"paragraphs (paragraphs.numbered) or make it one list to tell them apart.", + ) text = self.style.references.format.replace("{designation}", target.designation) return CrossRef(text, target.anchor or "") @@ -737,6 +755,9 @@ def _warn(self, rule: str, message: str) -> None: } +_COUNTER_KEYS = itertools.count() + + class _Counter: """Numbers sibling units in order (§13.2, §15.8). A unit joins the previous unit's number when it is an alternative to it: the same @@ -746,10 +767,19 @@ class _Counter: def __init__(self, exclusive: Callable[[Presence, Presence], bool]) -> None: self.exclusive = exclusive + #: Tells counters apart for as long as the render runs (an id() may + #: be reused once a counter is gone). + self.key = next(_COUNTER_KEYS) self.count = 0 self.identifier = "" self.holders: list[Presence] = [] + def new_list(self) -> None: + """Go on counting in another list: its first unit is no alternative + to the last unit of the list before, which is not its sibling.""" + self.identifier = "" + self.holders = [] + def next(self, identifier: str, presence: Presence | None) -> int: """The number for the next unit. *presence* is None for a unit with no valid condition of its own, which is never an alternative.""" diff --git a/tests/test_render.py b/tests/test_render.py index efd6886..44d0d68 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -778,16 +778,68 @@ def test_lists_and_quotes_nested_to_the_limit_render(output_format: str) -> None render(FRONT + "# A\n\n" + "> " * (MAX_NESTING + 1) + "deepest\n", format=output_format) -def test_nesting_past_the_limit_is_refused_up_front() -> None: - import time +@pytest.mark.parametrize("body", [ + "> " * 2000 + "x\n", + "> " * 2000 + "x\n", # markers four columns apart count too +]) +def test_deep_quotes_are_refused_before_the_validator_reads_them(body: str) -> None: + from unittest import mock + + from legaldown_render import build + with mock.patch.object(build, "block_quotes", wraps=build.block_quotes) as reading: + with pytest.raises(DocumentError, match="levels deep"): + render(FRONT + "# A\n\n" + body) + assert reading.call_count == 0 + +def test_nesting_past_the_limit_is_refused() -> None: from legaldown_render.build import MAX_NESTING with pytest.raises(DocumentError, match=f"more than {MAX_NESTING} levels"): render(FRONT + _nested_list(MAX_NESTING + 1)) - start = time.monotonic() + + +def test_lists_and_quotes_count_together() -> None: + # A list 60 deep whose deepest item holds a quote with another list 60 + # deep nests 121 levels. + deepest = " " * 60 + inner = "".join(f"{deepest}> {' ' * depth}- m{depth}\n" for depth in range(60)) + with pytest.raises(DocumentError, match="levels deep"): + render(FRONT + _nested_list(60) + inner, format="html") + + +def test_deep_inline_formatting_is_refused_and_shallow_renders() -> None: with pytest.raises(DocumentError, match="levels deep"): - render(FRONT + "# A\n\n" + "> " * 2000 + "x\n") - assert time.monotonic() - start < 10 + render(FRONT + "# A\n\n" + "*a " * 300 + "x" + "*" * 300 + "\n") + assert "x" in text("# A\n\n" + "*a " * 30 + "x" + "*" * 30 + "\n") + + +def test_quote_markers_in_code_do_not_count_as_depth() -> None: + assert ">>>" in render(FRONT + "# A\n\n> ```\n> " + ">" * 150 + "\n> ```\n", standalone=False).output + + +def test_a_reference_to_an_item_another_list_also_numbers_is_ambiguous() -> None: + result = render(FRONT + "# A\n\n- a {#x}\n\nPara.\n\n- b {#y}\n\nSee {{ref: x}}.\n", format="text") + assert "See 1(a)." in result.output + assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} + single = render(FRONT + "# A\n\n- a {#x}\n- b\n\nSee {{ref: x}}.\n", format="text") + assert "render-ref-ambiguous" not in {d.rule for d in single.diagnostics} + + +def test_alternatives_are_not_ambiguous() -> None: + body = ("# A\n\n- first {#alt when=forum:courts}\n- second {#alt when=forum:arbitration}\n\n" + "See {{ref: alt}}.\n") + front = FRONT.replace("---\n", "---\nquestions:\n forum:\n type: choice\n choices:\n" + " courts: Courts\n arbitration: Arbitration\n", 1) + result = render(front + body, format="text") + assert "render-ref-ambiguous" not in {d.rule for d in result.diagnostics} + + +def test_sibling_lists_are_told_apart_by_how_their_designations_read() -> None: + body = "# A\n\n- one\n 1. sub a {#x}\n - sub b {#y}\n\nSee {{ref: x}} and {{ref: y}}.\n" + levels = [{"counter": "lower-alpha", "label": "({n})", "ref": "({n})"}, + {"counter": "decimal", "label": "{n})", "ref": "({n})"}] + output = text(body, overrides={"enumeration.levels": levels}) + assert "See 1(a)(1) and 1(a)(2)." in output def test_a_list_of_empty_items_renders_nothing() -> None: From 2b55c12d12dfa710226715a9fd35d37e0211e253 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 06:03:41 +0000 Subject: [PATCH 10/19] Merge the nested with statements in a test (ruff SIM117) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- tests/test_render.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/test_render.py b/tests/test_render.py index 44d0d68..f470b11 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -786,9 +786,9 @@ def test_deep_quotes_are_refused_before_the_validator_reads_them(body: str) -> N from unittest import mock from legaldown_render import build - with mock.patch.object(build, "block_quotes", wraps=build.block_quotes) as reading: - with pytest.raises(DocumentError, match="levels deep"): - render(FRONT + "# A\n\n" + body) + with (mock.patch.object(build, "block_quotes", wraps=build.block_quotes) as reading, + pytest.raises(DocumentError, match="levels deep")): + render(FRONT + "# A\n\n" + body) assert reading.call_count == 0 From 7f2263ab9c7fbe145bc49138bc40a797ed994961 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 28 Sep 2026 05:11:56 +0000 Subject: [PATCH 11/19] Build lists and quotes from the validator's block model (validator#71, #72) The validator now gives list items that hold blocks (Block.items is a list of ListItem) and reads a quote's content as blocks (parser.quote_content); the flat items with depths the renderer nested (listed_items, list_runs) are gone. The builder follows: - A list is built item by item from its blocks, nested lists included. Items are numbered as the validator numbers them, in document order across nested lists, and a placed marker belongs to the innermost item its fragment is in (list_fragments) - A quote's content is quote_content's blocks, to the validator's quote depth; a drafting note is is_drafting_note's decision. The builder no longer re-reads container text with parse_document, and the quote-marker estimate and per-level block_quotes calls are gone - Empty items keep their place (validator#46) Fixed with it upstream and dropped from CONFORMANCE.md: content after a nested list (#64), nested marker changes (#65), indented code in quotes and items (#41), empty items (#46), tables without leading pipes (#44). A heading in a quote or item is now paragraph text in the model: filed as legaldown-validator#78 and listed. The validator caps lists at 64 levels and quotes at 16, so the renderer's 100-level limit now guards inline formatting and combinations; the tests follow. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 20 +-- docs/architecture.md | 9 +- docs/roadmap.md | 4 +- src/legaldown_render/build.py | 161 +++++++++-------------- src/legaldown_render/validator_bridge.py | 16 ++- tests/test_render.py | 77 ++++------- 6 files changed, 112 insertions(+), 175 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index d6a3a18..5f13dba 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -47,9 +47,10 @@ It is checked against the specification's own examples and fixtures corpus (`tes files, or `translations` is refused when rendered with answers, as §17.6 requires below Full. A template with Errors is refused too, since §15.7.2 defines no output for one. - **Diagnostics have no line numbers**, because the validator's diagnostics do not carry them yet. -- **Nesting limit.** Lists, quotes, and inline formatting (emphasis, links) nested more than 100 - levels deep, together, are refused (`DocumentError`). The check runs before the work that grows - with the depth, so a deeply nested document is refused quickly and cannot exhaust the stack. +- **Nesting limit.** The validator reads lists 64 levels deep and quotes 16; deeper, an item's or + quote's content is its text. The renderer follows it, and refuses (`DocumentError`) a document + whose lists, quotes, and inline formatting (emphasis, links) together nest more than 100 levels, + before building it, so no document can exhaust the stack. - **Ambiguous references.** Each list starts again at its first number, as contracts are usually drafted, so two lists in one section can both have an item "(a)". A `{{ref:}}` to such an item renders with a `render-ref-ambiguous` Warning; numbered paragraphs tell them apart. @@ -63,16 +64,9 @@ It is checked against the specification's own examples and fixtures corpus (`tes - **Line breaks** within a paragraph are joined, and a backslash hard break shows as a literal backslash. Link reference definitions (`[label]: url`) are not resolved ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)). - - **Content after a nested list** in an item (a paragraph, a quote, more nested items) ends - the list; what follows starts a new top-level list, renumbered - ([#64](https://github.com/ForLegalAI/legaldown-validator/issues/64)). A nested list whose - bullet character or ordered delimiter changes is kept as one list - ([#65](https://github.com/ForLegalAI/legaldown-validator/issues/65)). - - **Indented code inside a quote or a list item** is read as paragraph text - ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)). At the top level it is - code. - - **Empty list items** are left out ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)), - and a table row needs a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)). + - **A heading inside a quote or a list item** shows as paragraph text: `# Heading` keeps its + `#`, and a setext underline joins the paragraph + ([#78](https://github.com/ForLegalAI/legaldown-validator/issues/78)). ## Beyond this level (Full, §17.4) diff --git a/docs/architecture.md b/docs/architecture.md index 081f163..27df1d3 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -77,10 +77,11 @@ never decides a structural or LegalDown question itself: | Question | Answered by | |---|---| | Sections, headings, identifiers | `Document.sections`, `ValidationResult.sections` | -| Blocks: paragraphs, lists and items, quotes, tables, code, rules | `Document` blocks; quote content and code inside items are read by the validator's parser too | +| Blocks: paragraphs, lists and items, quotes, tables, code, rules | `Document` blocks. List items hold blocks (nested lists, code, quotes, tables), and a quote's content is the validator's `quote_content()` | | Where a marker (`{#id when=…}`) is placed, and what it means | The validator's `find_markers()`, with its own `placed(template)` | | Whether the document is a template | The validator's `is_template()`, over those markers | -| Whether a quote is a drafting note | The validator's `block_quotes()` | +| Whether a quote is a drafting note | The validator's `is_drafting_note()` | +| Which list item a marker belongs to | The validator's `list_fragments()`: the items each fragment is in, numbered in document order | | A lifted definition, `{{ref:}}` or `{{term:}}` block's source | The validator's `render_block()` | The imports beyond the validator's public API are all in `validator_bridge.py`, which is the @@ -106,9 +107,7 @@ The builder never works around the validator's model. Where the model loses or m something, the output follows the model, and the gap is listed in `CONFORMANCE.md` and filed on the validator: - a paragraph's line breaks are joined -- indented code inside a quote or a list item is paragraph text -- content after a nested list leaves its item, and a nested list that changes its marker stays - one list +- a heading inside a quote or a list item is paragraph text Guessing at lost structure was tried, and it traded each gap for new bugs. diff --git a/docs/roadmap.md b/docs/roadmap.md index 5efb262..84c5a51 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -12,8 +12,8 @@ here. |---|---|---|---| | U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Done in [validator#61](https://github.com/ForLegalAI/legaldown-validator/pull/61); the renderer nests lists from it (v0.2) | | U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | -| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, the CommonMark fence and indentation helpers, the answers-file reader, `listed_items` and `list_runs` for nested lists, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | -| U1b | validator | **Parser gaps** the renderer inherits. List markers (#21), tables (#22), comments and HTML blocks (#23), the signature-block cutoff (#24), empty comments (#28), and indented code (#9) are fixed in [validator#49](https://github.com/ForLegalAI/legaldown-validator/pull/49). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), indented code in quotes and items ([#41](https://github.com/ForLegalAI/legaldown-validator/issues/41)), tables without a leading pipe ([#44](https://github.com/ForLegalAI/legaldown-validator/issues/44)), empty list items ([#46](https://github.com/ForLegalAI/legaldown-validator/issues/46)), content after a nested list ([#64](https://github.com/ForLegalAI/legaldown-validator/issues/64)), nested marker changes ([#65](https://github.com/ForLegalAI/legaldown-validator/issues/65)) | Partly done. The renderer adopts #49 when it is released. Listed in `CONFORMANCE.md` | +| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, `is_drafting_note`, the CommonMark fence and indentation helpers, the answers-file reader, `list_items`, `list_fragments`, `quote_content` and `MAX_QUOTE_DEPTH`, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | +| U1b | validator | **Parser gaps** the renderer inherits. Fixed in validator `main`: list markers (#21), tables (#22, #44), comments and HTML blocks (#23, #59), the signature-block cutoff (#24), empty comments (#28), indented code (#9, #41), empty list items (#46), content after a nested list (#64), nested marker changes (#65). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), headings in quotes and items ([#78](https://github.com/ForLegalAI/legaldown-validator/issues/78)) | Mostly done. The renderer adopts them when the validator is released. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | | U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Done in [validator#55](https://github.com/ForLegalAI/legaldown-validator/pull/55); the renderer uses it (v0.2) | | U5 | specification | A **rendering fixtures corpus**: source + style settings → expected plain-text output | Not started. The plain-text writer is designed to be its oracle | diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index f1900ae..f11f02b 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -16,18 +16,21 @@ build, so no text in a document — written out, as an entity, or percent-encoded — can pass for a sentinel. -Lists nest as the validator reads them: its listed items, each with a depth -and the kind of its list (ForLegalAI/legaldown-validator#61). +Lists and quotes are built as the validator's model holds them: list items +hold blocks, nested lists included, and a quote's content is the blocks the +validator reads in it (ForLegalAI/legaldown-validator#71, #72). """ from __future__ import annotations +import itertools import re import secrets +from collections.abc import Iterator from dataclasses import dataclass, replace from urllib.parse import unquote from legaldown import Block as ModelBlock -from legaldown import Directive, Document, ValidationResult, find_definition_anchors, parse_document, render_block +from legaldown import Directive, Document, ValidationResult, find_definition_anchors, render_block from legaldown.markers import HTML_COMMENT_RE from markdown_it import MarkdownIt from markdown_it.tree import SyntaxTreeNode @@ -59,15 +62,17 @@ ) from .validator_bridge import ( FENCE_OPEN_RE, + MAX_QUOTE_DEPTH, block_fragments, - block_quotes, closes_fence, dedent, indent_width, + is_drafting_note, lex, - list_runs, - listed_items, + list_fragments, + list_items, placed_markers, + quote_content, ) _OPEN, _CLOSE = "\ue000", "\ue001" @@ -86,11 +91,6 @@ #: reach it. Deeper documents are refused up front, before the work that #: grows with the depth. MAX_NESTING = 100 -# The quote markers that open a line, however far apart: an upper bound on -# how deep the quotes in a text nest, cheap to take before the validator -# reads them (its reading costs more the deeper they go). Taken on the -# lexer's view, so code lines do not count. A safety limit, never a reading. -_QUOTE_MARKERS_RE = re.compile(r"^(?:[ \t]*>)+", re.MULTILINE) def normalize_source(source: str) -> str: @@ -416,31 +416,43 @@ def block(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]], *, t return Rule() raise InternalError(f"unexpected block kind '{block.kind}' in the validator's model") - def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> List | None: - """A list, nested as the validator reads it. Its items are the - validator's listed items, in order, each with its depth and the kind - of the list it is in; they are block_fragments' fragments too, as a - list block has no text, prefix, or suffix, so a marker's fragment - index is the item's index here. None when every item is empty.""" - listed = listed_items(block) - if not listed: - return None - nesting = [(level, kind) for _item, level, kind in listed] - self._check_depth(max(level for level, _kind in nesting) + 1) - items: list[ListItem] = [] - for index, (item, level, _kind) in enumerate(listed): - identifier = condition = "" - if index in placed: - offset, identifier, condition = placed[index] - item = _cut_marker(item, offset) - # The item's content is as deep as the item, for the nesting limit. - self.depth += level - try: - blocks = self._item_blocks(item) - finally: - self.depth -= level - items.append(ListItem(blocks=blocks, anchor_id=identifier, condition=condition)) - return _nest(items, nesting) + def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> List: + """A list, nested as the validator's model holds it: items that hold + blocks. The validator numbers a list's items in document order, + nested ones included, and each fragment says which items it is in + (``list_fragments``), so a marker placed at an item's first + paragraph belongs to the innermost of them.""" + fragments = list_fragments(block) + markers = {fragments[index][2][-1]: marker for index, marker in placed.items()} + return self._list(block, markers, itertools.count()) + + def _list(self, block: ModelBlock, markers: dict[int, tuple[int, str, str]], + numbers: Iterator[int] | None) -> List: + """*block*'s items, numbered from *numbers* as the validator numbers + them (None in a quote, whose items it does not number).""" + self._check_depth(1) + self.depth += 1 + try: + items: list[ListItem] = [] + for item in list_items(block): + number = next(numbers) if numbers is not None else None + children = list(item.blocks) + identifier = condition = "" + if number in markers: + offset, identifier, condition = markers[number] + children[0] = replace(children[0], text=_cut_marker(children[0].text, offset)) + blocks: list[Block] = [] + for child in children: + if child.kind in ("ordered_list", "unordered_list"): + blocks.append(self._list(child, markers, numbers)) + elif child.kind == "html": + self._html_block(child) + elif (built := self.block(child, {}, top_level=False)) is not None: + blocks.append(built) + items.append(ListItem(blocks=tuple(blocks), anchor_id=identifier, condition=condition)) + finally: + self.depth -= 1 + return List(ordered=block.kind == "ordered_list", items=tuple(items)) def _check_depth(self, more: int) -> None: """Refuse a document that nests lists, quotes, and inline formatting @@ -449,44 +461,25 @@ def _check_depth(self, more: int) -> None: raise DocumentError(f"The document nests lists, quotes, and inline formatting more than " f"{MAX_NESTING} levels deep, which is not rendered.") - def _item_blocks(self, item: str) -> tuple[Block, ...]: - first, _, rest = item.partition("\n") - if first.lstrip().startswith(">") or FENCE_OPEN_RE.match(first): - # The item opens with a quote or code, whose lines the validator - # keeps as written (a drafting note included). - return self.fragment(item) - return (Paragraph(self.text(first)),) + (self.fragment(rest) if rest.strip() else ()) - def quote(self, block: ModelBlock) -> Block: - """A block quote, its content read by the validator's parser. A - drafting note is decided by the validator's own test (§15.6).""" - text = block.text - view = self.markers.lex(text).view - self._check_depth(1 + max((match.group().count(">") for match in _QUOTE_MARKERS_RE.finditer(view)), - default=0)) - quotes = block_quotes(block) - drafting = bool(quotes) and quotes[0].start == 0 and quotes[0].is_drafting_note - if drafting: - text = text.partition("\n")[2] - blocks = self.fragment(text) - return DraftingNote(blocks) if drafting else Quote(blocks) - - def fragment(self, text: str) -> tuple[Block, ...]: - """Blocks for *text* inside a quote or a list item, read by the - validator's parser — as a body, so that text starting with ``---`` - is not frontmatter. No marker is placed there (§5.7), and a heading - is not a section (§4.1).""" + """A block quote, its content the blocks the validator reads in it + (``quote_content``). Past the validator's quote depth, it reads the + quote's text as one, and so does the builder. A drafting note is + decided by the validator's own test (§15.6); its first line, the + ``[!DRAFTING]`` marker, is not shown.""" + drafting = is_drafting_note(block) + text = block.text.partition("\n")[2] if drafting else block.text self._check_depth(1) self.depth += 1 try: - inner = parse_document("\n" + text) - blocks = self.blocks(inner.preamble, None, markers=False) - for section in inner.sections: - blocks += (Paragraph((Strong(self.text(section.title)),)),) - blocks += self.blocks(section.blocks, None, markers=False) + if self.depth <= MAX_QUOTE_DEPTH: + children, _spans = quote_content(text, self.depth) + blocks = self.blocks(list(children), None, markers=False) + else: + blocks = (Paragraph(self.text(text)),) if text.strip() else () finally: self.depth -= 1 - return blocks + return DraftingNote(blocks) if drafting else Quote(blocks) # -- document ------------------------------------------------------------- @@ -513,38 +506,6 @@ def tree(self) -> RenderTree: ) -def _nest(items: list[ListItem], nesting: list[tuple[int, str]]) -> List: - """*items*, with their depths and kinds (*nesting*), as the tree of lists - the validator's ``list_runs`` groups them into: each list nested in the - item before its first one, one level up. Built without recursion, so a - deep list costs no stack here.""" - runs = list_runs(nesting) - members: dict[int, list[int]] = {} # list -> its items, by index - parent: dict[int, int] = {} # nested list -> the item it is nested in - last: list[int] = [] # the last item at each depth - for index, ((level, _kind), run) in enumerate(zip(nesting, runs, strict=True)): - if run not in members: - members[run] = [] - if level: - parent[run] = last[level - 1] - members[run].append(index) - del last[level:] - last.append(index) - nested: dict[int, list[List]] = {} # item -> the lists nested in it, last first - # list_runs numbers lists in order of their first item, and a nested list - # starts after the item holding it: from the last list back, every - # nested list is built before that item. The last one built is the top. - for run in reversed(members): - built = List( - ordered=nesting[members[run][0]][1] == "ordered_list", - items=tuple(replace(items[i], blocks=items[i].blocks + tuple(reversed(nested[i]))) if i in nested - else items[i] for i in members[run]), - ) - if run in parent: - nested.setdefault(parent[run], []).append(built) - return built - - def _strip_markers(block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> tuple[ModelBlock, str, str]: """*block* without its placed marker, and the marker's identifier and condition. Fragments are numbered as block_fragments numbers them.""" diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index 33556b4..868fa28 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -17,27 +17,29 @@ from legaldown import Document from legaldown.cli import _read_answers as read_answers -from legaldown.definitions import block_fragments +from legaldown.definitions import block_fragments, list_fragments from legaldown.directives import Lexed, lex from legaldown.markdown import FENCE_OPEN_RE, closes_fence, dedent, indent_width -from legaldown.models import listed_items -from legaldown.serializer import list_runs +from legaldown.models import list_items +from legaldown.parser import MAX_QUOTE_DEPTH, quote_content from legaldown.validator.core import is_template -from legaldown.validator.templates import block_quotes +from legaldown.validator.templates import is_drafting_note from legaldown.validator.units import FoundMarker, find_markers __all__ = [ "FENCE_OPEN_RE", + "MAX_QUOTE_DEPTH", "PlacedMarkers", "block_fragments", - "block_quotes", "closes_fence", "dedent", "indent_width", + "is_drafting_note", "lex", - "list_runs", - "listed_items", + "list_fragments", + "list_items", "placed_markers", + "quote_content", "read_answers", ] diff --git a/tests/test_render.py b/tests/test_render.py index f470b11..3218281 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -522,10 +522,9 @@ def test_source_holding_sentinel_characters_renders_them() -> None: def test_item_opening_with_a_nested_list_keeps_its_label_first() -> None: - # The writer's own case, on a hand-built tree: the validator never gives - # an item that opens with a nested list (an empty item is dropped and - # its items move up, models.listed_items), but a tree built in code can. - # The item's label comes first; nested lists line up with its text. + # An item that opens with a nested list (`-` then an indented list, in + # the validator's model): the item's label comes first, on a line of its + # own, and nested lists line up with its text. from legaldown_render.tree import List, ListItem, Paragraph, RenderTree, Section, Text from legaldown_render.writers import TextWriter @@ -768,45 +767,6 @@ def _nested_list(levels: int) -> str: return "# A\n\n" + "".join(" " * depth + f"- l{depth}\n" for depth in range(levels)) -@pytest.mark.parametrize("output_format", ["html", "text"]) -def test_lists_and_quotes_nested_to_the_limit_render(output_format: str) -> None: - from legaldown_render.build import MAX_NESTING - assert "l99" in render(FRONT + _nested_list(MAX_NESTING), format=output_format).output - quotes = FRONT + "# A\n\n" + "> " * MAX_NESTING + "deepest\n" - assert "deepest" in render(quotes, format=output_format).output - with pytest.raises(DocumentError): - render(FRONT + "# A\n\n" + "> " * (MAX_NESTING + 1) + "deepest\n", format=output_format) - - -@pytest.mark.parametrize("body", [ - "> " * 2000 + "x\n", - "> " * 2000 + "x\n", # markers four columns apart count too -]) -def test_deep_quotes_are_refused_before_the_validator_reads_them(body: str) -> None: - from unittest import mock - - from legaldown_render import build - with (mock.patch.object(build, "block_quotes", wraps=build.block_quotes) as reading, - pytest.raises(DocumentError, match="levels deep")): - render(FRONT + "# A\n\n" + body) - assert reading.call_count == 0 - - -def test_nesting_past_the_limit_is_refused() -> None: - from legaldown_render.build import MAX_NESTING - with pytest.raises(DocumentError, match=f"more than {MAX_NESTING} levels"): - render(FRONT + _nested_list(MAX_NESTING + 1)) - - -def test_lists_and_quotes_count_together() -> None: - # A list 60 deep whose deepest item holds a quote with another list 60 - # deep nests 121 levels. - deepest = " " * 60 - inner = "".join(f"{deepest}> {' ' * depth}- m{depth}\n" for depth in range(60)) - with pytest.raises(DocumentError, match="levels deep"): - render(FRONT + _nested_list(60) + inner, format="html") - - def test_deep_inline_formatting_is_refused_and_shallow_renders() -> None: with pytest.raises(DocumentError, match="levels deep"): render(FRONT + "# A\n\n" + "*a " * 300 + "x" + "*" * 300 + "\n") @@ -842,9 +802,31 @@ def test_sibling_lists_are_told_apart_by_how_their_designations_read() -> None: assert "See 1(a)(1) and 1(a)(2)." in output -def test_a_list_of_empty_items_renders_nothing() -> None: - output = text("# A\n\n- \n - \n\nAfter.\n") - assert "1. A\n\nAfter." in output +def test_empty_list_items_keep_their_place() -> None: + # The validator keeps an empty item (CommonMark), so later items keep + # their numbers. + output = text("# A\n\n- \n - \n- b {#b}\n\nSee {{ref: b}}.\n") + assert "(a)\n (i)\n(b) b" in output + assert "See 1(b)." in output + + +@pytest.mark.parametrize("output_format", ["html", "text"]) +def test_deep_lists_and_quotes_render_as_the_validator_caps_them(output_format: str) -> None: + # The validator reads lists 64 levels deep and quotes 16; deeper, an + # item's or quote's content is text. So the renderer never nests past + # its own limit on lists and quotes alone. + lists = "# A\n\n" + "".join(" " * depth + f"- l{depth}\n" for depth in range(150)) + assert "l149" in render(FRONT + lists, format=output_format).output + for marker in ("> ", "> "): + assert "deepest" in render(FRONT + "# A\n\n" + marker * 2000 + "deepest\n", format=output_format).output + + +def test_lists_and_inline_formatting_count_together() -> None: + from legaldown_render.build import MAX_NESTING + lists = "".join(" " * depth + f"- l{depth}\n" for depth in range(60)) + deep = " " * 60 + "- " + "*a " * (MAX_NESTING - 50) + "x" + "*" * (MAX_NESTING - 50) + "\n" + with pytest.raises(DocumentError, match="levels deep"): + render(FRONT + "# A\n\n" + lists + deep) def test_sibling_lists_in_one_item_never_share_a_designation() -> None: @@ -963,8 +945,7 @@ def test_template_decision_is_the_validators() -> None: def test_quote_content_is_read_as_a_body() -> None: # Text starting with "---" inside a quote is not frontmatter. output = text("# A\n\n> ---\n> title: x\n> ---\n> Visible?\n") - for line in ("> title: x", "> Visible?"): - assert line in output + assert "title: x" in output and "Visible?" in output def test_a_signature_block_heading_in_a_quote_renders() -> None: From c8d51a45e16e089835d2f08baf7f2907d09bef29 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 28 Sep 2026 14:21:49 +0000 Subject: [PATCH 12/19] Follow validator#80 and #81: raw-html is the validator's; container headings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - The validator now reports raw HTML other than comments (raw-html, §8.7), once per block or text, as cmark-gfm reads it. The renderer's own count is gone, so the Warning is reported once; nothing is emitted, as before - A heading in a list item or a quote is a heading block in the model (fixes legaldown-validator#78). It is not a section (§4.1): it renders as a bold line, with no number and no anchor. A marker after it is text, as the validator reads it (§5.7) - CONFORMANCE.md, the architecture doc, and the roadmap follow Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 6 ++---- docs/architecture.md | 11 +++++------ docs/roadmap.md | 2 +- src/legaldown_render/api.py | 10 ++-------- src/legaldown_render/build.py | 36 +++++++++++------------------------ tests/test_render.py | 15 ++++++++++++--- 6 files changed, 33 insertions(+), 47 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 5f13dba..d782408 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -20,6 +20,7 @@ It is checked against the specification's own examples and fixtures corpus (`tes | `{{include:}}` below Full (§17.5) | ✅ Renders `[NOT PROCESSED: include path]` with a Warning (`render-not-processed`) | | Party and side display rules (§3.6) | ✅ | | At least one output format (§13.6) | ✅ HTML (recommended) and plain text (optional) | +| A heading in a list item or a quote | ✅ Shown as a bold line: it is not a section (§4.1), so it has no number or anchor | | Comment stripping (§8.6) | ✅ Inline comments, and comment blocks across any number of lines. As in CommonMark, a comment block left unclosed runs to the end of the document | | Templates with answers (§15.8) | ✅ Assembled first with `legaldown-validator`'s Assembly capability (§15.7), then rendered: `answers=` / `--answers answers.yaml`. A template that needs other files is refused (see below) | | Template view without answers (§15.8) | ✅ Conditions marked with style-defined labels; every `{{choose:}}` phrase shown as `[a / b]`; drafting notes styled; alternatives share a number | @@ -35,7 +36,7 @@ It is checked against the specification's own examples and fixtures corpus (`tes | Signature blocks (§2.2) | ✅ One per party, with `legal_name` and each representative | | Labels following the document language (§13.7) | ✅ Built in for `en`, `cs`, `de`, `fr`, `pl`, and `sk`; any label can be set by a style | | `ref-not-enumerated` Warning (§6.3, §16.3) | ✅ | -| `raw-html` Warning; raw HTML never emitted (§8.7) | ✅ An HTML block is dropped whole; an inline tag is dropped and its text kept. Nothing is emitted, and comments draw no Warning | +| `raw-html` Warning; raw HTML never emitted (§8.7) | ✅ Nothing is emitted: an HTML block is dropped whole, and an inline tag is dropped with its text kept. The Warning is the validator's, once per block or text that holds raw HTML other than comments | | Automatic term recognition (§7.4, MAY) | ❌ Not supported | | Extended tables through raw HTML (§9.2, MAY) | ❌ Not supported; the HTML table is dropped with the Warning | | Question prompts next to blanks (§15.8, MAY) | ✅ `placeholders.show_prompt` | @@ -64,9 +65,6 @@ It is checked against the specification's own examples and fixtures corpus (`tes - **Line breaks** within a paragraph are joined, and a backslash hard break shows as a literal backslash. Link reference definitions (`[label]: url`) are not resolved ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)). - - **A heading inside a quote or a list item** shows as paragraph text: `# Heading` keeps its - `#`, and a setext underline joins the paragraph - ([#78](https://github.com/ForLegalAI/legaldown-validator/issues/78)). ## Beyond this level (Full, §17.4) diff --git a/docs/architecture.md b/docs/architecture.md index 27df1d3..4dcf6ea 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -99,15 +99,14 @@ still closes. A defined term in a link title or alt text shows its term but is n definition's anchor. An HTML block in the model (`kind="html"`), a comment block included, renders nothing (§8.6). -One that holds anything besides comments counts once for the `raw-html` Warning (§8.7). Within a -block's text, inline comments and tags are dropped; where one stood between two spaces, one space -stays. Each text that had a tag (a paragraph, a title, a table cell) counts once for the Warning. +Within a block's text, inline comments and tags are dropped; where one stood between two spaces, +one space stays. The validator reports raw HTML other than comments (`raw-html`, §8.7). A heading +in a list item or a quote (`kind="heading"`) is not a section (§4.1) and shows as a bold line. The builder never works around the validator's model. Where the model loses or misreads something, the output follows the model, and the gap is listed in `CONFORMANCE.md` and filed on the validator: - a paragraph's line breaks are joined -- a heading inside a quote or a list item is paragraph text Guessing at lost structure was tried, and it traded each gap for new bugs. @@ -222,8 +221,8 @@ validator to export them publicly is roadmap item U3. - **Internal inconsistencies** raise `InternalError`, and the CLI exits with 70. Diagnostics reuse `legaldown.Diagnostic`. The renderer adds rules only it can evaluate: -`ref-not-enumerated` and `raw-html`, both specification rule ids, plus renderer-specific ids -prefixed `render-`: `render-not-processed` and `render-locale-fallback`. Validator diagnostics +`ref-not-enumerated`, a specification rule id, plus renderer-specific ids prefixed `render-`: +`render-not-processed`, `render-locale-fallback`, and `render-ref-ambiguous`. Validator diagnostics have no line numbers yet, and neither do the renderer's. ## Security diff --git a/docs/roadmap.md b/docs/roadmap.md index 84c5a51..40bd90c 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -13,7 +13,7 @@ here. | U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Done in [validator#61](https://github.com/ForLegalAI/legaldown-validator/pull/61); the renderer nests lists from it (v0.2) | | U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | | U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, `is_drafting_note`, the CommonMark fence and indentation helpers, the answers-file reader, `list_items`, `list_fragments`, `quote_content` and `MAX_QUOTE_DEPTH`, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | -| U1b | validator | **Parser gaps** the renderer inherits. Fixed in validator `main`: list markers (#21), tables (#22, #44), comments and HTML blocks (#23, #59), the signature-block cutoff (#24), empty comments (#28), indented code (#9, #41), empty list items (#46), content after a nested list (#64), nested marker changes (#65). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)), headings in quotes and items ([#78](https://github.com/ForLegalAI/legaldown-validator/issues/78)) | Mostly done. The renderer adopts them when the validator is released. Listed in `CONFORMANCE.md` | +| U1b | validator | **Parser gaps** the renderer inherits. Fixed in validator `main`: list markers (#21), tables (#22, #44), comments and HTML blocks (#23, #59), the signature-block cutoff (#24), empty comments (#28), indented code (#9, #41), empty list items (#46), content after a nested list (#64), nested marker changes (#65), headings in quotes and items (#78), raw-html (#40). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)) | Mostly done. The renderer adopts them when the validator is released. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | | U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Done in [validator#55](https://github.com/ForLegalAI/legaldown-validator/pull/55); the renderer uses it (v0.2) | | U5 | specification | A **rendering fixtures corpus**: source + style settings → expected plain-text output | Not started. The plain-text writer is designed to be its oracle | diff --git a/src/legaldown_render/api.py b/src/legaldown_render/api.py index bcd053d..e58ac79 100644 --- a/src/legaldown_render/api.py +++ b/src/legaldown_render/api.py @@ -157,14 +157,8 @@ def _parse(source: str) -> Document: def _resolve(document: Document, result: ValidationResult, style: Style) -> tuple[RenderTree, list[Diagnostic]]: diagnostics: list[Diagnostic] = [] - tree, raw_html = build_tree(document, result) - if raw_html: - diagnostics.append(Diagnostic( - rule="raw-html", - level="warning", - message=f"Raw HTML in {raw_html} place(s) in the body is not rendered; HTML does not render " - f"portably (§8.7).", - )) + # Raw HTML is never emitted; the validator reports it (raw-html, §8.7). + tree = build_tree(document, result) locale = parse_locale(style.locale) if style.locale else None if style.locale and locale is None: raise StyleError("setting", [f"locale: '{style.locale}' is not a known locale (e.g. en-US, cs-CZ)"]) diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index f11f02b..dcc857b 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -173,9 +173,6 @@ def __init__(self, document: Document, result: ValidationResult) -> None: self.md = MarkdownIt("commonmark", {"html": True}) self.env: dict = {} self.payloads: list[_Payload] = [] - # Texts in which raw HTML was dropped (§8.7), counted once each. - self.raw_html = 0 - self._dropped_html = False self.markers = placed_markers(document) self.depth = 0 # quotes and list items the builder is inside nonce = secrets.token_hex(8) @@ -273,11 +270,7 @@ def _restore_url(self, url: str) -> str: def text(self, source: str) -> tuple[Inline, ...]: """Inline nodes for *source*, one block's text as the validator holds it: directives protected, then parsed as inline Markdown.""" - self._dropped_html = False - inlines = _trim(self.inlines(self._protect(source))) - if self._dropped_html: - self.raw_html += 1 - return inlines + return _trim(self.inlines(self._protect(source))) def inlines(self, content: str) -> tuple[Inline, ...]: """Inline nodes for the inline Markdown *content* (sentinels included).""" @@ -317,7 +310,6 @@ def _inline(self, node: SyntaxTreeNode) -> list[Inline]: case "html_inline": if node.content.startswith(" b
c d Line
two Wordboldword\n", format="text") assert "a b c d Line two Wordboldword" in result.output - warnings = [d for d in result.diagnostics if d.rule == "raw-html"] - assert len(warnings) == 1 and "in 1 place" in warnings[0].message + # Reported once, by the validator (§8.7). + assert [d.rule for d in result.diagnostics].count("raw-html") == 1 def test_table_rows_are_as_wide_as_the_header() -> None: From 1ef8c6094c814370a817aab5fae441f3e9d73d28 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 28 Sep 2026 14:35:20 +0000 Subject: [PATCH 13/19] Fix review findings on block-model lists, quotes, and ambiguous refs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - render-ref-ambiguous warns only when another item reading the same can appear with the referenced one: each item's slot keeps its full presence, and alternatives or units under exclusive conditions (§15.4) never draw it. Lists in quotes and drafting notes, whose items the validator does not treat as units, are left out. Its advice no longer suggests numbered paragraphs, which do not change item designations - A drafting note's content is the validator's reading of the whole quote (quote_content), with the [!DRAFTING] marker taken off the first paragraph or heading, instead of a re-read of the text after the marker line: an indented line after the marker stays paragraph text, so its directives resolve - A list with no placed marker skips the list_fragments walk, and lists in quotes are not numbered for markers, as the docstring said - The counters for the ambiguity check are the counter objects themselves; the global key counter is gone - The nesting-limit comment, CONFORMANCE.md, and the resolver docstring say what the code does Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 5 +- src/legaldown_render/build.py | 36 +++++++++--- src/legaldown_render/resolve/resolver.py | 70 +++++++++++++++--------- src/legaldown_render/validator_bridge.py | 2 + tests/test_render.py | 36 ++++++++++++ 5 files changed, 113 insertions(+), 36 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index d782408..14bb52f 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -51,10 +51,11 @@ It is checked against the specification's own examples and fixtures corpus (`tes - **Nesting limit.** The validator reads lists 64 levels deep and quotes 16; deeper, an item's or quote's content is its text. The renderer follows it, and refuses (`DocumentError`) a document whose lists, quotes, and inline formatting (emphasis, links) together nest more than 100 levels, - before building it, so no document can exhaust the stack. + while building it and before resolving or writing it, so no document can exhaust the stack. - **Ambiguous references.** Each list starts again at its first number, as contracts are usually drafted, so two lists in one section can both have an item "(a)". A `{{ref:}}` to such an item - renders with a `render-ref-ambiguous` Warning; numbered paragraphs tell them apart. + renders with a `render-ref-ambiguous` Warning, unless the two can never appear together + (alternatives, exclusive conditions). Lists in quotes and drafting notes are not counted. - **Malformed directives** (a §11.2 grammar violation) render their type's marker without a value, for example `[INVALID AMOUNT]`. The specification defines no marker for this case. - **One parser.** The renderer builds from the validator's document model only diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index dcc857b..54aecbd 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -61,6 +61,7 @@ Text, ) from .validator_bridge import ( + DRAFTING_MARKER, FENCE_OPEN_RE, MAX_QUOTE_DEPTH, block_fragments, @@ -88,8 +89,9 @@ #: How deep lists, quotes, and inline formatting (emphasis, links) may nest, #: together. Far beyond any real document, and far enough below Python's #: recursion limit that building, resolving, and writing the tree never -#: reach it. Deeper documents are refused up front, before the work that -#: grows with the depth. +#: reach it. The builder refuses a deeper document as it reaches the limit, +#: before resolving or writing it. (The validator caps lists at 64 levels +#: and quotes at 16 itself.) MAX_NESTING = 100 @@ -411,6 +413,8 @@ def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> Li nested ones included, and each fragment says which items it is in (``list_fragments``), so a marker placed at an item's first paragraph belongs to the innermost of them.""" + if not placed: + return self._list(block, {}, None) fragments = list_fragments(block) markers = {fragments[index][2][-1]: marker for index, marker in placed.items()} return self._list(block, markers, itertools.count()) @@ -418,7 +422,8 @@ def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> Li def _list(self, block: ModelBlock, markers: dict[int, tuple[int, str, str]], numbers: Iterator[int] | None) -> List: """*block*'s items, numbered from *numbers* as the validator numbers - them (None in a quote, whose items it does not number).""" + them, to find the items *markers* belong to. None when no marker is + placed in the list, as in a quote, where the validator places none.""" self._check_depth(1) self.depth += 1 try: @@ -454,17 +459,20 @@ def quote(self, block: ModelBlock) -> Block: """A block quote, its content the blocks the validator reads in it (``quote_content``). Past the validator's quote depth, it reads the quote's text as one, and so does the builder. A drafting note is - decided by the validator's own test (§15.6); its first line, the - ``[!DRAFTING]`` marker, is not shown.""" + decided by the validator's own test (§15.6); its ``[!DRAFTING]`` + marker, which starts its first paragraph or heading in the + validator's reading, is not shown.""" drafting = is_drafting_note(block) - text = block.text.partition("\n")[2] if drafting else block.text self._check_depth(1) self.depth += 1 try: if self.depth <= MAX_QUOTE_DEPTH: - children, _spans = quote_content(text, self.depth) - blocks = self.blocks(list(children), None, markers=False) + children = list(quote_content(block.text, self.depth)[0]) + if drafting: + children = _without_drafting_marker(children) + blocks = self.blocks(children, None, markers=False) else: + text = block.text.partition("\n")[2] if drafting else block.text blocks = (Paragraph(self.text(text)),) if text.strip() else () finally: self.depth -= 1 @@ -495,6 +503,18 @@ def tree(self) -> RenderTree: ) +def _without_drafting_marker(children: list[ModelBlock]) -> list[ModelBlock]: + """A drafting note's blocks, as the validator reads them, without the + ``[!DRAFTING]`` marker that starts the first of them (its first line is + the marker, §15.6); a block that held only the marker goes.""" + first = children[0] if children else None + if first is None or first.kind not in ("paragraph", "heading") \ + or not first.text.upper().startswith(DRAFTING_MARKER): + return children + rest = first.text[len(DRAFTING_MARKER):].lstrip() + return ([replace(first, text=rest)] if rest else []) + children[1:] + + def _strip_markers(block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> tuple[ModelBlock, str, str]: """*block* without its placed marker, and the marker's identifier and condition. Fragments are numbered as block_fragments numbers them.""" diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index ed11c8c..1d8bfaa 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -15,12 +15,11 @@ The validator has already reported every document-level Error for the failures shown here, so the resolver only adds the diagnostics that need a -style template or a renderer: ``ref-not-enumerated``, ``raw-html``, and -constructs beyond the Rendering level (§17.5). +style template or a renderer: ``ref-not-enumerated``, ``render-ref-ambiguous``, +``render-locale-fallback``, and constructs beyond the Rendering level (§17.5). """ from __future__ import annotations -import itertools from collections.abc import Callable from dataclasses import dataclass, replace from typing import Any @@ -131,8 +130,11 @@ def __init__( self.diagnostics: list[Diagnostic] = [] self.targets: dict[str, _Target] = {} self.used_anchors: set[str] = set() - # Item designation -> the numbered slots that read that way (render-ref-ambiguous). - self.item_slots: dict[str, set[tuple[int, int]]] = {} + # For render-ref-ambiguous: each item designation's numbered slots + # (counter and number: alternatives share one) with the presence of + # the item holding it, and the slot of each anchored item. + self.item_slots: dict[str, list[tuple[_Counter, int, Presence]]] = {} + self.slot_of: dict[str, tuple[_Counter, int, Presence]] = {} questions = self.metadata.questions self.questions: dict[str, Any] = questions if isinstance(questions, dict) else {} self.inconsistent_placeholders: set[str] = set() @@ -364,22 +366,28 @@ def _number_paragraph(self, block: Paragraph, section: Section, number: int) -> self._register(block.anchor_id, replace(target, anchor=anchor)) return replace(block, label=label, anchor=anchor) - def _structure(self, block: Block, *, section: Section | None, depth: int, presence: Presence) -> Block: + def _structure(self, block: Block, *, section: Section | None, depth: int, presence: Presence, + quoted: bool = False) -> Block: + """*block* numbered and labelled. *quoted*: inside a quote, whose + items the validator does not treat as units (they hold no marker).""" match block: case List(): return self._structure_list(block, section=section, depth=depth, - parent=section.designation if section else "", presence=presence) + parent=section.designation if section else "", presence=presence, + quoted=quoted) case Quote(blocks=blocks): - return Quote(tuple(self._structure(child, section=section, depth=depth, presence=presence) - for child in blocks)) + return Quote(tuple(self._structure(child, section=section, depth=depth, presence=presence, + quoted=True) for child in blocks)) case DraftingNote(blocks=blocks): - return DraftingNote(tuple(self._structure(child, section=section, depth=depth, presence=presence) - for child in blocks), label=self.labels.drafting_note) + return DraftingNote(tuple(self._structure(child, section=section, depth=depth, presence=presence, + quoted=True) for child in blocks), + label=self.labels.drafting_note) case _: return block def _structure_list(self, block: List, *, section: Section | None, depth: int, parent: str, - presence: Presence, counters: dict[tuple[str, str] | None, _Counter] | None = None) -> List: + presence: Presence, counters: dict[tuple[str, str] | None, _Counter] | None = None, + quoted: bool = False) -> List: """*block* numbered under *parent*. Sibling lists nested in the same item share *counters*, one per way of writing a designation (counter style and reference form): a second list whose designations would @@ -409,8 +417,11 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p designation = extend(parent, fill(fmt.ref, n=n, section=base), textual=self.textual) label = fill(fmt.label, n=n, section=base, path=designation) target = _Target(designation, None) - # Which numbered slot reads this way; alternatives share one. - self.item_slots.setdefault(designation, set()).add((counter.key, index)) + if not quoted: + slot = (counter, index, inner) + self.item_slots.setdefault(designation, []).append(slot) + if item.anchor_id: + self.slot_of.setdefault(item.anchor_id, slot) else: designation, label, target = parent, None, _Target(base, None, enumerated=False) anchor = self._anchor(item.anchor_id) @@ -419,9 +430,9 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p siblings: dict[tuple[str, str] | None, _Counter] = {} children = tuple( self._structure_list(child, section=section, depth=depth + 1, parent=designation, presence=inner, - counters=siblings) + counters=siblings, quoted=quoted) if isinstance(child, List) - else self._structure(child, section=section, depth=depth + 1, presence=inner) + else self._structure(child, section=section, depth=depth + 1, presence=inner, quoted=quoted) for child in item.blocks ) items.append(replace(item, blocks=children, label=label, anchor=anchor)) @@ -510,16 +521,29 @@ def _ref(self, directive: Directive, target_id: str) -> Inline: f"'{{{{ref: {target_id}}}}}' targets an item or paragraph that the style does not number; " f"it renders as its section's designation, '{target.designation}' (§6.3).", ) - elif len(self.item_slots.get(target.designation, ())) > 1: + elif self._ambiguous(target_id, target.designation): self._warn( "render-ref-ambiguous", - f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which more than one list item " - f"in the document reads as: each list starts again at its first number. Number the " - f"paragraphs (paragraphs.numbered) or make it one list to tell them apart.", + f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which another list item that can " + f"appear with it also reads as: each list starts again at its first number. Make them one " + f"list, or refer to the item in words.", ) text = self.style.references.format.replace("{designation}", target.designation) return CrossRef(text, target.anchor or "") + def _ambiguous(self, target_id: str, designation: str) -> bool: + """True if another list item that can appear together with the one + *target_id* anchors reads as the same *designation*. Alternatives, + and units under exclusive conditions, never appear together (§15.4).""" + slot = self.slot_of.get(target_id) + if slot is None: + return False + counter, number, presence = slot + return any( + (other_counter, other_number) != (counter, number) and not self._exclusive(presence, other_presence) + for other_counter, other_number, other_presence in self.item_slots.get(designation, ()) + ) + def _term(self, directive: Directive, definition_id: str) -> Inline: term = self.result.definition_lookup.get(definition_id) if term is None: @@ -755,9 +779,6 @@ def _warn(self, rule: str, message: str) -> None: } -_COUNTER_KEYS = itertools.count() - - class _Counter: """Numbers sibling units in order (§13.2, §15.8). A unit joins the previous unit's number when it is an alternative to it: the same @@ -767,9 +788,6 @@ class _Counter: def __init__(self, exclusive: Callable[[Presence, Presence], bool]) -> None: self.exclusive = exclusive - #: Tells counters apart for as long as the render runs (an id() may - #: be reused once a counter is gone). - self.key = next(_COUNTER_KEYS) self.count = 0 self.identifier = "" self.holders: list[Presence] = [] diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index 868fa28..a2f3391 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -23,10 +23,12 @@ from legaldown.models import list_items from legaldown.parser import MAX_QUOTE_DEPTH, quote_content from legaldown.validator.core import is_template +from legaldown.validator.templates import _DRAFTING_MARKER as DRAFTING_MARKER from legaldown.validator.templates import is_drafting_note from legaldown.validator.units import FoundMarker, find_markers __all__ = [ + "DRAFTING_MARKER", "FENCE_OPEN_RE", "MAX_QUOTE_DEPTH", "PlacedMarkers", diff --git a/tests/test_render.py b/tests/test_render.py index 29f6494..1030751 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -794,6 +794,42 @@ def test_alternatives_are_not_ambiguous() -> None: assert "render-ref-ambiguous" not in {d.rule for d in result.diagnostics} +QUESTIONS = FRONT.replace("---\n", "---\nquestions:\n forum:\n type: choice\n choices:\n" + " courts: Courts\n arbitration: Arbitration\n", 1) + + +def test_lists_in_alternative_sections_are_not_ambiguous() -> None: + body = ("# Disputes {#d when=forum:courts}\n\n- court one {#c1}\n\nSee {{ref: c1}}.\n\n" + "# Disputes {#d when=forum:arbitration}\n\n- arbitration one\n") + result = render(QUESTIONS + body, format="text") + assert "See 1(a)." in result.output + assert "render-ref-ambiguous" not in {d.rule for d in result.diagnostics} + + +def test_lists_in_quotes_and_drafting_notes_are_not_ambiguous() -> None: + body = ("# A\n\n1. first {#a}\n2. second {#b}\n\n> [!DRAFTING]\n> Options:\n>\n> 1. keep\n> 2. drop\n\n" + "> 1. quoted\n> 2. quoted\n\nSee {{ref: b}}.\n") + result = render(FRONT + body, format="text") + assert "render-ref-ambiguous" not in {d.rule for d in result.diagnostics} + + +def test_an_ambiguous_reference_says_how_to_tell_the_items_apart() -> None: + result = render(FRONT + "# A\n\n- a {#x}\n\nPara.\n\n- b\n\nSee {{ref: x}}.\n", format="text") + message = next(d.message for d in result.diagnostics if d.rule == "render-ref-ambiguous") + assert "one list" in message and "in words" in message and "paragraphs" not in message + + +def test_a_drafting_note_reads_as_the_validator_reads_it() -> None: + # An indented line after the marker continues its paragraph (it cannot + # interrupt one), so the placeholder in it is live, not code. + output = render(FRONT + "# A\n\n> [!DRAFTING]\n> Use {{placeholder: x, type=text}} here\n", + format="text").output + assert "> [Drafting note]\n> Use [_____] here" in output + assert "{{placeholder" not in output and "[!DRAFTING]" not in output + # The marker underlined makes a heading of it; the note holds nothing else. + assert "[!DRAFTING]" not in text("# A\n\n> [!DRAFTING]\n> ---\n") + + def test_sibling_lists_are_told_apart_by_how_their_designations_read() -> None: body = "# A\n\n- one\n 1. sub a {#x}\n - sub b {#y}\n\nSee {{ref: x}} and {{ref: y}}.\n" levels = [{"counter": "lower-alpha", "label": "({n})", "ref": "({n})"}, From be0a0bdce01f6f1272db298c4a2f0a41be3b7376 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 03:45:24 +0000 Subject: [PATCH 14/19] Make render-ref-ambiguous a hint; keep drafting markers out of view MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - render-ref-ambiguous compares every alternative of the referenced unit, not only the first, so the result no longer depends on their order - Numbered paragraphs take part again: a paragraph and a list item reading the same designation are flagged - The Warning is a hint ("may"): units that can never appear together are left out, but the condition the reference itself stands under is not considered; CONFORMANCE.md says so - A drafting note whose marker line the validator reads as code (indented four columns) is the text after that line, as before, so the marker is never shown - The [!DRAFTING] marker text is defined in the renderer (§15.6) instead of imported from the validator's private name - The resolver's docstring lists the diagnostics it emits Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 6 ++-- src/legaldown_render/build.py | 18 +++++++--- src/legaldown_render/resolve/resolver.py | 45 ++++++++++++++---------- src/legaldown_render/validator_bridge.py | 2 -- tests/test_render.py | 23 ++++++++++++ 5 files changed, 67 insertions(+), 27 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 14bb52f..1271501 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -54,8 +54,10 @@ It is checked against the specification's own examples and fixtures corpus (`tes while building it and before resolving or writing it, so no document can exhaust the stack. - **Ambiguous references.** Each list starts again at its first number, as contracts are usually drafted, so two lists in one section can both have an item "(a)". A `{{ref:}}` to such an item - renders with a `render-ref-ambiguous` Warning, unless the two can never appear together - (alternatives, exclusive conditions). Lists in quotes and drafting notes are not counted. + or to a numbered paragraph reading the same renders with a `render-ref-ambiguous` Warning, a + hint: it is left out when the two can never appear together (alternatives, exclusive + conditions), but the condition the reference itself stands under is not considered. Lists in + quotes and drafting notes are not counted. - **Malformed directives** (a §11.2 grammar violation) render their type's marker without a value, for example `[INVALID AMOUNT]`. The specification defines no marker for this case. - **One parser.** The renderer builds from the validator's document model only diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index 54aecbd..f4da598 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -61,7 +61,6 @@ Text, ) from .validator_bridge import ( - DRAFTING_MARKER, FENCE_OPEN_RE, MAX_QUOTE_DEPTH, block_fragments, @@ -93,6 +92,9 @@ #: before resolving or writing it. (The validator caps lists at 64 levels #: and quotes at 16 itself.) MAX_NESTING = 100 +# A drafting note's first line (§15.6). Which quotes are drafting notes is +# the validator's decision (is_drafting_note); this text is taken off. +DRAFTING_MARKER = "[!DRAFTING]" def normalize_source(source: str) -> str: @@ -469,7 +471,12 @@ def quote(self, block: ModelBlock) -> Block: if self.depth <= MAX_QUOTE_DEPTH: children = list(quote_content(block.text, self.depth)[0]) if drafting: - children = _without_drafting_marker(children) + unmarked = _without_drafting_marker(children) + # When the marker does not start the first paragraph or + # heading (an indented marker line reads as code), the + # note is the text after its marker line. + children = unmarked if unmarked is not None else list( + quote_content(block.text.partition("\n")[2], self.depth)[0]) blocks = self.blocks(children, None, markers=False) else: text = block.text.partition("\n")[2] if drafting else block.text @@ -503,14 +510,15 @@ def tree(self) -> RenderTree: ) -def _without_drafting_marker(children: list[ModelBlock]) -> list[ModelBlock]: +def _without_drafting_marker(children: list[ModelBlock]) -> list[ModelBlock] | None: """A drafting note's blocks, as the validator reads them, without the ``[!DRAFTING]`` marker that starts the first of them (its first line is - the marker, §15.6); a block that held only the marker goes.""" + the marker, §15.6); a block that held only the marker goes. None when + the marker does not start a first paragraph or heading.""" first = children[0] if children else None if first is None or first.kind not in ("paragraph", "heading") \ or not first.text.upper().startswith(DRAFTING_MARKER): - return children + return None rest = first.text[len(DRAFTING_MARKER):].lstrip() return ([replace(first, text=rest)] if rest else []) + children[1:] diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index 1d8bfaa..30e9fbd 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -16,7 +16,7 @@ The validator has already reported every document-level Error for the failures shown here, so the resolver only adds the diagnostics that need a style template or a renderer: ``ref-not-enumerated``, ``render-ref-ambiguous``, -``render-locale-fallback``, and constructs beyond the Rendering level (§17.5). +and ``render-not-processed`` for constructs beyond the Rendering level (§17.5). """ from __future__ import annotations @@ -130,11 +130,12 @@ def __init__( self.diagnostics: list[Diagnostic] = [] self.targets: dict[str, _Target] = {} self.used_anchors: set[str] = set() - # For render-ref-ambiguous: each item designation's numbered slots - # (counter and number: alternatives share one) with the presence of - # the item holding it, and the slot of each anchored item. + # For render-ref-ambiguous: each designation of a numbered item or + # paragraph, with its numbered slots (counter and number: alternatives + # share one) and the presence of the unit holding it; and the slots + # of each anchored unit, one per alternative. self.item_slots: dict[str, list[tuple[_Counter, int, Presence]]] = {} - self.slot_of: dict[str, tuple[_Counter, int, Presence]] = {} + self.slot_of: dict[str, list[tuple[_Counter, int, Presence]]] = {} questions = self.metadata.questions self.questions: dict[str, Any] = questions if isinstance(questions, dict) else {} self.inconsistent_placeholders: set[str] = set() @@ -340,7 +341,8 @@ def _structure_section(self, section: Section, presence: Presence) -> tuple[Bloc if isinstance(block, Paragraph) and block.top_level: # Alternative paragraphs share a number, as sections do (§15.8). number = paragraphs.next(block.anchor_id, self._own_presence(block.condition, presence)) - block = self._number_paragraph(block, section, number) + block = self._number_paragraph(block, section, number, + slot=(paragraphs, number, presence | self._presence(block.condition))) out.append(self._structure(block, section=section, depth=0, presence=presence)) return tuple(out) @@ -350,7 +352,8 @@ def _own_presence(self, condition: str, enclosing: Presence) -> Presence | None: own = self._presence(condition) return enclosing | own if own else None - def _number_paragraph(self, block: Paragraph, section: Section, number: int) -> Paragraph: + def _number_paragraph(self, block: Paragraph, section: Section, number: int, + slot: tuple[_Counter, int, Presence]) -> Paragraph: numbering = self.style.paragraphs base = section.designation or "" if numbering.numbered: @@ -359,6 +362,9 @@ def _number_paragraph(self, block: Paragraph, section: Section, number: int) -> designation = extend(base, fill(fmt.ref, n=n, section=base), textual=self.textual) label = fill(fmt.label, n=n, section=base, path=designation) target = _Target(designation, None) + self.item_slots.setdefault(designation, []).append(slot) + if block.anchor_id: + self.slot_of.setdefault(block.anchor_id, []).append(slot) else: label, target = None, _Target(base, None, enumerated=False) anchor = self._anchor(block.anchor_id) @@ -421,7 +427,7 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p slot = (counter, index, inner) self.item_slots.setdefault(designation, []).append(slot) if item.anchor_id: - self.slot_of.setdefault(item.anchor_id, slot) + self.slot_of.setdefault(item.anchor_id, []).append(slot) else: designation, label, target = parent, None, _Target(base, None, enumerated=False) anchor = self._anchor(item.anchor_id) @@ -524,24 +530,27 @@ def _ref(self, directive: Directive, target_id: str) -> Inline: elif self._ambiguous(target_id, target.designation): self._warn( "render-ref-ambiguous", - f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which another list item that can " - f"appear with it also reads as: each list starts again at its first number. Make them one " + f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which may also be how another " + f"numbered item or paragraph reads: each list starts again at its first number. Make them one " f"list, or refer to the item in words.", ) text = self.style.references.format.replace("{designation}", target.designation) return CrossRef(text, target.anchor or "") def _ambiguous(self, target_id: str, designation: str) -> bool: - """True if another list item that can appear together with the one - *target_id* anchors reads as the same *designation*. Alternatives, - and units under exclusive conditions, never appear together (§15.4).""" - slot = self.slot_of.get(target_id) - if slot is None: + """True if another numbered item or paragraph may read as the same + *designation* as the one *target_id* anchors, in a document where + both appear. A hint, not a proof: units under exclusive conditions + never appear together (§15.4), but where the reference itself + stands is not considered.""" + own = self.slot_of.get(target_id, []) + if not own: return False - counter, number, presence = slot + own_numbers = {(counter, number) for counter, number, _presence in own} return any( - (other_counter, other_number) != (counter, number) and not self._exclusive(presence, other_presence) - for other_counter, other_number, other_presence in self.item_slots.get(designation, ()) + (counter, number) not in own_numbers + and any(not self._exclusive(presence, other) for _c, _n, presence in own) + for counter, number, other in self.item_slots.get(designation, ()) ) def _term(self, directive: Directive, definition_id: str) -> Inline: diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index a2f3391..868fa28 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -23,12 +23,10 @@ from legaldown.models import list_items from legaldown.parser import MAX_QUOTE_DEPTH, quote_content from legaldown.validator.core import is_template -from legaldown.validator.templates import _DRAFTING_MARKER as DRAFTING_MARKER from legaldown.validator.templates import is_drafting_note from legaldown.validator.units import FoundMarker, find_markers __all__ = [ - "DRAFTING_MARKER", "FENCE_OPEN_RE", "MAX_QUOTE_DEPTH", "PlacedMarkers", diff --git a/tests/test_render.py b/tests/test_render.py index 1030751..5d68b5b 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -813,6 +813,25 @@ def test_lists_in_quotes_and_drafting_notes_are_not_ambiguous() -> None: assert "render-ref-ambiguous" not in {d.rule for d in result.diagnostics} +@pytest.mark.parametrize("first, second", [("courts", "arbitration"), ("arbitration", "courts")]) +def test_every_alternative_is_checked_for_ambiguity(first: str, second: str) -> None: + # Under arbitration, x and y both read 1(a), whichever alternative of x + # comes first. + body = (f"# A\n\n- one {{#x when=forum:{first}}}\n- two {{#x when=forum:{second}}}\n\nPara.\n\n" + "- other {#y when=forum:arbitration}\n\nSee {{ref: x}}.\n") + result = render(QUESTIONS + body, format="text") + assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} + + +def test_a_numbered_paragraph_reading_like_a_list_item_is_ambiguous() -> None: + settings = {"paragraphs.numbered": True, + "paragraphs.format": {"counter": "lower-alpha", "label": "({n})", "ref": "({n})"}} + result = render(FRONT + "# A\n\nIntro {#p}\n\n- item\n\nSee {{ref: p}}.\n", format="text", + overrides=settings) + assert "See 1(a)." in result.output + assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} + + def test_an_ambiguous_reference_says_how_to_tell_the_items_apart() -> None: result = render(FRONT + "# A\n\n- a {#x}\n\nPara.\n\n- b\n\nSee {{ref: x}}.\n", format="text") message = next(d.message for d in result.diagnostics if d.rule == "render-ref-ambiguous") @@ -828,6 +847,10 @@ def test_a_drafting_note_reads_as_the_validator_reads_it() -> None: assert "{{placeholder" not in output and "[!DRAFTING]" not in output # The marker underlined makes a heading of it; the note holds nothing else. assert "[!DRAFTING]" not in text("# A\n\n> [!DRAFTING]\n> ---\n") + # An indented marker line reads as code in the validator; the note is + # then the text after it. + output = text("# A\n\n> [!DRAFTING]\n> Note text\n") + assert "> [Drafting note]\n> Note text" in output and "[!DRAFTING]" not in output def test_sibling_lists_are_told_apart_by_how_their_designations_read() -> None: From ad89f824bb70e13f5d9e3bf9edfc68e2a5bd49af Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 03:46:02 +0000 Subject: [PATCH 15/19] Point the drafting-marker stripping at legaldown-validator#88 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- src/legaldown_render/build.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index f4da598..d74a7f2 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -514,7 +514,8 @@ def _without_drafting_marker(children: list[ModelBlock]) -> list[ModelBlock] | N """A drafting note's blocks, as the validator reads them, without the ``[!DRAFTING]`` marker that starts the first of them (its first line is the marker, §15.6); a block that held only the marker goes. None when - the marker does not start a first paragraph or heading.""" + the marker does not start a first paragraph or heading. To be replaced by + the validator's own reading (ForLegalAI/legaldown-validator#88).""" first = children[0] if children else None if first is None or first.kind not in ("paragraph", "heading") \ or not first.text.upper().startswith(DRAFTING_MARKER): From ebe34b5aefc58f014b197e03b1c33d3eeb51a263 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 03:57:52 +0000 Subject: [PATCH 16/19] Check every numbered unit for ambiguous designations, in one place - Sections, numbered paragraphs, and list items all record the designation they read as, through one helper, so a list item or paragraph that reads like a subsection ("1.1" in the continental style) draws the hint too - The referenced unit is the first to take the anchor, with its alternatives (they share its number); a second use of the identifier is another unit, so duplicates no longer hide each other - The message no longer assumes two restarting lists: another section, paragraph, or list item may read the same; refer to it in words or number the parts apart - Under the none scheme a numbered paragraph's label is its designation, so the label and what {{ref:}} prints never differ - _number_paragraph takes the counter and presence once; item_slots is designation_slots; CONFORMANCE.md describes when the hint fires; tests cover sections, duplicates, paragraphs under conditions, and an item clashing only with a paragraph Across the test and specification documents, only features.lgd under the continental style draws the hint, and rightly: paragraph (2) of 2.1 and step 2 of its list both read "2.1(2)". Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 11 ++-- src/legaldown_render/resolve/resolver.py | 82 +++++++++++++++--------- tests/test_render.py | 37 ++++++++++- 3 files changed, 92 insertions(+), 38 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index 1271501..d85449f 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -53,11 +53,12 @@ It is checked against the specification's own examples and fixtures corpus (`tes whose lists, quotes, and inline formatting (emphasis, links) together nest more than 100 levels, while building it and before resolving or writing it, so no document can exhaust the stack. - **Ambiguous references.** Each list starts again at its first number, as contracts are usually - drafted, so two lists in one section can both have an item "(a)". A `{{ref:}}` to such an item - or to a numbered paragraph reading the same renders with a `render-ref-ambiguous` Warning, a - hint: it is left out when the two can never appear together (alternatives, exclusive - conditions), but the condition the reference itself stands under is not considered. Lists in - quotes and drafting notes are not counted. + drafted, so two lists in one section can both have an item "(a)", and a style can number a + paragraph or an item like a subsection ("1.1"). A `{{ref:}}` to a section, paragraph, or list + item whose designation another of them also reads as renders with a `render-ref-ambiguous` + Warning. It is a hint: it is left out when the two can never appear together (alternatives, + exclusive conditions), but the condition the reference itself stands under is not considered. + Lists in quotes and drafting notes are not counted. - **Malformed directives** (a §11.2 grammar violation) render their type's marker without a value, for example `[INVALID AMOUNT]`. The specification defines no marker for this case. - **One parser.** The renderer builds from the validator's document model only diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index 30e9fbd..c90f51b 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -130,12 +130,12 @@ def __init__( self.diagnostics: list[Diagnostic] = [] self.targets: dict[str, _Target] = {} self.used_anchors: set[str] = set() - # For render-ref-ambiguous: each designation of a numbered item or - # paragraph, with its numbered slots (counter and number: alternatives - # share one) and the presence of the unit holding it; and the slots - # of each anchored unit, one per alternative. - self.item_slots: dict[str, list[tuple[_Counter, int, Presence]]] = {} - self.slot_of: dict[str, list[tuple[_Counter, int, Presence]]] = {} + # For render-ref-ambiguous: every designation a numbered unit — a + # section, a paragraph, a list item — reads as, with each unit's slot + # (what numbers it and its number: alternatives share one) and the + # presence of the unit; and the slots of each anchored unit. + self.designation_slots: dict[str, list[_Slot]] = {} + self.slot_of: dict[str, list[_Slot]] = {} questions = self.metadata.questions self.questions: dict[str, Any] = questions if isinstance(questions, dict) else {} self.inconsistent_placeholders: set[str] = set() @@ -293,6 +293,8 @@ def _number_sections(self) -> list[Section]: label = fill(fmt.label, n=format_counter(parts[-1], fmt.counter), path=designation) anchor = self._anchor(section.identifier) self._register(section.identifier, _Target(designation, anchor)) + # Sections with one number are alternatives (the validator's). + self._record(designation, (_SECTIONS, indexed.number, presences[level]), section.identifier) out.append(replace(section, level=min(len(parts), 5), label=label, designation=designation, anchor=anchor)) return out @@ -340,9 +342,10 @@ def _structure_section(self, section: Section, presence: Presence) -> tuple[Bloc for block in section.blocks: if isinstance(block, Paragraph) and block.top_level: # Alternative paragraphs share a number, as sections do (§15.8). - number = paragraphs.next(block.anchor_id, self._own_presence(block.condition, presence)) - block = self._number_paragraph(block, section, number, - slot=(paragraphs, number, presence | self._presence(block.condition))) + own = self._own_presence(block.condition, presence) + number = paragraphs.next(block.anchor_id, own) + block = self._number_paragraph(block, section, paragraphs, number, + presence if own is None else own) out.append(self._structure(block, section=section, depth=0, presence=presence)) return tuple(out) @@ -352,19 +355,20 @@ def _own_presence(self, condition: str, enclosing: Presence) -> Presence | None: own = self._presence(condition) return enclosing | own if own else None - def _number_paragraph(self, block: Paragraph, section: Section, number: int, - slot: tuple[_Counter, int, Presence]) -> Paragraph: + def _number_paragraph(self, block: Paragraph, section: Section, counter: _Counter, number: int, + presence: Presence) -> Paragraph: numbering = self.style.paragraphs base = section.designation or "" if numbering.numbered: fmt = numbering.format n = format_counter(number, fmt.counter) designation = extend(base, fill(fmt.ref, n=n, section=base), textual=self.textual) - label = fill(fmt.label, n=n, section=base, path=designation) + # Under the none scheme the designation is heading text, which a + # label's "{section}" cannot rebuild: the label is what {{ref:}} + # prints, so the two never differ. + label = designation if self.textual else fill(fmt.label, n=n, section=base, path=designation) target = _Target(designation, None) - self.item_slots.setdefault(designation, []).append(slot) - if block.anchor_id: - self.slot_of.setdefault(block.anchor_id, []).append(slot) + self._record(designation, (counter, number, presence), block.anchor_id) else: label, target = None, _Target(base, None, enumerated=False) anchor = self._anchor(block.anchor_id) @@ -424,10 +428,7 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p label = fill(fmt.label, n=n, section=base, path=designation) target = _Target(designation, None) if not quoted: - slot = (counter, index, inner) - self.item_slots.setdefault(designation, []).append(slot) - if item.anchor_id: - self.slot_of.setdefault(item.anchor_id, []).append(slot) + self._record(designation, (counter, index, inner), item.anchor_id) else: designation, label, target = parent, None, _Target(base, None, enumerated=False) anchor = self._anchor(item.anchor_id) @@ -531,26 +532,34 @@ def _ref(self, directive: Directive, target_id: str) -> Inline: self._warn( "render-ref-ambiguous", f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which may also be how another " - f"numbered item or paragraph reads: each list starts again at its first number. Make them one " - f"list, or refer to the item in words.", + f"numbered part of the document reads (a section, a paragraph, or a list item). Refer to it in " + f"words, or number the parts apart.", ) text = self.style.references.format.replace("{designation}", target.designation) return CrossRef(text, target.anchor or "") + def _record(self, designation: str, slot: _Slot, anchor_id: str) -> None: + """Note that a numbered unit reads as *designation* (render-ref-ambiguous).""" + self.designation_slots.setdefault(designation, []).append(slot) + if anchor_id: + self.slot_of.setdefault(anchor_id, []).append(slot) + def _ambiguous(self, target_id: str, designation: str) -> bool: - """True if another numbered item or paragraph may read as the same - *designation* as the one *target_id* anchors, in a document where - both appear. A hint, not a proof: units under exclusive conditions - never appear together (§15.4), but where the reference itself - stands is not considered.""" - own = self.slot_of.get(target_id, []) - if not own: + """True if another numbered unit may read as the same *designation* + as the one *target_id* anchors, in a document where both appear. The + unit is the first that took the anchor, with its alternatives, which + share its number; any other unit, a second use of the identifier + included, is another. A hint, not a proof: units under exclusive + conditions never appear together (§15.4), but where the reference + itself stands is not considered.""" + slots = self.slot_of.get(target_id) + if not slots: return False - own_numbers = {(counter, number) for counter, number, _presence in own} + number = slots[0][:2] + presences = [presence for key, value, presence in slots if (key, value) == number] return any( - (counter, number) not in own_numbers - and any(not self._exclusive(presence, other) for _c, _n, presence in own) - for counter, number, other in self.item_slots.get(designation, ()) + (key, value) != number and any(not self._exclusive(presence, other) for presence in presences) + for key, value, other in self.designation_slots.get(designation, ()) ) def _term(self, directive: Directive, definition_id: str) -> Inline: @@ -788,6 +797,10 @@ def _warn(self, rule: str, message: str) -> None: } +#: What numbers sections, in a unit's slot: the validator (its numbers). +_SECTIONS = object() + + class _Counter: """Numbers sibling units in order (§13.2, §15.8). A unit joins the previous unit's number when it is an alternative to it: the same @@ -820,6 +833,11 @@ def next(self, identifier: str, presence: Presence | None) -> int: return self.count +#: A numbered unit's slot: what numbers it (a _Counter, or _SECTIONS), its +#: number, and the unit's presence (§15.3). +_Slot = tuple[object, object, Presence] + + def fill_condition(template: str, value: str, *, key: str = "condition") -> str: return template.replace("{" + key + "}", value) diff --git a/tests/test_render.py b/tests/test_render.py index 5d68b5b..c896bc0 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -832,10 +832,45 @@ def test_a_numbered_paragraph_reading_like_a_list_item_is_ambiguous() -> None: assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} +def test_an_item_reading_like_a_subsection_is_ambiguous() -> None: + # Continental items read "{section}.{n}", as subsection 1.1 does. + result = render(FRONT + "# A\n\n- a {#x}\n\n## B\n\nSee {{ref: x}}.\n", format="text", style="continental") + assert "See 1.1." in result.output + assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} + + +def test_a_second_use_of_an_identifier_is_another_unit() -> None: + result = render(FRONT + "# A\n\n- a {#x}\n\nPara.\n\n- b {#x}\n\nSee {{ref: x}}.\n", format="text") + assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} + + +PARAGRAPHS = {"paragraphs.numbered": True, + "paragraphs.format": {"counter": "lower-alpha", "label": "({n})", "ref": "({n})"}} + + +def test_a_list_item_clashing_only_with_a_paragraph_is_ambiguous() -> None: + result = render(FRONT + "# A\n\nIntro.\n\n- item {#i}\n\nSee {{ref: i}}.\n", format="text", + overrides=PARAGRAPHS) + assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} + + +def test_paragraphs_under_exclusive_conditions_are_not_ambiguous() -> None: + body = ("# A\n\nCourts only. {#p when=forum:courts}\n\n- only under arbitration {#i when=forum:arbitration}\n\n" + "See {{ref: p}}.\n") + result = render(QUESTIONS + body, format="text", overrides=PARAGRAPHS) + assert "render-ref-ambiguous" not in {d.rule for d in result.diagnostics} + + +def test_a_paragraph_label_is_what_its_reference_prints_under_the_none_scheme() -> None: + output = text("# A\n\nIntro {#p}\n\nSee {{ref: p}}.\n", + overrides={"numbering.scheme": "none", "paragraphs.numbered": True}) + assert "\n\nA 1 Intro\n\nA 2 See A 1.\n" in output + + def test_an_ambiguous_reference_says_how_to_tell_the_items_apart() -> None: result = render(FRONT + "# A\n\n- a {#x}\n\nPara.\n\n- b\n\nSee {{ref: x}}.\n", format="text") message = next(d.message for d in result.diagnostics if d.rule == "render-ref-ambiguous") - assert "one list" in message and "in words" in message and "paragraphs" not in message + assert "in words" in message and "number the parts apart" in message def test_a_drafting_note_reads_as_the_validator_reads_it() -> None: From bd1df48ae25a07a3ab661b77075c9805bd695924 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 04:26:14 +0000 Subject: [PATCH 17/19] Tailor render-ref-ambiguous advice; keep style labels under the none scheme MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Revert the none-scheme paragraph label override: labels follow the style again ("(1)", not the heading text). A label that shows {section} differs from the §13.3 designation {{ref:}} writes there, for items and paragraphs alike; CONFORMANCE.md says so - The hint's advice follows what clashes: two list items, "make them one list, or refer to the item in words"; a heading under the none scheme, "rename one"; otherwise, naming what else reads the same, "change the style's numbering" - A second use of an identifier is left to the validator's anchor-duplicate, not repeated as a hint - The referenced unit's slot is stored on its reference target when it is registered, so the check no longer depends on the order units are recorded in; the slot is a typed NamedTuple Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 6 +- src/legaldown_render/resolve/resolver.py | 111 ++++++++++++++--------- tests/test_render.py | 28 ++++-- 3 files changed, 92 insertions(+), 53 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index d85449f..d9bd7e7 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -58,7 +58,11 @@ It is checked against the specification's own examples and fixtures corpus (`tes item whose designation another of them also reads as renders with a `render-ref-ambiguous` Warning. It is a hint: it is left out when the two can never appear together (alternatives, exclusive conditions), but the condition the reference itself stands under is not considered. - Lists in quotes and drafting notes are not counted. + Lists in quotes and drafting notes are not counted, and a second use of an identifier is left + to the validator's `anchor-duplicate`. +- **Labels under the `none` scheme.** A style label that shows `{section}` (such as the default + paragraph label `{section}.{n}`) shows the section's heading text there, while `{{ref:}}` + writes the §13.3 designation ("Termination 1"), so the two can differ. - **Malformed directives** (a §11.2 grammar violation) render their type's marker without a value, for example `[INVALID AMOUNT]`. The specification defines no marker for this case. - **One parser.** The renderer builds from the validator's document model only diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index c90f51b..99351eb 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -22,7 +22,7 @@ from collections.abc import Callable from dataclasses import dataclass, replace -from typing import Any +from typing import Any, NamedTuple from legaldown import Diagnostic, Directive, Document, ValidationResult, slugify_identifier from legaldown.validator import KNOWN_CURRENCIES @@ -108,6 +108,8 @@ class _Target: #: False for an item or paragraph whose list or paragraphs the style does #: not number: the designation is then its section's (§6.3). enumerated: bool = True + #: The numbered unit the anchor is on, for render-ref-ambiguous. + slot: _Slot | None = None class Resolver: @@ -131,11 +133,8 @@ def __init__( self.targets: dict[str, _Target] = {} self.used_anchors: set[str] = set() # For render-ref-ambiguous: every designation a numbered unit — a - # section, a paragraph, a list item — reads as, with each unit's slot - # (what numbers it and its number: alternatives share one) and the - # presence of the unit; and the slots of each anchored unit. + # section, a paragraph, a list item — reads as, with each unit's slot. self.designation_slots: dict[str, list[_Slot]] = {} - self.slot_of: dict[str, list[_Slot]] = {} questions = self.metadata.questions self.questions: dict[str, Any] = questions if isinstance(questions, dict) else {} self.inconsistent_placeholders: set[str] = set() @@ -292,9 +291,10 @@ def _number_sections(self) -> list[Section]: fmt = levels[min(len(parts), len(levels)) - 1] label = fill(fmt.label, n=format_counter(parts[-1], fmt.counter), path=designation) anchor = self._anchor(section.identifier) - self._register(section.identifier, _Target(designation, anchor)) # Sections with one number are alternatives (the validator's). - self._record(designation, (_SECTIONS, indexed.number, presences[level]), section.identifier) + slot = _Slot(_SECTIONS, indexed.number, presences[level], "section", section.identifier) + self._record(designation, slot) + self._register(section.identifier, _Target(designation, anchor, slot=slot)) out.append(replace(section, level=min(len(parts), 5), label=label, designation=designation, anchor=anchor)) return out @@ -363,12 +363,10 @@ def _number_paragraph(self, block: Paragraph, section: Section, counter: _Counte fmt = numbering.format n = format_counter(number, fmt.counter) designation = extend(base, fill(fmt.ref, n=n, section=base), textual=self.textual) - # Under the none scheme the designation is heading text, which a - # label's "{section}" cannot rebuild: the label is what {{ref:}} - # prints, so the two never differ. - label = designation if self.textual else fill(fmt.label, n=n, section=base, path=designation) - target = _Target(designation, None) - self._record(designation, (counter, number, presence), block.anchor_id) + label = fill(fmt.label, n=n, section=base, path=designation) + slot = _Slot(counter, number, presence, "paragraph", block.anchor_id) + target = _Target(designation, None, slot=slot) + self._record(designation, slot) else: label, target = None, _Target(base, None, enumerated=False) anchor = self._anchor(block.anchor_id) @@ -428,7 +426,9 @@ def _structure_list(self, block: List, *, section: Section | None, depth: int, p label = fill(fmt.label, n=n, section=base, path=designation) target = _Target(designation, None) if not quoted: - self._record(designation, (counter, index, inner), item.anchor_id) + slot = _Slot(counter, index, inner, "item", item.anchor_id) + target = replace(target, slot=slot) + self._record(designation, slot) else: designation, label, target = parent, None, _Target(base, None, enumerated=False) anchor = self._anchor(item.anchor_id) @@ -528,39 +528,44 @@ def _ref(self, directive: Directive, target_id: str) -> Inline: f"'{{{{ref: {target_id}}}}}' targets an item or paragraph that the style does not number; " f"it renders as its section's designation, '{target.designation}' (§6.3).", ) - elif self._ambiguous(target_id, target.designation): - self._warn( - "render-ref-ambiguous", - f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which may also be how another " - f"numbered part of the document reads (a section, a paragraph, or a list item). Refer to it in " - f"words, or number the parts apart.", - ) + elif target.slot is not None and (clashes := self._clashes(target.slot, target.designation)): + self._warn("render-ref-ambiguous", self._ambiguity_message(target_id, target, clashes)) text = self.style.references.format.replace("{designation}", target.designation) return CrossRef(text, target.anchor or "") - def _record(self, designation: str, slot: _Slot, anchor_id: str) -> None: + def _record(self, designation: str, slot: _Slot) -> None: """Note that a numbered unit reads as *designation* (render-ref-ambiguous).""" self.designation_slots.setdefault(designation, []).append(slot) - if anchor_id: - self.slot_of.setdefault(anchor_id, []).append(slot) - - def _ambiguous(self, target_id: str, designation: str) -> bool: - """True if another numbered unit may read as the same *designation* - as the one *target_id* anchors, in a document where both appear. The - unit is the first that took the anchor, with its alternatives, which - share its number; any other unit, a second use of the identifier - included, is another. A hint, not a proof: units under exclusive - conditions never appear together (§15.4), but where the reference - itself stands is not considered.""" - slots = self.slot_of.get(target_id) - if not slots: - return False - number = slots[0][:2] - presences = [presence for key, value, presence in slots if (key, value) == number] - return any( - (key, value) != number and any(not self._exclusive(presence, other) for presence in presences) - for key, value, other in self.designation_slots.get(designation, ()) - ) + + def _clashes(self, slot: _Slot, designation: str) -> list[_Slot]: + """The other numbered units that may read as *designation*, the one + *slot*'s unit reads as, in a document where both appear. Its own + alternatives share its number and are not others; a second use of + its identifier is left to the validator (anchor-duplicate). A hint, + not a proof: units under exclusive conditions never appear together + (§15.4), but where the reference itself stands is not considered.""" + slots = self.designation_slots.get(designation, []) + own = [other.presence for other in slots if other.key == slot.key] + return [ + other for other in slots + if other.key != slot.key and not (slot.anchor and other.anchor == slot.anchor) + and any(not self._exclusive(presence, other.presence) for presence in own) + ] + + def _ambiguity_message(self, target_id: str, target: _Target, clashes: list[_Slot]) -> str: + """render-ref-ambiguous, with advice for what clashes.""" + start = f"'{{{{ref: {target_id}}}}}' renders as '{target.designation}', which " + kinds = {target.slot.kind} | {other.kind for other in clashes} if target.slot else set() + if kinds == {"item"}: + return (start + "another list item also reads as: each list starts again at its first number. " + "Make them one list, or refer to the item in words.") + if "section" in kinds and self.textual: + return (start + "another heading also reads as: the none scheme refers to a section by its " + "heading text. Rename one, or refer to it in words.") + names = sorted({{"section": "a section", "paragraph": "a numbered paragraph", "item": "a list item"}[kind] + for kind in (other.kind for other in clashes)}) + return (start + f"{' or '.join(names)} also reads as. Refer to it in words, or change the style's " + "numbering so that they differ.") def _term(self, directive: Directive, definition_id: str) -> Inline: term = self.result.definition_lookup.get(definition_id) @@ -833,9 +838,25 @@ def next(self, identifier: str, presence: Presence | None) -> int: return self.count -#: A numbered unit's slot: what numbers it (a _Counter, or _SECTIONS), its -#: number, and the unit's presence (§15.3). -_Slot = tuple[object, object, Presence] +class _Slot(NamedTuple): + """Where a numbered unit's designation comes from, for render-ref-ambiguous.""" + + #: What numbers the unit: its list's or paragraphs' _Counter, or + #: _SECTIONS for a section (the validator numbers those). + source: object + #: Its number there; alternatives share one (§15.8). + number: object + #: When the unit appears (§15.3). + presence: Presence + #: "section", "paragraph", or "item". + kind: str + #: The identifier it anchors, or "". + anchor: str + + @property + def key(self) -> tuple[object, object]: + """The unit and its alternatives: its source and number.""" + return (self.source, self.number) def fill_condition(template: str, value: str, *, key: str = "condition") -> str: diff --git a/tests/test_render.py b/tests/test_render.py index c896bc0..41cc352 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -839,9 +839,10 @@ def test_an_item_reading_like_a_subsection_is_ambiguous() -> None: assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} -def test_a_second_use_of_an_identifier_is_another_unit() -> None: +def test_a_second_use_of_an_identifier_is_left_to_the_validator() -> None: result = render(FRONT + "# A\n\n- a {#x}\n\nPara.\n\n- b {#x}\n\nSee {{ref: x}}.\n", format="text") - assert "render-ref-ambiguous" in {d.rule for d in result.diagnostics} + rules = {d.rule for d in result.diagnostics} + assert "anchor-duplicate" in rules and "render-ref-ambiguous" not in rules PARAGRAPHS = {"paragraphs.numbered": True, @@ -861,16 +862,29 @@ def test_paragraphs_under_exclusive_conditions_are_not_ambiguous() -> None: assert "render-ref-ambiguous" not in {d.rule for d in result.diagnostics} -def test_a_paragraph_label_is_what_its_reference_prints_under_the_none_scheme() -> None: - output = text("# A\n\nIntro {#p}\n\nSee {{ref: p}}.\n", - overrides={"numbering.scheme": "none", "paragraphs.numbered": True}) - assert "\n\nA 1 Intro\n\nA 2 See A 1.\n" in output +def test_paragraph_labels_follow_the_style_under_the_none_scheme() -> None: + output = render(FRONT + "# Confidentiality\n\nFirst.\n\nSecond.\n", format="text", style="continental", + overrides={"numbering.scheme": "none"}).output + assert "\n\n(1) First.\n\n(2) Second.\n" in output def test_an_ambiguous_reference_says_how_to_tell_the_items_apart() -> None: result = render(FRONT + "# A\n\n- a {#x}\n\nPara.\n\n- b\n\nSee {{ref: x}}.\n", format="text") message = next(d.message for d in result.diagnostics if d.rule == "render-ref-ambiguous") - assert "in words" in message and "number the parts apart" in message + assert "Make them one list, or refer to the item in words." in message + + +def test_a_heading_clash_under_the_none_scheme_advises_renaming() -> None: + result = render(FRONT + "# A\n\n## General {#g1}\n\n## General {#g2}\n\nSee {{ref: g1}}.\n", format="text", + overrides={"numbering.scheme": "none"}) + message = next(d.message for d in result.diagnostics if d.rule == "render-ref-ambiguous") + assert "Rename one" in message + + +def test_an_item_reading_like_a_subsection_advises_changing_the_numbering() -> None: + result = render(FRONT + "# A\n\n- a {#x}\n\n## B\n\nSee {{ref: x}}.\n", format="text", style="continental") + message = next(d.message for d in result.diagnostics if d.rule == "render-ref-ambiguous") + assert "a section also reads as" in message and "change the style's numbering" in message def test_a_drafting_note_reads_as_the_validator_reads_it() -> None: From 7c72018a9b8098e067cbe7972842808f8b847125 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 21:06:33 +0000 Subject: [PATCH 18/19] Adopt legaldown-validator 0.3.0 and its public API - Pin legaldown-validator>=0.3.0,<0.4. - Placed markers and the template decision come from the ValidationResult (placed_markers, is_template): each marker's field, offset, and list item are the validator's, so the builder no longer searches for them or numbers list items itself. - Import the lexer, list helpers, is_drafting_note, conditions, and value checks from the public API. validator_bridge.py keeps only quote_content, the fence helpers, and the answers-file reader (validator#93). - A paragraph keeps its lines (validator#87): hard breaks render as
and in text as a new line; a soft break is a newline in HTML, a space in text. A marker ends its fragment, whatever line it is on. - The CLI prints a validator diagnostic's line, as file:line:. - Docs: #25, #26, #27 adopted; link reference definitions remain a limitation (validator#92). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- CONFORMANCE.md | 15 +- docs/architecture.md | 32 ++-- .../0007-one-parser-validator-model.md | 6 +- docs/roadmap.md | 19 +-- pyproject.toml | 5 +- src/legaldown_render/build.py | 138 +++++++----------- src/legaldown_render/cli.py | 4 +- src/legaldown_render/resolve/resolver.py | 16 +- src/legaldown_render/validator_bridge.py | 63 ++------ tests/conformance/test_spec_corpus.py | 16 -- tests/conftest.py | 22 --- tests/golden/features.default.html | 22 ++- tests/test_cli.py | 5 +- tests/test_render.py | 25 +--- 14 files changed, 146 insertions(+), 242 deletions(-) diff --git a/CONFORMANCE.md b/CONFORMANCE.md index d9bd7e7..e1c369f 100644 --- a/CONFORMANCE.md +++ b/CONFORMANCE.md @@ -1,7 +1,7 @@ # Conformance -`legaldown-render` 0.1.0 targets **Level 2 — Rendering** of the LegalDown specification **0.2** -(§17.3). Core parsing and validation come from `legaldown-validator` 0.2.0, which claims Level 1 +`legaldown-render` 0.2.0 targets **Level 2 — Rendering** of the LegalDown specification **0.2** +(§17.3). Core parsing and validation come from `legaldown-validator` 0.3.0, which claims Level 1 — Core. Its own [CONFORMANCE.md](https://github.com/ForLegalAI/legaldown-validator/blob/main/CONFORMANCE.md) lists the Core rules it covers. @@ -47,7 +47,9 @@ It is checked against the specification's own examples and fixtures corpus (`tes - **Assembly reads no other files.** A template with include fragments, LegalDown attachment files, or `translations` is refused when rendered with answers, as §17.6 requires below Full. A template with Errors is refused too, since §15.7.2 defines no output for one. -- **Diagnostics have no line numbers**, because the validator's diagnostics do not carry them yet. +- **Line numbers.** The validator's diagnostics name their line, and the CLI prints it. The + renderer's own (`render-ref-ambiguous`, `ref-not-enumerated`, `render-not-processed`) do not + yet ([validator#93](https://github.com/ForLegalAI/legaldown-validator/issues/93)). - **Nesting limit.** The validator reads lists 64 levels deep and quotes 16; deeper, an item's or quote's content is its text. The renderer follows it, and refuses (`DocumentError`) a document whose lists, quotes, and inline formatting (emphasis, links) together nest more than 100 levels, @@ -69,10 +71,9 @@ It is checked against the specification's own examples and fixtures corpus (`tes ([ADR 0007](docs/decisions/0007-one-parser-validator-model.md)). Anchors, markers, and the template view therefore always agree with the validator's diagnostics. The price is that some structure renders as the validator's model holds it, until that model keeps more (roadmap - U1b–U2): - - **Line breaks** within a paragraph are joined, and a backslash hard break shows as a literal - backslash. Link reference definitions (`[label]: url`) are not resolved - ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)). + U1b): + - **Link reference definitions** (`[label]: url`) render as paragraphs, and the links that use + them as text ([#92](https://github.com/ForLegalAI/legaldown-validator/issues/92)). ## Beyond this level (Full, §17.4) diff --git a/docs/architecture.md b/docs/architecture.md index 4dcf6ea..0d227ee 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -78,14 +78,14 @@ never decides a structural or LegalDown question itself: |---|---| | Sections, headings, identifiers | `Document.sections`, `ValidationResult.sections` | | Blocks: paragraphs, lists and items, quotes, tables, code, rules | `Document` blocks. List items hold blocks (nested lists, code, quotes, tables), and a quote's content is the validator's `quote_content()` | -| Where a marker (`{#id when=…}`) is placed, and what it means | The validator's `find_markers()`, with its own `placed(template)` | -| Whether the document is a template | The validator's `is_template()`, over those markers | +| Where a marker (`{#id when=…}`) is placed, and what it means | `ValidationResult.placed_markers`: each marker's block, field, offset, identifier, and condition | +| Whether the document is a template | `ValidationResult.is_template` | | Whether a quote is a drafting note | The validator's `is_drafting_note()` | -| Which list item a marker belongs to | The validator's `list_fragments()`: the items each fragment is in, numbered in document order | +| Which list item a marker belongs to | `PlacedMarker.item`: the item's number among the list's items, in pre-order | | A lifted definition, `{{ref:}}` or `{{term:}}` block's source | The validator's `render_block()` | -The imports beyond the validator's public API are all in `validator_bridge.py`, which is the -list for roadmap item U3. +The few imports beyond the validator's public API (quote content, fence helpers, the +answers-file reader) are all in `validator_bridge.py`, the list for roadmap item U3. Within one block's text, **markdown-it-py parses inline Markdown only**: emphasis, links, code spans, inline HTML. Directives are protected first by **sentinels**. Every directive, and every @@ -106,7 +106,7 @@ in a list item or a quote (`kind="heading"`) is not a section (§4.1) and shows The builder never works around the validator's model. Where the model loses or misreads something, the output follows the model, and the gap is listed in `CONFORMANCE.md` and filed on the validator: -- a paragraph's line breaks are joined +- link reference definitions (`[label]: url`) render as paragraphs Guessing at lost structure was tried, and it traded each gap for new bugs. @@ -168,7 +168,7 @@ clamped to 1–5, as the validator numbers them. | Writer | Notes | |---|---| -| `TextWriter` | Plain text (§13.6). Line breaks inside paragraphs are joined, and lists are indented with their labels. It is the **test oracle** for resolution | +| `TextWriter` | Plain text (§13.6). A soft line break inside a paragraph is a space, a hard break a new line, and lists are indented with their labels. It is the **test oracle** for resolution | | `HtmlWriter` | A self-contained HTML5 page, or with `standalone=False` just the `
`. Semantic markup, with labels as real text rather than CSS counters, and a stylesheet generated from the style's presentation settings, including print rules | | DOCX, PDF | Planned ([ADR 0004](decisions/0004-html-first.md)) | @@ -198,19 +198,18 @@ src/legaldown_render/ | Dependency | Why | |---|---| -| `legaldown-validator>=0.2.0,<0.3` | The only LegalDown parser; Core validation | +| `legaldown-validator>=0.3.0,<0.4` | The only LegalDown parser; Core validation | | `markdown-it-py` | Inline Markdown inside one block's text (ADR 0007) | | `babel` | CLDR locale data (ADR 0005) | | `pyyaml` | Style templates | All of them are pure Python. The DOCX and PDF writers will bring their dependencies in as extras. -The renderer imports a few names from `legaldown` submodules that the validator does not -re-export at the top level: `legaldown.directives.lex`, `legaldown.markers`, -`legaldown.parser.FRONTMATTER_RE`, `legaldown.validator.helpers`, -`legaldown.validator.patterns`, `legaldown.validator.conditions`, and one private function, -`legaldown.validator.core._frontmatter_fields`. The pin to one minor version exists because of them. Asking the -validator to export them publicly is roadmap item U3. +The renderer uses the validator's public API (`legaldown`, `legaldown.validator`), except for +the few names in `validator_bridge.py`: `quote_content` and `MAX_QUOTE_DEPTH`, the fence +helpers, and the answers-file reader. The pin to one minor version exists because of them. +Asking the validator to export them publicly is roadmap item U3 +([validator#93](https://github.com/ForLegalAI/legaldown-validator/issues/93)). ## Errors and diagnostics @@ -222,8 +221,9 @@ validator to export them publicly is roadmap item U3. Diagnostics reuse `legaldown.Diagnostic`. The renderer adds rules only it can evaluate: `ref-not-enumerated`, a specification rule id, plus renderer-specific ids prefixed `render-`: -`render-not-processed`, `render-locale-fallback`, and `render-ref-ambiguous`. Validator diagnostics -have no line numbers yet, and neither do the renderer's. +`render-not-processed`, `render-locale-fallback`, and `render-ref-ambiguous`. The validator's +diagnostics carry their line (§16.9), and the CLI prints it as `file:line:`. The renderer's own +have none yet, because a block's line is not public in the validator (U2). ## Security diff --git a/docs/decisions/0007-one-parser-validator-model.md b/docs/decisions/0007-one-parser-validator-model.md index 817b5fa..1463f4b 100644 --- a/docs/decisions/0007-one-parser-validator-model.md +++ b/docs/decisions/0007-one-parser-validator-model.md @@ -29,9 +29,9 @@ The render tree is built **from the validator's `Document` only**: - **Block structure:** sections, blocks, list items, quotes, tables, and code all come from the validator's model. -- **Markers:** placed exactly where the validator's own `find_markers()` places them. -- **Template decision:** the validator's own formula over those markers. Since the validator - exposed it as `is_template()`, the renderer calls it instead of keeping a copy. +- **Markers:** placed exactly where the validator places them. Since validator 0.3.0 the + renderer reads them from `ValidationResult.placed_markers`. +- **Template decision:** the validator's own, `ValidationResult.is_template`. - **Quote content, and code inside list items:** read with the validator's parser too. markdown-it-py stays, in **inline mode only**, for the Markdown inside one block's text: emphasis, diff --git a/docs/roadmap.md b/docs/roadmap.md index 40bd90c..04468b4 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -11,16 +11,16 @@ here. | # | Where | Change | Status | |---|---|---|---| | U1 | validator | **Keep nested list structure** in the model ([#14](https://github.com/ForLegalAI/legaldown-validator/issues/14)) | Done in [validator#61](https://github.com/ForLegalAI/legaldown-validator/pull/61); the renderer nests lists from it (v0.2) | -| U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Open. Diagnostics have no line numbers in either package | -| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules (all in `validator_bridge.py`): the lexer, markers, value checks, conditions, `is_template`, `is_drafting_note`, the CommonMark fence and indentation helpers, the answers-file reader, `list_items`, `list_fragments`, `quote_content` and `MAX_QUOTE_DEPTH`, `IDENTIFIER_RE`, `KNOWN_CURRENCIES`. Better still, expose the validator's own **placed markers** and **template decision**, so the renderer uses them instead of mirroring those rules | Partly done: the template decision is the validator's `is_template()` (not yet public). Until the rest is public, the dependency is pinned to one minor version | -| U1b | validator | **Parser gaps** the renderer inherits. Fixed in validator `main`: list markers (#21), tables (#22, #44), comments and HTML blocks (#23, #59), the signature-block cutoff (#24), empty comments (#28), indented code (#9, #41), empty list items (#46), content after a nested list (#64), nested marker changes (#65), headings in quotes and items (#78), raw-html (#40). Still open: hard breaks ([#25](https://github.com/ForLegalAI/legaldown-validator/issues/25)) | Mostly done. The renderer adopts them when the validator is released. Listed in `CONFORMANCE.md` | +| U2 | validator | **Source positions** (line numbers) on sections, blocks, and diagnostics ([#27](https://github.com/ForLegalAI/legaldown-validator/issues/27)) | Done for the validator's diagnostics in 0.3.0 ([validator#85](https://github.com/ForLegalAI/legaldown-validator/pull/85)); the CLI prints them. Still open: a block's line, for the renderer's own diagnostics ([#93](https://github.com/ForLegalAI/legaldown-validator/issues/93)) | +| U3 | validator | **Public API** ([#26](https://github.com/ForLegalAI/legaldown-validator/issues/26)) for what the renderer imports from submodules, and the validator's own **placed markers** and **template decision** | Mostly done in 0.3.0 ([validator#90](https://github.com/ForLegalAI/legaldown-validator/pull/90)): the renderer reads `placed_markers` and `is_template` from the result. Left in `validator_bridge.py`: `quote_content`, the fence helpers, the answers-file reader ([#93](https://github.com/ForLegalAI/legaldown-validator/issues/93)), and a drafting note's content ([#88](https://github.com/ForLegalAI/legaldown-validator/issues/88)). Until then, the dependency is pinned to one minor version | +| U1b | validator | **Parser gaps** the renderer inherits. Fixed in validator `main`: list markers (#21), tables (#22, #44), comments and HTML blocks (#23, #59), the signature-block cutoff (#24), empty comments (#28), indented code (#9, #41), empty list items (#46), content after a nested list (#64), nested marker changes (#65), headings in quotes and items (#78), raw-html (#40). hard breaks (#25). Still open: link reference definitions ([#92](https://github.com/ForLegalAI/legaldown-validator/issues/92)) | Mostly done; adopted with 0.3.0. Listed in `CONFORMANCE.md` | | U4a | validator | **Specification 0.2** (templates, §15) | Done in 0.2.0 | | U4b | validator | Template **assembly** with an answers set (§15.7, [#30](https://github.com/ForLegalAI/legaldown-validator/issues/30)) | Done in [validator#55](https://github.com/ForLegalAI/legaldown-validator/pull/55); the renderer uses it (v0.2) | | U5 | specification | A **rendering fixtures corpus**: source + style settings → expected plain-text output | Not started. The plain-text writer is designed to be its oracle | ## Milestones -### v0.1 — HTML and text, specification 0.2 (this release) +### v0.1 — HTML and text, specification 0.2 - The full pipeline: parse and validate, build, resolve, write - All four numbering schemes; list enumeration; paragraph numbering; item and paragraph anchors @@ -34,13 +34,14 @@ here. - `text` and `html` writers; the `legaldown-render` CLI - Golden tests, and a conformance run over the specification's examples and fixtures -### v0.2 — Templates with answers +### v0.2 — Templates with answers (this release) -- Rendering with an answers set, through the core package's assembly (U4b): done -- `--answers answers.yaml` on the CLI: done +- Rendering with an answers set, through the core package's assembly (U4b) +- `--answers answers.yaml` on the CLI - Nested lists, with designations such as "2.1(b)(i)" (U1) -- Released together with the validator release that ships the CommonMark block model, nested - lists, and assembly +- The validator's CommonMark block model: headings in items and quotes, hard line breaks, and + diagnostics with line numbers +- Built on `legaldown-validator` 0.3.0; the first release on PyPI ### v0.3 — DOCX diff --git a/pyproject.toml b/pyproject.toml index e75dde9..6394fe0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -31,10 +31,7 @@ classifiers = [ dependencies = [ # The only LegalDown parser (docs/decisions/0002). A minor bump may change # the model, so the range is pinned to one minor version. - # NOT RELEASABLE YET: this code needs the validator release after 0.2.0 - # (the CommonMark block model, is_template, assemble). Move the range to - # that release when it is published. - "legaldown-validator>=0.2.0,<0.3", + "legaldown-validator>=0.3.0,<0.4", # Inline Markdown (emphasis, links, code spans) within the validator's # blocks, parsed in inline mode only (docs/decisions/0007). "markdown-it-py>=3.0,<5", diff --git a/src/legaldown_render/build.py b/src/legaldown_render/build.py index d74a7f2..3a90891 100644 --- a/src/legaldown_render/build.py +++ b/src/legaldown_render/build.py @@ -2,10 +2,10 @@ (docs/architecture.md, docs/decisions/0002). There is one parser: ``legaldown-validator``'s. Its ``Document`` gives the -sections, their blocks, and each block's text, and its own findings say -where markers are placed and whether the document is a template -(:mod:`.validator_bridge`). The builder never decides a structural or -LegalDown question itself. +sections, their blocks, and each block's text, and its ``ValidationResult`` +says where markers are placed and whether the document is a template +(``placed_markers``, ``is_template``). The builder never decides a +structural or LegalDown question itself. Within one block's text it still needs inline Markdown — emphasis, links, code spans — which markdown-it-py parses in inline mode only. Directives @@ -22,16 +22,26 @@ """ from __future__ import annotations -import itertools import re import secrets from collections.abc import Iterator from dataclasses import dataclass, replace +from functools import cache +from itertools import count from urllib.parse import unquote from legaldown import Block as ModelBlock -from legaldown import Directive, Document, ValidationResult, find_definition_anchors, render_block -from legaldown.markers import HTML_COMMENT_RE +from legaldown import ( + Directive, + Document, + PlacedMarker, + ValidationResult, + find_definition_anchors, + is_drafting_note, + lex, + list_items, + render_block, +) from markdown_it import MarkdownIt from markdown_it.tree import SyntaxTreeNode @@ -60,20 +70,7 @@ Table, Text, ) -from .validator_bridge import ( - FENCE_OPEN_RE, - MAX_QUOTE_DEPTH, - block_fragments, - closes_fence, - dedent, - indent_width, - is_drafting_note, - lex, - list_fragments, - list_items, - placed_markers, - quote_content, -) +from .validator_bridge import FENCE_OPEN_RE, MAX_QUOTE_DEPTH, closes_fence, dedent, indent_width, quote_content _OPEN, _CLOSE = "\ue000", "\ue001" # Leads the sentinel of source that renders nothing (a {{def:}}). It is @@ -177,7 +174,13 @@ def __init__(self, document: Document, result: ValidationResult) -> None: self.md = MarkdownIt("commonmark", {"html": True}) self.env: dict = {} self.payloads: list[_Payload] = [] - self.markers = placed_markers(document) + # The markers the validator placed, by (section index or None for + # the preamble, block index). + self.placed: dict[tuple[int | None, int], list[PlacedMarker]] = {} + for marker in result.placed_markers: + self.placed.setdefault((marker.section, marker.block), []).append(marker) + # Each text is lexed once in a render. + self.lex = cache(lex) self.depth = 0 # quotes and list items the builder is inside nonce = secrets.token_hex(8) # A hidden sentinel has its own form ("h"), so the lead character is @@ -202,7 +205,7 @@ def _protect(self, body: str) -> str: sentinel, so that Markdown cannot reinterpret directive syntax. A defined term becomes one sentinel and its {{def:}} another that renders nothing (see _DefinitionSpan). One pass over the directives, in order.""" - lexed = self.markers.lex(body) + lexed = self.lex(body) spans: dict[int, _DefinitionSpan] = {} for anchor in find_definition_anchors(body, language=self.language, lexed=lexed): if anchor.term is None or anchor.pair is None: @@ -351,32 +354,13 @@ def blocks(self, blocks: list[ModelBlock], section: int | None, *, markers: bool for index, block in enumerate(blocks): if block.kind == "html": continue # never emitted: a comment (§8.6) or raw HTML (§8.7) - placed = self._placed(block, section, index) if markers else {} + placed = self.placed.get((section, index), []) if markers else [] built = self.block(block, placed, top_level=top_level) if built is not None: out.append(built) return tuple(out) - def _placed(self, block: ModelBlock, section: int | None, index: int) -> dict[int, tuple[int, str, str]]: - """The markers the validator placed in *block*, by fragment index: - (offset of the marker in the fragment, identifier, condition).""" - placed: dict[int, tuple[int, str, str]] = {} - fragments = block_fragments(block) - for fragment_index, (fragment, _position) in enumerate(fragments): - found = self.markers.placed.get((section, index, fragment_index)) - if found is None: - continue - # The marker ends the first line; comments may follow it (§8.6), - # so a copy of its text inside one is not it. - first_line = HTML_COMMENT_RE.sub(lambda m: " " * len(m.group()), fragment.split("\n", 1)[0]) - offset = first_line.rfind(found.source) - if offset < 0: - raise InternalError(f"the validator placed '{found.source}' where the renderer cannot find it") - identifier = "" if found.include_only else found.marker.identifier - placed[fragment_index] = (offset, identifier, found.marker.condition) - return placed - - def block(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]], *, top_level: bool) -> Block | None: + def block(self, block: ModelBlock, placed: list[PlacedMarker], *, top_level: bool) -> Block | None: match block.kind: case "paragraph" | "definition" | "ref" | "term": block, identifier, condition = _strip_markers(block, placed) @@ -409,41 +393,34 @@ def block(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]], *, t return Paragraph((Strong(inlines),)) if inlines else None raise InternalError(f"unexpected block kind '{block.kind}' in the validator's model") - def list(self, block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> List: + def list(self, block: ModelBlock, placed: list[PlacedMarker]) -> List: """A list, nested as the validator's model holds it: items that hold - blocks. The validator numbers a list's items in document order, - nested ones included, and each fragment says which items it is in - (``list_fragments``), so a marker placed at an item's first - paragraph belongs to the innermost of them.""" - if not placed: - return self._list(block, {}, None) - fragments = list_fragments(block) - markers = {fragments[index][2][-1]: marker for index, marker in placed.items()} - return self._list(block, markers, itertools.count()) - - def _list(self, block: ModelBlock, markers: dict[int, tuple[int, str, str]], - numbers: Iterator[int] | None) -> List: - """*block*'s items, numbered from *numbers* as the validator numbers - them, to find the items *markers* belong to. None when no marker is - placed in the list, as in a quote, where the validator places none.""" + blocks. A placed marker names the item it marks by its pre-order + number among all the list's items, as the validator numbers them.""" + markers = {marker.item: marker for marker in placed} + return self._list(block, markers, count()) + + def _list(self, block: ModelBlock, markers: dict[int | None, PlacedMarker], numbers: Iterator[int]) -> List: + """*block*'s items, numbered from *numbers* in pre-order, to find the + items *markers* belong to.""" self._check_depth(1) self.depth += 1 try: items: list[ListItem] = [] for item in list_items(block): - number = next(numbers) if numbers is not None else None + marker = markers.get(next(numbers)) children = list(item.blocks) identifier = condition = "" - if number in markers: - offset, identifier, condition = markers[number] - children[0] = replace(children[0], text=_cut_marker(children[0].text, offset)) + if marker is not None: + identifier, condition = marker.identifier, marker.condition + children[0] = replace(children[0], text=_cut_marker(children[0].text, marker)) blocks: list[Block] = [] for child in children: if child.kind in ("ordered_list", "unordered_list"): blocks.append(self._list(child, markers, numbers)) elif child.kind == "html": continue # never emitted (§8.6, §8.7) - elif (built := self.block(child, {}, top_level=False)) is not None: + elif (built := self.block(child, [], top_level=False)) is not None: blocks.append(built) items.append(ListItem(blocks=tuple(blocks), anchor_id=identifier, condition=condition)) finally: @@ -506,7 +483,7 @@ def tree(self) -> RenderTree: language=self.language, preamble=self.blocks(document.preamble, None), sections=sections, - is_template=self.markers.template, + is_template=self.result.is_template, ) @@ -524,29 +501,26 @@ def _without_drafting_marker(children: list[ModelBlock]) -> list[ModelBlock] | N return ([replace(first, text=rest)] if rest else []) + children[1:] -def _strip_markers(block: ModelBlock, placed: dict[int, tuple[int, str, str]]) -> tuple[ModelBlock, str, str]: +def _strip_markers(block: ModelBlock, placed: list[PlacedMarker]) -> tuple[ModelBlock, str, str]: """*block* without its placed marker, and the marker's identifier and - condition. Fragments are numbered as block_fragments numbers them.""" - fields = [name for name in ("text", "prefix", "suffix") if getattr(block, name)] + condition. The validator says which field holds the marker.""" identifier = condition = "" - for fragment_index, (offset, marker_id, marker_condition) in placed.items(): - if fragment_index >= len(fields): - raise InternalError(f"the validator placed a marker in fragment {fragment_index} of a {block.kind} block") - name = fields[fragment_index] - block = replace(block, **{name: _cut_marker(getattr(block, name), offset)}) - identifier, condition = marker_id, marker_condition + for marker in placed: + block = replace(block, **{marker.field: _cut_marker(getattr(block, marker.field), marker)}) + identifier, condition = marker.identifier, marker.condition return block, identifier, condition -def _cut_marker(fragment: str, offset: int) -> str: - """*fragment* without the marker at *offset* on its first line; a - comment after the marker stays (it renders nothing).""" - first, newline, rest = fragment.partition("\n") - end = first.index("}", offset) + 1 +def _cut_marker(fragment: str, marker: PlacedMarker) -> str: + """*fragment* without *marker*, which ends it; a comment after the + marker stays (it renders nothing).""" + end = marker.offset + len(marker.source) + if fragment[marker.offset:end] != marker.source: + raise InternalError(f"the validator placed '{marker.source}' where the renderer cannot find it") # Only the spacing around the marker goes: text before it keeps its own # leading space (a lifted {{ref:}}'s suffix begins with one). - before, after = first[:offset].rstrip(), first[end:].strip() - return (f"{before} {after}" if after else before) + newline + rest + before, after = fragment[:marker.offset].rstrip(), fragment[end:].strip() + return f"{before} {after}" if after else before def _paragraph_source(block: ModelBlock) -> str: diff --git a/src/legaldown_render/cli.py b/src/legaldown_render/cli.py index 8a6dad3..1931fdd 100644 --- a/src/legaldown_render/cli.py +++ b/src/legaldown_render/cli.py @@ -128,7 +128,9 @@ def _report(name: str, diagnostics: list, *, quiet: bool) -> None: if quiet: return for diagnostic in diagnostics: - print(f"{name}: {diagnostic.level}: [{diagnostic.rule}] {diagnostic.message}", file=sys.stderr) + # As the validator's CLI prints them: the line when the finding has one. + where = f"{name}:{diagnostic.line}" if diagnostic.line else name + print(f"{where}: {diagnostic.level}: [{diagnostic.rule}] {diagnostic.message}", file=sys.stderr) if __name__ == "__main__": diff --git a/src/legaldown_render/resolve/resolver.py b/src/legaldown_render/resolve/resolver.py index 99351eb..105903f 100644 --- a/src/legaldown_render/resolve/resolver.py +++ b/src/legaldown_render/resolve/resolver.py @@ -25,10 +25,18 @@ from typing import Any, NamedTuple from legaldown import Diagnostic, Directive, Document, ValidationResult, slugify_identifier -from legaldown.validator import KNOWN_CURRENCIES -from legaldown.validator.conditions import ALWAYS, Presence, condition_problem, exclusive, parse_condition -from legaldown.validator.helpers import is_positive_numeric, is_valid_iso_date, is_valid_money_amount -from legaldown.validator.patterns import IDENTIFIER_RE +from legaldown.validator import ( + ALWAYS, + IDENTIFIER_RE, + KNOWN_CURRENCIES, + Presence, + condition_problem, + exclusive, + is_positive_numeric, + is_valid_iso_date, + is_valid_money_amount, + parse_condition, +) from ..build import plain_inlines from ..style import Style, effective_labels diff --git a/src/legaldown_render/validator_bridge.py b/src/legaldown_render/validator_bridge.py index 868fa28..9032d89 100644 --- a/src/legaldown_render/validator_bridge.py +++ b/src/legaldown_render/validator_bridge.py @@ -1,73 +1,28 @@ """Everything the renderer takes from ``legaldown-validator`` beyond its -top-level public API, in one place. +public API, in one place. The renderer never re-derives a LegalDown rule (docs/decisions/0002): where -the validator has decided something — where a marker is placed, whether a -document is a template — the renderer asks it. Some of those answers are not -part of the validator's public API yet, so they are imported here from its -modules. This module is the complete list of such imports, the surface that -roadmap item U3 asks the validator to make public; the dependency is pinned -to one minor version until it does. +the validator has decided something, the renderer asks it. Since +legaldown-validator 0.3.0 nearly all of it is public — where markers are +placed and whether a document is a template come with the +``ValidationResult`` (``placed_markers``, ``is_template``), and the model +helpers are exported from ``legaldown``. What is left here is how the +validator reads a block quote's content and a fenced code block, and how +its CLI reads an answers file. The dependency stays pinned to one minor +version while this list is not empty. """ from __future__ import annotations -from collections.abc import Callable -from dataclasses import dataclass -from functools import cache - -from legaldown import Document from legaldown.cli import _read_answers as read_answers -from legaldown.definitions import block_fragments, list_fragments -from legaldown.directives import Lexed, lex from legaldown.markdown import FENCE_OPEN_RE, closes_fence, dedent, indent_width -from legaldown.models import list_items from legaldown.parser import MAX_QUOTE_DEPTH, quote_content -from legaldown.validator.core import is_template -from legaldown.validator.templates import is_drafting_note -from legaldown.validator.units import FoundMarker, find_markers __all__ = [ "FENCE_OPEN_RE", "MAX_QUOTE_DEPTH", - "PlacedMarkers", - "block_fragments", "closes_fence", "dedent", "indent_width", - "is_drafting_note", - "lex", - "list_fragments", - "list_items", - "placed_markers", "quote_content", "read_answers", ] - - -@dataclass(frozen=True, slots=True) -class PlacedMarkers: - """The validator's reading of a document's markers (§5.7, §15.3).""" - - #: Whether the document is a template (§15.1): the validator's own - #: decision (``is_template``), over the same markers. - template: bool - #: The placed markers, by (section index or None for the preamble, - #: block index, fragment index in block_fragments(block)). - placed: dict[tuple[int | None, int, int], FoundMarker] - #: The lexer, cached for this one render: each text is lexed once. - lex: Callable[[str], Lexed] - - -def placed_markers(document: Document) -> PlacedMarkers: - """Which markers the validator places, and whether *document* is a - template — the two decisions the validator makes together in - ``validate_document`` (legaldown.validator.core, §15.1).""" - lexer = cache(lex) - markers = find_markers(document, lexer) - template = is_template(document, markers, lexer) - placed = { - (found.section, found.block, found.fragment): found - for found in markers - if found.marker is not None and found.placed(template) - } - return PlacedMarkers(template, placed, lexer) diff --git a/tests/conformance/test_spec_corpus.py b/tests/conformance/test_spec_corpus.py index 396481d..93dfb79 100644 --- a/tests/conformance/test_spec_corpus.py +++ b/tests/conformance/test_spec_corpus.py @@ -90,22 +90,6 @@ def test_raw_html_fixture_emits_no_html() -> None: assert "raw-html" in {d.rule for d in result.diagnostics} -@pytest.mark.parametrize("path", EXAMPLES + FIXTURES, ids=_name) -def test_template_decision_is_the_validators(path: Path) -> None: - """The renderer asks the validator's is_template over the markers it - found itself; the answer must be the one validate_document uses.""" - from conftest import validator_template - from legaldown import parse_document - - from legaldown_render.validator_bridge import placed_markers - - try: - document = parse_document(path.read_text(encoding="utf-8")) - except Exception: - pytest.skip("not a readable document") - assert placed_markers(document).template == validator_template(document) - - ASSEMBLY = sorted(path for path in (SPEC / "fixtures" / "assembly").iterdir() if path.is_dir()) \ if (SPEC / "fixtures" / "assembly").is_dir() else [] diff --git a/tests/conftest.py b/tests/conftest.py index 24bcae9..61a55d5 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -28,25 +28,3 @@ def spec_root() -> Path | None: if candidate and (Path(candidate) / "fixtures").is_dir(): return Path(candidate) return None - - -def validator_template(document: object) -> bool: - """The template decision validate_document itself makes: what its own - call to is_template returns, captured. The renderer asks is_template - too; this checks that validate_document still decides the same way.""" - from unittest import mock - - from legaldown import validate_document - from legaldown.validator import core - - seen: list[bool] = [] - original = core.is_template - - def spy(*args: object, **kwargs: object) -> bool: - seen.append(original(*args, **kwargs)) - return seen[-1] - - with mock.patch.object(core, "is_template", spy): - validate_document(document) - assert seen, "validate_document no longer calls is_template; update this helper" - return seen[-1] diff --git a/tests/golden/features.default.html b/tests/golden/features.default.html index e97dfaa..ae3af35 100644 --- a/tests/golden/features.default.html +++ b/tests/golden/features.default.html @@ -102,7 +102,8 @@

Services Agreement

-

This Services Agreement (this Agreement) is made between Acme and Beta Industries Inc. (together with its affiliates, the Client Group).

+

This Services Agreement (this Agreement) is made between Acme +and Beta Industries Inc. (together with its affiliates, the Client Group).

1. Definitions

@@ -113,7 +114,8 @@

1. 2. Services

2.1 Performance

-

The Provider shall perform the Services on each business day, and shall:

+

The Provider shall perform the Services on each +business day, and shall:

  1. (a)

    keep records of all work performed

  2. (b)

    notify the Client of any delay, including:

    @@ -131,12 +133,15 @@

    2.1

    2.2 Changes

    -

    A change under item 2.1(b)(i) or step 2.1(2) is made in writing under Section 2.1.

    +

    A change under item 2.1(b)(i) or step 2.1(2) is made in +writing under Section 2.1.

3. Fees

-

The Client shall pay $10,000.00 per month, and €1,234.50 for setup, within 30 days of July 1, 2026. The Provider's VAT number is DE123456789.

+

The Client shall pay $10,000.00 per month, and €1,234.50 +for setup, within 30 days of July 1, 2026. The Provider's VAT number is +DE123456789.

@@ -158,8 +163,13 @@

3.

4. Broken References

-

These are deliberately wrong: [BROKEN REF: nowhere], [UNDEFINED: nothing], [INVALID DATE: 2026-13-45], [INVALID AMOUNT], [UNKNOWN CURRENCY: XYZ], [UNKNOWN PARTY: nobody], [INVALID PARTY: Bad_Name], [UNKNOWN SIDE: nobody], [INVALID DURATION: -1], [INVALID DURATION UNIT: M], [INVALID FIELD: value], [UNKNOWN DIRECTIVE: trem], [UNKNOWN ATTACHMENT: schedule-z], [INVALID PLACEHOLDER: fee], and [NOT PROCESSED: include includes/extra.lgd].

-

Raw HTML is dropped. A safe link stays; [an unsafe one](javascript:alert(1)) does not.

+

These are deliberately wrong: [BROKEN REF: nowhere], [UNDEFINED: nothing], [INVALID DATE: 2026-13-45], +[INVALID AMOUNT], [UNKNOWN CURRENCY: XYZ], [UNKNOWN PARTY: nobody], +[INVALID PARTY: Bad_Name], [UNKNOWN SIDE: nobody], [INVALID DURATION: -1], [INVALID DURATION UNIT: M], +[INVALID FIELD: value], [UNKNOWN DIRECTIVE: trem], [UNKNOWN ATTACHMENT: schedule-z], [INVALID PLACEHOLDER: fee], +and [NOT PROCESSED: include includes/extra.lgd].

+

Raw HTML is dropped. A safe link stays; +[an unsafe one](javascript:alert(1)) does not.

Attachments

diff --git a/tests/test_cli.py b/tests/test_cli.py index c735a81..aad5ff3 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -14,7 +14,10 @@ def test_render_to_file_infers_the_format(tmp_path: Path, capsys: pytest.Capture out = tmp_path / "out.txt" assert main([FEATURES, "-o", str(out)]) == 0 assert out.read_text().startswith("Services Agreement") - assert "[ref-broken]" in capsys.readouterr().err + err = capsys.readouterr().err + assert "[ref-broken]" in err + # A validator finding names its line, as the validator's CLI prints it. + assert f"{FEATURES}:100: warning: [raw-html]" in err def test_style_set_and_locale(capsys: pytest.CaptureFixture[str]) -> None: diff --git a/tests/test_render.py b/tests/test_render.py index 41cc352..14ecc0e 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -33,6 +33,10 @@ def text(body: str, **settings: object) -> str: return render(FRONT + body, format="text", **settings).output +def html(body: str, **settings: object) -> str: + return render(FRONT + body, format="html", **settings).output + + def features(**settings: object) -> str: return render((DOCUMENTS / "features.lgd").read_text(encoding="utf-8"), **settings).output @@ -1118,12 +1122,10 @@ def test_an_unclosed_comment_inside_a_paragraph_is_text() -> None: assert "Shown." in output -def test_a_hard_break_keeps_both_lines() -> None: - # How the break shows depends on the validator's parser - # (ForLegalAI/legaldown-validator#25); only the text is pinned. - output = text("# A\n\nLine one\\\nline two\n") - assert "Line one" in output - assert "line two" in output +def test_hard_breaks_break_the_line_and_soft_breaks_do_not() -> None: + source = "# A\n\nLine one\\\nline two \nline three\nline four\n" + assert "Line one\nline two\nline three line four\n" in text(source) + assert "Line one
\nline two
\nline three\nline four

" in html(source) def test_a_comment_opener_in_code_or_a_directive_does_not_open() -> None: @@ -1151,14 +1153,3 @@ def test_a_dropped_comment_or_tag_leaves_one_space() -> None: def test_table_rows_are_as_wide_as_the_header() -> None: output = text("# A\n\n| a | b |\n|---|---|\n| 1 | 2 | 3 |\n| x |\n") assert "| 1 | 2 |\n| x | |" in output - - -def test_template_decision_matches_the_validator_on_the_test_documents() -> None: - from conftest import validator_template - from legaldown import parse_document - - from legaldown_render.validator_bridge import placed_markers - - for name in ("features", "template"): - document = parse_document((DOCUMENTS / f"{name}.lgd").read_text(encoding="utf-8")) - assert placed_markers(document).template == validator_template(document) From dd0aeba4d40ec005cb9980d65de1f1bbb70b2810 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 29 Sep 2026 21:06:44 +0000 Subject: [PATCH 19/19] Prepare the 0.2.0 release and publish to PyPI - Version 0.2.0. - .github/workflows/publish.yml, as legaldown-validator publishes: on a published GitHub Release, build the sdist and wheel, twine check, verify the tag matches __version__, smoke-test the wheel with a real render, and upload to PyPI with Trusted Publishing; a manual run uploads to TestPyPI. - .github/PUBLISHING.md: the one-time PyPI and GitHub setup, and how to cut a release. - README: install with pip install legaldown-render. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011ZFB1eaR4Fv8vUr2y8RXQt --- .github/PUBLISHING.md | 81 +++++++++++++++++++ .github/workflows/publish.yml | 120 +++++++++++++++++++++++++++++ README.md | 6 +- src/legaldown_render/__init__.py | 2 +- tests/golden/features.default.html | 2 +- tests/golden/template.default.html | 2 +- 6 files changed, 207 insertions(+), 6 deletions(-) create mode 100644 .github/PUBLISHING.md create mode 100644 .github/workflows/publish.yml diff --git a/.github/PUBLISHING.md b/.github/PUBLISHING.md new file mode 100644 index 0000000..aa0adc4 --- /dev/null +++ b/.github/PUBLISHING.md @@ -0,0 +1,81 @@ +# Publishing (maintainers) + +Release mechanics for this repository. Nothing here is needed to *use* the +renderer — see the [README](../README.md) for that. + +`legaldown-render` is published to PyPI by the [`publish`](workflows/publish.yml) +workflow using **Trusted Publishing** (OIDC). No API token is stored in this repository — PyPI +verifies the identity of the workflow run itself. + +## One-time PyPI setup + +This must exist before the first publish, and it must be done by a PyPI account that will own the +project. Because the project does not exist on PyPI yet, register it as a **pending** publisher. + +1. Sign in to → **Your account** → **Publishing** → + *Add a new pending publisher*. +2. Fill in exactly: + + | Field | Value | + |---|---| + | PyPI project name | `legaldown-render` | + | Owner | `ForLegalAI` | + | Repository name | `legaldown-render` | + | Workflow name | `publish.yml` | + | Environment name | `pypi` | + +3. Repeat on with environment name `testpypi` if you want the dry-run path. + +The environment names must match, or PyPI rejects the upload. + +## One-time GitHub setup + +Create the two environments under **Settings → Environments**: + +| Environment | Purpose | Suggested protection | +|---|---|---| +| `pypi` | Real releases | Required reviewers, and restrict to the `main` branch / tags | +| `testpypi` | Dry runs | None needed | + +Adding a required reviewer to `pypi` means a human approves each upload — worth it, since a PyPI +version number can never be reused once published. + +## Cutting a release + +1. Bump `__version__` in [`src/legaldown_render/__init__.py`](../src/legaldown_render/__init__.py) — it is the single + source of truth; `pyproject.toml` reads it via `[tool.hatch.version]`. +2. Commit, then publish a GitHub Release with the tag `vX.Y.Z`. Its notes are where this + project records what changed in a release. + +The workflow builds the sdist and wheel, runs `twine check --strict`, **verifies the tag matches +`__version__`** (a mismatch fails the run rather than publishing the wrong version), smoke-tests the +built wheel by installing it and invoking the CLI, and uploads to PyPI. + +## Dry run + +Before a first real release, exercise the whole path against TestPyPI: + +``` +Actions → Publish → Run workflow → target: testpypi +``` + +Then verify the result installs cleanly: + +```bash +pip install --index-url https://test.pypi.org/simple/ \ + --extra-index-url https://pypi.org/simple/ legaldown-render +legaldown-render --version +``` + +The extra index is needed because the dependencies (`legaldown-validator`, Babel, PyYAML, +markdown-it-py) come from PyPI. + +## Notes + +- **Version numbers are permanent.** PyPI does not allow re-uploading a version, even after + deletion. Use TestPyPI, or a `.devN`/`.rcN` suffix, to rehearse. +- The distribution is named `legaldown-render`; the import name is `legaldown_render` + (`pip install legaldown-render` → `import legaldown_render`). +- A release of `legaldown-validator` with a new minor version needs a renderer release too: the + dependency is pinned to one minor version (`pyproject.toml`). +- The package ships a `py.typed` marker, so type checkers use its annotations directly. diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..42f9a3d --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,120 @@ +name: Publish + +# Publishes the distribution to PyPI via Trusted Publishing (OIDC) — no API +# token is stored in this repository. See .github/PUBLISHING.md for the one-time PyPI +# setup that must exist before the first run. +# +# Triggers: +# - publishing a GitHub Release → build, verify, upload to PyPI +# - manual dispatch → build, verify, upload to TestPyPI + +on: + release: + types: [published] + workflow_dispatch: + inputs: + target: + description: "Where to upload the built distribution" + required: true + default: testpypi + type: choice + options: [testpypi, pypi] + +permissions: + contents: read + +concurrency: + group: publish-${{ github.ref }} + cancel-in-progress: false + +jobs: + build: + name: Build and verify distribution + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + cache: pip + + - name: Install build tooling + run: pip install build twine + + - name: Build sdist and wheel + run: python -m build + + - name: Check package metadata renders on PyPI + run: twine check --strict dist/* + + - name: Verify the version matches the release tag + if: github.event_name == 'release' + env: + TAG: ${{ github.event.release.tag_name }} + run: | + python - <<'PY' + import os, pathlib, re, sys + + tag = os.environ["TAG"].lstrip("v") + init = pathlib.Path("src/legaldown_render/__init__.py").read_text(encoding="utf-8") + version = re.search(r'__version__ = "([^"]+)"', init).group(1) + if version != tag: + sys.exit( + f"::error::Release tag {tag!r} does not match __version__ {version!r}. " + "Bump src/legaldown_render/__init__.py before tagging." + ) + print(f"version {version} matches tag {tag}") + PY + + - name: Smoke-test the built wheel + run: | + python -m venv /tmp/smoke + /tmp/smoke/bin/pip install --quiet dist/*.whl + /tmp/smoke/bin/legaldown-render --version + /tmp/smoke/bin/python -c "import legaldown_render as r; print(r.__version__, r.SPEC_VERSION, r.CONFORMANCE_LEVEL)" + # A real render: the built-in styles, labels, and locale data are in the wheel. + printf -- '---\ntitle: T\nlanguage: cs\n---\n\n# A\n\nText {{date: 2026-07-01}}.\n' > /tmp/smoke/t.lgd + /tmp/smoke/bin/legaldown-render /tmp/smoke/t.lgd -f text --style continental -o /tmp/smoke/t.txt + cat /tmp/smoke/t.txt + + - uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + testpypi: + name: Upload to TestPyPI + needs: build + if: github.event_name == 'workflow_dispatch' && inputs.target == 'testpypi' + runs-on: ubuntu-latest + environment: + name: testpypi + url: https://test.pypi.org/p/legaldown-render + permissions: + id-token: write # required for Trusted Publishing + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + - uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + + pypi: + name: Upload to PyPI + needs: build + if: github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && inputs.target == 'pypi') + runs-on: ubuntu-latest + environment: + name: pypi + url: https://pypi.org/p/legaldown-render + permissions: + id-token: write # required for Trusted Publishing + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + - uses: pypa/gh-action-pypi-publish@release/v1 diff --git a/README.md b/README.md index e834a14..3cb878a 100644 --- a/README.md +++ b/README.md @@ -45,10 +45,8 @@ This package never reimplements the LegalDown grammar. ## Install -`legaldown-render` is not on PyPI yet. Until its first release, install it from GitHub: - ```bash -pip install "legaldown-render @ git+https://github.com/ForLegalAI/legaldown-render" +pip install legaldown-render ``` Python 3.11 or newer. The dependencies are all pure Python: `legaldown-validator`, @@ -134,6 +132,8 @@ LEGALDOWN_SPEC_DIR=../LegalDown pytest -m conformance # the specification's ex ruff check . ``` +Releases go to PyPI from a GitHub Release; see [PUBLISHING.md](.github/PUBLISHING.md). + ## License [MIT](LICENSE) diff --git a/src/legaldown_render/__init__.py b/src/legaldown_render/__init__.py index 48adf90..2342fc3 100644 --- a/src/legaldown_render/__init__.py +++ b/src/legaldown_render/__init__.py @@ -21,7 +21,7 @@ from .errors import DocumentError, InternalError, RenderError, RenderRefused from .style import Style, StyleError, builtin_styles, dump_style, load_style -__version__ = "0.1.0" +__version__ = "0.2.0" #: The LegalDown specification version this renderer targets. SPEC_VERSION = "0.2" diff --git a/tests/golden/features.default.html b/tests/golden/features.default.html index ae3af35..f3ba52a 100644 --- a/tests/golden/features.default.html +++ b/tests/golden/features.default.html @@ -4,7 +4,7 @@ Services Agreement - +

Service