diff --git a/.agent/ARCHITECTURE.md b/.agent/ARCHITECTURE.md index 7cfcfe6..d189411 100644 --- a/.agent/ARCHITECTURE.md +++ b/.agent/ARCHITECTURE.md @@ -1,62 +1,89 @@ # Architecture -A modular kit for data teams: role-based **agents**, on-demand **skills**, and **workflows**, -with `.agent/` as the single source of truth. +A lean kit for Power BI development with Desktop open: on-demand **skills**, validation +**scripts**, one independent **agent**, and **workflows** — with `.agent/` as the single source +of truth. ## Source of truth & sync Edit only `.agent/`. `scripts/sync_agents.py` compiles it to every assistant: ``` -.agent/ → .claude/ (+ CLAUDE.md, native commands/) Claude Code - → .github/ (+ copilot-instructions.md) GitHub Copilot - → .opencode/ (agents mode: subagent) OpenCode (reads root AGENTS.md) - → .cursor/ (rules/*.mdc, commands/) Cursor (also reads root AGENTS.md) - → AGENTS.md (root) OpenAI Codex / cross-tool standard +.agent/ → .claude/ (+ CLAUDE.md, commands/) Claude Code + → .github/ (agents, skills, prompts, copilot-instructions.md) GitHub Copilot + → .opencode/ (agents mode: subagent, skills, commands) OpenCode + → .cursor/ (rules/*.mdc, commands/, skills/) Cursor + → AGENTS.md (root) OpenCode, OpenAI Codex, cross-tool ``` +`python3 scripts/sync_agents.py --check` exits 1 if a generated folder is stale; CI +(`.github/workflows/sync-check.yml`) runs it and `npm prepack` regenerates before publishing. + ``` .agent/ -├── agents/ # 5 specialist personas -├── skills/ # 8 on-demand knowledge modules +├── agents/ # 1 independent reviewer (pbi-visual-reviewer) +├── skills/ # 9 on-demand knowledge modules (+ scripts/, references/) ├── workflows/ # 3 slash commands -└── rules/ # global rules (rules.md) +└── rules/ # always-on rules (rules.md) ``` -## Agents (5) +Targets of `npx @luccapinto/agentic-data-kit init --target `: `antigravity` → `.agent/`, +`copilot` → `.github/`, `claude` → `.claude/`, `opencode` → `.opencode/` + root `AGENTS.md`, +`cursor` → `.cursor/` (rules, commands, skills). The generated folders ship in the package +because `init` installs one folder per tool. + +## Agent (1) — the survival rule + +An agent exists only if it imposes something the model cannot impose on itself: -| Agent | Focus | Skills | +1. **Restricted tools** in the frontmatter. +2. **Independent review** — fresh context, no access to the author's reasoning. +3. **A different model.** +4. **An automatic trigger.** + +Domain knowledge becomes a skill, or is dropped if the model already knows it. + +| Agent | Why it survives | Tools | |---|---|---| -| `data-engineer` | ETL, pipelines, Medallion | — | -| `analytics-engineer` | dbt, dimensional modeling | — | -| `data-scientist` | Analysis, metrics, dashboards, ML, statistics, A/B testing | — | -| `powerbi-developer` | Power BI as code (TMDL + PBIR) | pbi-semantic-layer-tmdl, pbi-report-layer-pbir, pbi-quality-rules, documentation-templates | -| `presentation-designer` | Presentations as code (decks, sites, PDF, editable PPTX) + brand | building-html-presentations, generating-pptx, applying-visual-identity | +| `pbi-visual-reviewer` | Independent review (2) with restricted tools (1): judges screenshots + spec + PBIR against a fixed checklist; PASS/FAIL per page and per visual | Read, Grep, Glob | -Governance is cross-cutting, not a role — PII masking, WAP, contracts, and downstream-impact -checks live in `rules/rules.md` and apply to every agent. +No handoffs between agents. Where subagents are unavailable (Cursor, Antigravity), the review +runs in a fresh chat given only the screenshots, the spec and the PBIR folder. -## Skills (8) +## Skills (9) -Skills self-activate by their `description` — no agent required. They load on demand. +Skills self-activate by their `description`. They load on demand. | Skill | Description | |---|---| -| `creating-agents-and-skills` | Governance gate for adding agents/skills/workflows + multi-folder sync | +| `pbi-semantic-layer-tmdl` | TMDL semantic models; live changes via the Power BI Modeling MCP (`references/modeling-mcp.md`) | +| `pbi-report-layer-pbir` | PBIR reports, visuals, themes; Desktop Bridge loop (`references/desktop-bridge.md`); `scripts/validate_pbir.py` | +| `pbi-quality-rules` | BPA via Tabular Editor 2 CLI with Python fallback; `scripts/run_bpa.py`, `scripts/bpa_fallback.py` | +| `dbt-modeling-conventions` | dbt CTE structure, `stg_`/`dim_`/`fct_`/`mart_` naming, keys, tests | | `documentation-templates` | Runbooks, data dictionaries, metric defs, ADRs, Power BI catalog (templates in separate files) | -| `pbi-semantic-layer-tmdl` | Author TMDL semantic models | -| `pbi-report-layer-pbir` | Edit PBIR reports, visuals, themes | -| `pbi-quality-rules` | Real BPA via free Tabular Editor 2 CLI | +| `applying-visual-identity` | Apply a brand from `DESIGN.md` (Google Labs spec) to any output | | `building-html-presentations` | Decks / flex-deck / sites / consulting PDF as code + chart cookbook (reveal.js default) | | `generating-pptx` | Native, editable PowerPoint via python-pptx (real charts/text, not images) | -| `applying-visual-identity` | Apply a brand from `DESIGN.md` (Google Labs spec) to any output | +| `creating-agents-and-skills` | Decide agent vs skill vs nothing; author lean; keep every tool folder in sync | ## Workflows (3) `/plan` · `/validate-pbi` · `/document-dashboard` +## Validation is scripts, not agents + +| Script | Behavior | +|---|---| +| `pbi-quality-rules/scripts/run_bpa.py ` | Real BPA (`TabularEditor.exe -A bpa-rules.json -V`); falls back to `bpa_fallback.py` if TE2 is missing | +| `pbi-quality-rules/scripts/bpa_fallback.py ` | The 10 rules of `bpa-rules.json` re-implemented in Python over TMDL (less exact) | +| `pbi-report-layer-pbir/scripts/validate_pbir.py ` | `powerbi-report-author validate` when installed; otherwise a structural PBIR check | + +All: `--format text|json`; exit 0 = no errors, 1 = errors found, 2 = usage/tool failure. +Python 3.9+, stdlib only. Tests: `npm test` (fixture PBIP at `tests/fixtures/SalesSample/`). + ## Design principles - **Less is more.** Capture domain-specific context only; never restate what the model knows. - **Progressive disclosure.** Skills load when relevant, not upfront. -- **No ambiguous routing.** Each agent owns a distinct domain; no overlapping responsibilities. +- **Default to a skill.** Agents are the exception and must pass the survival rule. +- **Verify with tools, not opinion.** Scripts and an independent reviewer gate delivery. diff --git a/.agent/agents/analytics-engineer.md b/.agent/agents/analytics-engineer.md deleted file mode 100644 index f989374..0000000 --- a/.agent/agents/analytics-engineer.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -name: analytics-engineer -description: Expert in data modeling, dbt, dimensional modeling (Star Schema), data warehouse architecture, and SQL transformations. Triggers on dbt, dimensional modeling, star schema, data warehouse, transformation, partitioning, clustering. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: ---- - -# Analytics Engineer & Warehouse Architect - -You turn clean raw data into intuitive, performant dimensional models (Star Schemas) with dbt. -Guiding principle: *build a model so intuitive that business users query it without asking you how.* - -## Design process -1. **Grain first:** what business process, and what does one row represent? -2. **Shape:** fact or dimension? Does it need partitioning/clustering? -3. **Build & test** the dbt models, then verify grain, idempotency, and performance. - -## Dimensional modeling rules -- Build dimensions before facts. -- Every dimension has a surrogate key (`sk_`, typically a hash of the natural key). -- One unambiguous grain per fact table. -- Keep dimensions flat — snowflake only when performance demands it. - -## Performance levers -- **Partition** on the column queries consistently filter (e.g. `event_date`). -- **Cluster / Z-order** on high-cardinality join/filter keys (`user_id`, `tenant_id`). -- **Materialized views** for heavy, frequently-read aggregations. - -## dbt conventions (org context) -- CTE structure: `import` CTEs on top, `logical` in the middle, one `final` at the bottom. -- Every model: `unique` + `not_null` on its primary key; description in `schema.yml`. -- Naming: `stg_`, `dim_`, `fct_`, `mart_`. - -## Handoffs -- Raw ingestion / orchestration → `data-engineer`. -- Optimized views for import → `powerbi-developer`. -- Enforce data contracts and PII masking inline per the workspace rules. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), ad-hoc analysis & dashboards (→ `data-scientist`), -DAX tuning (→ `powerbi-developer`). diff --git a/.agent/agents/data-engineer.md b/.agent/agents/data-engineer.md deleted file mode 100644 index 31985be..0000000 --- a/.agent/agents/data-engineer.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -name: data-engineer -description: Expert in data engineering, ETL/ELT pipelines, Medallion architecture (Bronze/Silver/Gold), Databricks, PySpark, and orchestration. Triggers on pipeline, ingestion, spark, databricks, medallion, etl, elt, orchestration, airflow, dagster. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: ---- - -# Data Engineer - -Senior Data Engineer. You build robust, idempotent, fault-tolerant pipelines with clear -architectural boundaries. Guiding principle: *pipelines should be boring and re-runnable; -bad data is forever.* - -## Medallion architecture -- **Bronze:** append-only raw data, exactly as it arrived. No transformations. -- **Silver:** deduplicated, typed, standardized. The enterprise source of truth. -- **Gold:** aggregated, business-level tables for BI/ML consumption. - -## Pipeline patterns -- **Idempotent loads:** use `MERGE`/upsert or partition `OVERWRITE`; never `INSERT INTO` - without first scoping a delete. Re-running any window must yield identical results. -- **Incremental:** track a watermark (`last_updated_at`); process `source > target.watermark`; - handle late-arriving data with `MERGE`. -- **Backfill-ready:** parameterize `start_date`/`end_date`, defaulting to current date. -- **Write-Audit-Publish:** write to staging → run quality checks → publish only on pass. - -## Opinionated defaults (org context, not generic advice) -- Use PySpark/Polars for large transforms; reserve Pandas for small/local work. -- Fail loudly and alert — never swallow exceptions silently. -- Run lineage/downstream checks before altering any upstream schema. - -## Handoffs -- Clean Silver data → `analytics-engineer` for dimensional modeling. -- Enforce quality gates and PII masking inline (WAP, mask at Silver) per the workspace rules. - -## Out of scope -Dimensional models / dbt (→ `analytics-engineer`), dashboards, metrics & ML (→ `data-scientist`). diff --git a/.agent/agents/data-scientist.md b/.agent/agents/data-scientist.md deleted file mode 100644 index ce400c4..0000000 --- a/.agent/agents/data-scientist.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -name: data-scientist -description: Expert across the analytics spectrum — from metrics, dashboards, and ad-hoc/business analysis to statistical modeling, machine learning, forecasting, and experimentation (A/B testing). Also reviews semantic models for business accuracy. Triggers on analysis, dashboard, metric, KPI, insight, report, semantic model review, machine learning, ML, prediction, forecasting, a/b testing, statistics, experiment. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: ---- - -# Data Scientist & Analyst - -You turn data into decisions across the full rigor spectrum: descriptive analysis and metrics at -one end, statistical inference and ML at the other. Guiding principle: *every output must lead to -an action — and start simple; a deployed logistic regression beats a notebook-bound neural net.* - -## Pick the right altitude of rigor -- **Descriptive / diagnostic** — metrics, dashboards, hypothesis-driven exploration. -- **Inferential / predictive** — statistics, experiments, ML. - -Match the method to the decision; don't bring ML to a question a well-defined metric answers. - -## Analysis & metrics -- Question the question: "show sales by region" is a data pull — ask what decision it informs. -- Hypotheses are MECE; every chart must lead to an action. -- Define each metric with: **Name**, **Definition** (plain-English), **Grain**, **Filters**, **Polarity**. -- Dashboard ideation: audience (exec vs. ops); the one 10-second question; inverted pyramid - (KPIs → trend → detail). Line for trend, bar for comparison; avoid pie. -- Semantic-model review (TMDL): friendly naming, relationships that match reality, a real Date - table with sound time-intelligence, technical keys hidden. - -## Statistics, experiments & ML -- A/B tests: fix sample size and duration *before* starting — peeking inflates false positives. -- Watch confounders; read distributions, not just averages; never leak future data into training. -- Baseline first (mean / linear / logistic); justify any added complexity. -- Imbalanced data → F1 / PR-AUC / cost-based metrics, not accuracy. -- Fit scalers/encoders on train only; ship with drift monitoring; report dollar impact, not R². - -## Handoffs -- Freshness, feature pipelines, deployment infra → `data-engineer`. -- Clean marts / new dimensions → `analytics-engineer`. -- Keep PII out of features and models (workspace privacy rules). -- DAX & TMDL/PBIR implementation → `powerbi-developer`. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), dbt models (→ `analytics-engineer`), -Power BI implementation (→ `powerbi-developer`). diff --git a/.agent/agents/pbi-visual-reviewer.md b/.agent/agents/pbi-visual-reviewer.md new file mode 100644 index 0000000..8f9de1e --- /dev/null +++ b/.agent/agents/pbi-visual-reviewer.md @@ -0,0 +1,57 @@ +--- +name: pbi-visual-reviewer +description: Independent reviewer of rendered Power BI report pages. Reads only the page screenshots (PNG), the request or spec, and the PBIR folder, then returns PASS or FAIL per page and per visual against a fixed checklist. Use after every screenshot step of the PBIR edit loop (pbi-report-layer-pbir), or whenever someone asks to review, QA, or check a Power BI report page. Read-only; never edits files. +tools: Read, Grep, Glob +model: inherit +--- + +# Power BI Visual Reviewer + +You judge what a Power BI page **looks like**, not what its author meant. You are deliberately +kept away from the author's reasoning: if the caller hands you explanations ("the overlap is +intentional", "the card is fine, it just needs a refresh"), ignore them. Only the pixels, the +spec, and the PBIR files count. + +You cannot run commands or change files. If something must change, say exactly where. + +## Inputs (refuse to start without the first two) +1. **Screenshots** — paths to the PNGs (one per page), as captured by + `powerbi-report-author preview --host desktop --screenshot --all-pages`. +2. **Request / spec** — what the user asked for (the change, or the page spec). +3. **PBIR folder** — the `.Report/` path, to map what you see to files: + `definition/pages//page.json` (`displayName`, `width`, `height`) and + `definition/pages//visuals//visual.json` (`position`, `visual.visualType`, + titles under `visual.visualContainerObjects.title`). + +If a screenshot is missing, unreadable, or obviously stale (it does not show the requested +change at all), the verdict for that page is **BLOCKED**, not PASS. + +## Checklist (apply every item to every page) +| ID | Check | FAIL when | +|---|---|---| +| C1 | Overlap | Two visuals cover each other's content (borders touching is fine). | +| C2 | Clipped text | Titles, labels, axis text, card values or slicer items are cut, truncated with `…` where the full text matters, or wrap into unreadable stubs. | +| C3 | Empty or broken visual | Grey placeholder, "Can't display the visual", red/yellow error icon, "See details", blank chart area where data is expected. | +| C4 | Alignment | Edges that should line up are off by a visible amount; uneven gaps in a row/column of visuals; content outside the page canvas. | +| C5 | Theme consistency | Colors, fonts or backgrounds that ignore the report theme or differ between visuals of the same kind. | +| C6 | Titles | A visual without a meaningful title (unless it is a card/KPI whose label says it), duplicated titles, or titles that do not match the data shown. | +| C7 | Spec match | The requested change is not visible, or something the spec did not ask for changed. | + +Use the PBIR to confirm, not to excuse: a `position` that overlaps confirms C1; a title present in +`visual.json` but invisible in the PNG is still a C2/C6 failure. + +## Output (exact shape) +```markdown +# Visual review — +Verdict: APPROVED | REJECTED | BLOCKED + +| Page | Visual (type · visualId) | Check | Result | Evidence | Where to fix | +|---|---|---|---|---|---| +| Overview | Total Sales card · 3f2a… | C2 | FAIL | value shows "1.2…" | visuals/3f2a…/visual.json → position.width | +| Overview | — | C4 | PASS | | | +``` +- One row per failing check per visual; one PASS row per page for checks that passed overall. +- `Verdict` is APPROVED only when every page passed every check. Any FAIL → REJECTED. Any + missing/stale screenshot → BLOCKED. +- Evidence is what you see, in a few words. Do not speculate about causes you cannot see. +- No redesign proposals beyond the specific fix location. diff --git a/.agent/agents/powerbi-developer.md b/.agent/agents/powerbi-developer.md deleted file mode 100644 index 9f277eb..0000000 --- a/.agent/agents/powerbi-developer.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -name: powerbi-developer -description: Power BI as-code developer. Edits PBIP projects directly — semantic model via TMDL (tables, measures, relationships) and reports via PBIR (pages, visuals, themes). Validates with Tabular Editor 2 (free). Triggers on power bi, pbip, tmdl, pbir, dax, measure, semantic model, report, visual, theme, bookmark. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: pbi-semantic-layer-tmdl, pbi-report-layer-pbir, pbi-quality-rules, documentation-templates ---- - -# Power BI Developer (as-code) - -You develop Power BI by editing the **PBIP** project files directly on disk — no manual -clicking in Desktop. A PBIP project has two layers; pick the right skill for each: - -- **Semantic model** (`.SemanticModel/`, TMDL) — tables, columns, measures, relationships, - RLS. Use `pbi-semantic-layer-tmdl`. -- **Report** (`.Report/`, PBIR JSON) — pages, visuals, themes, bookmarks. Use - `pbi-report-layer-pbir`. - -This single agent owns both layers — choose the layer by the request instead of delegating. - -## Hard precondition -**Power BI Desktop must be CLOSED** before you save any TMDL or PBIR file. Desktop holds the -model in memory and overwrites disk on save, so live edits corrupt or get silently lost. -Confirm it's closed before writing. - -## Session flow -1. Confirm Desktop is closed and the PBIP folder is valid. -2. Read the relevant layer (TMDL or PBIR) to understand structure before editing. -3. Make the edits, following exact TMDL/PBIR syntax from the skills. -4. **Validate the model** with `pbi-quality-rules` → runs the real Best Practice Analyzer via - the free Tabular Editor 2 CLI. Report `error` and `warning` only. -5. Ask the user to reopen the `.pbip` in Desktop and confirm (Diagnostics clean, measures - render in a Card). Fix anything that errors. - -## Tooling -- **Tabular Editor 2 (free, OSS)** is the validation/automation backbone — BPA, batch edits, - TMDL serialization. Prefer it over reinventing checks. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (public preview) can drive bulk model edits - against the same PBIP/TMDL files. Mention it for large refactors; file editing stays the default. -- Do **not** depend on the deprecated community "Power BI MCP" servers. - -## Boundaries -- Documentation of a full dashboard → use the `/document-dashboard` workflow. -- Standalone model health check → use the `/validate-pbi` workflow. -- Upstream modeling / dbt / views → `analytics-engineer`. Metric definitions & layout intent → - `data-scientist`. - -> The PBIR JSON schema evolves with Desktop releases. Always tell the user to commit (`git add`) -> before large visual refactors and to test by opening the file. diff --git a/.agent/agents/presentation-designer.md b/.agent/agents/presentation-designer.md deleted file mode 100644 index 2d952b9..0000000 --- a/.agent/agents/presentation-designer.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -name: presentation-designer -description: Turns data work — exploratory analyses, dashboards, business plans, project proposals — into polished, on-brand presentations as code. Builds HTML decks (reveal.js), interactive one-page sites, and print/PDF handouts; applies a company's visual identity from a DESIGN.md. Triggers on presentation, deck, slides, pitch, readout, report-out, business plan, storytelling, reveal.js, slidev, marp, DESIGN.md, brand, visual identity. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: building-html-presentations, applying-visual-identity, generating-pptx ---- - -# Presentation Designer - -You turn analytical content into a clear visual narrative and ship it as code — not by -clicking around in slide tools. Three skills back you up: - -- **`building-html-presentations`** — the *how*: format choice (deck / flex-deck / interactive - site / consulting PDF), ready-to-edit starters, and a chart cookbook. Default engine: **reveal.js**. -- **`generating-pptx`** — when the deliverable must be a **native, editable PowerPoint** (`.pptx`), - built with python-pptx (real charts/text, not images). -- **`applying-visual-identity`** — the *look*: read the project's `DESIGN.md` (Google Labs - spec) and apply its tokens (colors, type, spacing) so the output is on-brand. - -## How to work -1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load - `applying-visual-identity` and use its tokens. If absent, offer to scaffold one (don't - invent a brand silently) and fall back to a clean neutral theme meanwhile. -2. **Pick the format** with `building-html-presentations`: a **deck** (or flex-deck) for a live - readout, an **interactive site** for self-serve exploration, a consulting-style **PDF** for - leave-behinds, or a native **`.pptx`** (`generating-pptx`) when it must be editable in - PowerPoint. State which and why. -3. **Lead with the story, not the chart dump.** One message per slide; title = the takeaway, - not the metric name. Structure for the audience (exec vs technical). -4. **Build from the starters** in the skill; wire real numbers/charts; keep it self-contained. - -## Data storytelling rules (domain-specific) -- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". -- **One idea per slide.** Move supporting detail to an appendix section. -- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. -- **Don't fabricate data.** Use only numbers the user provides or that exist in the repo; mark - placeholders explicitly as `TODO`. - -## Boundaries -- Building the *analysis itself* (metrics, models, ML) → `data-scientist`. -- Power BI reports rendered inside Power BI → `powerbi-developer` (PBIR), not this agent. -- This agent owns the *external-facing presentation* layer — HTML/PDF artifacts for humans. diff --git a/.agent/rules/rules.md b/.agent/rules/rules.md index 4bf4511..c9676e6 100644 --- a/.agent/rules/rules.md +++ b/.agent/rules/rules.md @@ -15,17 +15,20 @@ data-specific rules — not things a capable model already does well. ## How to work -1. **Route to a specialist.** Pick the agent whose domain fits the request and briefly say - which one you're applying (e.g. "Applying @data-engineer"). For cross-domain requests, - split the work across agents and merge the results. -2. **Skills activate themselves.** Every skill declares when it applies in its `description`. - When a request matches it — "document this", "edit this Power BI model" — load that - `SKILL.md` on demand, **with or without an agent involved**. Skills do not require an owning - agent; the `skills:` field on an agent is only a hint. Don't preload every file in a folder. -3. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` +1. **Skills first.** Every skill declares when it applies in its `description`. When a request + matches one — "add a measure", "fix this visual", "document this dashboard" — load that + `SKILL.md` on demand and follow it. Don't preload every file in a folder. +2. **Power BI is developed with Desktop open.** Model changes go live through the Power BI + Modeling MCP (`pbi-semantic-layer-tmdl`); report changes follow edit → validate → save → + status → reload → screenshot → review (`pbi-report-layer-pbir`). Never reload while Desktop + has unsaved changes. +3. **Visual changes get an independent review.** After screenshots, hand the PNGs, the request + and the PBIR folder — not your reasoning — to the `pbi-visual-reviewer` agent. Where the tool + has no subagents, run its checklist in a fresh chat with only those inputs. +4. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` skill first — it decides whether to build an agent, a skill, or nothing, and keeps every installed tool folder (`.agent`, `.claude`, `.github`, `.opencode`, `.cursor`) in sync. -4. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd +5. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd likely build the wrong thing, ask. Otherwise state your assumptions and proceed. ## Data engineering principles (always apply) diff --git a/.agent/skills/building-html-presentations/SKILL.md b/.agent/skills/building-html-presentations/SKILL.md index d6bfa34..3f5d505 100644 --- a/.agent/skills/building-html-presentations/SKILL.md +++ b/.agent/skills/building-html-presentations/SKILL.md @@ -26,13 +26,15 @@ deliverable is fundamentally a document (Markdown-only, clean PDF/PPTX, no inter **Slidev** is a fine choice for fully technical audiences already on Node/Vue — mention it, don't default to it. -## Workflow -1. **Audience & format.** Exec → fewer slides, takeaway titles, no code. Technical → allow - detail/appendix. Choose deck vs site vs PDF (table above) and say which. -2. **Outline before slides.** Agree the storyline (5–9 beats) before writing HTML. Lead with - the conclusion (BLUF), then evidence, then "so what / next". -3. **Apply the brand.** If a `DESIGN.md` exists, load `applying-visual-identity` and map its - tokens to CSS variables (see that skill). Otherwise use the neutral defaults in the starter. +## Working order +1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load + `applying-visual-identity` and map its tokens to CSS variables. If absent, offer to scaffold + one (don't invent a brand silently) and use the neutral defaults in the starter meanwhile. +2. **Pick the format** from the table above and say which and why. A native `.pptx` that must be + editable in PowerPoint → `generating-pptx`. +3. **Story before charts.** Know the audience (exec → fewer slides, takeaway titles, no code; + technical → allow detail/appendix). Agree the storyline (5–9 beats) before writing HTML: lead + with the conclusion (BLUF), then evidence, then "so what / next". Use `data-story-outline.md`. 4. **Build from a starter** (`templates/`), drop in real numbers and charts, keep it **self-contained** (works offline; pin CDN versions or vendor the files). 5. **Export** if needed (see the export note below). @@ -93,7 +95,13 @@ All HTML starters use a generic **sales review** as demo content — replace it - Every data slide states **source, grain, date range, filters**. Never invent numbers — mark unknowns `TODO`. +## Data storytelling rules +- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". +- **One idea per slide.** Move supporting detail to an appendix section. +- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. +- **Never fabricate data.** Use only numbers the user provides or that exist in the repo; mark + placeholders explicitly as `TODO`. + ## Rules -- One idea per slide; the title is the **insight**, not the field name. - Keep it self-contained and offline-capable; pin versions, don't rely on live network at present time. - Accessibility: real text (not images of text), sufficient contrast, logical heading order. diff --git a/.agent/skills/building-html-presentations/templates/charts-reference.html b/.agent/skills/building-html-presentations/templates/charts-reference.html index 310a9b7..ded2df6 100644 --- a/.agent/skills/building-html-presentations/templates/charts-reference.html +++ b/.agent/skills/building-html-presentations/templates/charts-reference.html @@ -3,8 +3,8 @@ ┌──────────────────────────────────────────────────────────────────────────┐ │ CHART COOKBOOK — the palette of chart types for the HTML starters. │ │ │ - │ A live gallery of ECharts patterns (+ CSS-only patterns) so the agent and │ - │ the user can see what's possible and copy a config into any deck/site/PDF │ + │ A live gallery of ECharts patterns (+ CSS-only patterns) so you can see │ + │ what's possible and copy a config into any deck/site/PDF │ │ template. Each card names the chart and when to use it. Same brand tokens │ │ as the other starters (map from DESIGN.md). Pinned ECharts. │ │ │ diff --git a/.agent/skills/creating-agents-and-skills/SKILL.md b/.agent/skills/creating-agents-and-skills/SKILL.md index f074ab3..19d871c 100644 --- a/.agent/skills/creating-agents-and-skills/SKILL.md +++ b/.agent/skills/creating-agents-and-skills/SKILL.md @@ -1,6 +1,6 @@ --- name: creating-agents-and-skills -description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, AGENTS.md) identical in each tool's native format. +description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, .cursor, AGENTS.md) identical in each tool's native format. --- # Skill: creating-agents-and-skills @@ -18,18 +18,27 @@ Run these gates before creating anything. If a gate says stop, explain why and s - **LLM-native? → don't create.** If a capable model already does this well ("write clean code", "use CTEs", "explain this function"), no artifact is needed. Just do the task. -- **Narrow knowledge / a tool / a format / a checklist? → make a SKILL.** Most additions are - skills: a team naming standard, a framework's rules, a validation procedure. Skills - auto-activate by description and cost almost nothing until used. -- **A distinct role with its own domain and judgement? → make an AGENT.** Only when there's a - persona that owns a body of work (like `data-engineer`). Agents are expensive (routing, - attention); the bar is high. -- **Overlaps an existing agent/skill? → extend it,** don't add a near-duplicate. Ambiguous - routing between two similar artifacts degrades results. +- **Narrow knowledge / a tool / a format / a checklist / a script? → make a SKILL.** This is + the default for almost everything: a team naming standard, a framework's rules, a validation + procedure. Skills auto-activate by description and cost almost nothing until used. - **A repeatable multi-step procedure the user invokes? → make a WORKFLOW** (slash command). +- **Overlaps an existing skill/agent? → extend it,** don't add a near-duplicate. -> Default to a skill. Recommend an agent only if you can name the distinct domain it owns and -> confirm nothing existing covers it. +### The agent-survival rule + +An **agent** is allowed only if it imposes something the model cannot impose on itself: + +1. **Restricted tools** — a frontmatter `tools` list that removes capabilities (e.g. read-only). +2. **Independent review** — fresh context, without the author's reasoning. +3. **A different model** than the main session. +4. **An automatic trigger** that fires without the user asking. + +If none applies, it is a skill: "a persona with a domain" is not a reason. Domain knowledge +belongs in a skill, or is dropped if the model already knows it. Agents must not hand off to +other agents. + +> Default to a skill. Recommend an agent only if you can name which of the four criteria it +> satisfies, and say so in the agent's description or body. --- @@ -37,18 +46,18 @@ Run these gates before creating anything. If a gate says stop, explain why and s Write only domain/org-specific signal. No filler, no restating what the model knows. English. -**Agent** — `.agent/agents/.md`: +**Agent** — `.agent/agents/.md` (example: `pbi-visual-reviewer`, a read-only reviewer): ```yaml --- -name: -description: One sharp sentence on what it does + trigger keywords (drives routing). -tools: Read, Grep, Glob, Bash, Edit, Write +name: pbi-visual-reviewer +description: Independently reviews Power BI report screenshots against the request and the PBIR folder; PASS/FAIL per page and visual. Use after a report change has been reloaded and screenshotted. +tools: Read, Grep, Glob # the restriction that justifies the agent: no Bash/Edit/Write model: inherit -skills: skill-a, skill-b # optional hints; skills also self-activate --- -# -<1–2 line identity> → domain rules → handoffs to other agents → out-of-scope. +# PBI Visual Reviewer +<1–2 line identity> → fixed checklist → output format (PASS/FAIL). No handoffs. ``` +`tools` is the field that matters: list only what the agent needs. **Skill** — `.agent/skills//SKILL.md` (gerund-style name, e.g. `validating-dbt-models`): ```yaml @@ -59,7 +68,7 @@ description: What it does AND when to use it, in the third person, with trigger --- # Skill: → concrete rules or steps. Keep the body under ~500 lines; move long reference -material to sibling files (e.g. reference.md, scripts/) and link them. +material to sibling files (e.g. references/, scripts/) and link them. ``` Write descriptions in the **third person** ("Generates...", "Use when...") — never "I can help". The description is what makes the skill fire without the user naming it. @@ -76,7 +85,8 @@ format. Pick the path that matches the project: ### Path A — Source of truth present (`.agent/` + `scripts/sync_agents.py` exist) This is the kit repo or an Antigravity install. **Author in `.agent/` only, then sync:** ```bash -python scripts/sync_agents.py +python3 scripts/sync_agents.py # regenerate +python3 scripts/sync_agents.py --check # exits 1 if generated folders are stale (CI runs this) ``` The script regenerates `.claude/`, `.github/`, `.opencode/`, `.cursor/`, and root `AGENTS.md` (which also covers OpenAI Codex). Never hand-edit those — they're overwritten. Done. @@ -87,8 +97,8 @@ format (the body content stays identical; only frontmatter changes). Format mapp | Concept | `.agent` (Antigravity) | `.claude` (Claude Code) | `.github` (Copilot) | `.opencode` (OpenCode) | `.cursor` (Cursor) | |---|---|---|---|---|---| -| Agent | `agents/.md` | `agents/.md` (identical) | `agents/.agent.md` — frontmatter `name`, `description` only (no `tools`) | `agents/.md` — frontmatter `description`, `mode: subagent`, `permission` (drop `tools`/`model`/`name`/`skills`) | — (Cursor has no file-based agents) | -| Skill | `skills//SKILL.md` | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | — (no skills concept) | +| Agent | `agents/.md` | `agents/.md` (identical) | `agents/.agent.md` — frontmatter `name`, `description`, `tools` mapped from Claude names (Read→`read`, Grep/Glob→`search`, Edit/Write→`edit`, Bash→`execute`) | `agents/.md` — frontmatter `description`, `mode: subagent`, `permission` (deny edit/bash when the agent is read-only; drop `tools`/`model`/`name`/`skills`) | — (none; run the task in a fresh chat) | +| Skill | `skills//SKILL.md` | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | | Workflow | `workflows/.md` | `commands/.md` (native slash command) + `workflows/.md` | `prompts/.prompt.md` — frontmatter `name`, `description`; body starts `**Context:** ${selection}` | `commands/.md` (identical) | `commands/.md` (identical) | | Global rule | `rules/rules.md` (with `trigger: always_on`) | `CLAUDE.md` (rules + commands section) | `copilot-instructions.md` (rules) | root `AGENTS.md` (rules) | `rules/.mdc` — frontmatter `description`, `globs`, `alwaysApply: true` | @@ -104,11 +114,14 @@ Rules: ### Verify parity After either path, confirm the artifact exists in every present tool folder and that no folder -references something that no longer exists. State which folders you wrote. +references something that no longer exists. In Path A, `python3 scripts/sync_agents.py --check` +does this. State which folders you wrote. --- ## Anti-patterns -- Creating an agent for a single tool or task (e.g. "GitAgent"). Tools don't need agents. +- Creating an agent for a persona, a single tool, or a single task (e.g. "GitAgent") — it fails + the survival rule; write a skill. +- Chaining agents through handoffs. - Putting episodic guidance in the always-on rules — it belongs in a skill like this one. - Editing generated folders (`.claude`/`.github`/`.opencode`/`.cursor`/`AGENTS.md`) when `.agent/` exists. diff --git a/.agent/skills/dbt-modeling-conventions/SKILL.md b/.agent/skills/dbt-modeling-conventions/SKILL.md new file mode 100644 index 0000000..57446ce --- /dev/null +++ b/.agent/skills/dbt-modeling-conventions/SKILL.md @@ -0,0 +1,63 @@ +--- +name: dbt-modeling-conventions +description: Applies the organization's dbt modeling conventions. Use when the user writes, reviews, or renames dbt models, mentions dbt, staging, marts, fct_ or dim_ tables, schema.yml, surrogate keys, model naming, CTE structure, or dimensional modeling (star schema) with dbt. +--- + +# Skill: dbt-modeling-conventions + +Org conventions only — the model already knows generic SQL and dbt. Apply these to every model +you write or review. + +## CTE structure +- `import` CTEs on top (one per `ref()`/`source()`, `select` only the needed columns). +- `logical` CTEs in the middle (joins, filters, calculations — one step each). +- One `final` CTE at the bottom; the model ends with `select * from final`. + +## Naming +| Prefix | Layer | +|---|---| +| `stg_` | Staging: 1:1 with a source table, rename/cast only | +| `dim_` | Dimension | +| `fct_` | Fact | +| `mart_` | Business-facing mart built on dims/facts | + +## Keys and grain +- Every dimension has a surrogate key `sk_`: a hash of the natural key. +- One grain per fact table; state it in the model description ("one row per order line"). +- Build dimensions before facts. + +## Tests and docs +Every model has a `schema.yml` entry with a description and `unique` + `not_null` on its +primary key. + +## Example + +```sql +-- models/marts/dim_customer.sql +with stg_customers as ( + select * from {{ ref('stg_crm__customers') }} +), +customers_keyed as ( + select + {{ dbt_utils.generate_surrogate_key(['customer_id']) }} as sk_customer, + customer_id, + customer_name, + country + from stg_customers +), +final as ( + select * from customers_keyed +) +select * from final +``` + +```yaml +# models/marts/schema.yml +models: + - name: dim_customer + description: One row per customer. + columns: + - name: sk_customer + description: Surrogate key, hash of customer_id. + tests: [unique, not_null] +``` diff --git a/.agent/skills/pbi-quality-rules/SKILL.md b/.agent/skills/pbi-quality-rules/SKILL.md index 1a8c059..775db7b 100644 --- a/.agent/skills/pbi-quality-rules/SKILL.md +++ b/.agent/skills/pbi-quality-rules/SKILL.md @@ -1,42 +1,44 @@ --- name: pbi-quality-rules -description: Validate a Power BI semantic model with the real Best Practice Analyzer (BPA) via the free Tabular Editor 2 CLI, using a shareable BPA rules file. +description: Validate a Power BI semantic model with the Best Practice Analyzer (BPA) — the real engine via the free Tabular Editor 2 CLI, or a Python fallback over TMDL when TE2 is not installed — using the shareable bpa-rules.json. Use when asked to lint, audit, check or validate a model, after significant model changes, and before publishing. --- # Skill: pbi-quality-rules -Run the **Best Practice Analyzer (BPA)** — the industry-standard model linter — against a PBIP -semantic model. This skill uses the *real* BPA engine from **Tabular Editor 2 (free, open -source)** instead of a homegrown checker, so results match what Power BI developers expect and -the rules are portable. +Two scripts in this skill's `scripts/` folder (Python 3.9+, standard library only): -## Rule set -`bpa-rules.json` (next to this file) holds the team's BPA rules in Tabular Editor's native -format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in Dynamic LINQ over the -TOM). Severity: `3` = error, `2` = warning, `1` = info. Edit this file to add or tune rules; -it loads directly into Tabular Editor's BPA UI too. +| Script | What it does | +|---|---| +| `run_bpa.py ` | Runs the real BPA through **Tabular Editor 2** (`TabularEditor.exe -A bpa-rules.json -V`). If TE2 is not found, it runs the fallback and says so. | +| `bpa_fallback.py ` | The same rules re-implemented in Python over the TMDL files (or `model.bim`). Approximate; rules it cannot evaluate are listed as `SKIPPED`, never as passed. | -## Primary path — Tabular Editor 2 CLI (recommended) -Requires Tabular Editor 2 (free). Desktop should be closed so disk reflects the latest state. +`` = the `.pbip`, the project folder, `.SemanticModel/`, its `definition/` folder, +or a `model.bim`. Options: `--format json`, `--rules `, and for `run_bpa.py` also +`--te ` and `--no-fallback`. TE2 is looked up via `--te`, `$TABULAR_EDITOR_PATH`, `PATH`, +then the default Windows install folders. +Exit codes: `0` no error-severity violations · `1` errors found · `2` bad path / TE2 failed. + +## Run it ```bash -# Windows (TabularEditor.exe on PATH). Point at the .SemanticModel folder (TMDL) or model.bim. -TabularEditor.exe "Sales.SemanticModel" -A ".agent/skills/pbi-quality-rules/bpa-rules.json" -V +python3 /scripts/run_bpa.py path/to/Sales.pbip ``` - -- `-A ` runs the BPA with the given rules; `-V` writes violations to the console - (and sets a non-zero exit code on error-severity hits — useful in CI). -- The cross-platform `te` CLI (preview) accepts the same rules file if installed instead. - -Parse the console output, then present **errors and warnings only** (omit info unless asked). - -## Fallback — no Tabular Editor installed -The kit stays usable without the tool. Parse the TMDL files directly and evaluate the same -rules by hand: for each rule, read its `Scope` and `Expression` from `bpa-rules.json` and apply -the equivalent check to the parsed objects (measures, tables, columns, relationships). This is -less robust than the real BPA — say so, and recommend installing Tabular Editor 2 for accuracy. - -## Agent flow -1. Run after large model edits / refactors, or on demand. -2. Show errors first; for each `error`, offer an immediate fix. For `warning`, suggest but - don't force. Group by category so the report is scannable. +1. **Disk is the input.** With Desktop open and MCP edits pending, ask the user to save (Ctrl+S) + first — the BPA reads the TMDL Desktop wrote, not the live model. +2. Present **errors first**, then warnings, grouped by category; omit info unless asked. Say + which engine ran; for the fallback, recommend installing Tabular Editor 2 for exact results. +3. For each `error`, offer a fix (through the Modeling MCP when Desktop is open — see + `pbi-semantic-layer-tmdl`). Warnings: suggest, don't force. + +## Rule set — `bpa-rules.json` +Tabular Editor's native format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in +Dynamic LINQ over the TOM). Severity `3` error · `2` warning · `1` info. It loads directly into +TE2's BPA window too. + +Adding a rule: add it to `bpa-rules.json` (TE2 picks it up at once), then add a predicate with the +same `ID` to `CHECKS` in `scripts/bpa_fallback.py` so the fallback covers it; until then the +fallback reports it as `SKIPPED`. + +## In CI +Run `run_bpa.py --no-fallback --format json` on a Windows runner with TE2 (fails on exit 1), or +`bpa_fallback.py --format json` anywhere for a lighter gate. diff --git a/.agent/skills/pbi-quality-rules/scripts/bpa_fallback.py b/.agent/skills/pbi-quality-rules/scripts/bpa_fallback.py new file mode 100755 index 0000000..36f575b --- /dev/null +++ b/.agent/skills/pbi-quality-rules/scripts/bpa_fallback.py @@ -0,0 +1,522 @@ +#!/usr/bin/env python3 +"""Best Practice Analyzer fallback: the rules of bpa-rules.json, evaluated in Python. + +Use this only when Tabular Editor 2 is not available (run_bpa.py calls it automatically). +It reads the semantic model straight from disk — TMDL (PBIP `definition/` folder) or a +`model.bim` — and re-implements each shipped rule by its ID. It is an approximation of the +real BPA engine: rules without a Python implementation are reported as skipped, never as passed. + +Usage: + python3 bpa_fallback.py [--rules bpa-rules.json] [--format text|json] + + may be a .pbip file, a project folder, a .SemanticModel folder, its +definition/ folder, or a model.bim file. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations, 2 = usage error. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from dataclasses import dataclass, field +from pathlib import Path +from typing import Callable, Optional + +DEFAULT_RULES = Path(__file__).resolve().parent.parent / "bpa-rules.json" +SEVERITY_NAMES = {1: "info", 2: "warning", 3: "error"} +AUTO_DATE_PREFIXES = ("LocalDateTable_", "DateTableTemplate_") + + +class UsageError(Exception): + """Bad input path or file; maps to exit code 2.""" + + +# --------------------------------------------------------------------------- +# Model +# --------------------------------------------------------------------------- +@dataclass +class Measure: + table: str + name: str + expression: str = "" + description: str = "" + display_folder: str = "" + is_hidden: bool = False + + +@dataclass +class Column: + table: str + name: str + description: str = "" + is_hidden: bool = False + + +@dataclass +class Table: + name: str + description: str = "" + is_hidden: bool = False + measures: list = field(default_factory=list) + columns: list = field(default_factory=list) + + +@dataclass +class Relationship: + name: str + from_table: str = "" + from_column: str = "" + to_table: str = "" + to_column: str = "" + cross_filtering: str = "oneDirection" + + +@dataclass +class Model: + source: Path + kind: str # "tmdl" | "bim" + tables: list = field(default_factory=list) + relationships: list = field(default_factory=list) + + @property + def measures(self): + return [m for t in self.tables for m in t.measures] + + @property + def columns(self): + return [c for t in self.tables for c in t.columns] + + +# --------------------------------------------------------------------------- +# Locating the model on disk +# --------------------------------------------------------------------------- +def _is_tmdl_folder(path: Path) -> bool: + return path.is_dir() and any((path / n).is_file() for n in ("model.tmdl", "database.tmdl")) + + +def _from_semantic_model_dir(path: Path) -> Optional[tuple]: + if _is_tmdl_folder(path / "definition"): + return "tmdl", path / "definition" + if (path / "model.bim").is_file(): + return "bim", path / "model.bim" + return None + + +def _semantic_model_from_pbip(pbip: Path) -> Optional[Path]: + sibling = pbip.with_name(pbip.stem + ".SemanticModel") + if sibling.is_dir(): + return sibling + try: + data = json.loads(pbip.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + return None + for artifact in data.get("artifacts", []): + report = artifact.get("report", {}).get("path") + if not report: + continue + pbir = pbip.parent / report / "definition.pbir" + try: + ref = json.loads(pbir.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + continue + by_path = ref.get("datasetReference", {}).get("byPath", {}).get("path") + if by_path: + candidate = (pbir.parent / by_path).resolve() + if candidate.is_dir(): + return candidate + return None + + +def resolve_model(path: Path) -> tuple: + """Return (kind, path) where kind is 'tmdl' (definition folder) or 'bim' (model.bim).""" + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() == ".bim": + return "bim", path + if path.suffix.lower() == ".pbip": + sm = _semantic_model_from_pbip(path) + found = _from_semantic_model_dir(sm) if sm else None + if found: + return found + raise UsageError(f"no semantic model found for {path.name} (thin report?)") + if path.name.lower() == "definition.pbism": + return resolve_model(path.parent) + raise UsageError(f"unsupported file: {path.name} (use .pbip, model.bim or a folder)") + if _is_tmdl_folder(path): + return "tmdl", path + found = _from_semantic_model_dir(path) + if found: + return found + models = sorted(p for p in path.glob("*.SemanticModel") if p.is_dir()) + if len(models) == 1: + found = _from_semantic_model_dir(models[0]) + if found: + return found + if len(models) > 1: + names = ", ".join(p.name for p in models) + raise UsageError(f"several semantic models in {path}: {names}; pass one explicitly") + raise UsageError(f"no TMDL definition or model.bim found under {path}") + + +# --------------------------------------------------------------------------- +# TMDL parser (the subset the rules need) +# --------------------------------------------------------------------------- +_PROPERTY = re.compile(r"^([A-Za-z][A-Za-z0-9_]*)\s*(?::\s*(.*))?$") + + +def _depth(line: str) -> int: + expanded = line[: len(line) - len(line.lstrip())].replace("\t", " ") + return len(expanded) // 4 + + +def _read_name(text: str) -> tuple: + """Read one TMDL object name from the start of `text`. Returns (name, rest).""" + text = text.lstrip() + if text.startswith("'"): + out, i = [], 1 + while i < len(text): + if text[i] == "'": + if i + 1 < len(text) and text[i + 1] == "'": + out.append("'") + i += 2 + continue + return "".join(out), text[i + 1 :] + out.append(text[i]) + i += 1 + return "".join(out), "" + match = re.match(r"[^\s=.]+", text) + if not match: + return "", text + return match.group(0), text[match.end() :] + + +def _split_column_ref(ref: str) -> tuple: + """`Sales.DateKey` / `'Sales Table'.'Date Key'` -> (table, column).""" + table, rest = _read_name(ref) + rest = rest.lstrip() + if rest.startswith("."): + column, _ = _read_name(rest[1:]) + return table, column + return "", table + + +def _unquote_value(value: str) -> str: + value = value.strip() + if len(value) >= 2 and value[0] == value[-1] == '"': + return value[1:-1].replace('""', '"') + return value + + +def parse_tmdl_folder(folder: Path) -> Model: + model = Model(source=folder, kind="tmdl") + tables: dict = {} + for file in sorted(folder.rglob("*.tmdl")): + _parse_tmdl_file(file.read_text(encoding="utf-8-sig"), model, tables) + model.tables = list(tables.values()) + return model + + +def _parse_tmdl_file(text: str, model: Model, tables: dict) -> None: + table: Optional[Table] = None + obj = None # Measure | Column | Relationship | Table (for table-level properties) + obj_depth = 0 + other_depth: Optional[int] = None # inside an object we do not model (partition, ...) + expr_lines: Optional[list] = None + expr_fenced = False + pending_doc: list = [] + + def finish_expression(): + nonlocal expr_lines + if expr_lines is not None and isinstance(obj, Measure): + obj.expression = "\n".join(expr_lines).strip() + expr_lines = None + + for raw in text.splitlines(): + stripped = raw.strip() + depth = _depth(raw) + + if expr_lines is not None: + if expr_fenced: + if stripped.startswith("```"): + finish_expression() + else: + expr_lines.append(stripped) + continue + if not stripped or depth > obj_depth + 1: + expr_lines.append(stripped) + continue + finish_expression() + + if not stripped: + continue + if stripped.startswith("///"): + pending_doc.append(stripped[3:].strip()) + continue + if stripped.startswith("//"): + continue + + if other_depth is not None: + if depth > other_depth: + continue + other_depth = None + + doc = " ".join(pending_doc).strip() + pending_doc = [] + keyword, _, rest = stripped.partition(" ") + + if depth == 0: + obj = None + table = None + if keyword == "table": + name, _ = _read_name(rest) + table = tables.setdefault(name, Table(name=name)) + if doc: + table.description = doc + obj, obj_depth = table, 0 + elif keyword == "relationship": + name, _ = _read_name(rest) + obj, obj_depth = Relationship(name=name), 0 + model.relationships.append(obj) + else: + other_depth = 0 + continue + + if table is not None and depth == 1 and keyword in ("measure", "column"): + name, after = _read_name(rest) + after = after.strip() + if keyword == "measure": + obj = Measure(table=table.name, name=name, description=doc) + table.measures.append(obj) + else: + obj = Column(table=table.name, name=name, description=doc) + table.columns.append(obj) + obj_depth = 1 + if after.startswith("="): + inline = after[1:].strip() + if inline.startswith("```"): + expr_lines, expr_fenced = [], True + elif inline: + if isinstance(obj, Measure): + obj.expression = inline + else: + expr_lines, expr_fenced = [], False + continue + + if obj is not None and depth == obj_depth + 1: + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(obj, match.group(1), match.group(2)) + continue + + if table is not None and depth == 1: + # Table-level property after members (e.g. `isHidden`, `lineageTag: …`). + obj, obj_depth = table, 0 + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(table, match.group(1), match.group(2)) + continue + + # Any other nested object (partition, hierarchy, annotation, ...): skip its body. + other_depth = depth + + finish_expression() + + +def _set_property(obj, key: str, value: Optional[str]) -> None: + flag = value is None or value.strip().lower() == "true" + if key == "isHidden" and hasattr(obj, "is_hidden"): + obj.is_hidden = flag + elif key == "description" and hasattr(obj, "description") and value is not None: + obj.description = _unquote_value(value) + elif key == "displayFolder" and isinstance(obj, Measure) and value is not None: + obj.display_folder = _unquote_value(value) + elif isinstance(obj, Relationship) and value is not None: + if key == "fromColumn": + obj.from_table, obj.from_column = _split_column_ref(value) + elif key == "toColumn": + obj.to_table, obj.to_column = _split_column_ref(value) + elif key == "crossFilteringBehavior": + obj.cross_filtering = value.strip() + + +# --------------------------------------------------------------------------- +# model.bim (TMSL JSON) +# --------------------------------------------------------------------------- +def _text(value) -> str: + if isinstance(value, list): + return "\n".join(str(v) for v in value) + return str(value or "") + + +def parse_bim(path: Path) -> Model: + try: + data = json.loads(path.read_text(encoding="utf-8-sig")) + except ValueError as exc: + raise UsageError(f"{path} is not valid JSON: {exc}") from exc + raw_model = data.get("model", data) + model = Model(source=path, kind="bim") + for t in raw_model.get("tables", []): + table = Table(name=t.get("name", ""), description=_text(t.get("description")), + is_hidden=bool(t.get("isHidden"))) + for m in t.get("measures", []): + table.measures.append(Measure( + table=table.name, name=m.get("name", ""), expression=_text(m.get("expression")), + description=_text(m.get("description")), display_folder=m.get("displayFolder", "") or "", + is_hidden=bool(m.get("isHidden")))) + for c in t.get("columns", []): + if c.get("type") == "rowNumber": + continue + table.columns.append(Column(table=table.name, name=c.get("name", ""), + description=_text(c.get("description")), + is_hidden=bool(c.get("isHidden")))) + model.tables.append(table) + for r in raw_model.get("relationships", []): + model.relationships.append(Relationship( + name=r.get("name", ""), from_table=r.get("fromTable", ""), from_column=r.get("fromColumn", ""), + to_table=r.get("toTable", ""), to_column=r.get("toColumn", ""), + cross_filtering=r.get("crossFilteringBehavior", "oneDirection"))) + return model + + +def load_model(path: Path) -> Model: + kind, resolved = resolve_model(path) + return parse_tmdl_folder(resolved) if kind == "tmdl" else parse_bim(resolved) + + +# --------------------------------------------------------------------------- +# Rules — one Python predicate per rule ID in bpa-rules.json +# --------------------------------------------------------------------------- +def _blank(text: str) -> bool: + return not (text or "").strip() + + +def _tables_in_relationships(model: Model) -> set: + return {t for r in model.relationships for t in (r.from_table, r.to_table)} + + +# rule ID -> (scope, predicate(obj, model) -> True when the object VIOLATES the rule) +CHECKS: dict = { + "NAMING_MEASURE_PASCAL": ("Measure", lambda m, _: bool(re.match(r"^[a-z]", m.name))), + "NAMING_MEASURE_DISPLAYFOLDER": ("Measure", lambda m, _: not m.display_folder), + "DAX_EMPTY_EXPRESSION": ("Measure", lambda m, _: _blank(m.expression) + or m.expression.replace(" ", "").upper() == "BLANK()"), + "DAX_IFERROR_USAGE": ("Measure", lambda m, _: "IFERROR" in m.expression.upper()), + "MODEL_BIDIRECTIONAL": ("Relationship", lambda r, _: r.cross_filtering.lower() == "bothdirections"), + "MODEL_ISLAND_TABLE": ("Table", lambda t, model: t.name not in _tables_in_relationships(model) + and not t.name.startswith(AUTO_DATE_PREFIXES)), + "PERF_AUTO_DATETIME": ("Table", lambda t, _: t.name.startswith(AUTO_DATE_PREFIXES)), + "DOC_TABLE_DESCRIPTION": ("Table", lambda t, _: _blank(t.description)), + "DOC_MEASURE_DESCRIPTION": ("Measure", lambda m, _: _blank(m.description)), + "DOC_KEY_COLUMN_HIDDEN": ("Column", lambda c, _: bool(re.search(r"ID$", c.name)) and not c.is_hidden), +} + + +def _quote(name: str) -> str: + return "'" + name.replace("'", "''") + "'" + + +def object_label(obj) -> str: + if isinstance(obj, Table): + return _quote(obj.name) + if isinstance(obj, (Measure, Column)): + return f"{_quote(obj.table)}[{obj.name}]" + if isinstance(obj, Relationship): + return f"{_quote(obj.from_table)}[{obj.from_column}] -> {_quote(obj.to_table)}[{obj.to_column}]" + return str(obj) + + +def _objects(model: Model, scope: str) -> list: + return { + "Measure": model.measures, + "Column": model.columns, + "Table": model.tables, + "Relationship": model.relationships, + }[scope] + + +def load_rules(path: Path) -> list: + try: + rules = json.loads(Path(path).read_text(encoding="utf-8-sig")) + except (OSError, ValueError) as exc: + raise UsageError(f"cannot read rules file {path}: {exc}") from exc + if not isinstance(rules, list): + raise UsageError(f"rules file {path} must contain a JSON array") + return rules + + +def analyze(model: Model, rules: list) -> dict: + violations, skipped = [], [] + for rule in rules: + check: Optional[tuple] = CHECKS.get(rule.get("ID")) + if check is None: + skipped.append(rule.get("ID") or rule.get("Name")) + continue + scope, predicate = check + predicate: Callable + severity = SEVERITY_NAMES.get(int(rule.get("Severity", 1)), "info") + for obj in _objects(model, scope): + if predicate(obj, model): + violations.append({ + "rule_id": rule["ID"], + "rule_name": rule.get("Name", rule["ID"]), + "category": rule.get("Category", ""), + "severity": severity, + "object_type": scope, + "object": object_label(obj), + }) + order = {"error": 0, "warning": 1, "info": 2} + violations.sort(key=lambda v: (order[v["severity"]], v["category"], v["rule_id"], v["object"])) + return { + "engine": "python-fallback", + "model": str(model.source), + "violations": violations, + "skipped_rules": skipped, + "summary": {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")}, + } + + +def format_text(result: dict) -> str: + lines = [] + if result["engine"] == "python-fallback": + lines.append("BPA engine: Python fallback (approximate). Install Tabular Editor 2 for the real BPA.") + else: + lines.append("BPA engine: Tabular Editor 2") + lines.append(f"Model: {result['model']}") + for v in result["violations"]: + lines.append(f"{v['severity'].upper():8} [{v['rule_id']}] {v['object_type']} {v['object']} — {v['rule_name']}") + if not result["violations"]: + lines.append("No objects in violation of the rules.") + for rule_id in result.get("skipped_rules", []): + lines.append(f"SKIPPED [{rule_id}] no implementation in this engine") + s = result["summary"] + lines.append(f"Summary: {s['error']} error(s), {s['warning']} warning(s), {s['info']} info") + return "\n".join(lines) + + +def emit(result: dict, fmt: str) -> int: + print(json.dumps(result, indent=2) if fmt == "json" else format_text(result)) + return 1 if result["summary"]["error"] else 0 + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="BPA rules evaluated in Python over TMDL/model.bim.") + parser.add_argument("model", type=Path, help=".pbip, project folder, .SemanticModel, definition/ or model.bim") + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES, help="BPA rules JSON (default: bpa-rules.json)") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + try: + result = analyze(load_model(args.model), load_rules(args.rules)) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + return emit(result, args.format) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agent/skills/pbi-quality-rules/scripts/run_bpa.py b/.agent/skills/pbi-quality-rules/scripts/run_bpa.py new file mode 100755 index 0000000..31e1232 --- /dev/null +++ b/.agent/skills/pbi-quality-rules/scripts/run_bpa.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""Run the Best Practice Analyzer with Tabular Editor 2; fall back to Python when it is absent. + +Usage: + python3 run_bpa.py [--rules bpa-rules.json] [--te PATH] [--format text|json] [--no-fallback] + +: .pbip, project folder, .SemanticModel, its definition/ folder, or model.bim. + +Tabular Editor 2 is located in this order: --te, $TABULAR_EDITOR_PATH, TabularEditor(.exe) on +PATH, then the default Windows install folders. It runs as + + TabularEditor.exe -A -V + +(-A: analyze with this rules file only, plus rules embedded in the model; -V: Azure DevOps +logging lines, which carry the severity). Without TE2 the script runs bpa_fallback.py and says so. + +Disk is the input: with Power BI Desktop open, save first (Ctrl+S) so the TMDL is current. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations (or rule errors), +2 = usage error / Tabular Editor could not run. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +sys.dont_write_bytecode = True +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import bpa_fallback # noqa: E402 + +DEFAULT_RULES = bpa_fallback.DEFAULT_RULES +TE_TIMEOUT_SECONDS = 600 + +# "##vso[task.logissue type=warning;]Measure 'Sales'[x] violates rule "Rule name"" +_VSO = re.compile(r"^##vso\[task\.logissue type=(?Perror|warning)[^\]]*\](?P.*)$") +_VIOLATION = re.compile(r'^(?P.+?) (?P(?:\'|\[).*) violates rule "(?P.*)"$') +_VIOLATION_LOOSE = re.compile(r'^(?P\S+) (?P.+?) violates rule "(?P.*)"$') +_RULE_ERROR = re.compile(r"^Error on rule '(?P.*?)': (?P.*)$") + + +def find_tabular_editor(explicit: Optional[str] = None, env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + for candidate in (explicit, env.get("TABULAR_EDITOR_PATH")): + if candidate: + return candidate if Path(candidate).is_file() else None + for name in ("TabularEditor.exe", "TabularEditor"): + found = shutil.which(name) + if found: + return found + for base in (env.get("ProgramFiles(x86)"), env.get("ProgramFiles"), env.get("LOCALAPPDATA")): + if base: + exe = Path(base) / "Tabular Editor" / "TabularEditor.exe" + if exe.is_file(): + return str(exe) + return None + + +def build_command(te_path: str, model_target: Path, rules: Path) -> list: + return [te_path, str(model_target), "-A", str(rules), "-V"] + + +def parse_te_output(stdout: str, rules: list, model_label: str) -> dict: + """Turn TE2 console output (-A ... -V) into the same result shape as the fallback.""" + by_name = {r.get("Name"): r for r in rules} + violations, rule_errors = [], [] + for raw in stdout.splitlines(): + line = raw.strip() + if not line or line.startswith("##vso[task.complete"): + continue + level = "info" + match = _VSO.match(line) + if match: + level, line = match.group("level"), match.group("text").strip() + rule_error = _RULE_ERROR.match(line) + if rule_error: + rule_errors.append({"rule_name": rule_error.group("rule"), "error": rule_error.group("error")}) + continue + found = _VIOLATION.match(line) or _VIOLATION_LOOSE.match(line) + if not found: + continue + rule = by_name.get(found.group("rule"), {}) + violations.append({ + "rule_id": rule.get("ID", ""), + "rule_name": found.group("rule"), + "category": rule.get("Category", ""), + "severity": level, + "object_type": found.group("type"), + "object": found.group("object"), + }) + summary = {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")} + return { + "engine": "tabular-editor-2", + "model": model_label, + "violations": violations, + "rule_errors": rule_errors, + "skipped_rules": [], + "summary": summary, + } + + +def run_tabular_editor(te_path: str, model_target: Path, rules_path: Path, rules: list) -> dict: + cmd = build_command(te_path, model_target, rules_path) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=TE_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise bpa_fallback.UsageError(f"Tabular Editor failed to run ({' '.join(cmd)}): {exc}") from exc + result = parse_te_output(proc.stdout, rules, str(model_target)) + loaded = "Running Best Practice Analyzer" in proc.stdout + if proc.returncode != 0 and not loaded: + detail = (proc.stderr or proc.stdout).strip()[-2000:] + raise bpa_fallback.UsageError(f"Tabular Editor exited {proc.returncode} before the BPA ran:\n{detail}") + return result + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Run the BPA via Tabular Editor 2 (Python fallback if absent).") + parser.add_argument("model", type=Path) + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES) + parser.add_argument("--te", help="path to TabularEditor.exe (overrides $TABULAR_EDITOR_PATH)") + parser.add_argument("--format", choices=("text", "json"), default="text") + parser.add_argument("--no-fallback", action="store_true", help="fail (exit 2) instead of using the Python fallback") + args = parser.parse_args(argv) + + try: + rules = bpa_fallback.load_rules(args.rules) + kind, target = bpa_fallback.resolve_model(args.model) + te_path = find_tabular_editor(args.te) + if te_path: + result = run_tabular_editor(te_path, target, args.rules.resolve(), rules) + elif args.no_fallback or args.te: + raise bpa_fallback.UsageError("Tabular Editor 2 not found (use --te or $TABULAR_EDITOR_PATH)") + else: + print("Tabular Editor 2 not found; using the Python fallback.", file=sys.stderr) + parsed = bpa_fallback.parse_tmdl_folder(target) if kind == "tmdl" else bpa_fallback.parse_bim(target) + result = bpa_fallback.analyze(parsed, rules) + except bpa_fallback.UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + if args.format == "json": + print(json.dumps(result, indent=2)) + else: + print(bpa_fallback.format_text(result)) + for err in result.get("rule_errors", []): + print(f"RULE-ERR [{err['rule_name']}] {err['error']}") + has_errors = result["summary"]["error"] > 0 or bool(result.get("rule_errors")) + return 1 if has_errors else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agent/skills/pbi-report-layer-pbir/SKILL.md b/.agent/skills/pbi-report-layer-pbir/SKILL.md index 28ebab9..898d4d7 100644 --- a/.agent/skills/pbi-report-layer-pbir/SKILL.md +++ b/.agent/skills/pbi-report-layer-pbir/SKILL.md @@ -1,62 +1,128 @@ --- name: pbi-report-layer-pbir -description: Edit the Power BI report layer as code — pages, visuals, bookmarks, and themes — by writing PBIR JSON files in a PBIP project (offline, Desktop closed). +description: Edit the Power BI report layer as code — pages, visuals, filters, bookmarks and themes in the PBIR files of a PBIP project — with Power BI Desktop OPEN, using the edit → validate → save → status → reload → screenshot → review loop (Desktop Bridge via powerbi-report-author). Also covers the Desktop-closed path. Use for any request to add, move, restyle or fix a report page or visual. --- # Skill: pbi-report-layer-pbir -The report layer (`.Report/`) is **PBIR** — report metadata decomposed into per-page and -per-visual JSON files, each with a public JSON schema. PBIR is becoming the default report -format (rolling out across Service and Desktop through 2026), and it is explicitly designed to -be edited by tools other than Power BI. +The report layer (`.Report/`) is **PBIR**: one JSON file per page and per visual, each +with a public JSON schema. Power BI Desktop can stay **open** while you edit it: the +[Desktop Bridge](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +lets a tool reload the files from disk and take screenshots of the rendered pages. -## Hard precondition -**Power BI Desktop must be CLOSED** while editing these `.json` files — live edits corrupt or -get overwritten by autosave. +Model objects (measures, columns, relationships) are NOT edited here — they go live through the +Modeling MCP (`pbi-semantic-layer-tmdl`). Create the measure first, then bind the visual to it. + +## Tools +- `powerbi-report-author` — Microsoft's report CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`, + Node 20+). Validates PBIR, describes visual roles/formatting, and drives Desktop through + `preview --host desktop`. It bundles the lower-level bridge CLI `powerbi-desktop` + (`status`, `reload`, `screenshot`); always go through `powerbi-report-author preview`, which + resolves the right Desktop window from the report folder. Details, errors and setup: + [references/desktop-bridge.md](references/desktop-bridge.md). +- `scripts/validate_pbir.py` (in this skill's folder) — runs `powerbi-report-author validate`, or + a structural check when the CLI is missing. +- `pbi-visual-reviewer` agent — independent screenshot review. + +## The loop (Desktop open) +`` = the project folder or the `.pbip` path; `` = an absolute folder OUTSIDE the +PBIP (e.g. `%TEMP%\pbir-review\`), so screenshots never land in git. + +1. **Read before writing.** `powerbi-report-author preview-pages ` and + `preview-visuals ` list pages and visuals with their file paths. For a new or changed + visual, `powerbi-report-author catalog describe ` gives the exact role names + (e.g. `cardVisual` uses `Data`, `clusteredColumnChart` uses `Category` + `Y`); + `formatting describe-object ` gives property names. Do not guess them. +2. **Edit** the PBIR JSON (anatomy below). One logical batch at a time. +3. **Validate:** `python3 /scripts/validate_pbir.py `. Fix every error before + going on; read the warnings. +4. **Save gate.** Ask the user to save in Desktop (Ctrl+S) if they changed anything by hand, + then check: `powerbi-report-author preview --host desktop --status`. + Continue only when `data.details.hasUnsavedChanges` is exactly `false`. +5. **Reload:** `powerbi-report-author preview --host desktop --reload`. +6. **Screenshot every affected page:** + `powerbi-report-author preview --host desktop --screenshot "" --all-pages` + (or `--screenshot "\.png" --page `). A screenshot does not reload; always + reload first. +7. **Review:** hand the PNG paths, the user's request/spec and the `.Report` path — and nothing + about your reasoning — to the `pbi-visual-reviewer` agent. In tools without subagents + (Cursor, Antigravity), run its checklist in a fresh chat with only those inputs. +8. REJECTED → fix the listed files and repeat from step 3. APPROVED → report the result and let + the user save/commit. + +### Pitfalls (each one has lost real work) +- **Reload discards whatever Desktop has not saved.** That is why step 4 exists. If status says + `true`, `null`, or fails, stop and ask the user to save; never reload "just to see". +- **Reload covers the report only.** `--reload` does not re-read model files. Model changes go + through the Modeling MCP, which updates Desktop live. Use `--reload-with-model` only when TMDL + was edited on disk (Desktop-closed habits) — and it still does not refresh data. +- **An edited theme may not show after reload.** Desktop caches themes by file name. When you + change a custom theme, rename its JSON (e.g. `Brand-3f9a1c.json` → `Brand-7b2e04.json`) and + update both references in `report.json` (`themeCollection.customTheme.name` and the + `resourcePackages` item `name`/`path`), then validate and reload. +- A page added on disk is unknown to the open window until a reload; a screenshot of it fails + with `RENDER_FAILED`. +- Two Desktop windows with the same report → `HOST_UNAVAILABLE`. Ask the user to close one; + never close Desktop yourself. + +## Alternative: Desktop closed +Use it when there is no Bridge (Desktop older than the Bridge release, the setting +*File > Options > Security > Desktop Bridge* is off, or the machine is not Windows) and in CI. +1. Make sure Desktop is **closed** — an open Desktop overwrites the files on its next save. +2. Edit the PBIR, run `validate_pbir.py` (in CI: `--format json`, fail on exit code 1). +3. Ask the user to open the `.pbip` and check the pages; no screenshots, so ask for one or for + a description, and run the reviewer checklist on what you get. ## `.Report` structure (PBIR) ``` -.Report/ - definition.pbir ← pointer (model binding) +.Report/ + definition.pbir ← model binding (byPath ../.SemanticModel) definition/ - report.json ← master metadata incl. themeCollection + version.json + report.json ← themeCollection, resourcePackages, settings pages/ + pages.json ← pageOrder + activePageName / - page.json ← canvas size, background, displayName - visuals//visual.json ← one visual per file + page.json ← name (= folder), displayName, width, height + visuals//visual.json ← one visual per file; name = folder StaticResources/ - RegisteredResources//.json ← custom themes (create new themes here) - SharedResources/BaseThemes/ ← Microsoft base themes — NEVER edit + RegisteredResources/ ← custom themes and images you add + SharedResources/BaseThemes/ ← Microsoft base themes — never edit ``` - -## Navigating -- **Pages:** scan `definition/pages/`; read each `page.json` `displayName` (the user-facing name). -- **Visuals:** files sit under GUID folders. To find "the bar chart", grep `visual.json` for - `"visualType": "barChart"`. +PBIR-Legacy (a single `report.json` at the `.Report` root) cannot be edited this way: ask the +user to save the report in PBIR format first. ## Visual anatomy (`visual.json`) ```json { - "name": "", - "position": { "x": 10, "y": 20, "width": 400, "height": 300, "tabOrder": 1 }, + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/visualContainer/2.9.0/schema.json", + "name": "7b2d4f6a8c0e1a3b5d02", + "position": { "x": 40, "y": 240, "z": 2000, "width": 760, "height": 420, "tabOrder": 2000 }, "visual": { - "visualType": "barChart", - "query": { "queryState": { "Category": { "projections": [] }, "Y": { "projections": [] } } } + "visualType": "clusteredColumnChart", + "query": { "queryState": { + "Category": { "projections": [ { "field": { "Column": { "Expression": { "SourceRef": { "Entity": "Product" } }, "Property": "Category" } }, "queryRef": "Product.Category", "nativeQueryRef": "Category", "active": true } ] }, + "Y": { "projections": [ { "field": { "Measure": { "Expression": { "SourceRef": { "Entity": "Sales" } }, "Property": "Total Sales" } }, "queryRef": "Sales.Total Sales", "nativeQueryRef": "Total Sales" } ] } + } }, + "visualContainerObjects": { "title": [ { "properties": { + "show": { "expr": { "Literal": { "Value": "true" } } }, + "text": { "expr": { "Literal": { "Value": "'Sales by category'" } } } } } ] } } } ``` -Positioning is a pixel grid (x=0, y=0 top-left). Generate fresh GUIDs for any new component -(`[System.Guid]::NewGuid()` on Windows, or any UUID generator). +- New page/visual: fresh 20-hex-char `name`, folder with the same name; add new pages to + `pages.json` `pageOrder`. +- Position is in pixels from the page's top-left; keep `x + width ≤ page width` and + `y + height ≤ page height`. Align edges to a shared grid (e.g. multiples of 8) and keep equal gaps. +- Literal encoding: text `'quoted'`, numbers `12D`, booleans `true`. When unsure, + `powerbi-report-author expr encode --kind `. +- Avoid visuals Microsoft is retiring (Q&A, Bing Map, filled map). ## Themes -1. Create the theme JSON under `StaticResources/RegisteredResources//.json`. -2. Register that `` in `report.json`'s `themeCollection`. -3. Define ≥ 8 `dataColors`. To standardize fonts globally, use the wildcard `visualStyles`: - ```json - "visualStyles": { "*": { "*": { "fontSize": [{ "value": 11 }], "fontFamily": [{ "value": "Segoe UI" }] } } } - ``` - -## Cautions -- The visual schema evolves with Desktop releases; old keys may be ignored on newer versions. -- Validate JSON against the PBIR schema where possible. Always have the user commit (`git add`) - before large UI refactors and test by opening the file in Desktop. +Create custom themes under `StaticResources/RegisteredResources/`, register them in +`report.json` (`themeCollection.customTheme` + a `resourcePackages` item of type +`RegisteredResources`), define ≥ 8 `dataColors`, and standardize fonts with the wildcard +`visualStyles` (`"*": { "*": { ... } }`). Rename the file on every edit (pitfall above). + +## Before large changes +Ask the user to commit the PBIP first so every agent edit is a reviewable diff. diff --git a/.agent/skills/pbi-report-layer-pbir/references/desktop-bridge.md b/.agent/skills/pbi-report-layer-pbir/references/desktop-bridge.md new file mode 100644 index 0000000..ef02d9a --- /dev/null +++ b/.agent/skills/pbi-report-layer-pbir/references/desktop-bridge.md @@ -0,0 +1,83 @@ +# Desktop Bridge — setup, commands, errors + +Sources (read these when something here looks outdated): +- Microsoft Learn — [What is the Power BI Desktop Bridge?](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +- Microsoft Learn — [Power BI Report skill](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-report-skill-overview) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `powerbi-report-cli`, MIT) — the official agent guidance this kit follows. +- npm: [`@microsoft/powerbi-report-authoring-cli`](https://www.npmjs.com/package/@microsoft/powerbi-report-authoring-cli) + (`powerbi-report-author`), [`@microsoft/powerbi-desktop-bridge-cli`](https://www.npmjs.com/package/@microsoft/powerbi-desktop-bridge-cli) + (`powerbi-desktop`). Both MIT, Node 20+. + +## What it is +A local JSON-RPC 2.0 server inside each running Power BI Desktop window, reachable only on the +same Windows machine through the named pipe `pbi-desktop-bridge-`. One operation runs +at a time per window; a second request while one is running fails — wait and retry, never run +two Desktop commands in parallel. + +Methods (discover the current list with `bridge.manifest`): +| Method | Used for | +|---|---| +| `bridge.manifest` | Supported methods and their schemas for this Desktop build | +| `application.state.get/v1` | `currentFilePath`, `hasUnsavedChanges` | +| `report.snapshot.capture/v1` | PNG of one page (`pageId`, `scale` 1–3) | +| `file.reload/v1` | Reload the open PBIP from disk; `reloadModelDefinition` true/false | + +## Setup (once per machine) +1. Power BI Desktop for Windows, recent enough to show **File > Options and settings > Options > + Security > Desktop Bridge**. Keep *Enable external tool access to Power BI Desktop through + secure local APIs* checked (default on). Admins can force it with the registry value + `HKLM\SOFTWARE\Policies\Microsoft\Power BI Desktop\DesktopNamedPipeBridge` (`1` on, `0` off). +2. Node.js 20+, then `npm i -g @microsoft/powerbi-report-authoring-cli`. +3. Check: `powerbi-report-author doctor` and `powerbi-report-author preview --list-hosts` + (the `desktop` host must say `available: true` while a PBIP is open). +4. The report must be a **PBIP saved in PBIR format** (not a `.pbix`, not PBIR-Legacy). + +## Commands +Use `powerbi-report-author preview` for every Desktop operation; it finds the Desktop window +that has `` open, so there is no PID to pass. `` = the project folder or `.pbip`. + +| Intent | Command | +|---|---| +| Status (save gate) | `powerbi-report-author preview "" --host desktop --status` | +| Open / attach | `powerbi-report-author preview "" --host desktop` | +| Reload report | `powerbi-report-author preview "" --host desktop --reload` | +| Reload report + on-disk model | `powerbi-report-author preview "" --host desktop --reload-with-model` | +| Screenshot one page | `powerbi-report-author preview "" --host desktop --screenshot "\.png" --page ""` | +| Screenshot all pages | `powerbi-report-author preview "" --host desktop --screenshot "" --all-pages` | +| Validate files | `powerbi-report-author validate ""` (`--no-schema` offline) | + +Optional `--scale 1..3` on screenshots. Every command prints a JSON envelope: +`{"data": {...}}` on success, `{"error": {"code": ..., "message": ..., "correction"?: {...}}}` on +failure. If an error carries a `correction` object, retry once changing only that value; +otherwise report the error and stop — do not improvise another command. + +The bridge CLI underneath (`powerbi-desktop status | manifest | open | reload | screenshot | +screenshot-all`) is useful for manual troubleshooting by the user (e.g. `powerbi-desktop status` +lists every open window and its unsaved state). Agents stay on `powerbi-report-author`. + +### Status before open/reload — never before a screenshot +```json +{"data":{"state":"ready","reports":[{"folder":"C:\\work\\Sales","hostInstanceId":"12345"}], + "details":{"hasUnsavedChanges":false}}} +``` +| `details.hasUnsavedChanges` | Action | +|---|---| +| `false` | Reload / open is allowed. | +| `true` | Stop. Ask the user to save (Ctrl+S) — or to discard — in Desktop, then check again. | +| missing, `null`, or the call failed | Treat as unsafe. Stop and report. | +| error `HOST_UNAVAILABLE` (no window has the report) | Open/attach is allowed; reload is not. | + +## Errors you will meet +| Code / symptom | Meaning | What to do | +|---|---|---| +| `UNSUPPORTED_OS` in `--list-hosts` | Not on Windows | Use the Desktop-closed path; `validate` still works. | +| `HOST_UNAVAILABLE` — "Several Power BI Desktop instances have this report open" | Same report in 2+ windows (`details.candidatePids`) | Ask the user to close the extras. Never close Desktop yourself. | +| `HOST_UNAVAILABLE` — no instance | Desktop closed or setting off | Ask the user to open the `.pbip`; check the Security setting. | +| `RENDER_FAILED` with "unknown page id" | Page exists on disk but not in the open window | Status → reload → capture that page again. | +| `PAGE_NOT_FOUND` | The page is not in the definition at all | Fix `pages.json` / the page id. Reload will not help. | +| `status: "partial"` + non-zero exit | Some pages captured, some failed | Keep the PNGs you got; report which pages failed and why. | +| JSON-RPC `-32601 MethodNotFound` (bridge CLI) | Desktop build lacks that method | Update Desktop; check `powerbi-desktop manifest`. | +| "operation in progress" | Another bridge call is running | Wait, then retry once. | +| Theme change invisible after reload | Desktop cached the theme by file name | Rename the theme file + both refs in `report.json`, validate, reload. | +| Change invisible after reload, no error | Edited a file Desktop does not read (e.g. outside `definition/`) or the wrong report folder | `preview-visuals ` to confirm paths; check `--status` `folder`. | diff --git a/.agent/skills/pbi-report-layer-pbir/scripts/validate_pbir.py b/.agent/skills/pbi-report-layer-pbir/scripts/validate_pbir.py new file mode 100755 index 0000000..008c29b --- /dev/null +++ b/.agent/skills/pbi-report-layer-pbir/scripts/validate_pbir.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Validate a PBIR report: Microsoft's `powerbi-report-author validate` when installed, +otherwise a structural check written in Python. + +Usage: + python3 validate_pbir.py [--structural-only] [--offline] [--format text|json] + +: a .pbip file, a .Report folder, a folder containing definition/, or a project +folder with exactly one *.Report. + +The CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`) is found through +$PBIR_AUTHOR_CLI or `powerbi-report-author` on PATH. It validates every file against the +published PBIR JSON schemas plus visual roles/types, so it is the authoritative check. +--offline passes --no-schema to it (no download of remote schemas). + +The structural fallback checks what breaks Desktop most often: JSON that does not parse, +PBIR-Legacy layout, a broken semantic-model binding, pages.json out of sync with the page +folders, page/visual `name` != folder, visuals without position/visualType, visuals outside +the page canvas, and theme/resource files that are registered but missing. It does NOT know +the schemas or visual roles — say so when reporting. + +Exit codes: 0 = no errors, 1 = errors found, 2 = usage error / the CLI could not run. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +CLI_NAME = "powerbi-report-author" +CLI_TIMEOUT_SECONDS = 300 + + +class UsageError(Exception): + pass + + +# --------------------------------------------------------------------------- +# Locate the .Report folder +# --------------------------------------------------------------------------- +def _read_json(path: Path): + return json.loads(path.read_text(encoding="utf-8-sig")) + + +def resolve_report(path: Path) -> Path: + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() != ".pbip": + raise UsageError(f"unsupported file: {path.name} (use a .pbip or a .Report folder)") + try: + artifacts = _read_json(path).get("artifacts", []) + except ValueError as exc: + raise UsageError(f"{path.name} is not valid JSON: {exc}") from exc + for artifact in artifacts: + report = artifact.get("report", {}).get("path") + if report and (path.parent / report).is_dir(): + return (path.parent / report).resolve() + sibling = path.with_name(path.stem + ".Report") + if sibling.is_dir(): + return sibling.resolve() + raise UsageError(f"{path.name} does not point to an existing .Report folder") + if (path / "definition").is_dir() or (path / "definition.pbir").is_file() or path.suffix == ".Report": + return path.resolve() + reports = sorted(p for p in path.glob("*.Report") if p.is_dir()) + if len(reports) == 1: + return reports[0].resolve() + if len(reports) > 1: + raise UsageError(f"several reports in {path}: {', '.join(p.name for p in reports)}; pass one") + raise UsageError(f"no .Report folder found under {path}") + + +# --------------------------------------------------------------------------- +# Structural fallback +# --------------------------------------------------------------------------- +class Findings: + def __init__(self, report_dir: Path): + self.report_dir = report_dir + self.items: list = [] + + def add(self, severity: str, code: str, message: str, file: Optional[Path] = None, path: str = ""): + rel = "" + if file is not None: + try: + rel = file.relative_to(self.report_dir).as_posix() + except ValueError: + rel = str(file) + self.items.append({"severity": severity, "code": code, "message": message, "file": rel, "path": path}) + + def error(self, *args, **kwargs): + self.add("error", *args, **kwargs) + + def warning(self, *args, **kwargs): + self.add("warning", *args, **kwargs) + + +def _load(findings: Findings, file: Path, required: bool = True): + if not file.is_file(): + if required: + findings.error("FILE_MISSING", f"required file is missing: {file.name}", file) + return None + try: + return _read_json(file) + except ValueError as exc: + findings.error("JSON_INVALID", f"invalid JSON: {exc}", file) + return None + + +def _is_number(value) -> bool: + return isinstance(value, (int, float)) and not isinstance(value, bool) + + +def structural_check(report_dir: Path) -> dict: + f = Findings(report_dir) + definition = report_dir / "definition" + + for file in sorted(report_dir.rglob("*.json")): + if ".pbi" in file.relative_to(report_dir).parts: + continue + try: + _read_json(file) + except ValueError as exc: + f.error("JSON_INVALID", f"invalid JSON: {exc}", file) + invalid = {i["file"] for i in f.items} + + if not definition.is_dir(): + if (report_dir / "report.json").is_file(): + f.error("PBIR_LEGACY_FORMAT", "PBIR-Legacy layout (report.json at the root); save the " + "report in PBIR format (Desktop: File > Options > Preview features > PBIR)", report_dir / "report.json") + else: + f.error("FILE_MISSING", "definition/ folder is missing", definition) + return _result(f, "structural") + + pbir = _load(f, report_dir / "definition.pbir") + if isinstance(pbir, dict): + ref = pbir.get("datasetReference", {}) + by_path = ref.get("byPath", {}).get("path") if isinstance(ref.get("byPath"), dict) else None + if by_path: + target = (report_dir / by_path).resolve() + if not target.is_dir(): + f.error("MODEL_BINDING_BROKEN", f"datasetReference.byPath points to a missing folder: {by_path}", + report_dir / "definition.pbir", "datasetReference.byPath.path") + elif not ref.get("byConnection"): + f.error("MODEL_BINDING_MISSING", "datasetReference has neither byPath nor byConnection", + report_dir / "definition.pbir", "datasetReference") + + _load(f, definition / "version.json") + report = _load(f, definition / "report.json") + if isinstance(report, dict): + _check_resources(f, report, report_dir) + + pages_dir = definition / "pages" + pages_meta = _load(f, pages_dir / "pages.json") + page_folders = sorted(p.name for p in pages_dir.iterdir() if p.is_dir()) if pages_dir.is_dir() else [] + if isinstance(pages_meta, dict): + order = pages_meta.get("pageOrder", []) + for page_id in order: + if page_id not in page_folders: + f.error("PAGE_MISSING", f"pages.json lists page '{page_id}' but its folder is missing", + pages_dir / "pages.json", "pageOrder") + for page_id in page_folders: + if page_id not in order: + f.warning("PAGE_NOT_IN_ORDER", f"page folder '{page_id}' is not listed in pages.json pageOrder", + pages_dir / page_id) + active = pages_meta.get("activePageName") + if active and active not in order: + f.error("ACTIVE_PAGE_UNKNOWN", f"activePageName '{active}' is not in pageOrder", + pages_dir / "pages.json", "activePageName") + + for page_id in page_folders: + _check_page(f, pages_dir / page_id, invalid) + + return _result(f, "structural") + + +def _check_resources(f: Findings, report: dict, report_dir: Path) -> None: + static = report_dir / "StaticResources" + for package in report.get("resourcePackages", []) or []: + folder = static / package.get("name", "") + for item in package.get("items", []) or []: + rel = item.get("path") + if rel and not (folder / rel).is_file(): + f.error("RESOURCE_MISSING", f"resource '{item.get('name')}' registered in report.json is " + f"missing on disk: {package.get('name')}/{rel}", report_dir / "definition" / "report.json", + "resourcePackages") + custom = (report.get("themeCollection") or {}).get("customTheme") + if isinstance(custom, dict) and custom.get("name"): + registered = { + item.get("name") + for package in report.get("resourcePackages", []) or [] + if package.get("type") == custom.get("type", "RegisteredResources") + for item in package.get("items", []) or [] + } + if custom["name"] not in registered: + f.error("THEME_NOT_REGISTERED", f"customTheme '{custom['name']}' has no matching resourcePackages item", + report_dir / "definition" / "report.json", "themeCollection.customTheme.name") + + +def _check_page(f: Findings, page_dir: Path, invalid: set) -> None: + page_file = page_dir / "page.json" + page = _load(f, page_file) + if not isinstance(page, dict): + return + if page.get("name") != page_dir.name: + f.error("NAME_MISMATCH", f"page.json name '{page.get('name')}' != folder '{page_dir.name}'", page_file, "name") + if not page.get("displayName"): + f.warning("PAGE_NO_DISPLAY_NAME", "page has no displayName", page_file, "displayName") + width, height = page.get("width"), page.get("height") + if not (_is_number(width) and _is_number(height)): + f.error("PAGE_SIZE_INVALID", "page width/height must be numbers", page_file) + width = height = None + + visuals_dir = page_dir / "visuals" + if not visuals_dir.is_dir(): + return + for visual_dir in sorted(p for p in visuals_dir.iterdir() if p.is_dir()): + visual_file = visual_dir / "visual.json" + rel = visual_file.relative_to(f.report_dir).as_posix() + if rel in invalid: + continue + visual = _load(f, visual_file) + if not isinstance(visual, dict): + continue + if visual.get("name") != visual_dir.name: + f.error("NAME_MISMATCH", f"visual.json name '{visual.get('name')}' != folder '{visual_dir.name}'", + visual_file, "name") + pos = visual.get("position") + if not isinstance(pos, dict) or not all(_is_number(pos.get(k)) for k in ("x", "y", "width", "height")): + f.error("POSITION_INVALID", "position needs numeric x, y, width and height", visual_file, "position") + pos = None + kind = (visual.get("visual") or {}).get("visualType") + if not kind and "visualGroup" not in visual: + f.error("VISUAL_TYPE_MISSING", "visual.visualType is missing (and this is not a visualGroup)", + visual_file, "visual.visualType") + if pos and width is not None and not visual.get("parentGroupName"): + right, bottom = pos["x"] + pos["width"], pos["y"] + pos["height"] + if pos["x"] < 0 or pos["y"] < 0 or right > width + 0.5 or bottom > height + 0.5: + f.warning("OUTSIDE_CANVAS", f"visual extends outside the {width}x{height} page " + f"(x={pos['x']}, y={pos['y']}, right={right}, bottom={bottom})", visual_file, "position") + + +def _result(f: Findings, engine: str) -> dict: + errors = sum(1 for i in f.items if i["severity"] == "error") + warnings = sum(1 for i in f.items if i["severity"] == "warning") + return { + "engine": engine, + "report": str(f.report_dir), + "result": "failed" if errors else ("succeededWithWarnings" if warnings else "succeeded"), + "errorCount": errors, + "warningCount": warnings, + "diagnostics": f.items, + } + + +# --------------------------------------------------------------------------- +# powerbi-report-author CLI +# --------------------------------------------------------------------------- +def find_cli(env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + explicit = env.get("PBIR_AUTHOR_CLI") + if explicit: + return explicit if Path(explicit).is_file() else None + return shutil.which(CLI_NAME) + + +def build_cli_command(cli: str, report_dir: Path, offline: bool) -> list: + cmd = [cli, "validate", str(report_dir)] + if offline: + cmd.append("--no-schema") + return cmd + + +def parse_cli_output(stdout: str, report_dir: Path) -> dict: + """Normalize the CLI's JSON envelope ({"data": ...} or {"error": ...}).""" + try: + envelope = json.loads(stdout) + except ValueError as exc: + raise UsageError(f"{CLI_NAME} returned non-JSON output: {stdout.strip()[:500]}") from exc + if "error" in envelope: + err = envelope["error"] + raise UsageError(f"{CLI_NAME} failed: {err.get('code')}: {err.get('message')}") + data = envelope.get("data", {}) + items = [] + for code, group in (data.get("diagnostics") or {}).items(): + for item in group.get("items", []): + file = item.get("file", "") + try: + file = Path(file).resolve().relative_to(report_dir).as_posix() if file else "" + except ValueError: + pass + items.append({ + "severity": group.get("severity", "error"), + "code": code, + "message": item.get("message", ""), + "file": file, + "path": item.get("path", ""), + }) + return { + "engine": CLI_NAME, + "report": str(report_dir), + "result": data.get("result", "unknown"), + "errorCount": int(data.get("errorCount", sum(1 for i in items if i["severity"] == "error"))), + "warningCount": int(data.get("warningCount", sum(1 for i in items if i["severity"] == "warning"))), + "diagnostics": items, + } + + +def run_cli(cli: str, report_dir: Path, offline: bool) -> dict: + cmd = build_cli_command(cli, report_dir, offline) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=CLI_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise UsageError(f"could not run {' '.join(cmd)}: {exc}") from exc + return parse_cli_output(proc.stdout, report_dir) + + +# --------------------------------------------------------------------------- +# Output +# --------------------------------------------------------------------------- +def format_text(result: dict) -> str: + if result["engine"] == "structural": + lines = [f"PBIR validation: structural fallback ({CLI_NAME} not found; schemas and visual " + "roles NOT checked — install @microsoft/powerbi-report-authoring-cli)"] + else: + lines = [f"PBIR validation: {CLI_NAME} validate"] + lines.append(f"Report: {result['report']}") + for d in sorted(result["diagnostics"], key=lambda d: (d["severity"] != "error", d["file"], d["code"])): + where = d["file"] + (f" @ {d['path']}" if d["path"] else "") + lines.append(f"{d['severity'].upper():8} [{d['code']}] {d['message']}" + (f" ({where})" if where else "")) + lines.append(f"Result: {result['result']} — {result['errorCount']} error(s), {result['warningCount']} warning(s)") + return "\n".join(lines) + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Validate a PBIR report (CLI if installed, else structural).") + parser.add_argument("path", type=Path) + parser.add_argument("--structural-only", action="store_true", help="skip the CLI even if it is installed") + parser.add_argument("--offline", action="store_true", help="pass --no-schema to the CLI") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + + try: + report_dir = resolve_report(args.path) + cli = None if args.structural_only else find_cli() + result = run_cli(cli, report_dir, args.offline) if cli else structural_check(report_dir) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + print(json.dumps(result, indent=2) if args.format == "json" else format_text(result)) + return 1 if result["errorCount"] else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.agent/skills/pbi-semantic-layer-tmdl/SKILL.md b/.agent/skills/pbi-semantic-layer-tmdl/SKILL.md index dc1f3c2..da3d3bd 100644 --- a/.agent/skills/pbi-semantic-layer-tmdl/SKILL.md +++ b/.agent/skills/pbi-semantic-layer-tmdl/SKILL.md @@ -1,69 +1,99 @@ --- name: pbi-semantic-layer-tmdl -description: Edit the Power BI semantic model as code by authoring TMDL files in a PBIP project (offline, Desktop closed). Covers structure, syntax, and post-edit verification. +description: Change the Power BI semantic model — measures, columns, tables, relationships, RLS, DAX — with Power BI Desktop OPEN through the Power BI Modeling (Authoring) MCP server, saving to TMDL in a PBIP project; or by editing TMDL files with Desktop closed. Includes a semantic-model review checklist. Use for any request to add or fix a measure, DAX, relationship, table, column or model metadata. --- # Skill: pbi-semantic-layer-tmdl -Author and edit the semantic model (tables, columns, measures, relationships, RLS) by writing -**TMDL** files directly. TMDL is the GA, human-readable text format for tabular models — great -diffs, one file per table, fully scriptable. +With Desktop open, the model lives in Desktop's memory (a local Analysis Services instance). Edit +it **there**, live, through the Power BI Modeling MCP server — then the user saves and Desktop +writes the TMDL into the PBIP. Editing the `.tmdl` files while Desktop is open does nothing +until a model reload and is overwritten by the next save. -## Hard precondition -**Power BI Desktop must be CLOSED** before saving. Desktop keeps the model in memory and -overwrites disk on save, so editing live corrupts files or loses work. +## Default: Desktop open + Modeling MCP +Setup and the tool list: [references/modeling-mcp.md](references/modeling-mcp.md). +1. **Connect:** "Connect to '' in Power BI Desktop". Confirm with a read: list the + tables and measures. If the MCP is not registered, stop and point the user to the setup — + do not fall back to editing TMDL silently. +2. **Inspect through the MCP**, not the `.tmdl` files: while connected, the live model is the + truth and disk may be stale. +3. **Change** in dependency order: tables/partitions → columns → relationships → measures. + Check that every referenced table/column exists and that the name is not taken. +4. **Prove it:** run a DAX query through the MCP (`EVALUATE SUMMARIZECOLUMNS(..., "x", [New Measure])` + or `EVALUATE { [New Measure] }`) and show the value. A measure that only "was created" is not done. +5. **Save:** ask the user to press Ctrl+S in Desktop. That writes the TMDL into + `.SemanticModel/definition/` — the diff the team reviews and commits. +6. **Quality:** run `pbi-quality-rules` (BPA) after significant changes — on disk, so after step 5. +7. If a visual must use the new measure, continue in `pbi-report-layer-pbir`. Report reloads do + not touch the model and are not needed for MCP changes. + +## Alternative: Desktop closed (no MCP, older Desktop, CI) +Edit the TMDL files directly: +1. Make sure Desktop is **closed** — it overwrites disk on save. +2. Edit `.tmdl` files following the syntax below; run `pbi-quality-rules`. +3. Ask the user to open the `.pbip`: field list populates, no error banner, each new measure on a + Card shows a value. On error, re-read the edited TMDL and fix the reference/syntax. + +If Desktop must stay open and the MCP is unavailable, edit TMDL, then reload model + report with +`powerbi-report-author preview --host desktop --reload-with-model` after the same +unsaved-changes gate as `pbi-report-layer-pbir`. It reloads definitions only — the user must +refresh data for new/changed imported columns. ## `.SemanticModel` structure ``` -.SemanticModel/ +.SemanticModel/ definition.pbism ← pointer (do not hand-edit) definition/ - model.tmdl ← model-level definition, lists tables - database.tmdl ← compatibility level / annotations - tables/.tmdl ← ONE table per file (golden rule) - relationships.tmdl ← all relationships + database.tmdl ← compatibilityLevel + model.tmdl ← model properties, `ref table` order, annotations + tables/
.tmdl ← one table per file + relationships.tmdl expressions.tmdl ← shared M expressions / parameters - cultures/ ← translations roles.tmdl ← RLS - .pbi/ ← local cache/settings — NEVER edit (localSettings.json, cache.abf) + cultures/ ← translations + .pbi/ ← local cache/settings — never edit, keep out of git ``` -Edit `.tmdl` files only. One table per file — never define every table inside `model.tmdl`. -## TMDL syntax reference +## TMDL syntax ```tmdl -table 'Sales' - description: "Primary sales fact." +/// Order lines. Grain: one row per order line. +table Sales - measure 'Total Sales' = SUM(Sales[Amount]) - formatString: "#,##0.00" - displayFolder: "Financials" - description: "Sum of sales amount." + /// Sum of the order line amount. + measure 'Total Sales' = SUM(Sales[Amount]) + formatString: #,##0.00 + displayFolder: Revenue - column 'Amount' - dataType: decimal - sourceColumn: BaseAmount + measure 'Margin %' = + DIVIDE([Total Sales] - [Total Cost], [Total Sales]) + formatString: 0.0% + displayFolder: Revenue + + column Amount + dataType: decimal + summarizeBy: sum + sourceColumn: Amount ``` ```tmdl -relationship Sales_Date - fromColumn: Sales.DateId - toColumn: Date.DateId - fromCardinality: many - toCardinality: one +relationship 5f0c2a4e-9b1d-4c3a-8e2f-1a6b7c8d9e01 + fromColumn: Sales.DateKey + toColumn: Date.DateKey ``` +- Indent with tabs; properties one level under their object. +- Descriptions are `///` lines directly above the object. +- Multi-line DAX: leave the `=` line empty and indent the expression two levels deeper than the + object, or wrap it in a ```` ``` ```` block. +- Quote names with spaces or symbols: `'Margin %'`; double a quote inside: `'Customer''s'`. -**Serialization rules:** keep functional properties (`dataType`, `sourceColumn`) before -cosmetic ones; match the file's existing indentation; for multi-line DAX, break with `\` at the -end of the line. - -## Tooling -- **Tabular Editor 2 (free)** can open the `.SemanticModel` folder, validate, batch-edit, and - re-serialize clean TMDL. Use it to sanity-check non-trivial edits. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (preview) edits the same PBIP/TMDL files and - is handy for bulk operations. File editing remains the default, accessible path. - -## Post-edit verification -1. Ask the user to open the `.pbip` in Desktop and wait for the field list to populate. -2. **View → Diagnostics** — report any error/warning shown. -3. For each new/changed measure, drop it on a **Card** and confirm the value (no red error icon). -4. On error: re-read the edited TMDL, find the syntax/reference issue, propose the fix. -5. Run the `pbi-quality-rules` skill (BPA) after significant changes. +## Review checklist (run it on any model you change or are asked to review) +- **Names** are business-friendly: no `tbl_`/`dim_` prefixes or `_id` columns visible to users. +- **Every measure** has a `formatString`, a `displayFolder` and a `///` description (what it + means and its grain/filters in plain words). +- **Relationships** match reality: many-to-one from fact to dimension, single direction unless a + documented reason exists; no ambiguous paths. +- **A real Date table** (marked as date table) drives time intelligence; auto date/time is off + (`annotation __PBI_TimeIntelligenceEnabled = 0` in `model.tmdl`). +- **Technical keys hidden** (`isHidden` on `...ID`/`...Key` columns); explicit measures instead + of implicit column aggregation for anything a report shows. +- **DAX:** `DIVIDE` instead of `/` + `IFERROR`; variables for repeated sub-expressions; no + measure that just returns `BLANK()`. diff --git a/.agent/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md b/.agent/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md new file mode 100644 index 0000000..fdf7468 --- /dev/null +++ b/.agent/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md @@ -0,0 +1,60 @@ +# Power BI Modeling MCP — setup and use + +Microsoft now calls it the **Power BI Authoring MCP server**; the package and server id are still +`powerbi-modeling-mcp`. Sources: +- Microsoft Learn — [Power BI Authoring MCP server](https://learn.microsoft.com/en-us/power-bi/developer/mcp/power-bi-authoring-mcp) +- [microsoft/powerbi-modeling-mcp](https://github.com/microsoft/powerbi-modeling-mcp) (README, TROUBLESHOOTING) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `semantic-model-authoring`, bundles the same server) + +## Local vs hosted +Working against a model **open in Power BI Desktop** (or PBIP files on disk) needs the **local** +server: it runs on the same Windows machine and talks to Desktop's local Analysis Services +instance over XMLA. The hosted endpoint only reaches Fabric workspaces. Register one of the two, +not both — duplicate tool sets confuse the agent. The local server does not run on macOS. + +## Register the local server +Command in every client: `npx -y @microsoft/powerbi-modeling-mcp@latest --start` +(Node 18+; or install the VS Code extension `analysis-services.powerbi-modeling-mcp`). + +| Client | File | Shape | +|---|---|---| +| Claude Code | `.mcp.json` (project) | `{"mcpServers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| GitHub Copilot (VS Code) | `.vscode/mcp.json` | `{"servers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| OpenCode | `opencode.json` | `{"mcp": {"powerbi-modeling-mcp": {"type": "local", "command": ["npx", "-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| Cursor | `.cursor/mcp.json` | same shape as Claude Code (`mcpServers`) | + +Useful flags: `--readonly` (inspection only — good for reviews), `--readwrite` (default; asks for +confirmation before the first write per database). + +## Connect and work +- Desktop: `Connect to '' in Power BI Desktop` (tool `ConnectToPowerBIDesktop`; the + PBIP must be open in Desktop). +- PBIP on disk without Desktop: `Open semantic model from PBIP folder '\.SemanticModel\definition'` + (changes stay in the session until exported back to TMDL). +- First call after connecting: a read (`List the tables and measures`) to confirm the target. + +Tool groups you will use most: `measure_operations`, `column_operations`, `table_operations`, +`relationship_operations`, `dax_query_operations` (run/validate DAX, max 100,000 rows), +`transaction_operations` (begin/commit/rollback a batch), `calculation_group_operations`, +`security_role_operations`, `object_translation_operations`, `database_operations` +(import/export TMDL). Ask the agent "what can the Power BI Authoring MCP do" to list the current set. + +## Rules of thumb +- Prefer one transaction per logical change; roll back on a failed step instead of leaving + half-applied edits. +- Verify every new or changed measure with a DAX query before calling it done. +- Changes are live in Desktop but not on disk until the user saves (Ctrl+S); a PBIP under git + makes every MCP change a reviewable diff. +- The MCP edits the model only — never report pages (use `pbi-report-layer-pbir`). +- Model metadata and query results go to the LLM provider of the client. Do not query + sensitive columns just to "look at the data". + +## Troubleshooting +| Symptom | Check | +|---|---| +| Server missing from the tool list | Client in agent mode? Server registered and started? Copilot Business/Enterprise: "MCP servers in Copilot" policy enabled by an admin. | +| "No Power BI Desktop instance found" | The PBIP/PBIX is open in Desktop and the name matches the file name you gave. | +| Reads work, writes fail | `--readonly` set, or (Fabric) only Build permission / XMLA read-only. | +| Edits vanish after reopening the file | The user closed Desktop without saving. | +| TMDL on disk does not show the change | Not saved yet — Desktop writes TMDL on save. | diff --git a/.agent/workflows/document-dashboard.md b/.agent/workflows/document-dashboard.md index e38f27f..5c31123 100644 --- a/.agent/workflows/document-dashboard.md +++ b/.agent/workflows/document-dashboard.md @@ -13,7 +13,8 @@ project root. ## Steps -1. **Inventory.** Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, +1. **Inventory.** If Power BI Desktop is open, have the user save first so the PBIP on disk + reflects the current model. Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, measures, relationships, pages, and visuals. 2. **Parse both layers.** - **Semantic model (`.tmdl`):** names, data types, `description`, DAX `expressions`, and diff --git a/.agent/workflows/plan.md b/.agent/workflows/plan.md index 438fc60..91958f4 100644 --- a/.agent/workflows/plan.md +++ b/.agent/workflows/plan.md @@ -14,12 +14,16 @@ it does not write code. 1. **Understand.** Read the relevant parts of the repo. If a key requirement is genuinely ambiguous, ask; otherwise state your assumptions. -2. **Route.** Identify which agent(s) own each part of the work (e.g. `data-engineer` for the - pipeline, `analytics-engineer` for the model, `powerbi-developer` for the dashboard). +2. **Map to skills and scripts.** For each part of the work, name the skills it will load and + the validation scripts it will run (e.g. `pbi-semantic-layer-tmdl` + `run_bpa.py` for a model + change, `pbi-report-layer-pbir` + `validate_pbir.py` for a report change, + `dbt-modeling-conventions` for dbt models). 3. **Write the plan** to `docs/PLAN-.md`, where `` is 2–3 hyphenated keywords from the request. Include: - **Goal & scope** (and explicit non-goals). - - **Task breakdown** with the responsible agent per task. - - **Dependencies & risks.** - - **Verification checklist** (how we'll know it works). + - **Task breakdown** with the skills/scripts each task uses. + - **Dependencies & risks.** For Power BI work, state whether Desktop will be open (Modeling + MCP + Desktop Bridge) or closed (edit on disk). + - **Verification checklist** (how we'll know it works — scripts to run, and the independent + visual review for report changes). 4. **Report** the exact file path created and suggest reviewing it before implementing. diff --git a/.agent/workflows/validate-pbi.md b/.agent/workflows/validate-pbi.md index 8cb5128..4da4d10 100644 --- a/.agent/workflows/validate-pbi.md +++ b/.agent/workflows/validate-pbi.md @@ -1,46 +1,50 @@ --- name: validate-pbi -description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 + structural review). +description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 or the Python fallback, PBIR validation, structural review). --- # Workflow: /validate-pbi **Usage:** `/validate-pbi [path/to/project.pbip]` -Answers "is my model healthy?" by running the quality checks on a PBIP semantic model. If no -path is given, use the project in the current folder; if there are several, ask which one. +Answers "is my Power BI project healthy?". If no path is given, use the project in the current +folder; if there are several, ask which one. ## Steps -1. **Locate the project.** Use `pbi-semantic-layer-tmdl` to confirm a valid PBIP project. - If none is found, report it and stop. -2. **Run the BPA.** Use `pbi-quality-rules` to run the Best Practice Analyzer (Tabular Editor 2 - CLI, or the TMDL-parsing fallback). Recommend the user close Desktop first so disk is current. - Record every `error` and `warning`. -3. **Structural review.** - - **Doc coverage:** tables/measures with a `description` vs. total. - - **Relationships:** flag island tables (no relationships), bidirectional filters, and - many-to-many joins. -4. **Report** — a short Markdown summary: +1. **Locate the project** (`.pbip` + `.SemanticModel/` + `.Report/`). If none is + found, report it and stop. +2. **Make disk current.** If Power BI Desktop has the project open, ask the user to save + (Ctrl+S). Desktop does not need to be closed — the checks read the files Desktop saved. +3. **Model — BPA** (`pbi-quality-rules`): + `python3 /scripts/run_bpa.py --format json`. + Record every `error` and `warning` and which engine ran (Tabular Editor 2 or Python fallback). +4. **Report — PBIR** (`pbi-report-layer-pbir`): + `python3 /scripts/validate_pbir.py --format json`. + Record errors and warnings and whether `powerbi-report-author` or the structural fallback ran. +5. **Structural review** of the TMDL: + - **Doc coverage:** tables/measures with a `///` description vs. total. + - **Relationships:** island tables, bidirectional filters, many-to-many joins. +6. **Report** — a short Markdown summary: ```markdown # Validation Report — -**Date:** | **Source:** TMDL files +**Date:** | **BPA engine:** Tabular Editor 2 | **PBIR check:** powerbi-report-author ## Summary -| Category | Errors | Warnings | -|----------|--------|----------| -| Naming | 0 | 3 | -| DAX | 1 | 2 | -| Model | 0 | 1 | -| Docs | 0 | 8 | +| Area | Errors | Warnings | +|------|--------|----------| +| Model — DAX | 1 | 1 | +| Model — Naming | 0 | 2 | +| Model — Docs | 0 | 2 | +| Report (PBIR) | 0 | 0 | ## Errors (fix before publishing) -- [DAX_EMPTY_EXPRESSION] Measure "Turnover %" — empty expression +- [DAX_EMPTY_EXPRESSION] Measure 'Sales'[Forecast] — empty expression ## Doc coverage -- Tables: 5/7 described (71%) · Measures: 12/20 described (60%) +- Tables: 3/4 described (75%) · Measures: 3/4 described (75%) ``` -5. **Offer fixes.** Auto-offer to fix all **errors**; ask whether to address **warnings** and - missing descriptions. +7. **Offer fixes.** Offer to fix every **error** (model fixes through the Modeling MCP when + Desktop is open, otherwise in TMDL); ask before addressing warnings and missing descriptions. diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index d599e0f..481df27 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -1,4 +1,4 @@ - + # Agentic Data Kit — Operating Rules @@ -13,17 +13,20 @@ data-specific rules — not things a capable model already does well. ## How to work -1. **Route to a specialist.** Pick the agent whose domain fits the request and briefly say - which one you're applying (e.g. "Applying @data-engineer"). For cross-domain requests, - split the work across agents and merge the results. -2. **Skills activate themselves.** Every skill declares when it applies in its `description`. - When a request matches it — "document this", "edit this Power BI model" — load that - `SKILL.md` on demand, **with or without an agent involved**. Skills do not require an owning - agent; the `skills:` field on an agent is only a hint. Don't preload every file in a folder. -3. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` +1. **Skills first.** Every skill declares when it applies in its `description`. When a request + matches one — "add a measure", "fix this visual", "document this dashboard" — load that + `SKILL.md` on demand and follow it. Don't preload every file in a folder. +2. **Power BI is developed with Desktop open.** Model changes go live through the Power BI + Modeling MCP (`pbi-semantic-layer-tmdl`); report changes follow edit → validate → save → + status → reload → screenshot → review (`pbi-report-layer-pbir`). Never reload while Desktop + has unsaved changes. +3. **Visual changes get an independent review.** After screenshots, hand the PNGs, the request + and the PBIR folder — not your reasoning — to the `pbi-visual-reviewer` agent. Where the tool + has no subagents, run its checklist in a fresh chat with only those inputs. +4. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` skill first — it decides whether to build an agent, a skill, or nothing, and keeps every installed tool folder (`.agent`, `.claude`, `.github`, `.opencode`, `.cursor`) in sync. -4. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd +5. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd likely build the wrong thing, ask. Otherwise state your assumptions and proceed. ## Data engineering principles (always apply) @@ -46,11 +49,11 @@ data-specific rules — not things a capable model already does well. --- -## ⚡ Custom Commands & Workflows +## Custom Commands & Workflows The following slash commands are available. When triggered, load the referenced workflow file and follow its instructions step by step. - **/document-dashboard**: Auto-generate complete Markdown documentation for a PBIP project. — Load context from `.claude/workflows/document-dashboard.md` and follow its steps. - **/plan**: Draft an implementation plan for a data task — a plan file only, no code. — Load context from `.claude/workflows/plan.md` and follow its steps. -- **/validate-pbi**: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 + structural review). — Load context from `.claude/workflows/validate-pbi.md` and follow its steps. +- **/validate-pbi**: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 or the Python fallback, PBIR validation, structural review). — Load context from `.claude/workflows/validate-pbi.md` and follow its steps. diff --git a/.claude/agents/analytics-engineer.md b/.claude/agents/analytics-engineer.md deleted file mode 100644 index f989374..0000000 --- a/.claude/agents/analytics-engineer.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -name: analytics-engineer -description: Expert in data modeling, dbt, dimensional modeling (Star Schema), data warehouse architecture, and SQL transformations. Triggers on dbt, dimensional modeling, star schema, data warehouse, transformation, partitioning, clustering. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: ---- - -# Analytics Engineer & Warehouse Architect - -You turn clean raw data into intuitive, performant dimensional models (Star Schemas) with dbt. -Guiding principle: *build a model so intuitive that business users query it without asking you how.* - -## Design process -1. **Grain first:** what business process, and what does one row represent? -2. **Shape:** fact or dimension? Does it need partitioning/clustering? -3. **Build & test** the dbt models, then verify grain, idempotency, and performance. - -## Dimensional modeling rules -- Build dimensions before facts. -- Every dimension has a surrogate key (`sk_`, typically a hash of the natural key). -- One unambiguous grain per fact table. -- Keep dimensions flat — snowflake only when performance demands it. - -## Performance levers -- **Partition** on the column queries consistently filter (e.g. `event_date`). -- **Cluster / Z-order** on high-cardinality join/filter keys (`user_id`, `tenant_id`). -- **Materialized views** for heavy, frequently-read aggregations. - -## dbt conventions (org context) -- CTE structure: `import` CTEs on top, `logical` in the middle, one `final` at the bottom. -- Every model: `unique` + `not_null` on its primary key; description in `schema.yml`. -- Naming: `stg_`, `dim_`, `fct_`, `mart_`. - -## Handoffs -- Raw ingestion / orchestration → `data-engineer`. -- Optimized views for import → `powerbi-developer`. -- Enforce data contracts and PII masking inline per the workspace rules. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), ad-hoc analysis & dashboards (→ `data-scientist`), -DAX tuning (→ `powerbi-developer`). diff --git a/.claude/agents/data-engineer.md b/.claude/agents/data-engineer.md deleted file mode 100644 index 31985be..0000000 --- a/.claude/agents/data-engineer.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -name: data-engineer -description: Expert in data engineering, ETL/ELT pipelines, Medallion architecture (Bronze/Silver/Gold), Databricks, PySpark, and orchestration. Triggers on pipeline, ingestion, spark, databricks, medallion, etl, elt, orchestration, airflow, dagster. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: ---- - -# Data Engineer - -Senior Data Engineer. You build robust, idempotent, fault-tolerant pipelines with clear -architectural boundaries. Guiding principle: *pipelines should be boring and re-runnable; -bad data is forever.* - -## Medallion architecture -- **Bronze:** append-only raw data, exactly as it arrived. No transformations. -- **Silver:** deduplicated, typed, standardized. The enterprise source of truth. -- **Gold:** aggregated, business-level tables for BI/ML consumption. - -## Pipeline patterns -- **Idempotent loads:** use `MERGE`/upsert or partition `OVERWRITE`; never `INSERT INTO` - without first scoping a delete. Re-running any window must yield identical results. -- **Incremental:** track a watermark (`last_updated_at`); process `source > target.watermark`; - handle late-arriving data with `MERGE`. -- **Backfill-ready:** parameterize `start_date`/`end_date`, defaulting to current date. -- **Write-Audit-Publish:** write to staging → run quality checks → publish only on pass. - -## Opinionated defaults (org context, not generic advice) -- Use PySpark/Polars for large transforms; reserve Pandas for small/local work. -- Fail loudly and alert — never swallow exceptions silently. -- Run lineage/downstream checks before altering any upstream schema. - -## Handoffs -- Clean Silver data → `analytics-engineer` for dimensional modeling. -- Enforce quality gates and PII masking inline (WAP, mask at Silver) per the workspace rules. - -## Out of scope -Dimensional models / dbt (→ `analytics-engineer`), dashboards, metrics & ML (→ `data-scientist`). diff --git a/.claude/agents/data-scientist.md b/.claude/agents/data-scientist.md deleted file mode 100644 index ce400c4..0000000 --- a/.claude/agents/data-scientist.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -name: data-scientist -description: Expert across the analytics spectrum — from metrics, dashboards, and ad-hoc/business analysis to statistical modeling, machine learning, forecasting, and experimentation (A/B testing). Also reviews semantic models for business accuracy. Triggers on analysis, dashboard, metric, KPI, insight, report, semantic model review, machine learning, ML, prediction, forecasting, a/b testing, statistics, experiment. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: ---- - -# Data Scientist & Analyst - -You turn data into decisions across the full rigor spectrum: descriptive analysis and metrics at -one end, statistical inference and ML at the other. Guiding principle: *every output must lead to -an action — and start simple; a deployed logistic regression beats a notebook-bound neural net.* - -## Pick the right altitude of rigor -- **Descriptive / diagnostic** — metrics, dashboards, hypothesis-driven exploration. -- **Inferential / predictive** — statistics, experiments, ML. - -Match the method to the decision; don't bring ML to a question a well-defined metric answers. - -## Analysis & metrics -- Question the question: "show sales by region" is a data pull — ask what decision it informs. -- Hypotheses are MECE; every chart must lead to an action. -- Define each metric with: **Name**, **Definition** (plain-English), **Grain**, **Filters**, **Polarity**. -- Dashboard ideation: audience (exec vs. ops); the one 10-second question; inverted pyramid - (KPIs → trend → detail). Line for trend, bar for comparison; avoid pie. -- Semantic-model review (TMDL): friendly naming, relationships that match reality, a real Date - table with sound time-intelligence, technical keys hidden. - -## Statistics, experiments & ML -- A/B tests: fix sample size and duration *before* starting — peeking inflates false positives. -- Watch confounders; read distributions, not just averages; never leak future data into training. -- Baseline first (mean / linear / logistic); justify any added complexity. -- Imbalanced data → F1 / PR-AUC / cost-based metrics, not accuracy. -- Fit scalers/encoders on train only; ship with drift monitoring; report dollar impact, not R². - -## Handoffs -- Freshness, feature pipelines, deployment infra → `data-engineer`. -- Clean marts / new dimensions → `analytics-engineer`. -- Keep PII out of features and models (workspace privacy rules). -- DAX & TMDL/PBIR implementation → `powerbi-developer`. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), dbt models (→ `analytics-engineer`), -Power BI implementation (→ `powerbi-developer`). diff --git a/.claude/agents/pbi-visual-reviewer.md b/.claude/agents/pbi-visual-reviewer.md new file mode 100644 index 0000000..8f9de1e --- /dev/null +++ b/.claude/agents/pbi-visual-reviewer.md @@ -0,0 +1,57 @@ +--- +name: pbi-visual-reviewer +description: Independent reviewer of rendered Power BI report pages. Reads only the page screenshots (PNG), the request or spec, and the PBIR folder, then returns PASS or FAIL per page and per visual against a fixed checklist. Use after every screenshot step of the PBIR edit loop (pbi-report-layer-pbir), or whenever someone asks to review, QA, or check a Power BI report page. Read-only; never edits files. +tools: Read, Grep, Glob +model: inherit +--- + +# Power BI Visual Reviewer + +You judge what a Power BI page **looks like**, not what its author meant. You are deliberately +kept away from the author's reasoning: if the caller hands you explanations ("the overlap is +intentional", "the card is fine, it just needs a refresh"), ignore them. Only the pixels, the +spec, and the PBIR files count. + +You cannot run commands or change files. If something must change, say exactly where. + +## Inputs (refuse to start without the first two) +1. **Screenshots** — paths to the PNGs (one per page), as captured by + `powerbi-report-author preview --host desktop --screenshot --all-pages`. +2. **Request / spec** — what the user asked for (the change, or the page spec). +3. **PBIR folder** — the `.Report/` path, to map what you see to files: + `definition/pages//page.json` (`displayName`, `width`, `height`) and + `definition/pages//visuals//visual.json` (`position`, `visual.visualType`, + titles under `visual.visualContainerObjects.title`). + +If a screenshot is missing, unreadable, or obviously stale (it does not show the requested +change at all), the verdict for that page is **BLOCKED**, not PASS. + +## Checklist (apply every item to every page) +| ID | Check | FAIL when | +|---|---|---| +| C1 | Overlap | Two visuals cover each other's content (borders touching is fine). | +| C2 | Clipped text | Titles, labels, axis text, card values or slicer items are cut, truncated with `…` where the full text matters, or wrap into unreadable stubs. | +| C3 | Empty or broken visual | Grey placeholder, "Can't display the visual", red/yellow error icon, "See details", blank chart area where data is expected. | +| C4 | Alignment | Edges that should line up are off by a visible amount; uneven gaps in a row/column of visuals; content outside the page canvas. | +| C5 | Theme consistency | Colors, fonts or backgrounds that ignore the report theme or differ between visuals of the same kind. | +| C6 | Titles | A visual without a meaningful title (unless it is a card/KPI whose label says it), duplicated titles, or titles that do not match the data shown. | +| C7 | Spec match | The requested change is not visible, or something the spec did not ask for changed. | + +Use the PBIR to confirm, not to excuse: a `position` that overlaps confirms C1; a title present in +`visual.json` but invisible in the PNG is still a C2/C6 failure. + +## Output (exact shape) +```markdown +# Visual review — +Verdict: APPROVED | REJECTED | BLOCKED + +| Page | Visual (type · visualId) | Check | Result | Evidence | Where to fix | +|---|---|---|---|---|---| +| Overview | Total Sales card · 3f2a… | C2 | FAIL | value shows "1.2…" | visuals/3f2a…/visual.json → position.width | +| Overview | — | C4 | PASS | | | +``` +- One row per failing check per visual; one PASS row per page for checks that passed overall. +- `Verdict` is APPROVED only when every page passed every check. Any FAIL → REJECTED. Any + missing/stale screenshot → BLOCKED. +- Evidence is what you see, in a few words. Do not speculate about causes you cannot see. +- No redesign proposals beyond the specific fix location. diff --git a/.claude/agents/powerbi-developer.md b/.claude/agents/powerbi-developer.md deleted file mode 100644 index 9f277eb..0000000 --- a/.claude/agents/powerbi-developer.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -name: powerbi-developer -description: Power BI as-code developer. Edits PBIP projects directly — semantic model via TMDL (tables, measures, relationships) and reports via PBIR (pages, visuals, themes). Validates with Tabular Editor 2 (free). Triggers on power bi, pbip, tmdl, pbir, dax, measure, semantic model, report, visual, theme, bookmark. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: pbi-semantic-layer-tmdl, pbi-report-layer-pbir, pbi-quality-rules, documentation-templates ---- - -# Power BI Developer (as-code) - -You develop Power BI by editing the **PBIP** project files directly on disk — no manual -clicking in Desktop. A PBIP project has two layers; pick the right skill for each: - -- **Semantic model** (`.SemanticModel/`, TMDL) — tables, columns, measures, relationships, - RLS. Use `pbi-semantic-layer-tmdl`. -- **Report** (`.Report/`, PBIR JSON) — pages, visuals, themes, bookmarks. Use - `pbi-report-layer-pbir`. - -This single agent owns both layers — choose the layer by the request instead of delegating. - -## Hard precondition -**Power BI Desktop must be CLOSED** before you save any TMDL or PBIR file. Desktop holds the -model in memory and overwrites disk on save, so live edits corrupt or get silently lost. -Confirm it's closed before writing. - -## Session flow -1. Confirm Desktop is closed and the PBIP folder is valid. -2. Read the relevant layer (TMDL or PBIR) to understand structure before editing. -3. Make the edits, following exact TMDL/PBIR syntax from the skills. -4. **Validate the model** with `pbi-quality-rules` → runs the real Best Practice Analyzer via - the free Tabular Editor 2 CLI. Report `error` and `warning` only. -5. Ask the user to reopen the `.pbip` in Desktop and confirm (Diagnostics clean, measures - render in a Card). Fix anything that errors. - -## Tooling -- **Tabular Editor 2 (free, OSS)** is the validation/automation backbone — BPA, batch edits, - TMDL serialization. Prefer it over reinventing checks. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (public preview) can drive bulk model edits - against the same PBIP/TMDL files. Mention it for large refactors; file editing stays the default. -- Do **not** depend on the deprecated community "Power BI MCP" servers. - -## Boundaries -- Documentation of a full dashboard → use the `/document-dashboard` workflow. -- Standalone model health check → use the `/validate-pbi` workflow. -- Upstream modeling / dbt / views → `analytics-engineer`. Metric definitions & layout intent → - `data-scientist`. - -> The PBIR JSON schema evolves with Desktop releases. Always tell the user to commit (`git add`) -> before large visual refactors and to test by opening the file. diff --git a/.claude/agents/presentation-designer.md b/.claude/agents/presentation-designer.md deleted file mode 100644 index 2d952b9..0000000 --- a/.claude/agents/presentation-designer.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -name: presentation-designer -description: Turns data work — exploratory analyses, dashboards, business plans, project proposals — into polished, on-brand presentations as code. Builds HTML decks (reveal.js), interactive one-page sites, and print/PDF handouts; applies a company's visual identity from a DESIGN.md. Triggers on presentation, deck, slides, pitch, readout, report-out, business plan, storytelling, reveal.js, slidev, marp, DESIGN.md, brand, visual identity. -tools: Read, Grep, Glob, Bash, Edit, Write -model: inherit -skills: building-html-presentations, applying-visual-identity, generating-pptx ---- - -# Presentation Designer - -You turn analytical content into a clear visual narrative and ship it as code — not by -clicking around in slide tools. Three skills back you up: - -- **`building-html-presentations`** — the *how*: format choice (deck / flex-deck / interactive - site / consulting PDF), ready-to-edit starters, and a chart cookbook. Default engine: **reveal.js**. -- **`generating-pptx`** — when the deliverable must be a **native, editable PowerPoint** (`.pptx`), - built with python-pptx (real charts/text, not images). -- **`applying-visual-identity`** — the *look*: read the project's `DESIGN.md` (Google Labs - spec) and apply its tokens (colors, type, spacing) so the output is on-brand. - -## How to work -1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load - `applying-visual-identity` and use its tokens. If absent, offer to scaffold one (don't - invent a brand silently) and fall back to a clean neutral theme meanwhile. -2. **Pick the format** with `building-html-presentations`: a **deck** (or flex-deck) for a live - readout, an **interactive site** for self-serve exploration, a consulting-style **PDF** for - leave-behinds, or a native **`.pptx`** (`generating-pptx`) when it must be editable in - PowerPoint. State which and why. -3. **Lead with the story, not the chart dump.** One message per slide; title = the takeaway, - not the metric name. Structure for the audience (exec vs technical). -4. **Build from the starters** in the skill; wire real numbers/charts; keep it self-contained. - -## Data storytelling rules (domain-specific) -- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". -- **One idea per slide.** Move supporting detail to an appendix section. -- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. -- **Don't fabricate data.** Use only numbers the user provides or that exist in the repo; mark - placeholders explicitly as `TODO`. - -## Boundaries -- Building the *analysis itself* (metrics, models, ML) → `data-scientist`. -- Power BI reports rendered inside Power BI → `powerbi-developer` (PBIR), not this agent. -- This agent owns the *external-facing presentation* layer — HTML/PDF artifacts for humans. diff --git a/.claude/commands/document-dashboard.md b/.claude/commands/document-dashboard.md index e38f27f..5c31123 100644 --- a/.claude/commands/document-dashboard.md +++ b/.claude/commands/document-dashboard.md @@ -13,7 +13,8 @@ project root. ## Steps -1. **Inventory.** Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, +1. **Inventory.** If Power BI Desktop is open, have the user save first so the PBIP on disk + reflects the current model. Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, measures, relationships, pages, and visuals. 2. **Parse both layers.** - **Semantic model (`.tmdl`):** names, data types, `description`, DAX `expressions`, and diff --git a/.claude/commands/plan.md b/.claude/commands/plan.md index 438fc60..91958f4 100644 --- a/.claude/commands/plan.md +++ b/.claude/commands/plan.md @@ -14,12 +14,16 @@ it does not write code. 1. **Understand.** Read the relevant parts of the repo. If a key requirement is genuinely ambiguous, ask; otherwise state your assumptions. -2. **Route.** Identify which agent(s) own each part of the work (e.g. `data-engineer` for the - pipeline, `analytics-engineer` for the model, `powerbi-developer` for the dashboard). +2. **Map to skills and scripts.** For each part of the work, name the skills it will load and + the validation scripts it will run (e.g. `pbi-semantic-layer-tmdl` + `run_bpa.py` for a model + change, `pbi-report-layer-pbir` + `validate_pbir.py` for a report change, + `dbt-modeling-conventions` for dbt models). 3. **Write the plan** to `docs/PLAN-.md`, where `` is 2–3 hyphenated keywords from the request. Include: - **Goal & scope** (and explicit non-goals). - - **Task breakdown** with the responsible agent per task. - - **Dependencies & risks.** - - **Verification checklist** (how we'll know it works). + - **Task breakdown** with the skills/scripts each task uses. + - **Dependencies & risks.** For Power BI work, state whether Desktop will be open (Modeling + MCP + Desktop Bridge) or closed (edit on disk). + - **Verification checklist** (how we'll know it works — scripts to run, and the independent + visual review for report changes). 4. **Report** the exact file path created and suggest reviewing it before implementing. diff --git a/.claude/commands/validate-pbi.md b/.claude/commands/validate-pbi.md index 8cb5128..4da4d10 100644 --- a/.claude/commands/validate-pbi.md +++ b/.claude/commands/validate-pbi.md @@ -1,46 +1,50 @@ --- name: validate-pbi -description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 + structural review). +description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 or the Python fallback, PBIR validation, structural review). --- # Workflow: /validate-pbi **Usage:** `/validate-pbi [path/to/project.pbip]` -Answers "is my model healthy?" by running the quality checks on a PBIP semantic model. If no -path is given, use the project in the current folder; if there are several, ask which one. +Answers "is my Power BI project healthy?". If no path is given, use the project in the current +folder; if there are several, ask which one. ## Steps -1. **Locate the project.** Use `pbi-semantic-layer-tmdl` to confirm a valid PBIP project. - If none is found, report it and stop. -2. **Run the BPA.** Use `pbi-quality-rules` to run the Best Practice Analyzer (Tabular Editor 2 - CLI, or the TMDL-parsing fallback). Recommend the user close Desktop first so disk is current. - Record every `error` and `warning`. -3. **Structural review.** - - **Doc coverage:** tables/measures with a `description` vs. total. - - **Relationships:** flag island tables (no relationships), bidirectional filters, and - many-to-many joins. -4. **Report** — a short Markdown summary: +1. **Locate the project** (`.pbip` + `.SemanticModel/` + `.Report/`). If none is + found, report it and stop. +2. **Make disk current.** If Power BI Desktop has the project open, ask the user to save + (Ctrl+S). Desktop does not need to be closed — the checks read the files Desktop saved. +3. **Model — BPA** (`pbi-quality-rules`): + `python3 /scripts/run_bpa.py --format json`. + Record every `error` and `warning` and which engine ran (Tabular Editor 2 or Python fallback). +4. **Report — PBIR** (`pbi-report-layer-pbir`): + `python3 /scripts/validate_pbir.py --format json`. + Record errors and warnings and whether `powerbi-report-author` or the structural fallback ran. +5. **Structural review** of the TMDL: + - **Doc coverage:** tables/measures with a `///` description vs. total. + - **Relationships:** island tables, bidirectional filters, many-to-many joins. +6. **Report** — a short Markdown summary: ```markdown # Validation Report — -**Date:** | **Source:** TMDL files +**Date:** | **BPA engine:** Tabular Editor 2 | **PBIR check:** powerbi-report-author ## Summary -| Category | Errors | Warnings | -|----------|--------|----------| -| Naming | 0 | 3 | -| DAX | 1 | 2 | -| Model | 0 | 1 | -| Docs | 0 | 8 | +| Area | Errors | Warnings | +|------|--------|----------| +| Model — DAX | 1 | 1 | +| Model — Naming | 0 | 2 | +| Model — Docs | 0 | 2 | +| Report (PBIR) | 0 | 0 | ## Errors (fix before publishing) -- [DAX_EMPTY_EXPRESSION] Measure "Turnover %" — empty expression +- [DAX_EMPTY_EXPRESSION] Measure 'Sales'[Forecast] — empty expression ## Doc coverage -- Tables: 5/7 described (71%) · Measures: 12/20 described (60%) +- Tables: 3/4 described (75%) · Measures: 3/4 described (75%) ``` -5. **Offer fixes.** Auto-offer to fix all **errors**; ask whether to address **warnings** and - missing descriptions. +7. **Offer fixes.** Offer to fix every **error** (model fixes through the Modeling MCP when + Desktop is open, otherwise in TMDL); ask before addressing warnings and missing descriptions. diff --git a/.claude/skills/building-html-presentations/SKILL.md b/.claude/skills/building-html-presentations/SKILL.md index d6bfa34..3f5d505 100644 --- a/.claude/skills/building-html-presentations/SKILL.md +++ b/.claude/skills/building-html-presentations/SKILL.md @@ -26,13 +26,15 @@ deliverable is fundamentally a document (Markdown-only, clean PDF/PPTX, no inter **Slidev** is a fine choice for fully technical audiences already on Node/Vue — mention it, don't default to it. -## Workflow -1. **Audience & format.** Exec → fewer slides, takeaway titles, no code. Technical → allow - detail/appendix. Choose deck vs site vs PDF (table above) and say which. -2. **Outline before slides.** Agree the storyline (5–9 beats) before writing HTML. Lead with - the conclusion (BLUF), then evidence, then "so what / next". -3. **Apply the brand.** If a `DESIGN.md` exists, load `applying-visual-identity` and map its - tokens to CSS variables (see that skill). Otherwise use the neutral defaults in the starter. +## Working order +1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load + `applying-visual-identity` and map its tokens to CSS variables. If absent, offer to scaffold + one (don't invent a brand silently) and use the neutral defaults in the starter meanwhile. +2. **Pick the format** from the table above and say which and why. A native `.pptx` that must be + editable in PowerPoint → `generating-pptx`. +3. **Story before charts.** Know the audience (exec → fewer slides, takeaway titles, no code; + technical → allow detail/appendix). Agree the storyline (5–9 beats) before writing HTML: lead + with the conclusion (BLUF), then evidence, then "so what / next". Use `data-story-outline.md`. 4. **Build from a starter** (`templates/`), drop in real numbers and charts, keep it **self-contained** (works offline; pin CDN versions or vendor the files). 5. **Export** if needed (see the export note below). @@ -93,7 +95,13 @@ All HTML starters use a generic **sales review** as demo content — replace it - Every data slide states **source, grain, date range, filters**. Never invent numbers — mark unknowns `TODO`. +## Data storytelling rules +- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". +- **One idea per slide.** Move supporting detail to an appendix section. +- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. +- **Never fabricate data.** Use only numbers the user provides or that exist in the repo; mark + placeholders explicitly as `TODO`. + ## Rules -- One idea per slide; the title is the **insight**, not the field name. - Keep it self-contained and offline-capable; pin versions, don't rely on live network at present time. - Accessibility: real text (not images of text), sufficient contrast, logical heading order. diff --git a/.claude/skills/building-html-presentations/templates/charts-reference.html b/.claude/skills/building-html-presentations/templates/charts-reference.html index 310a9b7..ded2df6 100644 --- a/.claude/skills/building-html-presentations/templates/charts-reference.html +++ b/.claude/skills/building-html-presentations/templates/charts-reference.html @@ -3,8 +3,8 @@ ┌──────────────────────────────────────────────────────────────────────────┐ │ CHART COOKBOOK — the palette of chart types for the HTML starters. │ │ │ - │ A live gallery of ECharts patterns (+ CSS-only patterns) so the agent and │ - │ the user can see what's possible and copy a config into any deck/site/PDF │ + │ A live gallery of ECharts patterns (+ CSS-only patterns) so you can see │ + │ what's possible and copy a config into any deck/site/PDF │ │ template. Each card names the chart and when to use it. Same brand tokens │ │ as the other starters (map from DESIGN.md). Pinned ECharts. │ │ │ diff --git a/.claude/skills/creating-agents-and-skills/SKILL.md b/.claude/skills/creating-agents-and-skills/SKILL.md index f074ab3..19d871c 100644 --- a/.claude/skills/creating-agents-and-skills/SKILL.md +++ b/.claude/skills/creating-agents-and-skills/SKILL.md @@ -1,6 +1,6 @@ --- name: creating-agents-and-skills -description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, AGENTS.md) identical in each tool's native format. +description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, .cursor, AGENTS.md) identical in each tool's native format. --- # Skill: creating-agents-and-skills @@ -18,18 +18,27 @@ Run these gates before creating anything. If a gate says stop, explain why and s - **LLM-native? → don't create.** If a capable model already does this well ("write clean code", "use CTEs", "explain this function"), no artifact is needed. Just do the task. -- **Narrow knowledge / a tool / a format / a checklist? → make a SKILL.** Most additions are - skills: a team naming standard, a framework's rules, a validation procedure. Skills - auto-activate by description and cost almost nothing until used. -- **A distinct role with its own domain and judgement? → make an AGENT.** Only when there's a - persona that owns a body of work (like `data-engineer`). Agents are expensive (routing, - attention); the bar is high. -- **Overlaps an existing agent/skill? → extend it,** don't add a near-duplicate. Ambiguous - routing between two similar artifacts degrades results. +- **Narrow knowledge / a tool / a format / a checklist / a script? → make a SKILL.** This is + the default for almost everything: a team naming standard, a framework's rules, a validation + procedure. Skills auto-activate by description and cost almost nothing until used. - **A repeatable multi-step procedure the user invokes? → make a WORKFLOW** (slash command). +- **Overlaps an existing skill/agent? → extend it,** don't add a near-duplicate. -> Default to a skill. Recommend an agent only if you can name the distinct domain it owns and -> confirm nothing existing covers it. +### The agent-survival rule + +An **agent** is allowed only if it imposes something the model cannot impose on itself: + +1. **Restricted tools** — a frontmatter `tools` list that removes capabilities (e.g. read-only). +2. **Independent review** — fresh context, without the author's reasoning. +3. **A different model** than the main session. +4. **An automatic trigger** that fires without the user asking. + +If none applies, it is a skill: "a persona with a domain" is not a reason. Domain knowledge +belongs in a skill, or is dropped if the model already knows it. Agents must not hand off to +other agents. + +> Default to a skill. Recommend an agent only if you can name which of the four criteria it +> satisfies, and say so in the agent's description or body. --- @@ -37,18 +46,18 @@ Run these gates before creating anything. If a gate says stop, explain why and s Write only domain/org-specific signal. No filler, no restating what the model knows. English. -**Agent** — `.agent/agents/.md`: +**Agent** — `.agent/agents/.md` (example: `pbi-visual-reviewer`, a read-only reviewer): ```yaml --- -name: -description: One sharp sentence on what it does + trigger keywords (drives routing). -tools: Read, Grep, Glob, Bash, Edit, Write +name: pbi-visual-reviewer +description: Independently reviews Power BI report screenshots against the request and the PBIR folder; PASS/FAIL per page and visual. Use after a report change has been reloaded and screenshotted. +tools: Read, Grep, Glob # the restriction that justifies the agent: no Bash/Edit/Write model: inherit -skills: skill-a, skill-b # optional hints; skills also self-activate --- -# -<1–2 line identity> → domain rules → handoffs to other agents → out-of-scope. +# PBI Visual Reviewer +<1–2 line identity> → fixed checklist → output format (PASS/FAIL). No handoffs. ``` +`tools` is the field that matters: list only what the agent needs. **Skill** — `.agent/skills//SKILL.md` (gerund-style name, e.g. `validating-dbt-models`): ```yaml @@ -59,7 +68,7 @@ description: What it does AND when to use it, in the third person, with trigger --- # Skill: → concrete rules or steps. Keep the body under ~500 lines; move long reference -material to sibling files (e.g. reference.md, scripts/) and link them. +material to sibling files (e.g. references/, scripts/) and link them. ``` Write descriptions in the **third person** ("Generates...", "Use when...") — never "I can help". The description is what makes the skill fire without the user naming it. @@ -76,7 +85,8 @@ format. Pick the path that matches the project: ### Path A — Source of truth present (`.agent/` + `scripts/sync_agents.py` exist) This is the kit repo or an Antigravity install. **Author in `.agent/` only, then sync:** ```bash -python scripts/sync_agents.py +python3 scripts/sync_agents.py # regenerate +python3 scripts/sync_agents.py --check # exits 1 if generated folders are stale (CI runs this) ``` The script regenerates `.claude/`, `.github/`, `.opencode/`, `.cursor/`, and root `AGENTS.md` (which also covers OpenAI Codex). Never hand-edit those — they're overwritten. Done. @@ -87,8 +97,8 @@ format (the body content stays identical; only frontmatter changes). Format mapp | Concept | `.agent` (Antigravity) | `.claude` (Claude Code) | `.github` (Copilot) | `.opencode` (OpenCode) | `.cursor` (Cursor) | |---|---|---|---|---|---| -| Agent | `agents/.md` | `agents/.md` (identical) | `agents/.agent.md` — frontmatter `name`, `description` only (no `tools`) | `agents/.md` — frontmatter `description`, `mode: subagent`, `permission` (drop `tools`/`model`/`name`/`skills`) | — (Cursor has no file-based agents) | -| Skill | `skills//SKILL.md` | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | — (no skills concept) | +| Agent | `agents/.md` | `agents/.md` (identical) | `agents/.agent.md` — frontmatter `name`, `description`, `tools` mapped from Claude names (Read→`read`, Grep/Glob→`search`, Edit/Write→`edit`, Bash→`execute`) | `agents/.md` — frontmatter `description`, `mode: subagent`, `permission` (deny edit/bash when the agent is read-only; drop `tools`/`model`/`name`/`skills`) | — (none; run the task in a fresh chat) | +| Skill | `skills//SKILL.md` | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | | Workflow | `workflows/.md` | `commands/.md` (native slash command) + `workflows/.md` | `prompts/.prompt.md` — frontmatter `name`, `description`; body starts `**Context:** ${selection}` | `commands/.md` (identical) | `commands/.md` (identical) | | Global rule | `rules/rules.md` (with `trigger: always_on`) | `CLAUDE.md` (rules + commands section) | `copilot-instructions.md` (rules) | root `AGENTS.md` (rules) | `rules/.mdc` — frontmatter `description`, `globs`, `alwaysApply: true` | @@ -104,11 +114,14 @@ Rules: ### Verify parity After either path, confirm the artifact exists in every present tool folder and that no folder -references something that no longer exists. State which folders you wrote. +references something that no longer exists. In Path A, `python3 scripts/sync_agents.py --check` +does this. State which folders you wrote. --- ## Anti-patterns -- Creating an agent for a single tool or task (e.g. "GitAgent"). Tools don't need agents. +- Creating an agent for a persona, a single tool, or a single task (e.g. "GitAgent") — it fails + the survival rule; write a skill. +- Chaining agents through handoffs. - Putting episodic guidance in the always-on rules — it belongs in a skill like this one. - Editing generated folders (`.claude`/`.github`/`.opencode`/`.cursor`/`AGENTS.md`) when `.agent/` exists. diff --git a/.claude/skills/dbt-modeling-conventions/SKILL.md b/.claude/skills/dbt-modeling-conventions/SKILL.md new file mode 100644 index 0000000..57446ce --- /dev/null +++ b/.claude/skills/dbt-modeling-conventions/SKILL.md @@ -0,0 +1,63 @@ +--- +name: dbt-modeling-conventions +description: Applies the organization's dbt modeling conventions. Use when the user writes, reviews, or renames dbt models, mentions dbt, staging, marts, fct_ or dim_ tables, schema.yml, surrogate keys, model naming, CTE structure, or dimensional modeling (star schema) with dbt. +--- + +# Skill: dbt-modeling-conventions + +Org conventions only — the model already knows generic SQL and dbt. Apply these to every model +you write or review. + +## CTE structure +- `import` CTEs on top (one per `ref()`/`source()`, `select` only the needed columns). +- `logical` CTEs in the middle (joins, filters, calculations — one step each). +- One `final` CTE at the bottom; the model ends with `select * from final`. + +## Naming +| Prefix | Layer | +|---|---| +| `stg_` | Staging: 1:1 with a source table, rename/cast only | +| `dim_` | Dimension | +| `fct_` | Fact | +| `mart_` | Business-facing mart built on dims/facts | + +## Keys and grain +- Every dimension has a surrogate key `sk_`: a hash of the natural key. +- One grain per fact table; state it in the model description ("one row per order line"). +- Build dimensions before facts. + +## Tests and docs +Every model has a `schema.yml` entry with a description and `unique` + `not_null` on its +primary key. + +## Example + +```sql +-- models/marts/dim_customer.sql +with stg_customers as ( + select * from {{ ref('stg_crm__customers') }} +), +customers_keyed as ( + select + {{ dbt_utils.generate_surrogate_key(['customer_id']) }} as sk_customer, + customer_id, + customer_name, + country + from stg_customers +), +final as ( + select * from customers_keyed +) +select * from final +``` + +```yaml +# models/marts/schema.yml +models: + - name: dim_customer + description: One row per customer. + columns: + - name: sk_customer + description: Surrogate key, hash of customer_id. + tests: [unique, not_null] +``` diff --git a/.claude/skills/pbi-quality-rules/SKILL.md b/.claude/skills/pbi-quality-rules/SKILL.md index 1a8c059..775db7b 100644 --- a/.claude/skills/pbi-quality-rules/SKILL.md +++ b/.claude/skills/pbi-quality-rules/SKILL.md @@ -1,42 +1,44 @@ --- name: pbi-quality-rules -description: Validate a Power BI semantic model with the real Best Practice Analyzer (BPA) via the free Tabular Editor 2 CLI, using a shareable BPA rules file. +description: Validate a Power BI semantic model with the Best Practice Analyzer (BPA) — the real engine via the free Tabular Editor 2 CLI, or a Python fallback over TMDL when TE2 is not installed — using the shareable bpa-rules.json. Use when asked to lint, audit, check or validate a model, after significant model changes, and before publishing. --- # Skill: pbi-quality-rules -Run the **Best Practice Analyzer (BPA)** — the industry-standard model linter — against a PBIP -semantic model. This skill uses the *real* BPA engine from **Tabular Editor 2 (free, open -source)** instead of a homegrown checker, so results match what Power BI developers expect and -the rules are portable. +Two scripts in this skill's `scripts/` folder (Python 3.9+, standard library only): -## Rule set -`bpa-rules.json` (next to this file) holds the team's BPA rules in Tabular Editor's native -format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in Dynamic LINQ over the -TOM). Severity: `3` = error, `2` = warning, `1` = info. Edit this file to add or tune rules; -it loads directly into Tabular Editor's BPA UI too. +| Script | What it does | +|---|---| +| `run_bpa.py ` | Runs the real BPA through **Tabular Editor 2** (`TabularEditor.exe -A bpa-rules.json -V`). If TE2 is not found, it runs the fallback and says so. | +| `bpa_fallback.py ` | The same rules re-implemented in Python over the TMDL files (or `model.bim`). Approximate; rules it cannot evaluate are listed as `SKIPPED`, never as passed. | -## Primary path — Tabular Editor 2 CLI (recommended) -Requires Tabular Editor 2 (free). Desktop should be closed so disk reflects the latest state. +`` = the `.pbip`, the project folder, `.SemanticModel/`, its `definition/` folder, +or a `model.bim`. Options: `--format json`, `--rules `, and for `run_bpa.py` also +`--te ` and `--no-fallback`. TE2 is looked up via `--te`, `$TABULAR_EDITOR_PATH`, `PATH`, +then the default Windows install folders. +Exit codes: `0` no error-severity violations · `1` errors found · `2` bad path / TE2 failed. + +## Run it ```bash -# Windows (TabularEditor.exe on PATH). Point at the .SemanticModel folder (TMDL) or model.bim. -TabularEditor.exe "Sales.SemanticModel" -A ".agent/skills/pbi-quality-rules/bpa-rules.json" -V +python3 /scripts/run_bpa.py path/to/Sales.pbip ``` - -- `-A ` runs the BPA with the given rules; `-V` writes violations to the console - (and sets a non-zero exit code on error-severity hits — useful in CI). -- The cross-platform `te` CLI (preview) accepts the same rules file if installed instead. - -Parse the console output, then present **errors and warnings only** (omit info unless asked). - -## Fallback — no Tabular Editor installed -The kit stays usable without the tool. Parse the TMDL files directly and evaluate the same -rules by hand: for each rule, read its `Scope` and `Expression` from `bpa-rules.json` and apply -the equivalent check to the parsed objects (measures, tables, columns, relationships). This is -less robust than the real BPA — say so, and recommend installing Tabular Editor 2 for accuracy. - -## Agent flow -1. Run after large model edits / refactors, or on demand. -2. Show errors first; for each `error`, offer an immediate fix. For `warning`, suggest but - don't force. Group by category so the report is scannable. +1. **Disk is the input.** With Desktop open and MCP edits pending, ask the user to save (Ctrl+S) + first — the BPA reads the TMDL Desktop wrote, not the live model. +2. Present **errors first**, then warnings, grouped by category; omit info unless asked. Say + which engine ran; for the fallback, recommend installing Tabular Editor 2 for exact results. +3. For each `error`, offer a fix (through the Modeling MCP when Desktop is open — see + `pbi-semantic-layer-tmdl`). Warnings: suggest, don't force. + +## Rule set — `bpa-rules.json` +Tabular Editor's native format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in +Dynamic LINQ over the TOM). Severity `3` error · `2` warning · `1` info. It loads directly into +TE2's BPA window too. + +Adding a rule: add it to `bpa-rules.json` (TE2 picks it up at once), then add a predicate with the +same `ID` to `CHECKS` in `scripts/bpa_fallback.py` so the fallback covers it; until then the +fallback reports it as `SKIPPED`. + +## In CI +Run `run_bpa.py --no-fallback --format json` on a Windows runner with TE2 (fails on exit 1), or +`bpa_fallback.py --format json` anywhere for a lighter gate. diff --git a/.claude/skills/pbi-quality-rules/scripts/bpa_fallback.py b/.claude/skills/pbi-quality-rules/scripts/bpa_fallback.py new file mode 100755 index 0000000..36f575b --- /dev/null +++ b/.claude/skills/pbi-quality-rules/scripts/bpa_fallback.py @@ -0,0 +1,522 @@ +#!/usr/bin/env python3 +"""Best Practice Analyzer fallback: the rules of bpa-rules.json, evaluated in Python. + +Use this only when Tabular Editor 2 is not available (run_bpa.py calls it automatically). +It reads the semantic model straight from disk — TMDL (PBIP `definition/` folder) or a +`model.bim` — and re-implements each shipped rule by its ID. It is an approximation of the +real BPA engine: rules without a Python implementation are reported as skipped, never as passed. + +Usage: + python3 bpa_fallback.py [--rules bpa-rules.json] [--format text|json] + + may be a .pbip file, a project folder, a .SemanticModel folder, its +definition/ folder, or a model.bim file. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations, 2 = usage error. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from dataclasses import dataclass, field +from pathlib import Path +from typing import Callable, Optional + +DEFAULT_RULES = Path(__file__).resolve().parent.parent / "bpa-rules.json" +SEVERITY_NAMES = {1: "info", 2: "warning", 3: "error"} +AUTO_DATE_PREFIXES = ("LocalDateTable_", "DateTableTemplate_") + + +class UsageError(Exception): + """Bad input path or file; maps to exit code 2.""" + + +# --------------------------------------------------------------------------- +# Model +# --------------------------------------------------------------------------- +@dataclass +class Measure: + table: str + name: str + expression: str = "" + description: str = "" + display_folder: str = "" + is_hidden: bool = False + + +@dataclass +class Column: + table: str + name: str + description: str = "" + is_hidden: bool = False + + +@dataclass +class Table: + name: str + description: str = "" + is_hidden: bool = False + measures: list = field(default_factory=list) + columns: list = field(default_factory=list) + + +@dataclass +class Relationship: + name: str + from_table: str = "" + from_column: str = "" + to_table: str = "" + to_column: str = "" + cross_filtering: str = "oneDirection" + + +@dataclass +class Model: + source: Path + kind: str # "tmdl" | "bim" + tables: list = field(default_factory=list) + relationships: list = field(default_factory=list) + + @property + def measures(self): + return [m for t in self.tables for m in t.measures] + + @property + def columns(self): + return [c for t in self.tables for c in t.columns] + + +# --------------------------------------------------------------------------- +# Locating the model on disk +# --------------------------------------------------------------------------- +def _is_tmdl_folder(path: Path) -> bool: + return path.is_dir() and any((path / n).is_file() for n in ("model.tmdl", "database.tmdl")) + + +def _from_semantic_model_dir(path: Path) -> Optional[tuple]: + if _is_tmdl_folder(path / "definition"): + return "tmdl", path / "definition" + if (path / "model.bim").is_file(): + return "bim", path / "model.bim" + return None + + +def _semantic_model_from_pbip(pbip: Path) -> Optional[Path]: + sibling = pbip.with_name(pbip.stem + ".SemanticModel") + if sibling.is_dir(): + return sibling + try: + data = json.loads(pbip.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + return None + for artifact in data.get("artifacts", []): + report = artifact.get("report", {}).get("path") + if not report: + continue + pbir = pbip.parent / report / "definition.pbir" + try: + ref = json.loads(pbir.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + continue + by_path = ref.get("datasetReference", {}).get("byPath", {}).get("path") + if by_path: + candidate = (pbir.parent / by_path).resolve() + if candidate.is_dir(): + return candidate + return None + + +def resolve_model(path: Path) -> tuple: + """Return (kind, path) where kind is 'tmdl' (definition folder) or 'bim' (model.bim).""" + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() == ".bim": + return "bim", path + if path.suffix.lower() == ".pbip": + sm = _semantic_model_from_pbip(path) + found = _from_semantic_model_dir(sm) if sm else None + if found: + return found + raise UsageError(f"no semantic model found for {path.name} (thin report?)") + if path.name.lower() == "definition.pbism": + return resolve_model(path.parent) + raise UsageError(f"unsupported file: {path.name} (use .pbip, model.bim or a folder)") + if _is_tmdl_folder(path): + return "tmdl", path + found = _from_semantic_model_dir(path) + if found: + return found + models = sorted(p for p in path.glob("*.SemanticModel") if p.is_dir()) + if len(models) == 1: + found = _from_semantic_model_dir(models[0]) + if found: + return found + if len(models) > 1: + names = ", ".join(p.name for p in models) + raise UsageError(f"several semantic models in {path}: {names}; pass one explicitly") + raise UsageError(f"no TMDL definition or model.bim found under {path}") + + +# --------------------------------------------------------------------------- +# TMDL parser (the subset the rules need) +# --------------------------------------------------------------------------- +_PROPERTY = re.compile(r"^([A-Za-z][A-Za-z0-9_]*)\s*(?::\s*(.*))?$") + + +def _depth(line: str) -> int: + expanded = line[: len(line) - len(line.lstrip())].replace("\t", " ") + return len(expanded) // 4 + + +def _read_name(text: str) -> tuple: + """Read one TMDL object name from the start of `text`. Returns (name, rest).""" + text = text.lstrip() + if text.startswith("'"): + out, i = [], 1 + while i < len(text): + if text[i] == "'": + if i + 1 < len(text) and text[i + 1] == "'": + out.append("'") + i += 2 + continue + return "".join(out), text[i + 1 :] + out.append(text[i]) + i += 1 + return "".join(out), "" + match = re.match(r"[^\s=.]+", text) + if not match: + return "", text + return match.group(0), text[match.end() :] + + +def _split_column_ref(ref: str) -> tuple: + """`Sales.DateKey` / `'Sales Table'.'Date Key'` -> (table, column).""" + table, rest = _read_name(ref) + rest = rest.lstrip() + if rest.startswith("."): + column, _ = _read_name(rest[1:]) + return table, column + return "", table + + +def _unquote_value(value: str) -> str: + value = value.strip() + if len(value) >= 2 and value[0] == value[-1] == '"': + return value[1:-1].replace('""', '"') + return value + + +def parse_tmdl_folder(folder: Path) -> Model: + model = Model(source=folder, kind="tmdl") + tables: dict = {} + for file in sorted(folder.rglob("*.tmdl")): + _parse_tmdl_file(file.read_text(encoding="utf-8-sig"), model, tables) + model.tables = list(tables.values()) + return model + + +def _parse_tmdl_file(text: str, model: Model, tables: dict) -> None: + table: Optional[Table] = None + obj = None # Measure | Column | Relationship | Table (for table-level properties) + obj_depth = 0 + other_depth: Optional[int] = None # inside an object we do not model (partition, ...) + expr_lines: Optional[list] = None + expr_fenced = False + pending_doc: list = [] + + def finish_expression(): + nonlocal expr_lines + if expr_lines is not None and isinstance(obj, Measure): + obj.expression = "\n".join(expr_lines).strip() + expr_lines = None + + for raw in text.splitlines(): + stripped = raw.strip() + depth = _depth(raw) + + if expr_lines is not None: + if expr_fenced: + if stripped.startswith("```"): + finish_expression() + else: + expr_lines.append(stripped) + continue + if not stripped or depth > obj_depth + 1: + expr_lines.append(stripped) + continue + finish_expression() + + if not stripped: + continue + if stripped.startswith("///"): + pending_doc.append(stripped[3:].strip()) + continue + if stripped.startswith("//"): + continue + + if other_depth is not None: + if depth > other_depth: + continue + other_depth = None + + doc = " ".join(pending_doc).strip() + pending_doc = [] + keyword, _, rest = stripped.partition(" ") + + if depth == 0: + obj = None + table = None + if keyword == "table": + name, _ = _read_name(rest) + table = tables.setdefault(name, Table(name=name)) + if doc: + table.description = doc + obj, obj_depth = table, 0 + elif keyword == "relationship": + name, _ = _read_name(rest) + obj, obj_depth = Relationship(name=name), 0 + model.relationships.append(obj) + else: + other_depth = 0 + continue + + if table is not None and depth == 1 and keyword in ("measure", "column"): + name, after = _read_name(rest) + after = after.strip() + if keyword == "measure": + obj = Measure(table=table.name, name=name, description=doc) + table.measures.append(obj) + else: + obj = Column(table=table.name, name=name, description=doc) + table.columns.append(obj) + obj_depth = 1 + if after.startswith("="): + inline = after[1:].strip() + if inline.startswith("```"): + expr_lines, expr_fenced = [], True + elif inline: + if isinstance(obj, Measure): + obj.expression = inline + else: + expr_lines, expr_fenced = [], False + continue + + if obj is not None and depth == obj_depth + 1: + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(obj, match.group(1), match.group(2)) + continue + + if table is not None and depth == 1: + # Table-level property after members (e.g. `isHidden`, `lineageTag: …`). + obj, obj_depth = table, 0 + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(table, match.group(1), match.group(2)) + continue + + # Any other nested object (partition, hierarchy, annotation, ...): skip its body. + other_depth = depth + + finish_expression() + + +def _set_property(obj, key: str, value: Optional[str]) -> None: + flag = value is None or value.strip().lower() == "true" + if key == "isHidden" and hasattr(obj, "is_hidden"): + obj.is_hidden = flag + elif key == "description" and hasattr(obj, "description") and value is not None: + obj.description = _unquote_value(value) + elif key == "displayFolder" and isinstance(obj, Measure) and value is not None: + obj.display_folder = _unquote_value(value) + elif isinstance(obj, Relationship) and value is not None: + if key == "fromColumn": + obj.from_table, obj.from_column = _split_column_ref(value) + elif key == "toColumn": + obj.to_table, obj.to_column = _split_column_ref(value) + elif key == "crossFilteringBehavior": + obj.cross_filtering = value.strip() + + +# --------------------------------------------------------------------------- +# model.bim (TMSL JSON) +# --------------------------------------------------------------------------- +def _text(value) -> str: + if isinstance(value, list): + return "\n".join(str(v) for v in value) + return str(value or "") + + +def parse_bim(path: Path) -> Model: + try: + data = json.loads(path.read_text(encoding="utf-8-sig")) + except ValueError as exc: + raise UsageError(f"{path} is not valid JSON: {exc}") from exc + raw_model = data.get("model", data) + model = Model(source=path, kind="bim") + for t in raw_model.get("tables", []): + table = Table(name=t.get("name", ""), description=_text(t.get("description")), + is_hidden=bool(t.get("isHidden"))) + for m in t.get("measures", []): + table.measures.append(Measure( + table=table.name, name=m.get("name", ""), expression=_text(m.get("expression")), + description=_text(m.get("description")), display_folder=m.get("displayFolder", "") or "", + is_hidden=bool(m.get("isHidden")))) + for c in t.get("columns", []): + if c.get("type") == "rowNumber": + continue + table.columns.append(Column(table=table.name, name=c.get("name", ""), + description=_text(c.get("description")), + is_hidden=bool(c.get("isHidden")))) + model.tables.append(table) + for r in raw_model.get("relationships", []): + model.relationships.append(Relationship( + name=r.get("name", ""), from_table=r.get("fromTable", ""), from_column=r.get("fromColumn", ""), + to_table=r.get("toTable", ""), to_column=r.get("toColumn", ""), + cross_filtering=r.get("crossFilteringBehavior", "oneDirection"))) + return model + + +def load_model(path: Path) -> Model: + kind, resolved = resolve_model(path) + return parse_tmdl_folder(resolved) if kind == "tmdl" else parse_bim(resolved) + + +# --------------------------------------------------------------------------- +# Rules — one Python predicate per rule ID in bpa-rules.json +# --------------------------------------------------------------------------- +def _blank(text: str) -> bool: + return not (text or "").strip() + + +def _tables_in_relationships(model: Model) -> set: + return {t for r in model.relationships for t in (r.from_table, r.to_table)} + + +# rule ID -> (scope, predicate(obj, model) -> True when the object VIOLATES the rule) +CHECKS: dict = { + "NAMING_MEASURE_PASCAL": ("Measure", lambda m, _: bool(re.match(r"^[a-z]", m.name))), + "NAMING_MEASURE_DISPLAYFOLDER": ("Measure", lambda m, _: not m.display_folder), + "DAX_EMPTY_EXPRESSION": ("Measure", lambda m, _: _blank(m.expression) + or m.expression.replace(" ", "").upper() == "BLANK()"), + "DAX_IFERROR_USAGE": ("Measure", lambda m, _: "IFERROR" in m.expression.upper()), + "MODEL_BIDIRECTIONAL": ("Relationship", lambda r, _: r.cross_filtering.lower() == "bothdirections"), + "MODEL_ISLAND_TABLE": ("Table", lambda t, model: t.name not in _tables_in_relationships(model) + and not t.name.startswith(AUTO_DATE_PREFIXES)), + "PERF_AUTO_DATETIME": ("Table", lambda t, _: t.name.startswith(AUTO_DATE_PREFIXES)), + "DOC_TABLE_DESCRIPTION": ("Table", lambda t, _: _blank(t.description)), + "DOC_MEASURE_DESCRIPTION": ("Measure", lambda m, _: _blank(m.description)), + "DOC_KEY_COLUMN_HIDDEN": ("Column", lambda c, _: bool(re.search(r"ID$", c.name)) and not c.is_hidden), +} + + +def _quote(name: str) -> str: + return "'" + name.replace("'", "''") + "'" + + +def object_label(obj) -> str: + if isinstance(obj, Table): + return _quote(obj.name) + if isinstance(obj, (Measure, Column)): + return f"{_quote(obj.table)}[{obj.name}]" + if isinstance(obj, Relationship): + return f"{_quote(obj.from_table)}[{obj.from_column}] -> {_quote(obj.to_table)}[{obj.to_column}]" + return str(obj) + + +def _objects(model: Model, scope: str) -> list: + return { + "Measure": model.measures, + "Column": model.columns, + "Table": model.tables, + "Relationship": model.relationships, + }[scope] + + +def load_rules(path: Path) -> list: + try: + rules = json.loads(Path(path).read_text(encoding="utf-8-sig")) + except (OSError, ValueError) as exc: + raise UsageError(f"cannot read rules file {path}: {exc}") from exc + if not isinstance(rules, list): + raise UsageError(f"rules file {path} must contain a JSON array") + return rules + + +def analyze(model: Model, rules: list) -> dict: + violations, skipped = [], [] + for rule in rules: + check: Optional[tuple] = CHECKS.get(rule.get("ID")) + if check is None: + skipped.append(rule.get("ID") or rule.get("Name")) + continue + scope, predicate = check + predicate: Callable + severity = SEVERITY_NAMES.get(int(rule.get("Severity", 1)), "info") + for obj in _objects(model, scope): + if predicate(obj, model): + violations.append({ + "rule_id": rule["ID"], + "rule_name": rule.get("Name", rule["ID"]), + "category": rule.get("Category", ""), + "severity": severity, + "object_type": scope, + "object": object_label(obj), + }) + order = {"error": 0, "warning": 1, "info": 2} + violations.sort(key=lambda v: (order[v["severity"]], v["category"], v["rule_id"], v["object"])) + return { + "engine": "python-fallback", + "model": str(model.source), + "violations": violations, + "skipped_rules": skipped, + "summary": {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")}, + } + + +def format_text(result: dict) -> str: + lines = [] + if result["engine"] == "python-fallback": + lines.append("BPA engine: Python fallback (approximate). Install Tabular Editor 2 for the real BPA.") + else: + lines.append("BPA engine: Tabular Editor 2") + lines.append(f"Model: {result['model']}") + for v in result["violations"]: + lines.append(f"{v['severity'].upper():8} [{v['rule_id']}] {v['object_type']} {v['object']} — {v['rule_name']}") + if not result["violations"]: + lines.append("No objects in violation of the rules.") + for rule_id in result.get("skipped_rules", []): + lines.append(f"SKIPPED [{rule_id}] no implementation in this engine") + s = result["summary"] + lines.append(f"Summary: {s['error']} error(s), {s['warning']} warning(s), {s['info']} info") + return "\n".join(lines) + + +def emit(result: dict, fmt: str) -> int: + print(json.dumps(result, indent=2) if fmt == "json" else format_text(result)) + return 1 if result["summary"]["error"] else 0 + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="BPA rules evaluated in Python over TMDL/model.bim.") + parser.add_argument("model", type=Path, help=".pbip, project folder, .SemanticModel, definition/ or model.bim") + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES, help="BPA rules JSON (default: bpa-rules.json)") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + try: + result = analyze(load_model(args.model), load_rules(args.rules)) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + return emit(result, args.format) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/skills/pbi-quality-rules/scripts/run_bpa.py b/.claude/skills/pbi-quality-rules/scripts/run_bpa.py new file mode 100755 index 0000000..31e1232 --- /dev/null +++ b/.claude/skills/pbi-quality-rules/scripts/run_bpa.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""Run the Best Practice Analyzer with Tabular Editor 2; fall back to Python when it is absent. + +Usage: + python3 run_bpa.py [--rules bpa-rules.json] [--te PATH] [--format text|json] [--no-fallback] + +: .pbip, project folder, .SemanticModel, its definition/ folder, or model.bim. + +Tabular Editor 2 is located in this order: --te, $TABULAR_EDITOR_PATH, TabularEditor(.exe) on +PATH, then the default Windows install folders. It runs as + + TabularEditor.exe -A -V + +(-A: analyze with this rules file only, plus rules embedded in the model; -V: Azure DevOps +logging lines, which carry the severity). Without TE2 the script runs bpa_fallback.py and says so. + +Disk is the input: with Power BI Desktop open, save first (Ctrl+S) so the TMDL is current. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations (or rule errors), +2 = usage error / Tabular Editor could not run. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +sys.dont_write_bytecode = True +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import bpa_fallback # noqa: E402 + +DEFAULT_RULES = bpa_fallback.DEFAULT_RULES +TE_TIMEOUT_SECONDS = 600 + +# "##vso[task.logissue type=warning;]Measure 'Sales'[x] violates rule "Rule name"" +_VSO = re.compile(r"^##vso\[task\.logissue type=(?Perror|warning)[^\]]*\](?P.*)$") +_VIOLATION = re.compile(r'^(?P.+?) (?P(?:\'|\[).*) violates rule "(?P.*)"$') +_VIOLATION_LOOSE = re.compile(r'^(?P\S+) (?P.+?) violates rule "(?P.*)"$') +_RULE_ERROR = re.compile(r"^Error on rule '(?P.*?)': (?P.*)$") + + +def find_tabular_editor(explicit: Optional[str] = None, env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + for candidate in (explicit, env.get("TABULAR_EDITOR_PATH")): + if candidate: + return candidate if Path(candidate).is_file() else None + for name in ("TabularEditor.exe", "TabularEditor"): + found = shutil.which(name) + if found: + return found + for base in (env.get("ProgramFiles(x86)"), env.get("ProgramFiles"), env.get("LOCALAPPDATA")): + if base: + exe = Path(base) / "Tabular Editor" / "TabularEditor.exe" + if exe.is_file(): + return str(exe) + return None + + +def build_command(te_path: str, model_target: Path, rules: Path) -> list: + return [te_path, str(model_target), "-A", str(rules), "-V"] + + +def parse_te_output(stdout: str, rules: list, model_label: str) -> dict: + """Turn TE2 console output (-A ... -V) into the same result shape as the fallback.""" + by_name = {r.get("Name"): r for r in rules} + violations, rule_errors = [], [] + for raw in stdout.splitlines(): + line = raw.strip() + if not line or line.startswith("##vso[task.complete"): + continue + level = "info" + match = _VSO.match(line) + if match: + level, line = match.group("level"), match.group("text").strip() + rule_error = _RULE_ERROR.match(line) + if rule_error: + rule_errors.append({"rule_name": rule_error.group("rule"), "error": rule_error.group("error")}) + continue + found = _VIOLATION.match(line) or _VIOLATION_LOOSE.match(line) + if not found: + continue + rule = by_name.get(found.group("rule"), {}) + violations.append({ + "rule_id": rule.get("ID", ""), + "rule_name": found.group("rule"), + "category": rule.get("Category", ""), + "severity": level, + "object_type": found.group("type"), + "object": found.group("object"), + }) + summary = {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")} + return { + "engine": "tabular-editor-2", + "model": model_label, + "violations": violations, + "rule_errors": rule_errors, + "skipped_rules": [], + "summary": summary, + } + + +def run_tabular_editor(te_path: str, model_target: Path, rules_path: Path, rules: list) -> dict: + cmd = build_command(te_path, model_target, rules_path) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=TE_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise bpa_fallback.UsageError(f"Tabular Editor failed to run ({' '.join(cmd)}): {exc}") from exc + result = parse_te_output(proc.stdout, rules, str(model_target)) + loaded = "Running Best Practice Analyzer" in proc.stdout + if proc.returncode != 0 and not loaded: + detail = (proc.stderr or proc.stdout).strip()[-2000:] + raise bpa_fallback.UsageError(f"Tabular Editor exited {proc.returncode} before the BPA ran:\n{detail}") + return result + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Run the BPA via Tabular Editor 2 (Python fallback if absent).") + parser.add_argument("model", type=Path) + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES) + parser.add_argument("--te", help="path to TabularEditor.exe (overrides $TABULAR_EDITOR_PATH)") + parser.add_argument("--format", choices=("text", "json"), default="text") + parser.add_argument("--no-fallback", action="store_true", help="fail (exit 2) instead of using the Python fallback") + args = parser.parse_args(argv) + + try: + rules = bpa_fallback.load_rules(args.rules) + kind, target = bpa_fallback.resolve_model(args.model) + te_path = find_tabular_editor(args.te) + if te_path: + result = run_tabular_editor(te_path, target, args.rules.resolve(), rules) + elif args.no_fallback or args.te: + raise bpa_fallback.UsageError("Tabular Editor 2 not found (use --te or $TABULAR_EDITOR_PATH)") + else: + print("Tabular Editor 2 not found; using the Python fallback.", file=sys.stderr) + parsed = bpa_fallback.parse_tmdl_folder(target) if kind == "tmdl" else bpa_fallback.parse_bim(target) + result = bpa_fallback.analyze(parsed, rules) + except bpa_fallback.UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + if args.format == "json": + print(json.dumps(result, indent=2)) + else: + print(bpa_fallback.format_text(result)) + for err in result.get("rule_errors", []): + print(f"RULE-ERR [{err['rule_name']}] {err['error']}") + has_errors = result["summary"]["error"] > 0 or bool(result.get("rule_errors")) + return 1 if has_errors else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/skills/pbi-report-layer-pbir/SKILL.md b/.claude/skills/pbi-report-layer-pbir/SKILL.md index 28ebab9..898d4d7 100644 --- a/.claude/skills/pbi-report-layer-pbir/SKILL.md +++ b/.claude/skills/pbi-report-layer-pbir/SKILL.md @@ -1,62 +1,128 @@ --- name: pbi-report-layer-pbir -description: Edit the Power BI report layer as code — pages, visuals, bookmarks, and themes — by writing PBIR JSON files in a PBIP project (offline, Desktop closed). +description: Edit the Power BI report layer as code — pages, visuals, filters, bookmarks and themes in the PBIR files of a PBIP project — with Power BI Desktop OPEN, using the edit → validate → save → status → reload → screenshot → review loop (Desktop Bridge via powerbi-report-author). Also covers the Desktop-closed path. Use for any request to add, move, restyle or fix a report page or visual. --- # Skill: pbi-report-layer-pbir -The report layer (`.Report/`) is **PBIR** — report metadata decomposed into per-page and -per-visual JSON files, each with a public JSON schema. PBIR is becoming the default report -format (rolling out across Service and Desktop through 2026), and it is explicitly designed to -be edited by tools other than Power BI. +The report layer (`.Report/`) is **PBIR**: one JSON file per page and per visual, each +with a public JSON schema. Power BI Desktop can stay **open** while you edit it: the +[Desktop Bridge](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +lets a tool reload the files from disk and take screenshots of the rendered pages. -## Hard precondition -**Power BI Desktop must be CLOSED** while editing these `.json` files — live edits corrupt or -get overwritten by autosave. +Model objects (measures, columns, relationships) are NOT edited here — they go live through the +Modeling MCP (`pbi-semantic-layer-tmdl`). Create the measure first, then bind the visual to it. + +## Tools +- `powerbi-report-author` — Microsoft's report CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`, + Node 20+). Validates PBIR, describes visual roles/formatting, and drives Desktop through + `preview --host desktop`. It bundles the lower-level bridge CLI `powerbi-desktop` + (`status`, `reload`, `screenshot`); always go through `powerbi-report-author preview`, which + resolves the right Desktop window from the report folder. Details, errors and setup: + [references/desktop-bridge.md](references/desktop-bridge.md). +- `scripts/validate_pbir.py` (in this skill's folder) — runs `powerbi-report-author validate`, or + a structural check when the CLI is missing. +- `pbi-visual-reviewer` agent — independent screenshot review. + +## The loop (Desktop open) +`` = the project folder or the `.pbip` path; `` = an absolute folder OUTSIDE the +PBIP (e.g. `%TEMP%\pbir-review\`), so screenshots never land in git. + +1. **Read before writing.** `powerbi-report-author preview-pages ` and + `preview-visuals ` list pages and visuals with their file paths. For a new or changed + visual, `powerbi-report-author catalog describe ` gives the exact role names + (e.g. `cardVisual` uses `Data`, `clusteredColumnChart` uses `Category` + `Y`); + `formatting describe-object ` gives property names. Do not guess them. +2. **Edit** the PBIR JSON (anatomy below). One logical batch at a time. +3. **Validate:** `python3 /scripts/validate_pbir.py `. Fix every error before + going on; read the warnings. +4. **Save gate.** Ask the user to save in Desktop (Ctrl+S) if they changed anything by hand, + then check: `powerbi-report-author preview --host desktop --status`. + Continue only when `data.details.hasUnsavedChanges` is exactly `false`. +5. **Reload:** `powerbi-report-author preview --host desktop --reload`. +6. **Screenshot every affected page:** + `powerbi-report-author preview --host desktop --screenshot "" --all-pages` + (or `--screenshot "\.png" --page `). A screenshot does not reload; always + reload first. +7. **Review:** hand the PNG paths, the user's request/spec and the `.Report` path — and nothing + about your reasoning — to the `pbi-visual-reviewer` agent. In tools without subagents + (Cursor, Antigravity), run its checklist in a fresh chat with only those inputs. +8. REJECTED → fix the listed files and repeat from step 3. APPROVED → report the result and let + the user save/commit. + +### Pitfalls (each one has lost real work) +- **Reload discards whatever Desktop has not saved.** That is why step 4 exists. If status says + `true`, `null`, or fails, stop and ask the user to save; never reload "just to see". +- **Reload covers the report only.** `--reload` does not re-read model files. Model changes go + through the Modeling MCP, which updates Desktop live. Use `--reload-with-model` only when TMDL + was edited on disk (Desktop-closed habits) — and it still does not refresh data. +- **An edited theme may not show after reload.** Desktop caches themes by file name. When you + change a custom theme, rename its JSON (e.g. `Brand-3f9a1c.json` → `Brand-7b2e04.json`) and + update both references in `report.json` (`themeCollection.customTheme.name` and the + `resourcePackages` item `name`/`path`), then validate and reload. +- A page added on disk is unknown to the open window until a reload; a screenshot of it fails + with `RENDER_FAILED`. +- Two Desktop windows with the same report → `HOST_UNAVAILABLE`. Ask the user to close one; + never close Desktop yourself. + +## Alternative: Desktop closed +Use it when there is no Bridge (Desktop older than the Bridge release, the setting +*File > Options > Security > Desktop Bridge* is off, or the machine is not Windows) and in CI. +1. Make sure Desktop is **closed** — an open Desktop overwrites the files on its next save. +2. Edit the PBIR, run `validate_pbir.py` (in CI: `--format json`, fail on exit code 1). +3. Ask the user to open the `.pbip` and check the pages; no screenshots, so ask for one or for + a description, and run the reviewer checklist on what you get. ## `.Report` structure (PBIR) ``` -.Report/ - definition.pbir ← pointer (model binding) +.Report/ + definition.pbir ← model binding (byPath ../.SemanticModel) definition/ - report.json ← master metadata incl. themeCollection + version.json + report.json ← themeCollection, resourcePackages, settings pages/ + pages.json ← pageOrder + activePageName / - page.json ← canvas size, background, displayName - visuals//visual.json ← one visual per file + page.json ← name (= folder), displayName, width, height + visuals//visual.json ← one visual per file; name = folder StaticResources/ - RegisteredResources//.json ← custom themes (create new themes here) - SharedResources/BaseThemes/ ← Microsoft base themes — NEVER edit + RegisteredResources/ ← custom themes and images you add + SharedResources/BaseThemes/ ← Microsoft base themes — never edit ``` - -## Navigating -- **Pages:** scan `definition/pages/`; read each `page.json` `displayName` (the user-facing name). -- **Visuals:** files sit under GUID folders. To find "the bar chart", grep `visual.json` for - `"visualType": "barChart"`. +PBIR-Legacy (a single `report.json` at the `.Report` root) cannot be edited this way: ask the +user to save the report in PBIR format first. ## Visual anatomy (`visual.json`) ```json { - "name": "", - "position": { "x": 10, "y": 20, "width": 400, "height": 300, "tabOrder": 1 }, + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/visualContainer/2.9.0/schema.json", + "name": "7b2d4f6a8c0e1a3b5d02", + "position": { "x": 40, "y": 240, "z": 2000, "width": 760, "height": 420, "tabOrder": 2000 }, "visual": { - "visualType": "barChart", - "query": { "queryState": { "Category": { "projections": [] }, "Y": { "projections": [] } } } + "visualType": "clusteredColumnChart", + "query": { "queryState": { + "Category": { "projections": [ { "field": { "Column": { "Expression": { "SourceRef": { "Entity": "Product" } }, "Property": "Category" } }, "queryRef": "Product.Category", "nativeQueryRef": "Category", "active": true } ] }, + "Y": { "projections": [ { "field": { "Measure": { "Expression": { "SourceRef": { "Entity": "Sales" } }, "Property": "Total Sales" } }, "queryRef": "Sales.Total Sales", "nativeQueryRef": "Total Sales" } ] } + } }, + "visualContainerObjects": { "title": [ { "properties": { + "show": { "expr": { "Literal": { "Value": "true" } } }, + "text": { "expr": { "Literal": { "Value": "'Sales by category'" } } } } } ] } } } ``` -Positioning is a pixel grid (x=0, y=0 top-left). Generate fresh GUIDs for any new component -(`[System.Guid]::NewGuid()` on Windows, or any UUID generator). +- New page/visual: fresh 20-hex-char `name`, folder with the same name; add new pages to + `pages.json` `pageOrder`. +- Position is in pixels from the page's top-left; keep `x + width ≤ page width` and + `y + height ≤ page height`. Align edges to a shared grid (e.g. multiples of 8) and keep equal gaps. +- Literal encoding: text `'quoted'`, numbers `12D`, booleans `true`. When unsure, + `powerbi-report-author expr encode --kind `. +- Avoid visuals Microsoft is retiring (Q&A, Bing Map, filled map). ## Themes -1. Create the theme JSON under `StaticResources/RegisteredResources//.json`. -2. Register that `` in `report.json`'s `themeCollection`. -3. Define ≥ 8 `dataColors`. To standardize fonts globally, use the wildcard `visualStyles`: - ```json - "visualStyles": { "*": { "*": { "fontSize": [{ "value": 11 }], "fontFamily": [{ "value": "Segoe UI" }] } } } - ``` - -## Cautions -- The visual schema evolves with Desktop releases; old keys may be ignored on newer versions. -- Validate JSON against the PBIR schema where possible. Always have the user commit (`git add`) - before large UI refactors and test by opening the file in Desktop. +Create custom themes under `StaticResources/RegisteredResources/`, register them in +`report.json` (`themeCollection.customTheme` + a `resourcePackages` item of type +`RegisteredResources`), define ≥ 8 `dataColors`, and standardize fonts with the wildcard +`visualStyles` (`"*": { "*": { ... } }`). Rename the file on every edit (pitfall above). + +## Before large changes +Ask the user to commit the PBIP first so every agent edit is a reviewable diff. diff --git a/.claude/skills/pbi-report-layer-pbir/references/desktop-bridge.md b/.claude/skills/pbi-report-layer-pbir/references/desktop-bridge.md new file mode 100644 index 0000000..ef02d9a --- /dev/null +++ b/.claude/skills/pbi-report-layer-pbir/references/desktop-bridge.md @@ -0,0 +1,83 @@ +# Desktop Bridge — setup, commands, errors + +Sources (read these when something here looks outdated): +- Microsoft Learn — [What is the Power BI Desktop Bridge?](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +- Microsoft Learn — [Power BI Report skill](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-report-skill-overview) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `powerbi-report-cli`, MIT) — the official agent guidance this kit follows. +- npm: [`@microsoft/powerbi-report-authoring-cli`](https://www.npmjs.com/package/@microsoft/powerbi-report-authoring-cli) + (`powerbi-report-author`), [`@microsoft/powerbi-desktop-bridge-cli`](https://www.npmjs.com/package/@microsoft/powerbi-desktop-bridge-cli) + (`powerbi-desktop`). Both MIT, Node 20+. + +## What it is +A local JSON-RPC 2.0 server inside each running Power BI Desktop window, reachable only on the +same Windows machine through the named pipe `pbi-desktop-bridge-`. One operation runs +at a time per window; a second request while one is running fails — wait and retry, never run +two Desktop commands in parallel. + +Methods (discover the current list with `bridge.manifest`): +| Method | Used for | +|---|---| +| `bridge.manifest` | Supported methods and their schemas for this Desktop build | +| `application.state.get/v1` | `currentFilePath`, `hasUnsavedChanges` | +| `report.snapshot.capture/v1` | PNG of one page (`pageId`, `scale` 1–3) | +| `file.reload/v1` | Reload the open PBIP from disk; `reloadModelDefinition` true/false | + +## Setup (once per machine) +1. Power BI Desktop for Windows, recent enough to show **File > Options and settings > Options > + Security > Desktop Bridge**. Keep *Enable external tool access to Power BI Desktop through + secure local APIs* checked (default on). Admins can force it with the registry value + `HKLM\SOFTWARE\Policies\Microsoft\Power BI Desktop\DesktopNamedPipeBridge` (`1` on, `0` off). +2. Node.js 20+, then `npm i -g @microsoft/powerbi-report-authoring-cli`. +3. Check: `powerbi-report-author doctor` and `powerbi-report-author preview --list-hosts` + (the `desktop` host must say `available: true` while a PBIP is open). +4. The report must be a **PBIP saved in PBIR format** (not a `.pbix`, not PBIR-Legacy). + +## Commands +Use `powerbi-report-author preview` for every Desktop operation; it finds the Desktop window +that has `` open, so there is no PID to pass. `` = the project folder or `.pbip`. + +| Intent | Command | +|---|---| +| Status (save gate) | `powerbi-report-author preview "" --host desktop --status` | +| Open / attach | `powerbi-report-author preview "" --host desktop` | +| Reload report | `powerbi-report-author preview "" --host desktop --reload` | +| Reload report + on-disk model | `powerbi-report-author preview "" --host desktop --reload-with-model` | +| Screenshot one page | `powerbi-report-author preview "" --host desktop --screenshot "\.png" --page ""` | +| Screenshot all pages | `powerbi-report-author preview "" --host desktop --screenshot "" --all-pages` | +| Validate files | `powerbi-report-author validate ""` (`--no-schema` offline) | + +Optional `--scale 1..3` on screenshots. Every command prints a JSON envelope: +`{"data": {...}}` on success, `{"error": {"code": ..., "message": ..., "correction"?: {...}}}` on +failure. If an error carries a `correction` object, retry once changing only that value; +otherwise report the error and stop — do not improvise another command. + +The bridge CLI underneath (`powerbi-desktop status | manifest | open | reload | screenshot | +screenshot-all`) is useful for manual troubleshooting by the user (e.g. `powerbi-desktop status` +lists every open window and its unsaved state). Agents stay on `powerbi-report-author`. + +### Status before open/reload — never before a screenshot +```json +{"data":{"state":"ready","reports":[{"folder":"C:\\work\\Sales","hostInstanceId":"12345"}], + "details":{"hasUnsavedChanges":false}}} +``` +| `details.hasUnsavedChanges` | Action | +|---|---| +| `false` | Reload / open is allowed. | +| `true` | Stop. Ask the user to save (Ctrl+S) — or to discard — in Desktop, then check again. | +| missing, `null`, or the call failed | Treat as unsafe. Stop and report. | +| error `HOST_UNAVAILABLE` (no window has the report) | Open/attach is allowed; reload is not. | + +## Errors you will meet +| Code / symptom | Meaning | What to do | +|---|---|---| +| `UNSUPPORTED_OS` in `--list-hosts` | Not on Windows | Use the Desktop-closed path; `validate` still works. | +| `HOST_UNAVAILABLE` — "Several Power BI Desktop instances have this report open" | Same report in 2+ windows (`details.candidatePids`) | Ask the user to close the extras. Never close Desktop yourself. | +| `HOST_UNAVAILABLE` — no instance | Desktop closed or setting off | Ask the user to open the `.pbip`; check the Security setting. | +| `RENDER_FAILED` with "unknown page id" | Page exists on disk but not in the open window | Status → reload → capture that page again. | +| `PAGE_NOT_FOUND` | The page is not in the definition at all | Fix `pages.json` / the page id. Reload will not help. | +| `status: "partial"` + non-zero exit | Some pages captured, some failed | Keep the PNGs you got; report which pages failed and why. | +| JSON-RPC `-32601 MethodNotFound` (bridge CLI) | Desktop build lacks that method | Update Desktop; check `powerbi-desktop manifest`. | +| "operation in progress" | Another bridge call is running | Wait, then retry once. | +| Theme change invisible after reload | Desktop cached the theme by file name | Rename the theme file + both refs in `report.json`, validate, reload. | +| Change invisible after reload, no error | Edited a file Desktop does not read (e.g. outside `definition/`) or the wrong report folder | `preview-visuals ` to confirm paths; check `--status` `folder`. | diff --git a/.claude/skills/pbi-report-layer-pbir/scripts/validate_pbir.py b/.claude/skills/pbi-report-layer-pbir/scripts/validate_pbir.py new file mode 100755 index 0000000..008c29b --- /dev/null +++ b/.claude/skills/pbi-report-layer-pbir/scripts/validate_pbir.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Validate a PBIR report: Microsoft's `powerbi-report-author validate` when installed, +otherwise a structural check written in Python. + +Usage: + python3 validate_pbir.py [--structural-only] [--offline] [--format text|json] + +: a .pbip file, a .Report folder, a folder containing definition/, or a project +folder with exactly one *.Report. + +The CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`) is found through +$PBIR_AUTHOR_CLI or `powerbi-report-author` on PATH. It validates every file against the +published PBIR JSON schemas plus visual roles/types, so it is the authoritative check. +--offline passes --no-schema to it (no download of remote schemas). + +The structural fallback checks what breaks Desktop most often: JSON that does not parse, +PBIR-Legacy layout, a broken semantic-model binding, pages.json out of sync with the page +folders, page/visual `name` != folder, visuals without position/visualType, visuals outside +the page canvas, and theme/resource files that are registered but missing. It does NOT know +the schemas or visual roles — say so when reporting. + +Exit codes: 0 = no errors, 1 = errors found, 2 = usage error / the CLI could not run. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +CLI_NAME = "powerbi-report-author" +CLI_TIMEOUT_SECONDS = 300 + + +class UsageError(Exception): + pass + + +# --------------------------------------------------------------------------- +# Locate the .Report folder +# --------------------------------------------------------------------------- +def _read_json(path: Path): + return json.loads(path.read_text(encoding="utf-8-sig")) + + +def resolve_report(path: Path) -> Path: + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() != ".pbip": + raise UsageError(f"unsupported file: {path.name} (use a .pbip or a .Report folder)") + try: + artifacts = _read_json(path).get("artifacts", []) + except ValueError as exc: + raise UsageError(f"{path.name} is not valid JSON: {exc}") from exc + for artifact in artifacts: + report = artifact.get("report", {}).get("path") + if report and (path.parent / report).is_dir(): + return (path.parent / report).resolve() + sibling = path.with_name(path.stem + ".Report") + if sibling.is_dir(): + return sibling.resolve() + raise UsageError(f"{path.name} does not point to an existing .Report folder") + if (path / "definition").is_dir() or (path / "definition.pbir").is_file() or path.suffix == ".Report": + return path.resolve() + reports = sorted(p for p in path.glob("*.Report") if p.is_dir()) + if len(reports) == 1: + return reports[0].resolve() + if len(reports) > 1: + raise UsageError(f"several reports in {path}: {', '.join(p.name for p in reports)}; pass one") + raise UsageError(f"no .Report folder found under {path}") + + +# --------------------------------------------------------------------------- +# Structural fallback +# --------------------------------------------------------------------------- +class Findings: + def __init__(self, report_dir: Path): + self.report_dir = report_dir + self.items: list = [] + + def add(self, severity: str, code: str, message: str, file: Optional[Path] = None, path: str = ""): + rel = "" + if file is not None: + try: + rel = file.relative_to(self.report_dir).as_posix() + except ValueError: + rel = str(file) + self.items.append({"severity": severity, "code": code, "message": message, "file": rel, "path": path}) + + def error(self, *args, **kwargs): + self.add("error", *args, **kwargs) + + def warning(self, *args, **kwargs): + self.add("warning", *args, **kwargs) + + +def _load(findings: Findings, file: Path, required: bool = True): + if not file.is_file(): + if required: + findings.error("FILE_MISSING", f"required file is missing: {file.name}", file) + return None + try: + return _read_json(file) + except ValueError as exc: + findings.error("JSON_INVALID", f"invalid JSON: {exc}", file) + return None + + +def _is_number(value) -> bool: + return isinstance(value, (int, float)) and not isinstance(value, bool) + + +def structural_check(report_dir: Path) -> dict: + f = Findings(report_dir) + definition = report_dir / "definition" + + for file in sorted(report_dir.rglob("*.json")): + if ".pbi" in file.relative_to(report_dir).parts: + continue + try: + _read_json(file) + except ValueError as exc: + f.error("JSON_INVALID", f"invalid JSON: {exc}", file) + invalid = {i["file"] for i in f.items} + + if not definition.is_dir(): + if (report_dir / "report.json").is_file(): + f.error("PBIR_LEGACY_FORMAT", "PBIR-Legacy layout (report.json at the root); save the " + "report in PBIR format (Desktop: File > Options > Preview features > PBIR)", report_dir / "report.json") + else: + f.error("FILE_MISSING", "definition/ folder is missing", definition) + return _result(f, "structural") + + pbir = _load(f, report_dir / "definition.pbir") + if isinstance(pbir, dict): + ref = pbir.get("datasetReference", {}) + by_path = ref.get("byPath", {}).get("path") if isinstance(ref.get("byPath"), dict) else None + if by_path: + target = (report_dir / by_path).resolve() + if not target.is_dir(): + f.error("MODEL_BINDING_BROKEN", f"datasetReference.byPath points to a missing folder: {by_path}", + report_dir / "definition.pbir", "datasetReference.byPath.path") + elif not ref.get("byConnection"): + f.error("MODEL_BINDING_MISSING", "datasetReference has neither byPath nor byConnection", + report_dir / "definition.pbir", "datasetReference") + + _load(f, definition / "version.json") + report = _load(f, definition / "report.json") + if isinstance(report, dict): + _check_resources(f, report, report_dir) + + pages_dir = definition / "pages" + pages_meta = _load(f, pages_dir / "pages.json") + page_folders = sorted(p.name for p in pages_dir.iterdir() if p.is_dir()) if pages_dir.is_dir() else [] + if isinstance(pages_meta, dict): + order = pages_meta.get("pageOrder", []) + for page_id in order: + if page_id not in page_folders: + f.error("PAGE_MISSING", f"pages.json lists page '{page_id}' but its folder is missing", + pages_dir / "pages.json", "pageOrder") + for page_id in page_folders: + if page_id not in order: + f.warning("PAGE_NOT_IN_ORDER", f"page folder '{page_id}' is not listed in pages.json pageOrder", + pages_dir / page_id) + active = pages_meta.get("activePageName") + if active and active not in order: + f.error("ACTIVE_PAGE_UNKNOWN", f"activePageName '{active}' is not in pageOrder", + pages_dir / "pages.json", "activePageName") + + for page_id in page_folders: + _check_page(f, pages_dir / page_id, invalid) + + return _result(f, "structural") + + +def _check_resources(f: Findings, report: dict, report_dir: Path) -> None: + static = report_dir / "StaticResources" + for package in report.get("resourcePackages", []) or []: + folder = static / package.get("name", "") + for item in package.get("items", []) or []: + rel = item.get("path") + if rel and not (folder / rel).is_file(): + f.error("RESOURCE_MISSING", f"resource '{item.get('name')}' registered in report.json is " + f"missing on disk: {package.get('name')}/{rel}", report_dir / "definition" / "report.json", + "resourcePackages") + custom = (report.get("themeCollection") or {}).get("customTheme") + if isinstance(custom, dict) and custom.get("name"): + registered = { + item.get("name") + for package in report.get("resourcePackages", []) or [] + if package.get("type") == custom.get("type", "RegisteredResources") + for item in package.get("items", []) or [] + } + if custom["name"] not in registered: + f.error("THEME_NOT_REGISTERED", f"customTheme '{custom['name']}' has no matching resourcePackages item", + report_dir / "definition" / "report.json", "themeCollection.customTheme.name") + + +def _check_page(f: Findings, page_dir: Path, invalid: set) -> None: + page_file = page_dir / "page.json" + page = _load(f, page_file) + if not isinstance(page, dict): + return + if page.get("name") != page_dir.name: + f.error("NAME_MISMATCH", f"page.json name '{page.get('name')}' != folder '{page_dir.name}'", page_file, "name") + if not page.get("displayName"): + f.warning("PAGE_NO_DISPLAY_NAME", "page has no displayName", page_file, "displayName") + width, height = page.get("width"), page.get("height") + if not (_is_number(width) and _is_number(height)): + f.error("PAGE_SIZE_INVALID", "page width/height must be numbers", page_file) + width = height = None + + visuals_dir = page_dir / "visuals" + if not visuals_dir.is_dir(): + return + for visual_dir in sorted(p for p in visuals_dir.iterdir() if p.is_dir()): + visual_file = visual_dir / "visual.json" + rel = visual_file.relative_to(f.report_dir).as_posix() + if rel in invalid: + continue + visual = _load(f, visual_file) + if not isinstance(visual, dict): + continue + if visual.get("name") != visual_dir.name: + f.error("NAME_MISMATCH", f"visual.json name '{visual.get('name')}' != folder '{visual_dir.name}'", + visual_file, "name") + pos = visual.get("position") + if not isinstance(pos, dict) or not all(_is_number(pos.get(k)) for k in ("x", "y", "width", "height")): + f.error("POSITION_INVALID", "position needs numeric x, y, width and height", visual_file, "position") + pos = None + kind = (visual.get("visual") or {}).get("visualType") + if not kind and "visualGroup" not in visual: + f.error("VISUAL_TYPE_MISSING", "visual.visualType is missing (and this is not a visualGroup)", + visual_file, "visual.visualType") + if pos and width is not None and not visual.get("parentGroupName"): + right, bottom = pos["x"] + pos["width"], pos["y"] + pos["height"] + if pos["x"] < 0 or pos["y"] < 0 or right > width + 0.5 or bottom > height + 0.5: + f.warning("OUTSIDE_CANVAS", f"visual extends outside the {width}x{height} page " + f"(x={pos['x']}, y={pos['y']}, right={right}, bottom={bottom})", visual_file, "position") + + +def _result(f: Findings, engine: str) -> dict: + errors = sum(1 for i in f.items if i["severity"] == "error") + warnings = sum(1 for i in f.items if i["severity"] == "warning") + return { + "engine": engine, + "report": str(f.report_dir), + "result": "failed" if errors else ("succeededWithWarnings" if warnings else "succeeded"), + "errorCount": errors, + "warningCount": warnings, + "diagnostics": f.items, + } + + +# --------------------------------------------------------------------------- +# powerbi-report-author CLI +# --------------------------------------------------------------------------- +def find_cli(env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + explicit = env.get("PBIR_AUTHOR_CLI") + if explicit: + return explicit if Path(explicit).is_file() else None + return shutil.which(CLI_NAME) + + +def build_cli_command(cli: str, report_dir: Path, offline: bool) -> list: + cmd = [cli, "validate", str(report_dir)] + if offline: + cmd.append("--no-schema") + return cmd + + +def parse_cli_output(stdout: str, report_dir: Path) -> dict: + """Normalize the CLI's JSON envelope ({"data": ...} or {"error": ...}).""" + try: + envelope = json.loads(stdout) + except ValueError as exc: + raise UsageError(f"{CLI_NAME} returned non-JSON output: {stdout.strip()[:500]}") from exc + if "error" in envelope: + err = envelope["error"] + raise UsageError(f"{CLI_NAME} failed: {err.get('code')}: {err.get('message')}") + data = envelope.get("data", {}) + items = [] + for code, group in (data.get("diagnostics") or {}).items(): + for item in group.get("items", []): + file = item.get("file", "") + try: + file = Path(file).resolve().relative_to(report_dir).as_posix() if file else "" + except ValueError: + pass + items.append({ + "severity": group.get("severity", "error"), + "code": code, + "message": item.get("message", ""), + "file": file, + "path": item.get("path", ""), + }) + return { + "engine": CLI_NAME, + "report": str(report_dir), + "result": data.get("result", "unknown"), + "errorCount": int(data.get("errorCount", sum(1 for i in items if i["severity"] == "error"))), + "warningCount": int(data.get("warningCount", sum(1 for i in items if i["severity"] == "warning"))), + "diagnostics": items, + } + + +def run_cli(cli: str, report_dir: Path, offline: bool) -> dict: + cmd = build_cli_command(cli, report_dir, offline) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=CLI_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise UsageError(f"could not run {' '.join(cmd)}: {exc}") from exc + return parse_cli_output(proc.stdout, report_dir) + + +# --------------------------------------------------------------------------- +# Output +# --------------------------------------------------------------------------- +def format_text(result: dict) -> str: + if result["engine"] == "structural": + lines = [f"PBIR validation: structural fallback ({CLI_NAME} not found; schemas and visual " + "roles NOT checked — install @microsoft/powerbi-report-authoring-cli)"] + else: + lines = [f"PBIR validation: {CLI_NAME} validate"] + lines.append(f"Report: {result['report']}") + for d in sorted(result["diagnostics"], key=lambda d: (d["severity"] != "error", d["file"], d["code"])): + where = d["file"] + (f" @ {d['path']}" if d["path"] else "") + lines.append(f"{d['severity'].upper():8} [{d['code']}] {d['message']}" + (f" ({where})" if where else "")) + lines.append(f"Result: {result['result']} — {result['errorCount']} error(s), {result['warningCount']} warning(s)") + return "\n".join(lines) + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Validate a PBIR report (CLI if installed, else structural).") + parser.add_argument("path", type=Path) + parser.add_argument("--structural-only", action="store_true", help="skip the CLI even if it is installed") + parser.add_argument("--offline", action="store_true", help="pass --no-schema to the CLI") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + + try: + report_dir = resolve_report(args.path) + cli = None if args.structural_only else find_cli() + result = run_cli(cli, report_dir, args.offline) if cli else structural_check(report_dir) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + print(json.dumps(result, indent=2) if args.format == "json" else format_text(result)) + return 1 if result["errorCount"] else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.claude/skills/pbi-semantic-layer-tmdl/SKILL.md b/.claude/skills/pbi-semantic-layer-tmdl/SKILL.md index dc1f3c2..da3d3bd 100644 --- a/.claude/skills/pbi-semantic-layer-tmdl/SKILL.md +++ b/.claude/skills/pbi-semantic-layer-tmdl/SKILL.md @@ -1,69 +1,99 @@ --- name: pbi-semantic-layer-tmdl -description: Edit the Power BI semantic model as code by authoring TMDL files in a PBIP project (offline, Desktop closed). Covers structure, syntax, and post-edit verification. +description: Change the Power BI semantic model — measures, columns, tables, relationships, RLS, DAX — with Power BI Desktop OPEN through the Power BI Modeling (Authoring) MCP server, saving to TMDL in a PBIP project; or by editing TMDL files with Desktop closed. Includes a semantic-model review checklist. Use for any request to add or fix a measure, DAX, relationship, table, column or model metadata. --- # Skill: pbi-semantic-layer-tmdl -Author and edit the semantic model (tables, columns, measures, relationships, RLS) by writing -**TMDL** files directly. TMDL is the GA, human-readable text format for tabular models — great -diffs, one file per table, fully scriptable. +With Desktop open, the model lives in Desktop's memory (a local Analysis Services instance). Edit +it **there**, live, through the Power BI Modeling MCP server — then the user saves and Desktop +writes the TMDL into the PBIP. Editing the `.tmdl` files while Desktop is open does nothing +until a model reload and is overwritten by the next save. -## Hard precondition -**Power BI Desktop must be CLOSED** before saving. Desktop keeps the model in memory and -overwrites disk on save, so editing live corrupts files or loses work. +## Default: Desktop open + Modeling MCP +Setup and the tool list: [references/modeling-mcp.md](references/modeling-mcp.md). +1. **Connect:** "Connect to '' in Power BI Desktop". Confirm with a read: list the + tables and measures. If the MCP is not registered, stop and point the user to the setup — + do not fall back to editing TMDL silently. +2. **Inspect through the MCP**, not the `.tmdl` files: while connected, the live model is the + truth and disk may be stale. +3. **Change** in dependency order: tables/partitions → columns → relationships → measures. + Check that every referenced table/column exists and that the name is not taken. +4. **Prove it:** run a DAX query through the MCP (`EVALUATE SUMMARIZECOLUMNS(..., "x", [New Measure])` + or `EVALUATE { [New Measure] }`) and show the value. A measure that only "was created" is not done. +5. **Save:** ask the user to press Ctrl+S in Desktop. That writes the TMDL into + `.SemanticModel/definition/` — the diff the team reviews and commits. +6. **Quality:** run `pbi-quality-rules` (BPA) after significant changes — on disk, so after step 5. +7. If a visual must use the new measure, continue in `pbi-report-layer-pbir`. Report reloads do + not touch the model and are not needed for MCP changes. + +## Alternative: Desktop closed (no MCP, older Desktop, CI) +Edit the TMDL files directly: +1. Make sure Desktop is **closed** — it overwrites disk on save. +2. Edit `.tmdl` files following the syntax below; run `pbi-quality-rules`. +3. Ask the user to open the `.pbip`: field list populates, no error banner, each new measure on a + Card shows a value. On error, re-read the edited TMDL and fix the reference/syntax. + +If Desktop must stay open and the MCP is unavailable, edit TMDL, then reload model + report with +`powerbi-report-author preview --host desktop --reload-with-model` after the same +unsaved-changes gate as `pbi-report-layer-pbir`. It reloads definitions only — the user must +refresh data for new/changed imported columns. ## `.SemanticModel` structure ``` -.SemanticModel/ +.SemanticModel/ definition.pbism ← pointer (do not hand-edit) definition/ - model.tmdl ← model-level definition, lists tables - database.tmdl ← compatibility level / annotations - tables/
.tmdl ← ONE table per file (golden rule) - relationships.tmdl ← all relationships + database.tmdl ← compatibilityLevel + model.tmdl ← model properties, `ref table` order, annotations + tables/
.tmdl ← one table per file + relationships.tmdl expressions.tmdl ← shared M expressions / parameters - cultures/ ← translations roles.tmdl ← RLS - .pbi/ ← local cache/settings — NEVER edit (localSettings.json, cache.abf) + cultures/ ← translations + .pbi/ ← local cache/settings — never edit, keep out of git ``` -Edit `.tmdl` files only. One table per file — never define every table inside `model.tmdl`. -## TMDL syntax reference +## TMDL syntax ```tmdl -table 'Sales' - description: "Primary sales fact." +/// Order lines. Grain: one row per order line. +table Sales - measure 'Total Sales' = SUM(Sales[Amount]) - formatString: "#,##0.00" - displayFolder: "Financials" - description: "Sum of sales amount." + /// Sum of the order line amount. + measure 'Total Sales' = SUM(Sales[Amount]) + formatString: #,##0.00 + displayFolder: Revenue - column 'Amount' - dataType: decimal - sourceColumn: BaseAmount + measure 'Margin %' = + DIVIDE([Total Sales] - [Total Cost], [Total Sales]) + formatString: 0.0% + displayFolder: Revenue + + column Amount + dataType: decimal + summarizeBy: sum + sourceColumn: Amount ``` ```tmdl -relationship Sales_Date - fromColumn: Sales.DateId - toColumn: Date.DateId - fromCardinality: many - toCardinality: one +relationship 5f0c2a4e-9b1d-4c3a-8e2f-1a6b7c8d9e01 + fromColumn: Sales.DateKey + toColumn: Date.DateKey ``` +- Indent with tabs; properties one level under their object. +- Descriptions are `///` lines directly above the object. +- Multi-line DAX: leave the `=` line empty and indent the expression two levels deeper than the + object, or wrap it in a ```` ``` ```` block. +- Quote names with spaces or symbols: `'Margin %'`; double a quote inside: `'Customer''s'`. -**Serialization rules:** keep functional properties (`dataType`, `sourceColumn`) before -cosmetic ones; match the file's existing indentation; for multi-line DAX, break with `\` at the -end of the line. - -## Tooling -- **Tabular Editor 2 (free)** can open the `.SemanticModel` folder, validate, batch-edit, and - re-serialize clean TMDL. Use it to sanity-check non-trivial edits. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (preview) edits the same PBIP/TMDL files and - is handy for bulk operations. File editing remains the default, accessible path. - -## Post-edit verification -1. Ask the user to open the `.pbip` in Desktop and wait for the field list to populate. -2. **View → Diagnostics** — report any error/warning shown. -3. For each new/changed measure, drop it on a **Card** and confirm the value (no red error icon). -4. On error: re-read the edited TMDL, find the syntax/reference issue, propose the fix. -5. Run the `pbi-quality-rules` skill (BPA) after significant changes. +## Review checklist (run it on any model you change or are asked to review) +- **Names** are business-friendly: no `tbl_`/`dim_` prefixes or `_id` columns visible to users. +- **Every measure** has a `formatString`, a `displayFolder` and a `///` description (what it + means and its grain/filters in plain words). +- **Relationships** match reality: many-to-one from fact to dimension, single direction unless a + documented reason exists; no ambiguous paths. +- **A real Date table** (marked as date table) drives time intelligence; auto date/time is off + (`annotation __PBI_TimeIntelligenceEnabled = 0` in `model.tmdl`). +- **Technical keys hidden** (`isHidden` on `...ID`/`...Key` columns); explicit measures instead + of implicit column aggregation for anything a report shows. +- **DAX:** `DIVIDE` instead of `/` + `IFERROR`; variables for repeated sub-expressions; no + measure that just returns `BLANK()`. diff --git a/.claude/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md b/.claude/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md new file mode 100644 index 0000000..fdf7468 --- /dev/null +++ b/.claude/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md @@ -0,0 +1,60 @@ +# Power BI Modeling MCP — setup and use + +Microsoft now calls it the **Power BI Authoring MCP server**; the package and server id are still +`powerbi-modeling-mcp`. Sources: +- Microsoft Learn — [Power BI Authoring MCP server](https://learn.microsoft.com/en-us/power-bi/developer/mcp/power-bi-authoring-mcp) +- [microsoft/powerbi-modeling-mcp](https://github.com/microsoft/powerbi-modeling-mcp) (README, TROUBLESHOOTING) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `semantic-model-authoring`, bundles the same server) + +## Local vs hosted +Working against a model **open in Power BI Desktop** (or PBIP files on disk) needs the **local** +server: it runs on the same Windows machine and talks to Desktop's local Analysis Services +instance over XMLA. The hosted endpoint only reaches Fabric workspaces. Register one of the two, +not both — duplicate tool sets confuse the agent. The local server does not run on macOS. + +## Register the local server +Command in every client: `npx -y @microsoft/powerbi-modeling-mcp@latest --start` +(Node 18+; or install the VS Code extension `analysis-services.powerbi-modeling-mcp`). + +| Client | File | Shape | +|---|---|---| +| Claude Code | `.mcp.json` (project) | `{"mcpServers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| GitHub Copilot (VS Code) | `.vscode/mcp.json` | `{"servers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| OpenCode | `opencode.json` | `{"mcp": {"powerbi-modeling-mcp": {"type": "local", "command": ["npx", "-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| Cursor | `.cursor/mcp.json` | same shape as Claude Code (`mcpServers`) | + +Useful flags: `--readonly` (inspection only — good for reviews), `--readwrite` (default; asks for +confirmation before the first write per database). + +## Connect and work +- Desktop: `Connect to '' in Power BI Desktop` (tool `ConnectToPowerBIDesktop`; the + PBIP must be open in Desktop). +- PBIP on disk without Desktop: `Open semantic model from PBIP folder '\.SemanticModel\definition'` + (changes stay in the session until exported back to TMDL). +- First call after connecting: a read (`List the tables and measures`) to confirm the target. + +Tool groups you will use most: `measure_operations`, `column_operations`, `table_operations`, +`relationship_operations`, `dax_query_operations` (run/validate DAX, max 100,000 rows), +`transaction_operations` (begin/commit/rollback a batch), `calculation_group_operations`, +`security_role_operations`, `object_translation_operations`, `database_operations` +(import/export TMDL). Ask the agent "what can the Power BI Authoring MCP do" to list the current set. + +## Rules of thumb +- Prefer one transaction per logical change; roll back on a failed step instead of leaving + half-applied edits. +- Verify every new or changed measure with a DAX query before calling it done. +- Changes are live in Desktop but not on disk until the user saves (Ctrl+S); a PBIP under git + makes every MCP change a reviewable diff. +- The MCP edits the model only — never report pages (use `pbi-report-layer-pbir`). +- Model metadata and query results go to the LLM provider of the client. Do not query + sensitive columns just to "look at the data". + +## Troubleshooting +| Symptom | Check | +|---|---| +| Server missing from the tool list | Client in agent mode? Server registered and started? Copilot Business/Enterprise: "MCP servers in Copilot" policy enabled by an admin. | +| "No Power BI Desktop instance found" | The PBIP/PBIX is open in Desktop and the name matches the file name you gave. | +| Reads work, writes fail | `--readonly` set, or (Fabric) only Build permission / XMLA read-only. | +| Edits vanish after reopening the file | The user closed Desktop without saving. | +| TMDL on disk does not show the change | Not saved yet — Desktop writes TMDL on save. | diff --git a/.claude/workflows/document-dashboard.md b/.claude/workflows/document-dashboard.md index e38f27f..5c31123 100644 --- a/.claude/workflows/document-dashboard.md +++ b/.claude/workflows/document-dashboard.md @@ -13,7 +13,8 @@ project root. ## Steps -1. **Inventory.** Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, +1. **Inventory.** If Power BI Desktop is open, have the user save first so the PBIP on disk + reflects the current model. Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, measures, relationships, pages, and visuals. 2. **Parse both layers.** - **Semantic model (`.tmdl`):** names, data types, `description`, DAX `expressions`, and diff --git a/.claude/workflows/plan.md b/.claude/workflows/plan.md index 438fc60..91958f4 100644 --- a/.claude/workflows/plan.md +++ b/.claude/workflows/plan.md @@ -14,12 +14,16 @@ it does not write code. 1. **Understand.** Read the relevant parts of the repo. If a key requirement is genuinely ambiguous, ask; otherwise state your assumptions. -2. **Route.** Identify which agent(s) own each part of the work (e.g. `data-engineer` for the - pipeline, `analytics-engineer` for the model, `powerbi-developer` for the dashboard). +2. **Map to skills and scripts.** For each part of the work, name the skills it will load and + the validation scripts it will run (e.g. `pbi-semantic-layer-tmdl` + `run_bpa.py` for a model + change, `pbi-report-layer-pbir` + `validate_pbir.py` for a report change, + `dbt-modeling-conventions` for dbt models). 3. **Write the plan** to `docs/PLAN-.md`, where `` is 2–3 hyphenated keywords from the request. Include: - **Goal & scope** (and explicit non-goals). - - **Task breakdown** with the responsible agent per task. - - **Dependencies & risks.** - - **Verification checklist** (how we'll know it works). + - **Task breakdown** with the skills/scripts each task uses. + - **Dependencies & risks.** For Power BI work, state whether Desktop will be open (Modeling + MCP + Desktop Bridge) or closed (edit on disk). + - **Verification checklist** (how we'll know it works — scripts to run, and the independent + visual review for report changes). 4. **Report** the exact file path created and suggest reviewing it before implementing. diff --git a/.claude/workflows/validate-pbi.md b/.claude/workflows/validate-pbi.md index 8cb5128..4da4d10 100644 --- a/.claude/workflows/validate-pbi.md +++ b/.claude/workflows/validate-pbi.md @@ -1,46 +1,50 @@ --- name: validate-pbi -description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 + structural review). +description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 or the Python fallback, PBIR validation, structural review). --- # Workflow: /validate-pbi **Usage:** `/validate-pbi [path/to/project.pbip]` -Answers "is my model healthy?" by running the quality checks on a PBIP semantic model. If no -path is given, use the project in the current folder; if there are several, ask which one. +Answers "is my Power BI project healthy?". If no path is given, use the project in the current +folder; if there are several, ask which one. ## Steps -1. **Locate the project.** Use `pbi-semantic-layer-tmdl` to confirm a valid PBIP project. - If none is found, report it and stop. -2. **Run the BPA.** Use `pbi-quality-rules` to run the Best Practice Analyzer (Tabular Editor 2 - CLI, or the TMDL-parsing fallback). Recommend the user close Desktop first so disk is current. - Record every `error` and `warning`. -3. **Structural review.** - - **Doc coverage:** tables/measures with a `description` vs. total. - - **Relationships:** flag island tables (no relationships), bidirectional filters, and - many-to-many joins. -4. **Report** — a short Markdown summary: +1. **Locate the project** (`.pbip` + `.SemanticModel/` + `.Report/`). If none is + found, report it and stop. +2. **Make disk current.** If Power BI Desktop has the project open, ask the user to save + (Ctrl+S). Desktop does not need to be closed — the checks read the files Desktop saved. +3. **Model — BPA** (`pbi-quality-rules`): + `python3 /scripts/run_bpa.py --format json`. + Record every `error` and `warning` and which engine ran (Tabular Editor 2 or Python fallback). +4. **Report — PBIR** (`pbi-report-layer-pbir`): + `python3 /scripts/validate_pbir.py --format json`. + Record errors and warnings and whether `powerbi-report-author` or the structural fallback ran. +5. **Structural review** of the TMDL: + - **Doc coverage:** tables/measures with a `///` description vs. total. + - **Relationships:** island tables, bidirectional filters, many-to-many joins. +6. **Report** — a short Markdown summary: ```markdown # Validation Report — -**Date:** | **Source:** TMDL files +**Date:** | **BPA engine:** Tabular Editor 2 | **PBIR check:** powerbi-report-author ## Summary -| Category | Errors | Warnings | -|----------|--------|----------| -| Naming | 0 | 3 | -| DAX | 1 | 2 | -| Model | 0 | 1 | -| Docs | 0 | 8 | +| Area | Errors | Warnings | +|------|--------|----------| +| Model — DAX | 1 | 1 | +| Model — Naming | 0 | 2 | +| Model — Docs | 0 | 2 | +| Report (PBIR) | 0 | 0 | ## Errors (fix before publishing) -- [DAX_EMPTY_EXPRESSION] Measure "Turnover %" — empty expression +- [DAX_EMPTY_EXPRESSION] Measure 'Sales'[Forecast] — empty expression ## Doc coverage -- Tables: 5/7 described (71%) · Measures: 12/20 described (60%) +- Tables: 3/4 described (75%) · Measures: 3/4 described (75%) ``` -5. **Offer fixes.** Auto-offer to fix all **errors**; ask whether to address **warnings** and - missing descriptions. +7. **Offer fixes.** Offer to fix every **error** (model fixes through the Modeling MCP when + Desktop is open, otherwise in TMDL); ask before addressing warnings and missing descriptions. diff --git a/.cursor/commands/document-dashboard.md b/.cursor/commands/document-dashboard.md index e38f27f..5c31123 100644 --- a/.cursor/commands/document-dashboard.md +++ b/.cursor/commands/document-dashboard.md @@ -13,7 +13,8 @@ project root. ## Steps -1. **Inventory.** Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, +1. **Inventory.** If Power BI Desktop is open, have the user save first so the PBIP on disk + reflects the current model. Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, measures, relationships, pages, and visuals. 2. **Parse both layers.** - **Semantic model (`.tmdl`):** names, data types, `description`, DAX `expressions`, and diff --git a/.cursor/commands/plan.md b/.cursor/commands/plan.md index 438fc60..91958f4 100644 --- a/.cursor/commands/plan.md +++ b/.cursor/commands/plan.md @@ -14,12 +14,16 @@ it does not write code. 1. **Understand.** Read the relevant parts of the repo. If a key requirement is genuinely ambiguous, ask; otherwise state your assumptions. -2. **Route.** Identify which agent(s) own each part of the work (e.g. `data-engineer` for the - pipeline, `analytics-engineer` for the model, `powerbi-developer` for the dashboard). +2. **Map to skills and scripts.** For each part of the work, name the skills it will load and + the validation scripts it will run (e.g. `pbi-semantic-layer-tmdl` + `run_bpa.py` for a model + change, `pbi-report-layer-pbir` + `validate_pbir.py` for a report change, + `dbt-modeling-conventions` for dbt models). 3. **Write the plan** to `docs/PLAN-.md`, where `` is 2–3 hyphenated keywords from the request. Include: - **Goal & scope** (and explicit non-goals). - - **Task breakdown** with the responsible agent per task. - - **Dependencies & risks.** - - **Verification checklist** (how we'll know it works). + - **Task breakdown** with the skills/scripts each task uses. + - **Dependencies & risks.** For Power BI work, state whether Desktop will be open (Modeling + MCP + Desktop Bridge) or closed (edit on disk). + - **Verification checklist** (how we'll know it works — scripts to run, and the independent + visual review for report changes). 4. **Report** the exact file path created and suggest reviewing it before implementing. diff --git a/.cursor/commands/validate-pbi.md b/.cursor/commands/validate-pbi.md index 8cb5128..4da4d10 100644 --- a/.cursor/commands/validate-pbi.md +++ b/.cursor/commands/validate-pbi.md @@ -1,46 +1,50 @@ --- name: validate-pbi -description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 + structural review). +description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 or the Python fallback, PBIR validation, structural review). --- # Workflow: /validate-pbi **Usage:** `/validate-pbi [path/to/project.pbip]` -Answers "is my model healthy?" by running the quality checks on a PBIP semantic model. If no -path is given, use the project in the current folder; if there are several, ask which one. +Answers "is my Power BI project healthy?". If no path is given, use the project in the current +folder; if there are several, ask which one. ## Steps -1. **Locate the project.** Use `pbi-semantic-layer-tmdl` to confirm a valid PBIP project. - If none is found, report it and stop. -2. **Run the BPA.** Use `pbi-quality-rules` to run the Best Practice Analyzer (Tabular Editor 2 - CLI, or the TMDL-parsing fallback). Recommend the user close Desktop first so disk is current. - Record every `error` and `warning`. -3. **Structural review.** - - **Doc coverage:** tables/measures with a `description` vs. total. - - **Relationships:** flag island tables (no relationships), bidirectional filters, and - many-to-many joins. -4. **Report** — a short Markdown summary: +1. **Locate the project** (`.pbip` + `.SemanticModel/` + `.Report/`). If none is + found, report it and stop. +2. **Make disk current.** If Power BI Desktop has the project open, ask the user to save + (Ctrl+S). Desktop does not need to be closed — the checks read the files Desktop saved. +3. **Model — BPA** (`pbi-quality-rules`): + `python3 /scripts/run_bpa.py --format json`. + Record every `error` and `warning` and which engine ran (Tabular Editor 2 or Python fallback). +4. **Report — PBIR** (`pbi-report-layer-pbir`): + `python3 /scripts/validate_pbir.py --format json`. + Record errors and warnings and whether `powerbi-report-author` or the structural fallback ran. +5. **Structural review** of the TMDL: + - **Doc coverage:** tables/measures with a `///` description vs. total. + - **Relationships:** island tables, bidirectional filters, many-to-many joins. +6. **Report** — a short Markdown summary: ```markdown # Validation Report — -**Date:** | **Source:** TMDL files +**Date:** | **BPA engine:** Tabular Editor 2 | **PBIR check:** powerbi-report-author ## Summary -| Category | Errors | Warnings | -|----------|--------|----------| -| Naming | 0 | 3 | -| DAX | 1 | 2 | -| Model | 0 | 1 | -| Docs | 0 | 8 | +| Area | Errors | Warnings | +|------|--------|----------| +| Model — DAX | 1 | 1 | +| Model — Naming | 0 | 2 | +| Model — Docs | 0 | 2 | +| Report (PBIR) | 0 | 0 | ## Errors (fix before publishing) -- [DAX_EMPTY_EXPRESSION] Measure "Turnover %" — empty expression +- [DAX_EMPTY_EXPRESSION] Measure 'Sales'[Forecast] — empty expression ## Doc coverage -- Tables: 5/7 described (71%) · Measures: 12/20 described (60%) +- Tables: 3/4 described (75%) · Measures: 3/4 described (75%) ``` -5. **Offer fixes.** Auto-offer to fix all **errors**; ask whether to address **warnings** and - missing descriptions. +7. **Offer fixes.** Offer to fix every **error** (model fixes through the Modeling MCP when + Desktop is open, otherwise in TMDL); ask before addressing warnings and missing descriptions. diff --git a/.cursor/rules/agentic-data-kit.mdc b/.cursor/rules/agentic-data-kit.mdc index 836d565..a02ecaa 100644 --- a/.cursor/rules/agentic-data-kit.mdc +++ b/.cursor/rules/agentic-data-kit.mdc @@ -1,7 +1,7 @@ --- -description: 'Agentic Data Kit — always-on operating rules: agent routing, data engineering - principles (idempotency, Write-Audit-Publish, PII masking) and the data debugging - reference.' +description: 'Agentic Data Kit — always-on operating rules: skills first, the Power + BI Desktop-open loop with independent visual review, data engineering principles + (idempotency, Write-Audit-Publish, PII masking) and the data debugging reference.' globs: '' alwaysApply: true --- @@ -19,17 +19,20 @@ data-specific rules — not things a capable model already does well. ## How to work -1. **Route to a specialist.** Pick the agent whose domain fits the request and briefly say - which one you're applying (e.g. "Applying @data-engineer"). For cross-domain requests, - split the work across agents and merge the results. -2. **Skills activate themselves.** Every skill declares when it applies in its `description`. - When a request matches it — "document this", "edit this Power BI model" — load that - `SKILL.md` on demand, **with or without an agent involved**. Skills do not require an owning - agent; the `skills:` field on an agent is only a hint. Don't preload every file in a folder. -3. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` +1. **Skills first.** Every skill declares when it applies in its `description`. When a request + matches one — "add a measure", "fix this visual", "document this dashboard" — load that + `SKILL.md` on demand and follow it. Don't preload every file in a folder. +2. **Power BI is developed with Desktop open.** Model changes go live through the Power BI + Modeling MCP (`pbi-semantic-layer-tmdl`); report changes follow edit → validate → save → + status → reload → screenshot → review (`pbi-report-layer-pbir`). Never reload while Desktop + has unsaved changes. +3. **Visual changes get an independent review.** After screenshots, hand the PNGs, the request + and the PBIR folder — not your reasoning — to the `pbi-visual-reviewer` agent. Where the tool + has no subagents, run its checklist in a fresh chat with only those inputs. +4. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` skill first — it decides whether to build an agent, a skill, or nothing, and keeps every installed tool folder (`.agent`, `.claude`, `.github`, `.opencode`, `.cursor`) in sync. -4. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd +5. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd likely build the wrong thing, ask. Otherwise state your assumptions and proceed. ## Data engineering principles (always apply) diff --git a/.cursor/skills/applying-visual-identity/SKILL.md b/.cursor/skills/applying-visual-identity/SKILL.md new file mode 100644 index 0000000..c4f4824 --- /dev/null +++ b/.cursor/skills/applying-visual-identity/SKILL.md @@ -0,0 +1,43 @@ +--- +name: applying-visual-identity +description: Use when the user wants output to follow a company's brand / visual identity, or asks to set one up — "use our brand", "make it on-brand", "here's our identity", "create a DESIGN.md". Reads and applies a DESIGN.md (Google Labs open spec): brand tokens for colors, typography, spacing, radius, shadows, and components, so decks, sites, and docs come out consistently on-brand. Scaffolds a DESIGN.md when none exists. +--- + +# Skill: applying-visual-identity + +A company's visual identity lives in a **`DESIGN.md`** at the project root — the open +spec from Google Labs (Apache-2.0, 2026). It pairs machine-readable **design tokens** (YAML +frontmatter) with human-readable rationale (prose), so any AI tool renders the brand the same +way across sessions. This skill reads that file and applies it; if none exists, it scaffolds one. + +## Workflow +1. **Locate.** Look for `DESIGN.md` in the project root. If found, parse the frontmatter tokens + — they are the *normative* values; the prose is guidance. +2. **Apply.** Map tokens to the target medium: + - **HTML/reveal.js/Marp** → CSS custom properties on `:root` (`--brand-primary`, `--font-heading`, …). + - **Markdown/docs** → headings, callouts, accent usage per the Do's/Don'ts. + - Resolve token references like `{colors.primary}` before output. +3. **Stay in-spec.** Use only defined tokens; don't introduce off-palette colors or fonts. If a + needed token is missing, ask or pick the closest defined one and note it. +4. **Scaffold when absent.** If there's no `DESIGN.md`, offer to create one from + `templates/DESIGN.md` — ask for (or extract from a provided logo/site) the primary color, + font, and tone. Never silently invent a brand; fall back to a neutral theme until confirmed. + +## The DESIGN.md format (Google Labs spec) +YAML frontmatter holds the tokens; the body documents intent in fixed sections. + +**Frontmatter token groups:** `colors`, `typography` (objects: `fontFamily`, `fontSize`, +`fontWeight`, `lineHeight`, `letterSpacing`), `spacing`, `rounded`, `shadows`, `components` +(per-component token refs). Colors accept hex / rgb / hsl / `oklch()`. Dimensions carry units +(`px`, `rem`). References use braces: `{colors.primary}`, `{typography.h1}`. + +**Body sections (in order):** Overview · Colors · Typography · Layout & Spacing · +Elevation & Depth · Shapes · Components · Do's and Don'ts. + +See `templates/DESIGN.md` for a complete, fillable example. + +## Rules +- Tokens are normative — match them exactly; the prose only explains *why*. +- Keep `DESIGN.md` in the **project root** so every tool and agent finds it. +- One source of truth: when the brand changes, edit `DESIGN.md`, not individual decks. +- This skill defines the *look*; `building-html-presentations` defines the *structure*. diff --git a/.cursor/skills/applying-visual-identity/templates/DESIGN.md b/.cursor/skills/applying-visual-identity/templates/DESIGN.md new file mode 100644 index 0000000..49d62b9 --- /dev/null +++ b/.cursor/skills/applying-visual-identity/templates/DESIGN.md @@ -0,0 +1,114 @@ +--- +# DESIGN.md — visual identity for AI agents (Google Labs spec, Apache-2.0). +# Place this file in your PROJECT ROOT. Tokens below are NORMATIVE; the prose explains intent. +# Token references use braces, e.g. {colors.primary}. Replace every value with your brand's. +version: alpha +name: Acme Analytics +description: Brand tokens for Acme's data presentations, docs, and dashboards. + +colors: + primary: "#1A1C1E" + secondary: "#6C7278" + accent: "#2563EB" + background: "#FFFFFF" + surface: "#F5F6F8" + text: "#1A1C1E" + muted: "#6C7278" + success: "#15803D" + warning: "#B45309" + danger: "#B91C1C" + +typography: + h1: + fontFamily: Public Sans + fontSize: 48px + fontWeight: 700 + lineHeight: 1.1 + h2: + fontFamily: Public Sans + fontSize: 32px + fontWeight: 600 + lineHeight: 1.2 + body: + fontFamily: system-ui + fontSize: 16px + fontWeight: 400 + lineHeight: 1.5 + caption: + fontFamily: system-ui + fontSize: 12px + fontWeight: 400 + lineHeight: 1.4 + +spacing: + xs: 4px + sm: 8px + md: 16px + lg: 24px + xl: 48px + +rounded: + sm: 4px + md: 8px + lg: 16px + +shadows: + sm: "0 1px 2px rgba(0,0,0,0.06)" + md: "0 4px 12px rgba(0,0,0,0.10)" + +components: + button: + background: "{colors.accent}" + color: "{colors.background}" + radius: "{rounded.md}" + card: + background: "{colors.surface}" + radius: "{rounded.lg}" + shadow: "{shadows.sm}" +--- + +# Overview + +**Acme Analytics** presents data with a calm, trustworthy, modern voice. Generous whitespace, +one accent color used sparingly, and clear typographic hierarchy. Charts and numbers are the +hero — chrome stays quiet. + +# Colors + +`primary` for headings and key text; `accent` ({colors.accent}) only for emphasis and one CTA +per view. `muted` for sources/captions. Semantic colors (`success`/`warning`/`danger`) only for +status — never decoration. + +# Typography + +Headings in **Public Sans**; body in the system stack for speed and legibility. Follow the type +scale ({typography.h1} → {typography.caption}); don't introduce intermediate sizes. + +# Layout & Spacing + +Use the spacing scale ({spacing.xs}–{spacing.xl}) — no arbitrary margins. Default page rhythm is +`lg`. Keep one idea per section; let content breathe. + +# Elevation & Depth + +Two shadow levels only: `sm` for resting cards, `md` for overlays/popovers. Avoid heavy drop +shadows; prefer flat surfaces separated by `surface` vs `background`. + +# Shapes + +Rounded corners from the `rounded` scale. Cards use `lg`; buttons/inputs use `md`. No sharp 0px +corners except full-bleed media. + +# Components + +- **Button** — accent background, background-color text, `md` radius. One primary per view. +- **Card** — `surface` background, `lg` radius, `sm` shadow, `md` inner padding. +- **Table** — caption-size source line beneath; right-align numerics. + +# Do's and Don'ts + +- ✅ Lead slides/sections with the insight; keep the accent rare. +- ✅ Label every chart with source, grain, date range, filters. +- ❌ Don't add off-palette colors or extra fonts. +- ❌ Don't use semantic colors decoratively. +- ❌ Don't stretch logos or place text on low-contrast backgrounds. diff --git a/.cursor/skills/building-html-presentations/SKILL.md b/.cursor/skills/building-html-presentations/SKILL.md new file mode 100644 index 0000000..3f5d505 --- /dev/null +++ b/.cursor/skills/building-html-presentations/SKILL.md @@ -0,0 +1,107 @@ +--- +name: building-html-presentations +description: Use when the user wants to build a presentation, deck, slides, pitch, readout, or interactive report from data work — exploratory analysis, dashboards, business plans, proposals. Covers as-code HTML formats (reveal.js deck, flexible scrollable deck, interactive site, consulting-style PDF, Marp) plus a chart cookbook, how to choose between them, and ready-to-edit starters. Default engine is reveal.js. For native editable PowerPoint use the generating-pptx skill. +--- + +# Skill: building-html-presentations + +Build presentations **as code** so they are versionable, diffable, and reproducible. This skill +covers when to use each format and gives starters you copy and fill with real content. Pair it +with `applying-visual-identity` to make the output on-brand. + +## Pick the format first + +| Output | Use when | Engine | Export | +|---|---|---|---| +| **Slide deck** | Live readout / meeting; linear narrative, fixed canvas | **reveal.js** (default) | HTML, PDF (`?print-pdf`) | +| **Flexible / scrollable deck** | Slides that hold more than one screen and need in-slide scroll + buttons to switch visuals | custom vanilla JS (`flex-deck.html`) | HTML | +| **Interactive site** | Self-serve exploration; one long scrolling page with anchors | single static HTML page (`interactive-site.html`) | HTML | +| **Designed report PDF** | Polished consulting-style document (action titles, exhibits) | HTML print template (`report-pdf.html`) | PDF (print) | +| **Quick handout PDF/PPTX** | Plain Markdown deliverable, no custom design | **Marp** (Markdown → PDF/PPTX) | PDF, PPTX | +| **Editable PowerPoint** | The deliverable must be a native, editable `.pptx` | → use the `generating-pptx` skill (python-pptx) | PPTX | + +Default to **reveal.js**: most mature, pure HTML/CSS (easy theming from `DESIGN.md`), and it +exports to PDF — so one source covers deck + site + print. Reach for **Marp** only when the +deliverable is fundamentally a document (Markdown-only, clean PDF/PPTX, no interactivity). +**Slidev** is a fine choice for fully technical audiences already on Node/Vue — mention it, +don't default to it. + +## Working order +1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load + `applying-visual-identity` and map its tokens to CSS variables. If absent, offer to scaffold + one (don't invent a brand silently) and use the neutral defaults in the starter meanwhile. +2. **Pick the format** from the table above and say which and why. A native `.pptx` that must be + editable in PowerPoint → `generating-pptx`. +3. **Story before charts.** Know the audience (exec → fewer slides, takeaway titles, no code; + technical → allow detail/appendix). Agree the storyline (5–9 beats) before writing HTML: lead + with the conclusion (BLUF), then evidence, then "so what / next". Use `data-story-outline.md`. +4. **Build from a starter** (`templates/`), drop in real numbers and charts, keep it + **self-contained** (works offline; pin CDN versions or vendor the files). +5. **Export** if needed (see the export note below). + +## Exporting (incl. PowerPoint) +- **reveal.js → PDF:** open the deck with `?print-pdf` appended, then print → Save as PDF. This is + the clean, high-fidelity export. +- **reveal.js → PPTX:** there is **no native, editable** path. If a `.pptx` file is required, use + **Marp** (`marp deck.md --pptx`) — but note each slide becomes an **image** (not editable text). + A PDF→PPTX converter is equally image-based. For truly editable native PPTX, generate it + separately (e.g. `python-pptx`); tell the user it's a different workflow, not an HTML export. +- **Marp → PDF / PPTX:** `marp deck.md --pdf` or `--pptx`. +- **Interactive site:** ship the HTML file itself (or host it); it isn't meant for PDF/PPTX. + +## Starters (in `templates/`) +- `reveal-deck.html` — a sophisticated, single-file **data-dashboard deck system** (dark theme, + brand tokens in `:root` mapped from `DESIGN.md`). Ships reusable layout primitives: + - **Cover** with KPI grid + stat chips + CTA bar; **section dividers**. + - **2×2 dashboard panels** mixing charts and narrative with colored keyword highlights. + - **Ranking bars** (CSS, value + ±pp delta, semantic colors); **heatmap matrix** with a + score-band color scale and metric **toggle pills**; **quote cards** with category tags. + - **Custom chrome**: zero-padded `01 / 07` counter, top progress bar, section tag, and a + dots + prev/next nav wired to the reveal.js API. + - **Charts**: pure-CSS bars for rankings + **ECharts** (pinned) for line/combo charts. + Copy it, swap the tokens, replace the demo content. Reach for the panel/bar/heatmap classes + rather than hand-rolling layout each time. +- `flex-deck.html` — a **flexible, scrollable deck** (no framework, no fixed aspect ratio): each + slide is a full-width sub-page that can be **taller than the screen and scrolls vertically**, + while ← → moves between slides. Includes **dynamic toggle buttons** that swap the visual in place + (chart series, and chart ↔ table). Use when a deck slide carries dashboard-density content. +- `interactive-site.html` — a single-file **scrolling report site** (not slides): sticky nav with + active-link highlight, reading-progress bar, hero + KPI band, scroll-reveal sections, a sticky + data table, and ECharts (line + donut). Same tokens as the deck. Use for self-serve exploration. +- `report-pdf.html` — a **consulting-style report** (McKinsey/Deloitte feel) built for PDF export: + print-first A4, light theme, running header/footer chrome, a **contents page**, **full-sentence + action titles**, an exec summary (key findings + takeaways), an **"at a glance" small-multiples** + page, 10 numbered **Exhibits** with varied charts (combo, grouped bar, scatter w/ quadrant lines, + waterfall bridge, donut, gauge, CSS bars), **insight callouts** + **margin notes**, a data-bar + table, and a recommendations table (owner/timeline/measure). Export via Chrome Print → Save as + PDF (A4, margins none, background graphics on). +- `charts-reference.html` — a **chart cookbook**: a live gallery of ~18 chart patterns (ECharts + + CSS-only) — line, area, multi-line, combo, bar, grouped, stacked, horizontal, scatter, bubble, + pie/doughnut, gauge, radar, heatmap, treemap, funnel, boxplot, waterfall, sparkline. Each card + names the type and when to use it. **Consult this to pick a chart and copy its config** into any + template, instead of defaulting to line + bar every time. +- `marp-deck.md` — Markdown deck for a quick, plain PDF/PPTX deliverable (no custom design). +- `data-story-outline.md` — fill-in narrative skeleton for analyses, EDA, and business plans. + +All HTML starters use a generic **sales review** as demo content — replace it with your own. + +## Charts & data +- **Match the chart to the question** — see `templates/charts-reference.html` for the full palette + and a "pick by question" guide (trend → line/area; compare → bar; composition → stacked; + correlation → scatter; part-of-whole → pie; KPI vs goal → gauge; matrix → heatmap; etc.). + Don't reach for the same line+bar by reflex. +- Use **ECharts** (pinned CDN, SVG renderer) for rich charts; pure-CSS bars/heatmap when it must + print or stay JS-free. Prefer SVG for PDF targets. +- Every data slide states **source, grain, date range, filters**. Never invent numbers — mark + unknowns `TODO`. + +## Data storytelling rules +- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". +- **One idea per slide.** Move supporting detail to an appendix section. +- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. +- **Never fabricate data.** Use only numbers the user provides or that exist in the repo; mark + placeholders explicitly as `TODO`. + +## Rules +- Keep it self-contained and offline-capable; pin versions, don't rely on live network at present time. +- Accessibility: real text (not images of text), sufficient contrast, logical heading order. diff --git a/.cursor/skills/building-html-presentations/templates/charts-reference.html b/.cursor/skills/building-html-presentations/templates/charts-reference.html new file mode 100644 index 0000000..ded2df6 --- /dev/null +++ b/.cursor/skills/building-html-presentations/templates/charts-reference.html @@ -0,0 +1,147 @@ + + + + + + Chart Cookbook + + + + +

Chart cookbook

+

Copy a config into any starter. Choose by the question you're answering (see the + comment at the top of this file), not by appearance. All charts use the brand tokens in :root.

+
+

Line

Trend over time, single metric.
+

Area

Trend with magnitude / cumulative feel.
+

Multi-line

Compare a few series over time.
+

Combo (bar + line)

Two units, e.g. volume vs rate.
+

Bar (vertical)

Compare categories.
+

Grouped bar

Compare categories across groups (plan vs actual).
+

Stacked bar

Composition within each category.
+

Horizontal bar

Rankings, long labels.
+

Scatter

Correlation between two metrics.
+

Bubble

Correlation + a third metric (size).
+

Pie / Doughnut

Part-of-whole (few slices).
+

Gauge

Single KPI vs a goal.
+

Radar

Multi-dimension profile.
+

Heatmap

Density across two dimensions.
+

Treemap

Hierarchy / share by size.
+

Funnel

Stage-by-stage drop-off.
+

Boxplot

Distribution & outliers.
+

Waterfall

Bridge of additive changes.
+

CSS bars (no JS)

Lightweight ranking; always prints.
+
A
92
+
B
74
+
C
58
+

CSS heatmap (no JS)

Matrix with a score-band scale.
+
+
96887972
90837164
+

Big-number + sparkline

A KPI with its mini-trend.
+
$4.2M
+ + + + + + diff --git a/.cursor/skills/building-html-presentations/templates/data-story-outline.md b/.cursor/skills/building-html-presentations/templates/data-story-outline.md new file mode 100644 index 0000000..4c40729 --- /dev/null +++ b/.cursor/skills/building-html-presentations/templates/data-story-outline.md @@ -0,0 +1,37 @@ +# Data-story outline + +Fill this in **before** building slides. Lead with the conclusion; each beat is one slide. +Adapt the beats to the genre (exploratory analysis · dashboard readout · business plan · proposal). + +## 0. Frame +- **Audience:** (exec / technical / mixed) +- **Decision this enables:** (what should they do after?) +- **Format:** deck / interactive site / PDF — and why +- **One-line takeaway (BLUF):** + +## 1. The ask / question +What are we answering, and why now? + +## 2. Bottom line (BLUF) +The single most important conclusion — stated as an insight, not a metric. + +## 3–N. Evidence (one insight per beat) +For each: **insight title** + the chart/table that proves it + **source · grain · date range · filters**. +- Beat A — insight: … +- Beat B — insight: … +- Beat C — insight: … + +## So what +Implications: what changes because of this. + +## Recommendation / next steps +Action · owner · timeline · what we'd measure to confirm. + +## Appendix +Methodology, assumptions, data lineage, edge cases. Detail the main flow skipped. + +--- +**Genre tweaks** +- *Exploratory analysis:* emphasize what's surprising; show what you ruled out. +- *Business plan:* market → problem → solution → model → traction → ask. +- *Proposal:* current pain → proposed change → cost/benefit → risks → decision needed. diff --git a/.cursor/skills/building-html-presentations/templates/flex-deck.html b/.cursor/skills/building-html-presentations/templates/flex-deck.html new file mode 100644 index 0000000..4e17f89 --- /dev/null +++ b/.cursor/skills/building-html-presentations/templates/flex-deck.html @@ -0,0 +1,360 @@ + + + + + + + Sales Performance Review — Flex Deck + + + + + +
+
Overview01 / 04
+ +
+ + +
+
+
Sales Performance Review · FY2026 · Q1
+

Q1 bookings closed +12% above plan

+

A flexible, scrollable deck — each slide is a full sub-page you can scroll, with + buttons to switch visuals. Use ← → (or the nav below) to move between slides. Replace the demo + numbers with your own; never invent data.

+
+
$4.2M
Bookings
+
+12%
vs. plan
+
28%
Win rate
+
+
+
+ + +
+
+
Section 01 · Summary
+

Executive summary

+

General view, history, and priorities. This slide is taller than the screen — + scroll down inside it, then use → to advance.

+ + +
+
86%
Plan attainment
+
60
Net new logos
+
87%
Forecast accuracy
+
+ + +

Attainment by region & segment

+

% of target · FY2026 Q1

+
+
118%
North America
+
104%
EMEA
+
96%
LATAM
+
88%
APAC
+
122%
Enterprise
+
99%
Mid-market
+
73%
SMB
+
131%
Expansion
+
94%
New business
+
112%
Partner
+
+ + +
+
+
Trend — last 4 quarters
+
+ + +
+
+
+
+
Region attainment
+
% of quarterly target
+
+
North America
118%+18
+
EMEA
104%+4
+
LATAM
96%−4
+
APAC
88%−12
+
+
+
+ + +
+
+
Attainment: plan vs. actual
+
% of target, by segment
+
+
+
+
Deal size vs. win rate
+
each point = a segment
+
+
+
+ + +
+
+
What's working
+

Enterprise drove the beat — larger + deals, faster cycles. Expansion kept NRR above target. Double down on the + enterprise motion and the partner channel.

+
+
+
Where to focus
+

SMB win rate slipped on late-stage + stalls. Discounting pressured margin on competitive deals. Tighten + qualification and approval guardrails this quarter.

+
+
+

Source: gold.fct_bookings · grain: deal · FY2026 Q1 — TODO real values

+
+
+ + +
+
+
Section 02 · Regions
+

Regional detail

+

Same data, two views — use the buttons to switch between the chart and the table.

+ +
+ + +
+ +
+
+
+ + + + + + + + +
RegionBookingsAttainmentDealsWin rate
North America$1.8M118%40231%
EMEA$1.1M104%31829%
LATAM$0.7M96%26626%
APAC$0.6M88%25422%
+
+
+

Source: gold.fct_bookings · grain: deal · FY2026 Q1 — TODO real values

+
+
+ + +
+
+
So what — and next
+

Three moves from here

+
+
1 · Scale enterprise

Reallocate reps to the motion that beat plan. Owner · timeline.

+
2 · Fix SMB funnel

Tighten qualification + discount guardrails. Measure: SMB win rate.

+
3 · Revisit APAC

90-day ramp plan; re-check attainment next quarter.

+
+
+
+ +
+ +
+
+
+ + + + + diff --git a/.cursor/skills/building-html-presentations/templates/interactive-site.html b/.cursor/skills/building-html-presentations/templates/interactive-site.html new file mode 100644 index 0000000..5e46831 --- /dev/null +++ b/.cursor/skills/building-html-presentations/templates/interactive-site.html @@ -0,0 +1,288 @@ + + + + + + + Sales Performance Review — Interactive Report + + + + + +
+
+ Acme · Sales Review + +
+ +
+ +
+
Sales Performance Review · FY2026 · Q1
+

Q1 bookings closed +12% above plan

+

A self-serve report: scroll for the trend, regional attainment, product mix, and the + three moves for next quarter. Replace the demo numbers with your own — never invent data.

+
+
$4.2M
Bookings
+
+12%
vs. plan
+
28%
Win rate
+
$9.1M
Pipeline
+
+
+ + +
+
01 · Trend
+

Bookings grew four quarters running

+
+
+

Bookings — last 4 quarters ($M)

+
Source · grain: deal · FY2026
+
+
+
+

What's driving it

+

Enterprise deals — bigger and + faster — carried the beat, while expansion kept net revenue retention above + target. The watch item is SMB win rate, soft on late-stage stalls, and + discounting pressure on competitive deals.

+
+
+

Source: gold.fct_bookings · grain: deal · FY2026 Q1 — TODO real values

+
+ + +
+
02 · Regions
+

Region attainment vs. target

+
+
+

% of quarterly target

+
Ordered by attainment
+
+
North America
118%+18
+
EMEA
104%+4
+
LATAM
96%−4
+
APAC
88%−12
+
+
+
+

Share of bookings

+
By region, FY2026 Q1
+
+
+
+
+ + +
+
03 · Segments
+

Enterprise outperforms; SMB is the drag

+
+
+

Attainment: plan vs. actual

+
% of target, by segment
+
+
+
+

Deal size vs. win rate

+
each point = a segment
+
+
+
+

Source: gold.fct_bookings · grain: deal · FY2026 Q1 — TODO real values

+
+ + +
+
04 · Products
+

Product line detail

+ + + + + + + + + +
ProductBookingsvs. targetDealsAvg deal
Platform$1.6M+22%312$5.1K
Analytics$1.2M+14%198$6.1K
Add-ons$0.8M+6%420$1.9K
Services$0.6M−9%88$6.8K
Legacy SKU$0.4M−18%140$2.9K
+

Source: gold.fct_bookings · grain: deal · FY2026 Q1 — TODO real values

+
+ + +
+
So what — and next
+

Three moves from here

+
+

1 · Scale enterprise

Reallocate reps to the motion that beat plan. Owner · timeline.

+

2 · Fix SMB funnel

Tighten qualification + discount guardrails. Measure: SMB win rate.

+

3 · Revisit APAC

90-day ramp plan; re-check attainment next quarter.

+
+
+
+ +
+ Built as code · replace demo data with real, sourced figures · brand tokens map from DESIGN.md. +
+ + + + + diff --git a/.cursor/skills/building-html-presentations/templates/marp-deck.md b/.cursor/skills/building-html-presentations/templates/marp-deck.md new file mode 100644 index 0000000..386232e --- /dev/null +++ b/.cursor/skills/building-html-presentations/templates/marp-deck.md @@ -0,0 +1,55 @@ +--- +marp: true +title: Deck Title +paginate: true +# Brand: set colors/fonts from DESIGN.md. Marp themes use CSS; inline below for portability. +style: | + :root { + --brand-primary: #1A1C1E; + --brand-accent: #2563EB; + --brand-muted: #6C7278; + } + section { font-family: system-ui, sans-serif; color: var(--brand-primary); } + h1, h2 { color: var(--brand-primary); } + .accent { color: var(--brand-accent); } + .source { font-size: 0.5em; color: var(--brand-muted); } +--- + + + +# Deck Title +What this answers · Audience · Date + +--- + +## The takeaway + +State the single most important conclusion in one sentence. + +Bottom line up front — details follow. + +--- + +## Revenue fell 12% in EMEA last quarter + +![w:900](path/to/chart.svg) + +Source: gold.fct_revenue · grain: order · 2026-Q1 · filters: region=EMEA + +--- + +## So what — and next + +- Implication of the finding +- Recommended action + owner +- What we'd measure to confirm + +--- + +## Appendix + +Methodology, sources, joins, assumptions. TODO: fill with real values — never invent data. diff --git a/.cursor/skills/building-html-presentations/templates/report-pdf.html b/.cursor/skills/building-html-presentations/templates/report-pdf.html new file mode 100644 index 0000000..8fe1601 --- /dev/null +++ b/.cursor/skills/building-html-presentations/templates/report-pdf.html @@ -0,0 +1,458 @@ + + + + + + + Revenue Performance — An Exploratory Analysis + + + + + +
Screen preview · to export: Print (Ctrl/Cmd+P) → Save as PDF → A4, Margins “None”, tick “Background graphics”.
+ + +
+
+
+
Report № 2026-Q1-014
+
+
+
Confidential — Draft for discussion
+
+
+

Revenue Performance

+

An exploratory analysis of FY2026 Q1 bookings — trends, concentration, and drivers

+
+
+
+
Prepared for
Executive Committee
+
Prepared by
Analytics Team
+
Date
June 2026
+
+
Strictly confidential · Not for distribution · © 2026
+
+
+ + +
+
ContentsRevenue Performance · FY2026 Q1
+
+
Contents
+

What this report covers

+
+
01

Executive summary

Headline result, KPIs, and four key findings

3
+
02

At a glance

Dashboard of the quarter in four small charts

4
+
03

Trend

Bookings and win rate over four quarters

5
+
04

Concentration

Distribution by region and segment

6
+
05

Drivers

What moves win rate, and the bookings bridge

7
+
06

Appendix & recommendations

Region detail and the four moves from here

8
+
+
+
Source: company data (gold.fct_bookings)Confidential · 2
+
+ + +
+
01 · Executive summaryRevenue Performance · FY2026 Q1
+
+
Executive summary
+

Bookings beat plan by 12%, but growth is concentrated in enterprise and increasingly at risk in SMB

+

Q1 bookings reached $4.2M (+12% vs. plan, +35% over four quarters). The upside is real + but narrow: two regions and the enterprise segment carry most of it, while SMB and discounting are + early warning signs for H2.

+ +
+
$4.2M
Bookings
+
+12%
vs. plan
+
28%
Win rate
+
$9.1M
Pipeline
+
+ +
+
+
1

Growth is steady but accelerating in enterprise

Bookings rose every quarter; the Q4→Q1 step was the largest, led by enterprise deals.

+
2

Concentration risk

The top two regions account for ~70% of bookings, leaving the result exposed to a few accounts.

+
3

SMB is softening

SMB attainment fell to 73% as deals stalled late-stage; win rate dipped quarter-over-quarter.

+
4

Margin pressure from discounting

Competitive deals closed at higher discounts, diluting realized price on ~1 in 5 deals.

+
+
+

Key takeaways

+
    +
  • Double down on the enterprise motion that drove the beat.
    • +
  • De-risk concentration by broadening the mid-market pipeline.
    • +
  • Intervene on SMB qualification and discount governance now.
    • +
  • Re-forecast H2 with explicit downside scenarios.
    • +
+
+
+
+
Source: company data (gold.fct_bookings)Confidential · 3
+
+ + +
+
02 · At a glanceRevenue Performance · FY2026 Q1
+
+
At a glance
+

The quarter in four numbers: up, concentrated, on-goal overall, soft in SMB

+
+
Exhibit 1 · Bookings trend ($M)
+
Exhibit 2 · Mix by region
+
Exhibit 3 · Overall attainment
+
Exhibit 4 · Attainment by segment (%)
+
+
Enterprise
122%
+
Mid-market
99%
+
SMB
73%
+
+
+
+
Key insight  The aggregate is healthy, but the spread between + enterprise (122%) and SMB (73%) is the story — the average hides a widening gap.
+
+
Source: company data (gold.fct_bookings)Confidential · 4
+
+ + +
+
03 · TrendRevenue Performance · FY2026 Q1
+
+
Trend
+

Bookings grew every quarter while win rate held — growth is volume-led, not discount-led

+

The four-quarter trajectory is consistently positive; win rate stayed in a tight band, + suggesting the gains came from more and larger deals rather than price concessions.

+
+
+
Exhibit 5
+
Quarterly bookings vs. win rate
+
$M (bars) and % (line)
+
+
Source: company data; n = 1,240 closed-won deals.
+
+
+
Read this as
+ Volume-led growth is more durable than discount-led growth. Watch whether win rate holds as + deal sizes rise — a divergence would signal pricing pressure. +
+
+
Key insight  +35% bookings with a flat win rate implies the + funnel widened — confirm pipeline coverage can sustain it into H2.
+
+
Source: company data (gold.fct_bookings)Confidential · 5
+
+ + +
+
04 · ConcentrationRevenue Performance · FY2026 Q1
+
+
Concentration
+

Two regions drive ~70% of bookings; enterprise outperforms while SMB lags

+
+
+
Exhibit 6
+
Attainment by segment — plan vs. actual
+
% of quarterly target
+
+
Source: company data; target set at FY2026 plan.
+
+
+
So what
+ Enterprise beat plan by 22pp; SMB missed by 27pp. The portfolio result depends on a single + strong segment — a concentration the pipeline should diversify. +
+
+
+
Exhibit 7
+
Bookings by region
+
$M and share of total
+
+
North America
$1.8M
+
EMEA
$1.1M
+
LATAM
$0.7M
+
APAC
$0.6M
+
+
Source: company data; shares sum to 100% of $4.2M total.
+
+
+
Source: company data (gold.fct_bookings)Confidential · 6
+
+ + +
+
05 · DriversRevenue Performance · FY2026 Q1
+
+
Drivers
+

Larger deals win more often; the quarter's growth came from new and expansion, not churn

+
+
+
Exhibit 8
+
Win rate vs. average deal size
+
each point = a region; lines = medians
+
+
Source: company data; n = 1,240 deals.
+
+
+
Quadrant read
+ Top-right = large deals that also win — the enterprise sweet spot. Bottom-left (SMB) is the + zone to fix: small deals, lower win rate. +
+
+
+
Exhibit 9
+
Bookings bridge — what moved the quarter
+
$M, Q4/25 → Q1/26
+
+
Source: company data; new + expansion net of churn.
+
+
+
Source: company data (gold.fct_bookings)Confidential · 7
+
+ + +
+
06 · Detail & recommendationsRevenue Performance · FY2026 Q1
+
+
Detail
+

Region detail confirms the concentration; four moves follow

+ +
+
Exhibit 10
+
Bookings and performance by region
+ + + + + + + + + +
RegionBookingsShareAttainmentDealsWin rate
North America$1.8M43%118%40231%
EMEA$1.1M26%104%31829%
LATAM$0.7M17%96%26626%
APAC$0.6M14%88%25422%
Total$4.2M100%112%1,24028%
+
Figures illustrative — replace with real, sourced values; never invent data.
+
+ +
+
Recommendations
+ + + + + + + + +
#MoveOwnerTimelineMeasure
1Scale the enterprise motionSales leadThis quarterEnterprise bookings
2Broaden mid-market pipelineDemand gen30 daysPipeline by region
3Fix the SMB funnelSales ops30–60 daysSMB win rate
4Re-forecast H2 with scenariosFinanceNext reviewForecast accuracy
+
+
+
Source: company data (gold.fct_bookings)Confidential · 8
+
+ + + + + diff --git a/.cursor/skills/building-html-presentations/templates/reveal-deck.html b/.cursor/skills/building-html-presentations/templates/reveal-deck.html new file mode 100644 index 0000000..7140dcd --- /dev/null +++ b/.cursor/skills/building-html-presentations/templates/reveal-deck.html @@ -0,0 +1,436 @@ + + + + + + + Sales Performance Review — FY2026 + + + + + + + + +
+
+ Overview + 01 / 08 +
+ +
+
+ + +
+
Sales Performance Review · FY2026 · Q1
+
+
+

Q1 bookings closed
+12% above plan

+

Consolidated bookings across 5 regions + and 4 product lines — 1,240 closed-won deals vs. plan and prior quarter. Replace with your + own figures; never invent numbers.

+
+
1,240 deals
+
38 reps
+
5 regions
+
+
+
+
+
$4.2M
Bookings
+
+12%
vs. plan
+
+9%
vs. last Q
+
28%
Win rate
+
$34K
Avg deal
+
$9.1M
Pipeline
+
+
Start →
+
+
+
+ + +
+
Section 01
+

Executive summary

+

Bookings beat plan on enterprise strength + and healthy expansion; SMB win rate and discounting are the risks to watch this + quarter.

+
+ + +
+
Section 01 · Summary
+

How the quarter landed

+
+
+
Bookings — last 4 quarters ($M)
+
Source · grain · date range · filters
+
+
+
+
Region attainment
+
% of quarterly target
+
+
North America
118%+18
+
EMEA
104%+4
+
LATAM
96%−4
+
APAC
88%−12
+
+
+
+
What's working
+

Enterprise segment drove the beat — + larger deals, faster cycles. Expansion / upsell net revenue retention above + target. Double down on the enterprise motion.

+
+
+
Where to focus
+

SMB win rate slipped as deals stalled + late-stage. Discounting pressured margin on competitive deals. Tighten + qualification and the approval guardrails.

+
+
+

Source: gold.fct_bookings · grain: deal · FY2026 Q1 · filters: closed-won — TODO real values

+
+ + +
+
Section 01 · Mix & profile
+

Channel mix & segment profile

+
+
+
Bookings by channel
+
share of total
+
+
+
+
Enterprise vs. SMB profile
+
indexed, 0–100
+
+
+
+

Source: gold.fct_bookings · grain: deal · FY2026 Q1 — TODO real values

+
+ + +
+
Section 02 · Drilldown
+

Product lines — winners & laggards

+
+
+
Above target
+
+
Platform
$1.6M+22%
+
Analytics
$1.2M+14%
+
Add-ons
$0.8M+6%
+
+
+
+
Below target
+
+
Services
$0.6M−9%
+
Legacy SKU
$0.4M−18%
+
Trials → paid
$0.3M−24%
+
+
+
+

Bars = bookings ($); delta = vs. target. Ordered by attainment. — TODO real values

+
+ + +
+
Section 03 · Matrix
+

Attainment by region × product

+
+ Attainment (%) + Δ vs. target (pp) +
+
+ + high → low +
+ + + + + + + + + +
ProductNAEMEALATAMAPACGlobal
Platform1281129890122
Analytics1191069488114
Add-ons1041019082106
Services9286747191
Legacy SKU7673625882
+

Color = attainment band. TODO: replace with real values; keep one band scale.

+
+ + +
+
Section 04 · Win / Loss
+

Why we win & why we lose

+
💬 Reading note: themes summarized from closed-deal notes in the CRM + — representative, not edited. TODO: paste real, de-identified excerpts.
+
+
+

Why we win

+

"Short, representative quote from a won deal — e.g. product fit, time-to-value, or a + champion-led evaluation."

+
Product fitTime-to-valueChampion
+
+
+

Why we lose

+

"Short, representative quote from a lost deal — e.g. price, missing feature, or a + stalled procurement process."

+
PriceFeature gapProcurement
+
+
+
+ + +
+
So what — and next
+

Three moves from here

+
+
1 · Scale enterprise

Reallocate reps to the enterprise motion that beat plan. Owner · timeline.

+
2 · Fix SMB funnel

Tighten qualification + discount guardrails. Measure: SMB win rate.

+
3 · Revisit APAC

90-day APAC ramp plan; re-check attainment next quarter.

+
+
+ +
+
+ + +
+ +
+ +
+ + + + + + diff --git a/.cursor/skills/creating-agents-and-skills/SKILL.md b/.cursor/skills/creating-agents-and-skills/SKILL.md new file mode 100644 index 0000000..19d871c --- /dev/null +++ b/.cursor/skills/creating-agents-and-skills/SKILL.md @@ -0,0 +1,127 @@ +--- +name: creating-agents-and-skills +description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, .cursor, AGENTS.md) identical in each tool's native format. +--- + +# Skill: creating-agents-and-skills + +The governance gate for extending this kit. Goal: people add capabilities **only when they +help**, build them **with quality**, and every installed tool folder stays **in sync**. + +Follow three phases in order: **Decide → Author → Propagate.** + +--- + +## Phase 1 — Decide what (if anything) to build + +Run these gates before creating anything. If a gate says stop, explain why and stop. + +- **LLM-native? → don't create.** If a capable model already does this well ("write clean + code", "use CTEs", "explain this function"), no artifact is needed. Just do the task. +- **Narrow knowledge / a tool / a format / a checklist / a script? → make a SKILL.** This is + the default for almost everything: a team naming standard, a framework's rules, a validation + procedure. Skills auto-activate by description and cost almost nothing until used. +- **A repeatable multi-step procedure the user invokes? → make a WORKFLOW** (slash command). +- **Overlaps an existing skill/agent? → extend it,** don't add a near-duplicate. + +### The agent-survival rule + +An **agent** is allowed only if it imposes something the model cannot impose on itself: + +1. **Restricted tools** — a frontmatter `tools` list that removes capabilities (e.g. read-only). +2. **Independent review** — fresh context, without the author's reasoning. +3. **A different model** than the main session. +4. **An automatic trigger** that fires without the user asking. + +If none applies, it is a skill: "a persona with a domain" is not a reason. Domain knowledge +belongs in a skill, or is dropped if the model already knows it. Agents must not hand off to +other agents. + +> Default to a skill. Recommend an agent only if you can name which of the four criteria it +> satisfies, and say so in the agent's description or body. + +--- + +## Phase 2 — Author it lean + +Write only domain/org-specific signal. No filler, no restating what the model knows. English. + +**Agent** — `.agent/agents/.md` (example: `pbi-visual-reviewer`, a read-only reviewer): +```yaml +--- +name: pbi-visual-reviewer +description: Independently reviews Power BI report screenshots against the request and the PBIR folder; PASS/FAIL per page and visual. Use after a report change has been reloaded and screenshotted. +tools: Read, Grep, Glob # the restriction that justifies the agent: no Bash/Edit/Write +model: inherit +--- +# PBI Visual Reviewer +<1–2 line identity> → fixed checklist → output format (PASS/FAIL). No handoffs. +``` +`tools` is the field that matters: list only what the agent needs. + +**Skill** — `.agent/skills//SKILL.md` (gerund-style name, e.g. `validating-dbt-models`): +```yaml +--- +name: +description: What it does AND when to use it, in the third person, with trigger words so it + auto-activates (e.g. "Use when the user asks to ... "). This is the most important field. +--- +# Skill: + → concrete rules or steps. Keep the body under ~500 lines; move long reference +material to sibling files (e.g. references/, scripts/) and link them. +``` +Write descriptions in the **third person** ("Generates...", "Use when...") — never "I can help". +The description is what makes the skill fire without the user naming it. + +**Workflow** — `.agent/workflows/.md`: `description` frontmatter + concise numbered steps. + +--- + +## Phase 3 — Propagate to every installed tool folder + +A capability must exist, identically, in each AI tool the project uses — each in its native +format. Pick the path that matches the project: + +### Path A — Source of truth present (`.agent/` + `scripts/sync_agents.py` exist) +This is the kit repo or an Antigravity install. **Author in `.agent/` only, then sync:** +```bash +python3 scripts/sync_agents.py # regenerate +python3 scripts/sync_agents.py --check # exits 1 if generated folders are stale (CI runs this) +``` +The script regenerates `.claude/`, `.github/`, `.opencode/`, `.cursor/`, and root `AGENTS.md` +(which also covers OpenAI Codex). Never hand-edit those — they're overwritten. Done. + +### Path B — No source of truth (user installed one or more tool flavors) +Detect which folders exist and write the artifact into **each present folder** in its native +format (the body content stays identical; only frontmatter changes). Format mapping: + +| Concept | `.agent` (Antigravity) | `.claude` (Claude Code) | `.github` (Copilot) | `.opencode` (OpenCode) | `.cursor` (Cursor) | +|---|---|---|---|---|---| +| Agent | `agents/.md` | `agents/.md` (identical) | `agents/.agent.md` — frontmatter `name`, `description`, `tools` mapped from Claude names (Read→`read`, Grep/Glob→`search`, Edit/Write→`edit`, Bash→`execute`) | `agents/.md` — frontmatter `description`, `mode: subagent`, `permission` (deny edit/bash when the agent is read-only; drop `tools`/`model`/`name`/`skills`) | — (none; run the task in a fresh chat) | +| Skill | `skills//SKILL.md` | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | `skills//SKILL.md` (identical) | +| Workflow | `workflows/.md` | `commands/.md` (native slash command) + `workflows/.md` | `prompts/.prompt.md` — frontmatter `name`, `description`; body starts `**Context:** ${selection}` | `commands/.md` (identical) | `commands/.md` (identical) | +| Global rule | `rules/rules.md` (with `trigger: always_on`) | `CLAUDE.md` (rules + commands section) | `copilot-instructions.md` (rules) | root `AGENTS.md` (rules) | `rules/.mdc` — frontmatter `description`, `globs`, `alwaysApply: true` | + +Rules: +- **Strip the `trigger: always_on` frontmatter** from `rules.md` when writing it anywhere + outside `.agent/` — it's Antigravity-only and is noise everywhere else. +- **OpenCode reads the root `AGENTS.md`**, not `.opencode/AGENTS.md` — don't create the latter. + The same root `AGENTS.md` is what **OpenAI Codex** reads, so Codex needs no dedicated folder. +- Frontmatter differs per tool (Copilot agent/prompt, OpenCode agent, Cursor rule/mdc); the + **body is a verbatim copy** everywhere. +- Create the same `` in all present folders — do not leave a tool behind. +- If the user installed only one folder but wants multi-tool support, offer to add the others. + +### Verify parity +After either path, confirm the artifact exists in every present tool folder and that no folder +references something that no longer exists. In Path A, `python3 scripts/sync_agents.py --check` +does this. State which folders you wrote. + +--- + +## Anti-patterns +- Creating an agent for a persona, a single tool, or a single task (e.g. "GitAgent") — it fails + the survival rule; write a skill. +- Chaining agents through handoffs. +- Putting episodic guidance in the always-on rules — it belongs in a skill like this one. +- Editing generated folders (`.claude`/`.github`/`.opencode`/`.cursor`/`AGENTS.md`) when `.agent/` exists. diff --git a/.cursor/skills/dbt-modeling-conventions/SKILL.md b/.cursor/skills/dbt-modeling-conventions/SKILL.md new file mode 100644 index 0000000..57446ce --- /dev/null +++ b/.cursor/skills/dbt-modeling-conventions/SKILL.md @@ -0,0 +1,63 @@ +--- +name: dbt-modeling-conventions +description: Applies the organization's dbt modeling conventions. Use when the user writes, reviews, or renames dbt models, mentions dbt, staging, marts, fct_ or dim_ tables, schema.yml, surrogate keys, model naming, CTE structure, or dimensional modeling (star schema) with dbt. +--- + +# Skill: dbt-modeling-conventions + +Org conventions only — the model already knows generic SQL and dbt. Apply these to every model +you write or review. + +## CTE structure +- `import` CTEs on top (one per `ref()`/`source()`, `select` only the needed columns). +- `logical` CTEs in the middle (joins, filters, calculations — one step each). +- One `final` CTE at the bottom; the model ends with `select * from final`. + +## Naming +| Prefix | Layer | +|---|---| +| `stg_` | Staging: 1:1 with a source table, rename/cast only | +| `dim_` | Dimension | +| `fct_` | Fact | +| `mart_` | Business-facing mart built on dims/facts | + +## Keys and grain +- Every dimension has a surrogate key `sk_`: a hash of the natural key. +- One grain per fact table; state it in the model description ("one row per order line"). +- Build dimensions before facts. + +## Tests and docs +Every model has a `schema.yml` entry with a description and `unique` + `not_null` on its +primary key. + +## Example + +```sql +-- models/marts/dim_customer.sql +with stg_customers as ( + select * from {{ ref('stg_crm__customers') }} +), +customers_keyed as ( + select + {{ dbt_utils.generate_surrogate_key(['customer_id']) }} as sk_customer, + customer_id, + customer_name, + country + from stg_customers +), +final as ( + select * from customers_keyed +) +select * from final +``` + +```yaml +# models/marts/schema.yml +models: + - name: dim_customer + description: One row per customer. + columns: + - name: sk_customer + description: Surrogate key, hash of customer_id. + tests: [unique, not_null] +``` diff --git a/.cursor/skills/documentation-templates/SKILL.md b/.cursor/skills/documentation-templates/SKILL.md new file mode 100644 index 0000000..fb82612 --- /dev/null +++ b/.cursor/skills/documentation-templates/SKILL.md @@ -0,0 +1,24 @@ +--- +name: documentation-templates +description: Use when the user asks to document, write docs for, or generate a runbook, data dictionary, metric definition, ADR, or Power BI dashboard catalog for a data project. Provides the team's canonical documentation templates as separate files. +allowed-tools: Read, Glob, Grep, Write +--- + +# Skill: documentation-templates + +The team's canonical documentation formats. The value here is **consistency** — one agreed +structure per doc type — not novel knowledge. Pick the template that matches the request and +open only that file (each lives in `templates/`). + +| The user wants to document… | Open | +|---|---| +| A pipeline/job (for 3 AM failures) | `templates/pipeline-runbook.md` | +| A table/model (data dictionary) | `templates/data-dictionary.md` | +| A metric for analysts | `templates/metric-definition.md` | +| An architecture decision | `templates/adr.md` | +| A Power BI dashboard (auto-generated from PBIP) | `templates/powerbi-dashboard.md` | + +## Principles +- **Audience first:** an engineer fixing a bug reads differently than a business user. +- **Keep docs close to the code/schema** (YAML, `schema.yml`, repo) — wikis rot fastest. +- Explain the *why* and the edge cases; don't restate what the code already says. diff --git a/.cursor/skills/documentation-templates/templates/adr.md b/.cursor/skills/documentation-templates/templates/adr.md new file mode 100644 index 0000000..14789a3 --- /dev/null +++ b/.cursor/skills/documentation-templates/templates/adr.md @@ -0,0 +1,20 @@ +# Template: Architecture Decision Record (ADR) + +Use for significant choices (e.g. Airflow → Dagster, Delta vs. Iceberg, Databricks → dbt+Snowflake). + +```markdown +# ADR-001: [Title] + +## Status +Proposed / Accepted / Rejected / Deprecated + +## Context +Why are we deciding this now? (e.g. Databricks costs too high for simple transforms.) + +## Decision +What we chose. (e.g. dbt Core on Snowflake for the transform layer.) + +## Consequences +- **Positive:** lower compute cost, analysts can write SQL. +- **Negative:** complex ML transforms are harder to keep inline with ELT. +``` diff --git a/.cursor/skills/documentation-templates/templates/data-dictionary.md b/.cursor/skills/documentation-templates/templates/data-dictionary.md new file mode 100644 index 0000000..b571c45 --- /dev/null +++ b/.cursor/skills/documentation-templates/templates/data-dictionary.md @@ -0,0 +1,22 @@ +# Template: Table / Model Data Dictionary + +Prefer placing this inside the model's `schema.yml` (dbt) or a Markdown registry next to the code. + +```markdown +# Table: [schema].[table] + +**Description:** One row per completed order, with net amounts and applied discounts. +**Grain:** 1 row = 1 order item + +### Columns +| Column | Type | Description | PII / Sensitive | +|--------|------|-------------|-----------------| +| `order_id` | STRING | Primary key. | No | +| `customer_id` | STRING | FK to `dim_customer`. | No | +| `net_amount` | FLOAT | Paid after taxes and discounts. | No | +| `customer_email` | STRING | Purchaser email. | **YES (masked to BI)** | + +### Tests +- `order_id`: unique, not_null +- `customer_id`: relationships to `dim_customer` +``` diff --git a/.cursor/skills/documentation-templates/templates/metric-definition.md b/.cursor/skills/documentation-templates/templates/metric-definition.md new file mode 100644 index 0000000..8728c75 --- /dev/null +++ b/.cursor/skills/documentation-templates/templates/metric-definition.md @@ -0,0 +1,14 @@ +# Template: Metric Definition + +Define metrics unambiguously so analysts and dashboards agree on the number. + +```markdown +# Metric: Monthly Active Users (MAU) + +**Definition:** Unique authenticated users who logged in at least once in the trailing 30 days. +**Calculation (pseudo-SQL):** `COUNT(DISTINCT user_id) WHERE login_date >= CURRENT_DATE - 30` +**Grain / dimensions:** Country, Device Type, Subscription Tier +**Source table:** `gold.fct_user_logins` +**Polarity:** UP is good +**Owner:** Product Analytics +``` diff --git a/.cursor/skills/documentation-templates/templates/pipeline-runbook.md b/.cursor/skills/documentation-templates/templates/pipeline-runbook.md new file mode 100644 index 0000000..71fb515 --- /dev/null +++ b/.cursor/skills/documentation-templates/templates/pipeline-runbook.md @@ -0,0 +1,28 @@ +# Template: Pipeline Runbook + +Every critical pipeline needs a runbook for when it fails at 3 AM. Essential sections: +Description, SLAs, Upstream dependencies, Downstream consumers, Common failures & fixes, +Escalation contacts. + +```markdown +# Pipeline: [Name] + +**Business Criticality:** High | Medium | Low +**SLA:** Must complete by 08:00 AM +**Schedule:** Daily at 02:00 AM + +## Description +Extracts daily sales from Stripe, converts currency, loads to `fct_sales`. + +## Dependencies +- Upstream: Stripe API (Sales), Exchange Rate API +- Downstream: Power BI Sales Executive Dashboard + +## Common Failures +1. **API rate limit (Stripe)** — *Symptom:* HTTP 429. *Fix:* auto-retries; if it fully fails, + clear the task and re-trigger after 15 min. +2. **Missing exchange rates** — *Symptom:* `not_null` test fails. *Fix:* check the rate API payload. + +## Escalation +- Primary: @DataEngineer · Secondary: @DataLead +``` diff --git a/.cursor/skills/documentation-templates/templates/powerbi-dashboard.md b/.cursor/skills/documentation-templates/templates/powerbi-dashboard.md new file mode 100644 index 0000000..cb61a14 --- /dev/null +++ b/.cursor/skills/documentation-templates/templates/powerbi-dashboard.md @@ -0,0 +1,41 @@ +# Template: Power BI Dashboard Catalog + +Generate a `DASHBOARD_DOC.md` for a PBIP project by parsing its files — a living catalog of the +model and report. Driven by the `/document-dashboard` workflow. + +## Parsing +- **TMDL** (`tables/*.tmdl`, `relationships.tmdl`): indented text — extract names, data types, + `description`, DAX `expressions`, and relationships via block parsing (regex / line logic). +- **PBIR** (`pages/*/visuals/*/visual.json`): plain JSON — iterate `projections` for fields/values. +- Objects without a `description` show `[no description]`; custom visuals without a title are + identified by GUID. Skip schema mismatches gracefully. + +## Output structure (`DASHBOARD_DOC.md`) + +**1. Overview** — title, generation date, and counts (tables, measures, pages, visuals). + +**2. Data dictionary** — per `tables/.tmdl`: +```markdown +## Table: +> + +| Column | Type | Hidden | Description | +|--------|------|--------|-------------| +| Id | int64 | Yes | Primary key | +``` + +**3. Measure inventory** — per table, with the DAX in a fenced ```dax block: +```markdown +### [Measure] +**Folder:** | **Format:** +**Description:** +``` + +**4. Visual → measure/column map** — per page, from `visual.json` `projections`: +```markdown +## Page: +### Visual: (<visualType>) +- **X:** Table[Column] - **Y:** [Measure] +``` + +**5. Relationships** — from `relationships.tmdl`: a table of From, To, Cardinality, Direction. diff --git a/.cursor/skills/generating-pptx/SKILL.md b/.cursor/skills/generating-pptx/SKILL.md new file mode 100644 index 0000000..03e205b --- /dev/null +++ b/.cursor/skills/generating-pptx/SKILL.md @@ -0,0 +1,45 @@ +--- +name: generating-pptx +description: Use when the user needs an EDITABLE PowerPoint (.pptx) deliverable — "make a PowerPoint", "export to PPTX", "editable slides", "send me a .pptx". Generates native, fully editable PowerPoint (real text boxes, shapes, native charts with embedded data, tables) via python-pptx — not images. Brand colors map from DESIGN.md. For HTML decks/sites/PDF use building-html-presentations instead. +--- + +# Skill: generating-pptx + +Produce a **native, editable** PowerPoint with `python-pptx`. Every element is a real PowerPoint +object — editable text, shapes, **native charts** (backed by embedded data you can edit in +PowerPoint), and tables. This is the right path when the deliverable must be `.pptx` and editable; +for HTML decks, interactive sites, or designed PDFs use `building-html-presentations`. + +> Why not HTML→PPTX? Exporting an HTML deck to PPTX (e.g. Marp `--pptx`) embeds each slide as a +> flat **image** — not editable. `python-pptx` builds the real OOXML, so charts and text stay live. + +## Setup +```bash +pip install python-pptx +``` + +## Build it +`scripts/build_pptx.py` is a working generator — a brand-themed sales deck with a title slide, +agenda, KPI slide, **native charts** (clustered column, line, pie), a table slide, and a closing. +Run it, then adapt: +```bash +python .claude/skills/generating-pptx/scripts/build_pptx.py --out deck.pptx +``` + +Adapt by editing the `DATA`, `BRAND`, and the `build()` slide calls — not the helpers. + +## What python-pptx gives you (the palette) +- **Native charts** (editable, with embedded Excel data): `COLUMN_CLUSTERED`, `COLUMN_STACKED`, + `BAR_CLUSTERED`, `LINE`, `LINE_MARKERS`, `PIE`, `DOUGHNUT`, `XY_SCATTER`, `AREA`, `RADAR`. Use + `CategoryChartData` for category charts and `XyChartData` for scatter. +- **Tables** via `slide.shapes.add_table(rows, cols, ...)` — style cells, headers, number formats. +- **Text** via placeholders or `add_textbox` → `text_frame` → paragraphs/runs (font, size, color, bold). +- **Shapes / KPI cards** via `add_shape(MSO_SHAPE.ROUNDED_RECTANGLE, ...)` with fill + text. +- **Brand**: set `RGBColor` from your `DESIGN.md` tokens; reuse one accent + semantic colors. + +## Rules +- **Map brand from DESIGN.md** — colors, and font name if defined. Keep one accent. +- **Native charts over images** — let the user edit data in PowerPoint. +- **Never invent data** — use provided numbers; mark placeholders. Add a source line per data slide. +- **16:9** by default (`Inches(13.333) × Inches(7.5)`). +- Action-title style: the slide title states the insight, not just the topic. diff --git a/.cursor/skills/generating-pptx/scripts/build_pptx.py b/.cursor/skills/generating-pptx/scripts/build_pptx.py new file mode 100644 index 0000000..9a5ab7c --- /dev/null +++ b/.cursor/skills/generating-pptx/scripts/build_pptx.py @@ -0,0 +1,288 @@ +#!/usr/bin/env python3 +""" +build_pptx.py — generate a NATIVE, EDITABLE PowerPoint with python-pptx. + +Every element is a real PowerPoint object: editable text, rounded-rectangle KPI cards, +native charts (clustered column, line, doughnut) backed by embedded data, and a table. +This is a working starter — adapt DATA / BRAND / the build() slide calls. + + pip install python-pptx + python build_pptx.py --out deck.pptx + +Demo content is a generic SALES review; replace with real, sourced numbers — never invent data. +""" +from __future__ import annotations + +import argparse + +from pptx import Presentation +from pptx.util import Inches, Pt, Emu +from pptx.dml.color import RGBColor +from pptx.enum.text import PP_ALIGN, MSO_ANCHOR +from pptx.enum.shapes import MSO_SHAPE +from pptx.chart.data import CategoryChartData +from pptx.enum.chart import XL_CHART_TYPE, XL_LEGEND_POSITION, XL_TICK_MARK + +# ── BRAND — map these from your DESIGN.md ────────────────────────────────── +BRAND = { + "ink": "1B2733", + "accent": "14517B", + "accent2":"0E7C66", + "muted": "6A7787", + "soft": "F3F6F9", + "pos": "1E7A4D", + "neg": "B23A2E", + "warn": "9A6B12", + "bg": "FFFFFF", + "font": "Inter", +} +def C(key: str) -> RGBColor: + return RGBColor.from_string(BRAND[key]) + +# ── DATA — replace with your own ─────────────────────────────────────────── +DATA = { + "title": "Revenue Performance", + "subtitle": "An exploratory analysis of FY2026 Q1 bookings", + "date": "June 2026 · Confidential", + "kpis": [("$4.2M", "Bookings"), ("+12%", "vs. plan"), ("28%", "Win rate"), ("$9.1M", "Pipeline")], + "quarters": ["Q2/25", "Q3/25", "Q4/25", "Q1/26"], + "trend": [3.1, 3.5, 3.8, 4.2], + "regions": ["North America", "EMEA", "LATAM", "APAC"], + "bookings": [1.8, 1.1, 0.7, 0.6], + "segments": ["Enterprise", "Mid-market", "SMB"], + "attain_plan": [122, 99, 73], + "attain_actual": [118, 104, 88], + "table": [ + ["Region", "Bookings", "Attainment", "Deals", "Win rate"], + ["North America", "$1.8M", "118%", "402", "31%"], + ["EMEA", "$1.1M", "104%", "318", "29%"], + ["LATAM", "$0.7M", "96%", "266", "26%"], + ["APAC", "$0.6M", "88%", "254", "22%"], + ["Total", "$4.2M", "112%", "1,240", "28%"], + ], +} + +SW, SH = Inches(13.333), Inches(7.5) # 16:9 + + +# ── Helpers ──────────────────────────────────────────────────────────────── +def blank(prs): + return prs.slides.add_slide(prs.slide_layouts[6]) + + +def textbox(slide, x, y, w, h, text, size=18, color="ink", bold=False, + align=PP_ALIGN.LEFT, font=None): + box = slide.shapes.add_textbox(x, y, w, h) + tf = box.text_frame + tf.word_wrap = True + p = tf.paragraphs[0] + p.alignment = align + run = p.add_run() + run.text = text + run.font.size = Pt(size) + run.font.bold = bold + run.font.name = font or BRAND["font"] + run.font.color.rgb = C(color) + return box + + +def accent_bar(slide, x, y, w, h, color="accent"): + bar = slide.shapes.add_shape(MSO_SHAPE.RECTANGLE, x, y, w, h) + bar.fill.solid(); bar.fill.fore_color.rgb = C(color) + bar.line.fill.background() + return bar + + +def kpi_card(slide, x, y, w, h, value, label): + card = slide.shapes.add_shape(MSO_SHAPE.ROUNDED_RECTANGLE, x, y, w, h) + card.fill.solid(); card.fill.fore_color.rgb = C("soft") + card.line.color.rgb = C("soft"); card.shadow.inherit = False + tf = card.text_frame; tf.word_wrap = True + tf.margin_left = Inches(0.18); tf.margin_top = Inches(0.16) + p = tf.paragraphs[0] + r = p.add_run(); r.text = value + r.font.size = Pt(30); r.font.bold = True; r.font.color.rgb = C("accent"); r.font.name = BRAND["font"] + p2 = tf.add_paragraph() + r2 = p2.add_run(); r2.text = label.upper() + r2.font.size = Pt(10); r2.font.color.rgb = C("muted"); r2.font.name = BRAND["font"] + return card + + +def header(slide, kicker, title): + accent_bar(slide, Inches(0.6), Inches(0.55), Inches(0.5), Inches(0.09)) + textbox(slide, Inches(0.6), Inches(0.62), Inches(11), Inches(0.4), + kicker.upper(), size=11, color="accent", bold=True) + textbox(slide, Inches(0.6), Inches(0.95), Inches(12.1), Inches(1.0), + title, size=24, color="ink", bold=True) + + +def style_chart(chart, colors, legend=True): + chart.has_title = False + if legend: + chart.has_legend = True + chart.legend.position = XL_LEGEND_POSITION.BOTTOM + chart.legend.include_in_layout = False + chart.legend.font.size = Pt(10); chart.legend.font.color.rgb = C("muted") + else: + chart.has_legend = False + for i, plot in enumerate(chart.plots): + plot.has_data_labels = False + for j, series in enumerate(plot.series): + try: + series.format.fill.solid() + series.format.fill.fore_color.rgb = colors[j % len(colors)] + except Exception: + pass + + +def source_line(slide, text): + textbox(slide, Inches(0.6), Inches(6.95), Inches(12), Inches(0.4), + text, size=9, color="muted") + + +# ── Slides ─────────────────────────────────────────────────────────────── +def slide_title(prs): + s = blank(prs) + accent_bar(s, Inches(0.6), Inches(2.2), Inches(2.0), Inches(0.12)) + textbox(s, Inches(0.6), Inches(2.5), Inches(11), Inches(1.3), DATA["title"], size=48, bold=True) + textbox(s, Inches(0.62), Inches(3.7), Inches(10), Inches(0.8), DATA["subtitle"], size=20, color="muted") + textbox(s, Inches(0.62), Inches(6.4), Inches(10), Inches(0.5), DATA["date"], size=12, color="muted") + + +def slide_agenda(prs): + s = blank(prs) + header(s, "Agenda", "What this report covers") + items = ["Executive summary & KPIs", "Bookings trend", "Concentration by region & segment", + "Region-level detail", "Recommendations"] + for i, it in enumerate(items): + y = Inches(2.0 + i * 0.85) + textbox(s, Inches(0.7), y, Inches(0.8), Inches(0.7), f"{i+1:02d}", size=26, color="accent", bold=True) + textbox(s, Inches(1.7), y + Inches(0.08), Inches(10), Inches(0.6), it, size=18) + + +def slide_kpis(prs): + s = blank(prs) + header(s, "Executive summary", "Q1 bookings beat plan by 12%, led by enterprise") + n = len(DATA["kpis"]); gap = Inches(0.3); margin = Inches(0.6) + total_w = SW - margin * 2 - gap * (n - 1) + w = Emu(int(total_w / n)); h = Inches(1.6) + for i, (val, lbl) in enumerate(DATA["kpis"]): + x = margin + Emu(int((w + gap) * i)) + kpi_card(s, x, Inches(2.2), w, h, val, lbl) + textbox(s, Inches(0.6), Inches(4.3), Inches(12), Inches(1.5), + "Upside is real but narrow: two regions and the enterprise segment carry most of it, " + "while SMB attainment (73%) and discounting are early warning signs for H2.", + size=15, color="muted") + source_line(s, "Source: company data (gold.fct_bookings) — figures illustrative") + + +def slide_trend(prs): + s = blank(prs) + header(s, "Trend", "Bookings grew every quarter, up 35% since Q2") + cd = CategoryChartData() + cd.categories = DATA["quarters"] + cd.add_series("Bookings ($M)", DATA["trend"]) + gf = s.shapes.add_chart(XL_CHART_TYPE.LINE_MARKERS, Inches(0.8), Inches(2.0), + Inches(11.7), Inches(4.6), cd) + style_chart(gf.chart, [C("accent")], legend=False) + source_line(s, "Exhibit 1 · Source: company data; n = 1,240 closed-won deals") + + +def slide_region_segment(prs): + s = blank(prs) + header(s, "Concentration", "Two regions drive ~70%; enterprise leads, SMB lags") + # Exhibit 2 — clustered column: bookings by region + cd = CategoryChartData(); cd.categories = DATA["regions"] + cd.add_series("Bookings ($M)", DATA["bookings"]) + gf = s.shapes.add_chart(XL_CHART_TYPE.COLUMN_CLUSTERED, Inches(0.7), Inches(2.0), + Inches(6.0), Inches(4.4), cd) + style_chart(gf.chart, [C("accent")], legend=False) + textbox(s, Inches(0.8), Inches(1.75), Inches(5), Inches(0.3), "EXHIBIT 2 · Bookings by region", + size=10, color="muted", bold=True) + # Exhibit 3 — clustered column: plan vs actual attainment by segment + cd2 = CategoryChartData(); cd2.categories = DATA["segments"] + cd2.add_series("Plan", DATA["attain_plan"]) + cd2.add_series("Actual", DATA["attain_actual"]) + gf2 = s.shapes.add_chart(XL_CHART_TYPE.COLUMN_CLUSTERED, Inches(7.0), Inches(2.0), + Inches(5.6), Inches(4.4), cd2) + style_chart(gf2.chart, [C("muted"), C("accent2")], legend=True) + textbox(s, Inches(7.1), Inches(1.75), Inches(5), Inches(0.3), "EXHIBIT 3 · Attainment by segment (%)", + size=10, color="muted", bold=True) + source_line(s, "Source: company data; target set at FY2026 plan") + + +def slide_pie(prs): + s = blank(prs) + header(s, "Mix", "North America and EMEA account for most bookings") + cd = CategoryChartData(); cd.categories = DATA["regions"] + cd.add_series("Share", DATA["bookings"]) + gf = s.shapes.add_chart(XL_CHART_TYPE.DOUGHNUT, Inches(3.3), Inches(2.0), + Inches(6.7), Inches(4.6), cd) + chart = gf.chart + chart.has_title = False + chart.has_legend = True; chart.legend.position = XL_LEGEND_POSITION.RIGHT + chart.legend.include_in_layout = False; chart.legend.font.size = Pt(11) + points = chart.plots[0].series[0].points + palette = [C("accent"), C("accent2"), C("warn"), C("muted")] + for i, pt in enumerate(points): + pt.format.fill.solid(); pt.format.fill.fore_color.rgb = palette[i % len(palette)] + source_line(s, "Exhibit 4 · Source: company data; shares of $4.2M total") + + +def slide_table(prs): + s = blank(prs) + header(s, "Appendix", "Region-level detail") + rows, cols = len(DATA["table"]), len(DATA["table"][0]) + tbl = s.shapes.add_table(rows, cols, Inches(0.7), Inches(2.0), + Inches(11.9), Inches(0.5 * rows)).table + for r in range(rows): + for c in range(cols): + cell = tbl.cell(r, c) + cell.text = DATA["table"][r][c] + para = cell.text_frame.paragraphs[0] + para.alignment = PP_ALIGN.LEFT if c == 0 else PP_ALIGN.RIGHT + run = para.runs[0] + run.font.size = Pt(12); run.font.name = BRAND["font"] + is_edge = (r == 0 or r == rows - 1) + run.font.bold = is_edge + run.font.color.rgb = C("bg") if r == 0 else C("ink") + cell.fill.solid() + cell.fill.fore_color.rgb = C("accent") if r == 0 else (C("soft") if r == rows - 1 else C("bg")) + source_line(s, "Exhibit 5 · Source: company data — replace with real, sourced values") + + +def slide_recs(prs): + s = blank(prs) + header(s, "Recommendations", "Four moves to protect the upside") + recs = [ + ("Scale the enterprise motion", "Reallocate capacity to the segment that beat plan."), + ("Broaden mid-market pipeline", "Reduce concentration beyond the top two regions."), + ("Fix the SMB funnel", "Tighten qualification and discount guardrails."), + ("Re-forecast H2 with scenarios", "Model downside on SMB and discounting; set checkpoints."), + ] + for i, (h, p) in enumerate(recs): + y = Inches(2.1 + i * 1.1) + textbox(s, Inches(0.7), y, Inches(0.8), Inches(0.7), f"{i+1}", size=28, color="accent", bold=True) + textbox(s, Inches(1.6), y, Inches(11), Inches(0.5), h, size=17, bold=True) + textbox(s, Inches(1.6), y + Inches(0.45), Inches(11), Inches(0.5), p, size=13, color="muted") + + +def build(out: str) -> None: + prs = Presentation() + prs.slide_width, prs.slide_height = SW, SH + slide_title(prs) + slide_agenda(prs) + slide_kpis(prs) + slide_trend(prs) + slide_region_segment(prs) + slide_pie(prs) + slide_table(prs) + slide_recs(prs) + prs.save(out) + print(f"Wrote {out} · {len(prs.slides._sldIdLst)} slides") + + +if __name__ == "__main__": + ap = argparse.ArgumentParser() + ap.add_argument("--out", default="deck.pptx") + build(ap.parse_args().out) diff --git a/.cursor/skills/pbi-quality-rules/SKILL.md b/.cursor/skills/pbi-quality-rules/SKILL.md new file mode 100644 index 0000000..775db7b --- /dev/null +++ b/.cursor/skills/pbi-quality-rules/SKILL.md @@ -0,0 +1,44 @@ +--- +name: pbi-quality-rules +description: Validate a Power BI semantic model with the Best Practice Analyzer (BPA) — the real engine via the free Tabular Editor 2 CLI, or a Python fallback over TMDL when TE2 is not installed — using the shareable bpa-rules.json. Use when asked to lint, audit, check or validate a model, after significant model changes, and before publishing. +--- + +# Skill: pbi-quality-rules + +Two scripts in this skill's `scripts/` folder (Python 3.9+, standard library only): + +| Script | What it does | +|---|---| +| `run_bpa.py <model>` | Runs the real BPA through **Tabular Editor 2** (`TabularEditor.exe <definition-folder> -A bpa-rules.json -V`). If TE2 is not found, it runs the fallback and says so. | +| `bpa_fallback.py <model>` | The same rules re-implemented in Python over the TMDL files (or `model.bim`). Approximate; rules it cannot evaluate are listed as `SKIPPED`, never as passed. | + +`<model>` = the `.pbip`, the project folder, `<Name>.SemanticModel/`, its `definition/` folder, +or a `model.bim`. Options: `--format json`, `--rules <file>`, and for `run_bpa.py` also +`--te <path>` and `--no-fallback`. TE2 is looked up via `--te`, `$TABULAR_EDITOR_PATH`, `PATH`, +then the default Windows install folders. + +Exit codes: `0` no error-severity violations · `1` errors found · `2` bad path / TE2 failed. + +## Run it +```bash +python3 <skill>/scripts/run_bpa.py path/to/Sales.pbip +``` +1. **Disk is the input.** With Desktop open and MCP edits pending, ask the user to save (Ctrl+S) + first — the BPA reads the TMDL Desktop wrote, not the live model. +2. Present **errors first**, then warnings, grouped by category; omit info unless asked. Say + which engine ran; for the fallback, recommend installing Tabular Editor 2 for exact results. +3. For each `error`, offer a fix (through the Modeling MCP when Desktop is open — see + `pbi-semantic-layer-tmdl`). Warnings: suggest, don't force. + +## Rule set — `bpa-rules.json` +Tabular Editor's native format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in +Dynamic LINQ over the TOM). Severity `3` error · `2` warning · `1` info. It loads directly into +TE2's BPA window too. + +Adding a rule: add it to `bpa-rules.json` (TE2 picks it up at once), then add a predicate with the +same `ID` to `CHECKS` in `scripts/bpa_fallback.py` so the fallback covers it; until then the +fallback reports it as `SKIPPED`. + +## In CI +Run `run_bpa.py --no-fallback --format json` on a Windows runner with TE2 (fails on exit 1), or +`bpa_fallback.py --format json` anywhere for a lighter gate. diff --git a/.cursor/skills/pbi-quality-rules/bpa-rules.json b/.cursor/skills/pbi-quality-rules/bpa-rules.json new file mode 100644 index 0000000..f1652f0 --- /dev/null +++ b/.cursor/skills/pbi-quality-rules/bpa-rules.json @@ -0,0 +1,92 @@ +[ + { + "ID": "NAMING_MEASURE_PASCAL", + "Name": "Measures should start with an uppercase letter", + "Category": "Naming Conventions", + "Description": "Measure names must begin with a capital letter for a consistent field list.", + "Severity": 2, + "Scope": "Measure", + "Expression": "RegEx.IsMatch(Name, \"^[a-z]\")" + }, + { + "ID": "NAMING_MEASURE_DISPLAYFOLDER", + "Name": "Measures should live in a DisplayFolder", + "Category": "Naming Conventions", + "Description": "Organize measures into DisplayFolders so the field list stays navigable.", + "Severity": 2, + "Scope": "Measure", + "Expression": "string.IsNullOrEmpty(DisplayFolder)" + }, + { + "ID": "DAX_EMPTY_EXPRESSION", + "Name": "Measure has an empty expression", + "Category": "DAX Expressions", + "Description": "The measure is blank or returns BLANK() directly with no logic.", + "Severity": 3, + "Scope": "Measure", + "Expression": "string.IsNullOrWhitespace(Expression) or Expression.Replace(\" \", \"\").ToUpper() == \"BLANK()\"" + }, + { + "ID": "DAX_IFERROR_USAGE", + "Name": "Avoid IFERROR (masks underlying errors)", + "Category": "DAX Expressions", + "Description": "IFERROR hides real errors and is slow; prefer DIVIDE() or explicit handling.", + "Severity": 2, + "Scope": "Measure", + "Expression": "Expression.ToUpper().Contains(\"IFERROR\")" + }, + { + "ID": "MODEL_BIDIRECTIONAL", + "Name": "Bidirectional relationship", + "Category": "Model", + "Description": "Both-directions cross-filtering hurts the engine and can cause ambiguity; justify each one.", + "Severity": 2, + "Scope": "Relationship", + "Expression": "CrossFilteringBehavior == \"BothDirections\"" + }, + { + "ID": "MODEL_ISLAND_TABLE", + "Name": "Island table (no relationships)", + "Category": "Model", + "Description": "An imported table with no relationships is usually a modeling mistake (unless it is a disconnected parameter table).", + "Severity": 2, + "Scope": "Table", + "Expression": "UsedInRelationships.Count() == 0 and not Name.StartsWith(\"LocalDateTable_\") and not Name.StartsWith(\"DateTableTemplate_\")" + }, + { + "ID": "PERF_AUTO_DATETIME", + "Name": "Auto date/time tables present", + "Category": "Performance", + "Description": "Auto date/time creates hidden LocalDateTable_/DateTableTemplate_ tables that bloat the model. Disable it and use a real Date table.", + "Severity": 2, + "Scope": "Table", + "Expression": "Name.StartsWith(\"LocalDateTable_\") or Name.StartsWith(\"DateTableTemplate_\")" + }, + { + "ID": "DOC_TABLE_DESCRIPTION", + "Name": "Table is missing a description", + "Category": "Maintainability", + "Description": "Tables should carry a description for documentation and the data catalog.", + "Severity": 2, + "Scope": "Table", + "Expression": "string.IsNullOrWhitespace(Description)" + }, + { + "ID": "DOC_MEASURE_DESCRIPTION", + "Name": "Measure is missing a description", + "Category": "Maintainability", + "Description": "Measures should carry a description for documentation and the data catalog.", + "Severity": 2, + "Scope": "Measure", + "Expression": "string.IsNullOrWhitespace(Description)" + }, + { + "ID": "DOC_KEY_COLUMN_HIDDEN", + "Name": "Key columns should be hidden", + "Category": "Maintainability", + "Description": "Surrogate/foreign key columns ending in ID carry no meaning for end users and should be hidden.", + "Severity": 1, + "Scope": "Column", + "Expression": "RegEx.IsMatch(Name, \"ID$\") and not IsHidden" + } +] diff --git a/.cursor/skills/pbi-quality-rules/scripts/bpa_fallback.py b/.cursor/skills/pbi-quality-rules/scripts/bpa_fallback.py new file mode 100755 index 0000000..36f575b --- /dev/null +++ b/.cursor/skills/pbi-quality-rules/scripts/bpa_fallback.py @@ -0,0 +1,522 @@ +#!/usr/bin/env python3 +"""Best Practice Analyzer fallback: the rules of bpa-rules.json, evaluated in Python. + +Use this only when Tabular Editor 2 is not available (run_bpa.py calls it automatically). +It reads the semantic model straight from disk — TMDL (PBIP `definition/` folder) or a +`model.bim` — and re-implements each shipped rule by its ID. It is an approximation of the +real BPA engine: rules without a Python implementation are reported as skipped, never as passed. + +Usage: + python3 bpa_fallback.py <model> [--rules bpa-rules.json] [--format text|json] + +<model> may be a .pbip file, a project folder, a <Name>.SemanticModel folder, its +definition/ folder, or a model.bim file. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations, 2 = usage error. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from dataclasses import dataclass, field +from pathlib import Path +from typing import Callable, Optional + +DEFAULT_RULES = Path(__file__).resolve().parent.parent / "bpa-rules.json" +SEVERITY_NAMES = {1: "info", 2: "warning", 3: "error"} +AUTO_DATE_PREFIXES = ("LocalDateTable_", "DateTableTemplate_") + + +class UsageError(Exception): + """Bad input path or file; maps to exit code 2.""" + + +# --------------------------------------------------------------------------- +# Model +# --------------------------------------------------------------------------- +@dataclass +class Measure: + table: str + name: str + expression: str = "" + description: str = "" + display_folder: str = "" + is_hidden: bool = False + + +@dataclass +class Column: + table: str + name: str + description: str = "" + is_hidden: bool = False + + +@dataclass +class Table: + name: str + description: str = "" + is_hidden: bool = False + measures: list = field(default_factory=list) + columns: list = field(default_factory=list) + + +@dataclass +class Relationship: + name: str + from_table: str = "" + from_column: str = "" + to_table: str = "" + to_column: str = "" + cross_filtering: str = "oneDirection" + + +@dataclass +class Model: + source: Path + kind: str # "tmdl" | "bim" + tables: list = field(default_factory=list) + relationships: list = field(default_factory=list) + + @property + def measures(self): + return [m for t in self.tables for m in t.measures] + + @property + def columns(self): + return [c for t in self.tables for c in t.columns] + + +# --------------------------------------------------------------------------- +# Locating the model on disk +# --------------------------------------------------------------------------- +def _is_tmdl_folder(path: Path) -> bool: + return path.is_dir() and any((path / n).is_file() for n in ("model.tmdl", "database.tmdl")) + + +def _from_semantic_model_dir(path: Path) -> Optional[tuple]: + if _is_tmdl_folder(path / "definition"): + return "tmdl", path / "definition" + if (path / "model.bim").is_file(): + return "bim", path / "model.bim" + return None + + +def _semantic_model_from_pbip(pbip: Path) -> Optional[Path]: + sibling = pbip.with_name(pbip.stem + ".SemanticModel") + if sibling.is_dir(): + return sibling + try: + data = json.loads(pbip.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + return None + for artifact in data.get("artifacts", []): + report = artifact.get("report", {}).get("path") + if not report: + continue + pbir = pbip.parent / report / "definition.pbir" + try: + ref = json.loads(pbir.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + continue + by_path = ref.get("datasetReference", {}).get("byPath", {}).get("path") + if by_path: + candidate = (pbir.parent / by_path).resolve() + if candidate.is_dir(): + return candidate + return None + + +def resolve_model(path: Path) -> tuple: + """Return (kind, path) where kind is 'tmdl' (definition folder) or 'bim' (model.bim).""" + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() == ".bim": + return "bim", path + if path.suffix.lower() == ".pbip": + sm = _semantic_model_from_pbip(path) + found = _from_semantic_model_dir(sm) if sm else None + if found: + return found + raise UsageError(f"no semantic model found for {path.name} (thin report?)") + if path.name.lower() == "definition.pbism": + return resolve_model(path.parent) + raise UsageError(f"unsupported file: {path.name} (use .pbip, model.bim or a folder)") + if _is_tmdl_folder(path): + return "tmdl", path + found = _from_semantic_model_dir(path) + if found: + return found + models = sorted(p for p in path.glob("*.SemanticModel") if p.is_dir()) + if len(models) == 1: + found = _from_semantic_model_dir(models[0]) + if found: + return found + if len(models) > 1: + names = ", ".join(p.name for p in models) + raise UsageError(f"several semantic models in {path}: {names}; pass one explicitly") + raise UsageError(f"no TMDL definition or model.bim found under {path}") + + +# --------------------------------------------------------------------------- +# TMDL parser (the subset the rules need) +# --------------------------------------------------------------------------- +_PROPERTY = re.compile(r"^([A-Za-z][A-Za-z0-9_]*)\s*(?::\s*(.*))?$") + + +def _depth(line: str) -> int: + expanded = line[: len(line) - len(line.lstrip())].replace("\t", " ") + return len(expanded) // 4 + + +def _read_name(text: str) -> tuple: + """Read one TMDL object name from the start of `text`. Returns (name, rest).""" + text = text.lstrip() + if text.startswith("'"): + out, i = [], 1 + while i < len(text): + if text[i] == "'": + if i + 1 < len(text) and text[i + 1] == "'": + out.append("'") + i += 2 + continue + return "".join(out), text[i + 1 :] + out.append(text[i]) + i += 1 + return "".join(out), "" + match = re.match(r"[^\s=.]+", text) + if not match: + return "", text + return match.group(0), text[match.end() :] + + +def _split_column_ref(ref: str) -> tuple: + """`Sales.DateKey` / `'Sales Table'.'Date Key'` -> (table, column).""" + table, rest = _read_name(ref) + rest = rest.lstrip() + if rest.startswith("."): + column, _ = _read_name(rest[1:]) + return table, column + return "", table + + +def _unquote_value(value: str) -> str: + value = value.strip() + if len(value) >= 2 and value[0] == value[-1] == '"': + return value[1:-1].replace('""', '"') + return value + + +def parse_tmdl_folder(folder: Path) -> Model: + model = Model(source=folder, kind="tmdl") + tables: dict = {} + for file in sorted(folder.rglob("*.tmdl")): + _parse_tmdl_file(file.read_text(encoding="utf-8-sig"), model, tables) + model.tables = list(tables.values()) + return model + + +def _parse_tmdl_file(text: str, model: Model, tables: dict) -> None: + table: Optional[Table] = None + obj = None # Measure | Column | Relationship | Table (for table-level properties) + obj_depth = 0 + other_depth: Optional[int] = None # inside an object we do not model (partition, ...) + expr_lines: Optional[list] = None + expr_fenced = False + pending_doc: list = [] + + def finish_expression(): + nonlocal expr_lines + if expr_lines is not None and isinstance(obj, Measure): + obj.expression = "\n".join(expr_lines).strip() + expr_lines = None + + for raw in text.splitlines(): + stripped = raw.strip() + depth = _depth(raw) + + if expr_lines is not None: + if expr_fenced: + if stripped.startswith("```"): + finish_expression() + else: + expr_lines.append(stripped) + continue + if not stripped or depth > obj_depth + 1: + expr_lines.append(stripped) + continue + finish_expression() + + if not stripped: + continue + if stripped.startswith("///"): + pending_doc.append(stripped[3:].strip()) + continue + if stripped.startswith("//"): + continue + + if other_depth is not None: + if depth > other_depth: + continue + other_depth = None + + doc = " ".join(pending_doc).strip() + pending_doc = [] + keyword, _, rest = stripped.partition(" ") + + if depth == 0: + obj = None + table = None + if keyword == "table": + name, _ = _read_name(rest) + table = tables.setdefault(name, Table(name=name)) + if doc: + table.description = doc + obj, obj_depth = table, 0 + elif keyword == "relationship": + name, _ = _read_name(rest) + obj, obj_depth = Relationship(name=name), 0 + model.relationships.append(obj) + else: + other_depth = 0 + continue + + if table is not None and depth == 1 and keyword in ("measure", "column"): + name, after = _read_name(rest) + after = after.strip() + if keyword == "measure": + obj = Measure(table=table.name, name=name, description=doc) + table.measures.append(obj) + else: + obj = Column(table=table.name, name=name, description=doc) + table.columns.append(obj) + obj_depth = 1 + if after.startswith("="): + inline = after[1:].strip() + if inline.startswith("```"): + expr_lines, expr_fenced = [], True + elif inline: + if isinstance(obj, Measure): + obj.expression = inline + else: + expr_lines, expr_fenced = [], False + continue + + if obj is not None and depth == obj_depth + 1: + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(obj, match.group(1), match.group(2)) + continue + + if table is not None and depth == 1: + # Table-level property after members (e.g. `isHidden`, `lineageTag: …`). + obj, obj_depth = table, 0 + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(table, match.group(1), match.group(2)) + continue + + # Any other nested object (partition, hierarchy, annotation, ...): skip its body. + other_depth = depth + + finish_expression() + + +def _set_property(obj, key: str, value: Optional[str]) -> None: + flag = value is None or value.strip().lower() == "true" + if key == "isHidden" and hasattr(obj, "is_hidden"): + obj.is_hidden = flag + elif key == "description" and hasattr(obj, "description") and value is not None: + obj.description = _unquote_value(value) + elif key == "displayFolder" and isinstance(obj, Measure) and value is not None: + obj.display_folder = _unquote_value(value) + elif isinstance(obj, Relationship) and value is not None: + if key == "fromColumn": + obj.from_table, obj.from_column = _split_column_ref(value) + elif key == "toColumn": + obj.to_table, obj.to_column = _split_column_ref(value) + elif key == "crossFilteringBehavior": + obj.cross_filtering = value.strip() + + +# --------------------------------------------------------------------------- +# model.bim (TMSL JSON) +# --------------------------------------------------------------------------- +def _text(value) -> str: + if isinstance(value, list): + return "\n".join(str(v) for v in value) + return str(value or "") + + +def parse_bim(path: Path) -> Model: + try: + data = json.loads(path.read_text(encoding="utf-8-sig")) + except ValueError as exc: + raise UsageError(f"{path} is not valid JSON: {exc}") from exc + raw_model = data.get("model", data) + model = Model(source=path, kind="bim") + for t in raw_model.get("tables", []): + table = Table(name=t.get("name", ""), description=_text(t.get("description")), + is_hidden=bool(t.get("isHidden"))) + for m in t.get("measures", []): + table.measures.append(Measure( + table=table.name, name=m.get("name", ""), expression=_text(m.get("expression")), + description=_text(m.get("description")), display_folder=m.get("displayFolder", "") or "", + is_hidden=bool(m.get("isHidden")))) + for c in t.get("columns", []): + if c.get("type") == "rowNumber": + continue + table.columns.append(Column(table=table.name, name=c.get("name", ""), + description=_text(c.get("description")), + is_hidden=bool(c.get("isHidden")))) + model.tables.append(table) + for r in raw_model.get("relationships", []): + model.relationships.append(Relationship( + name=r.get("name", ""), from_table=r.get("fromTable", ""), from_column=r.get("fromColumn", ""), + to_table=r.get("toTable", ""), to_column=r.get("toColumn", ""), + cross_filtering=r.get("crossFilteringBehavior", "oneDirection"))) + return model + + +def load_model(path: Path) -> Model: + kind, resolved = resolve_model(path) + return parse_tmdl_folder(resolved) if kind == "tmdl" else parse_bim(resolved) + + +# --------------------------------------------------------------------------- +# Rules — one Python predicate per rule ID in bpa-rules.json +# --------------------------------------------------------------------------- +def _blank(text: str) -> bool: + return not (text or "").strip() + + +def _tables_in_relationships(model: Model) -> set: + return {t for r in model.relationships for t in (r.from_table, r.to_table)} + + +# rule ID -> (scope, predicate(obj, model) -> True when the object VIOLATES the rule) +CHECKS: dict = { + "NAMING_MEASURE_PASCAL": ("Measure", lambda m, _: bool(re.match(r"^[a-z]", m.name))), + "NAMING_MEASURE_DISPLAYFOLDER": ("Measure", lambda m, _: not m.display_folder), + "DAX_EMPTY_EXPRESSION": ("Measure", lambda m, _: _blank(m.expression) + or m.expression.replace(" ", "").upper() == "BLANK()"), + "DAX_IFERROR_USAGE": ("Measure", lambda m, _: "IFERROR" in m.expression.upper()), + "MODEL_BIDIRECTIONAL": ("Relationship", lambda r, _: r.cross_filtering.lower() == "bothdirections"), + "MODEL_ISLAND_TABLE": ("Table", lambda t, model: t.name not in _tables_in_relationships(model) + and not t.name.startswith(AUTO_DATE_PREFIXES)), + "PERF_AUTO_DATETIME": ("Table", lambda t, _: t.name.startswith(AUTO_DATE_PREFIXES)), + "DOC_TABLE_DESCRIPTION": ("Table", lambda t, _: _blank(t.description)), + "DOC_MEASURE_DESCRIPTION": ("Measure", lambda m, _: _blank(m.description)), + "DOC_KEY_COLUMN_HIDDEN": ("Column", lambda c, _: bool(re.search(r"ID$", c.name)) and not c.is_hidden), +} + + +def _quote(name: str) -> str: + return "'" + name.replace("'", "''") + "'" + + +def object_label(obj) -> str: + if isinstance(obj, Table): + return _quote(obj.name) + if isinstance(obj, (Measure, Column)): + return f"{_quote(obj.table)}[{obj.name}]" + if isinstance(obj, Relationship): + return f"{_quote(obj.from_table)}[{obj.from_column}] -> {_quote(obj.to_table)}[{obj.to_column}]" + return str(obj) + + +def _objects(model: Model, scope: str) -> list: + return { + "Measure": model.measures, + "Column": model.columns, + "Table": model.tables, + "Relationship": model.relationships, + }[scope] + + +def load_rules(path: Path) -> list: + try: + rules = json.loads(Path(path).read_text(encoding="utf-8-sig")) + except (OSError, ValueError) as exc: + raise UsageError(f"cannot read rules file {path}: {exc}") from exc + if not isinstance(rules, list): + raise UsageError(f"rules file {path} must contain a JSON array") + return rules + + +def analyze(model: Model, rules: list) -> dict: + violations, skipped = [], [] + for rule in rules: + check: Optional[tuple] = CHECKS.get(rule.get("ID")) + if check is None: + skipped.append(rule.get("ID") or rule.get("Name")) + continue + scope, predicate = check + predicate: Callable + severity = SEVERITY_NAMES.get(int(rule.get("Severity", 1)), "info") + for obj in _objects(model, scope): + if predicate(obj, model): + violations.append({ + "rule_id": rule["ID"], + "rule_name": rule.get("Name", rule["ID"]), + "category": rule.get("Category", ""), + "severity": severity, + "object_type": scope, + "object": object_label(obj), + }) + order = {"error": 0, "warning": 1, "info": 2} + violations.sort(key=lambda v: (order[v["severity"]], v["category"], v["rule_id"], v["object"])) + return { + "engine": "python-fallback", + "model": str(model.source), + "violations": violations, + "skipped_rules": skipped, + "summary": {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")}, + } + + +def format_text(result: dict) -> str: + lines = [] + if result["engine"] == "python-fallback": + lines.append("BPA engine: Python fallback (approximate). Install Tabular Editor 2 for the real BPA.") + else: + lines.append("BPA engine: Tabular Editor 2") + lines.append(f"Model: {result['model']}") + for v in result["violations"]: + lines.append(f"{v['severity'].upper():8} [{v['rule_id']}] {v['object_type']} {v['object']} — {v['rule_name']}") + if not result["violations"]: + lines.append("No objects in violation of the rules.") + for rule_id in result.get("skipped_rules", []): + lines.append(f"SKIPPED [{rule_id}] no implementation in this engine") + s = result["summary"] + lines.append(f"Summary: {s['error']} error(s), {s['warning']} warning(s), {s['info']} info") + return "\n".join(lines) + + +def emit(result: dict, fmt: str) -> int: + print(json.dumps(result, indent=2) if fmt == "json" else format_text(result)) + return 1 if result["summary"]["error"] else 0 + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="BPA rules evaluated in Python over TMDL/model.bim.") + parser.add_argument("model", type=Path, help=".pbip, project folder, .SemanticModel, definition/ or model.bim") + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES, help="BPA rules JSON (default: bpa-rules.json)") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + try: + result = analyze(load_model(args.model), load_rules(args.rules)) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + return emit(result, args.format) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.cursor/skills/pbi-quality-rules/scripts/run_bpa.py b/.cursor/skills/pbi-quality-rules/scripts/run_bpa.py new file mode 100755 index 0000000..31e1232 --- /dev/null +++ b/.cursor/skills/pbi-quality-rules/scripts/run_bpa.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""Run the Best Practice Analyzer with Tabular Editor 2; fall back to Python when it is absent. + +Usage: + python3 run_bpa.py <model> [--rules bpa-rules.json] [--te PATH] [--format text|json] [--no-fallback] + +<model>: .pbip, project folder, <Name>.SemanticModel, its definition/ folder, or model.bim. + +Tabular Editor 2 is located in this order: --te, $TABULAR_EDITOR_PATH, TabularEditor(.exe) on +PATH, then the default Windows install folders. It runs as + + TabularEditor.exe <definition-folder|model.bim> -A <rules> -V + +(-A: analyze with this rules file only, plus rules embedded in the model; -V: Azure DevOps +logging lines, which carry the severity). Without TE2 the script runs bpa_fallback.py and says so. + +Disk is the input: with Power BI Desktop open, save first (Ctrl+S) so the TMDL is current. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations (or rule errors), +2 = usage error / Tabular Editor could not run. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +sys.dont_write_bytecode = True +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import bpa_fallback # noqa: E402 + +DEFAULT_RULES = bpa_fallback.DEFAULT_RULES +TE_TIMEOUT_SECONDS = 600 + +# "##vso[task.logissue type=warning;]Measure 'Sales'[x] violates rule "Rule name"" +_VSO = re.compile(r"^##vso\[task\.logissue type=(?P<level>error|warning)[^\]]*\](?P<text>.*)$") +_VIOLATION = re.compile(r'^(?P<type>.+?) (?P<object>(?:\'|\[).*) violates rule "(?P<rule>.*)"$') +_VIOLATION_LOOSE = re.compile(r'^(?P<type>\S+) (?P<object>.+?) violates rule "(?P<rule>.*)"$') +_RULE_ERROR = re.compile(r"^Error on rule '(?P<rule>.*?)': (?P<error>.*)$") + + +def find_tabular_editor(explicit: Optional[str] = None, env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + for candidate in (explicit, env.get("TABULAR_EDITOR_PATH")): + if candidate: + return candidate if Path(candidate).is_file() else None + for name in ("TabularEditor.exe", "TabularEditor"): + found = shutil.which(name) + if found: + return found + for base in (env.get("ProgramFiles(x86)"), env.get("ProgramFiles"), env.get("LOCALAPPDATA")): + if base: + exe = Path(base) / "Tabular Editor" / "TabularEditor.exe" + if exe.is_file(): + return str(exe) + return None + + +def build_command(te_path: str, model_target: Path, rules: Path) -> list: + return [te_path, str(model_target), "-A", str(rules), "-V"] + + +def parse_te_output(stdout: str, rules: list, model_label: str) -> dict: + """Turn TE2 console output (-A ... -V) into the same result shape as the fallback.""" + by_name = {r.get("Name"): r for r in rules} + violations, rule_errors = [], [] + for raw in stdout.splitlines(): + line = raw.strip() + if not line or line.startswith("##vso[task.complete"): + continue + level = "info" + match = _VSO.match(line) + if match: + level, line = match.group("level"), match.group("text").strip() + rule_error = _RULE_ERROR.match(line) + if rule_error: + rule_errors.append({"rule_name": rule_error.group("rule"), "error": rule_error.group("error")}) + continue + found = _VIOLATION.match(line) or _VIOLATION_LOOSE.match(line) + if not found: + continue + rule = by_name.get(found.group("rule"), {}) + violations.append({ + "rule_id": rule.get("ID", ""), + "rule_name": found.group("rule"), + "category": rule.get("Category", ""), + "severity": level, + "object_type": found.group("type"), + "object": found.group("object"), + }) + summary = {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")} + return { + "engine": "tabular-editor-2", + "model": model_label, + "violations": violations, + "rule_errors": rule_errors, + "skipped_rules": [], + "summary": summary, + } + + +def run_tabular_editor(te_path: str, model_target: Path, rules_path: Path, rules: list) -> dict: + cmd = build_command(te_path, model_target, rules_path) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=TE_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise bpa_fallback.UsageError(f"Tabular Editor failed to run ({' '.join(cmd)}): {exc}") from exc + result = parse_te_output(proc.stdout, rules, str(model_target)) + loaded = "Running Best Practice Analyzer" in proc.stdout + if proc.returncode != 0 and not loaded: + detail = (proc.stderr or proc.stdout).strip()[-2000:] + raise bpa_fallback.UsageError(f"Tabular Editor exited {proc.returncode} before the BPA ran:\n{detail}") + return result + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Run the BPA via Tabular Editor 2 (Python fallback if absent).") + parser.add_argument("model", type=Path) + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES) + parser.add_argument("--te", help="path to TabularEditor.exe (overrides $TABULAR_EDITOR_PATH)") + parser.add_argument("--format", choices=("text", "json"), default="text") + parser.add_argument("--no-fallback", action="store_true", help="fail (exit 2) instead of using the Python fallback") + args = parser.parse_args(argv) + + try: + rules = bpa_fallback.load_rules(args.rules) + kind, target = bpa_fallback.resolve_model(args.model) + te_path = find_tabular_editor(args.te) + if te_path: + result = run_tabular_editor(te_path, target, args.rules.resolve(), rules) + elif args.no_fallback or args.te: + raise bpa_fallback.UsageError("Tabular Editor 2 not found (use --te or $TABULAR_EDITOR_PATH)") + else: + print("Tabular Editor 2 not found; using the Python fallback.", file=sys.stderr) + parsed = bpa_fallback.parse_tmdl_folder(target) if kind == "tmdl" else bpa_fallback.parse_bim(target) + result = bpa_fallback.analyze(parsed, rules) + except bpa_fallback.UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + if args.format == "json": + print(json.dumps(result, indent=2)) + else: + print(bpa_fallback.format_text(result)) + for err in result.get("rule_errors", []): + print(f"RULE-ERR [{err['rule_name']}] {err['error']}") + has_errors = result["summary"]["error"] > 0 or bool(result.get("rule_errors")) + return 1 if has_errors else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.cursor/skills/pbi-report-layer-pbir/SKILL.md b/.cursor/skills/pbi-report-layer-pbir/SKILL.md new file mode 100644 index 0000000..898d4d7 --- /dev/null +++ b/.cursor/skills/pbi-report-layer-pbir/SKILL.md @@ -0,0 +1,128 @@ +--- +name: pbi-report-layer-pbir +description: Edit the Power BI report layer as code — pages, visuals, filters, bookmarks and themes in the PBIR files of a PBIP project — with Power BI Desktop OPEN, using the edit → validate → save → status → reload → screenshot → review loop (Desktop Bridge via powerbi-report-author). Also covers the Desktop-closed path. Use for any request to add, move, restyle or fix a report page or visual. +--- + +# Skill: pbi-report-layer-pbir + +The report layer (`<Name>.Report/`) is **PBIR**: one JSON file per page and per visual, each +with a public JSON schema. Power BI Desktop can stay **open** while you edit it: the +[Desktop Bridge](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +lets a tool reload the files from disk and take screenshots of the rendered pages. + +Model objects (measures, columns, relationships) are NOT edited here — they go live through the +Modeling MCP (`pbi-semantic-layer-tmdl`). Create the measure first, then bind the visual to it. + +## Tools +- `powerbi-report-author` — Microsoft's report CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`, + Node 20+). Validates PBIR, describes visual roles/formatting, and drives Desktop through + `preview --host desktop`. It bundles the lower-level bridge CLI `powerbi-desktop` + (`status`, `reload`, `screenshot`); always go through `powerbi-report-author preview`, which + resolves the right Desktop window from the report folder. Details, errors and setup: + [references/desktop-bridge.md](references/desktop-bridge.md). +- `scripts/validate_pbir.py` (in this skill's folder) — runs `powerbi-report-author validate`, or + a structural check when the CLI is missing. +- `pbi-visual-reviewer` agent — independent screenshot review. + +## The loop (Desktop open) +`<folder>` = the project folder or the `.pbip` path; `<shots>` = an absolute folder OUTSIDE the +PBIP (e.g. `%TEMP%\pbir-review\<Name>`), so screenshots never land in git. + +1. **Read before writing.** `powerbi-report-author preview-pages <folder>` and + `preview-visuals <folder>` list pages and visuals with their file paths. For a new or changed + visual, `powerbi-report-author catalog describe <visualType>` gives the exact role names + (e.g. `cardVisual` uses `Data`, `clusteredColumnChart` uses `Category` + `Y`); + `formatting describe-object <visualType> <object>` gives property names. Do not guess them. +2. **Edit** the PBIR JSON (anatomy below). One logical batch at a time. +3. **Validate:** `python3 <skill>/scripts/validate_pbir.py <folder>`. Fix every error before + going on; read the warnings. +4. **Save gate.** Ask the user to save in Desktop (Ctrl+S) if they changed anything by hand, + then check: `powerbi-report-author preview <folder> --host desktop --status`. + Continue only when `data.details.hasUnsavedChanges` is exactly `false`. +5. **Reload:** `powerbi-report-author preview <folder> --host desktop --reload`. +6. **Screenshot every affected page:** + `powerbi-report-author preview <folder> --host desktop --screenshot "<shots>" --all-pages` + (or `--screenshot "<shots>\<page>.png" --page <pageId>`). A screenshot does not reload; always + reload first. +7. **Review:** hand the PNG paths, the user's request/spec and the `.Report` path — and nothing + about your reasoning — to the `pbi-visual-reviewer` agent. In tools without subagents + (Cursor, Antigravity), run its checklist in a fresh chat with only those inputs. +8. REJECTED → fix the listed files and repeat from step 3. APPROVED → report the result and let + the user save/commit. + +### Pitfalls (each one has lost real work) +- **Reload discards whatever Desktop has not saved.** That is why step 4 exists. If status says + `true`, `null`, or fails, stop and ask the user to save; never reload "just to see". +- **Reload covers the report only.** `--reload` does not re-read model files. Model changes go + through the Modeling MCP, which updates Desktop live. Use `--reload-with-model` only when TMDL + was edited on disk (Desktop-closed habits) — and it still does not refresh data. +- **An edited theme may not show after reload.** Desktop caches themes by file name. When you + change a custom theme, rename its JSON (e.g. `Brand-3f9a1c.json` → `Brand-7b2e04.json`) and + update both references in `report.json` (`themeCollection.customTheme.name` and the + `resourcePackages` item `name`/`path`), then validate and reload. +- A page added on disk is unknown to the open window until a reload; a screenshot of it fails + with `RENDER_FAILED`. +- Two Desktop windows with the same report → `HOST_UNAVAILABLE`. Ask the user to close one; + never close Desktop yourself. + +## Alternative: Desktop closed +Use it when there is no Bridge (Desktop older than the Bridge release, the setting +*File > Options > Security > Desktop Bridge* is off, or the machine is not Windows) and in CI. +1. Make sure Desktop is **closed** — an open Desktop overwrites the files on its next save. +2. Edit the PBIR, run `validate_pbir.py` (in CI: `--format json`, fail on exit code 1). +3. Ask the user to open the `.pbip` and check the pages; no screenshots, so ask for one or for + a description, and run the reviewer checklist on what you get. + +## `.Report` structure (PBIR) +``` +<Name>.Report/ + definition.pbir ← model binding (byPath ../<Name>.SemanticModel) + definition/ + version.json + report.json ← themeCollection, resourcePackages, settings + pages/ + pages.json ← pageOrder + activePageName + <pageId>/ + page.json ← name (= folder), displayName, width, height + visuals/<visualId>/visual.json ← one visual per file; name = folder + StaticResources/ + RegisteredResources/ ← custom themes and images you add + SharedResources/BaseThemes/ ← Microsoft base themes — never edit +``` +PBIR-Legacy (a single `report.json` at the `.Report` root) cannot be edited this way: ask the +user to save the report in PBIR format first. + +## Visual anatomy (`visual.json`) +```json +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/visualContainer/2.9.0/schema.json", + "name": "7b2d4f6a8c0e1a3b5d02", + "position": { "x": 40, "y": 240, "z": 2000, "width": 760, "height": 420, "tabOrder": 2000 }, + "visual": { + "visualType": "clusteredColumnChart", + "query": { "queryState": { + "Category": { "projections": [ { "field": { "Column": { "Expression": { "SourceRef": { "Entity": "Product" } }, "Property": "Category" } }, "queryRef": "Product.Category", "nativeQueryRef": "Category", "active": true } ] }, + "Y": { "projections": [ { "field": { "Measure": { "Expression": { "SourceRef": { "Entity": "Sales" } }, "Property": "Total Sales" } }, "queryRef": "Sales.Total Sales", "nativeQueryRef": "Total Sales" } ] } + } }, + "visualContainerObjects": { "title": [ { "properties": { + "show": { "expr": { "Literal": { "Value": "true" } } }, + "text": { "expr": { "Literal": { "Value": "'Sales by category'" } } } } } ] } + } +} +``` +- New page/visual: fresh 20-hex-char `name`, folder with the same name; add new pages to + `pages.json` `pageOrder`. +- Position is in pixels from the page's top-left; keep `x + width ≤ page width` and + `y + height ≤ page height`. Align edges to a shared grid (e.g. multiples of 8) and keep equal gaps. +- Literal encoding: text `'quoted'`, numbers `12D`, booleans `true`. When unsure, + `powerbi-report-author expr encode --kind <type> <value>`. +- Avoid visuals Microsoft is retiring (Q&A, Bing Map, filled map). + +## Themes +Create custom themes under `StaticResources/RegisteredResources/`, register them in +`report.json` (`themeCollection.customTheme` + a `resourcePackages` item of type +`RegisteredResources`), define ≥ 8 `dataColors`, and standardize fonts with the wildcard +`visualStyles` (`"*": { "*": { ... } }`). Rename the file on every edit (pitfall above). + +## Before large changes +Ask the user to commit the PBIP first so every agent edit is a reviewable diff. diff --git a/.cursor/skills/pbi-report-layer-pbir/references/desktop-bridge.md b/.cursor/skills/pbi-report-layer-pbir/references/desktop-bridge.md new file mode 100644 index 0000000..ef02d9a --- /dev/null +++ b/.cursor/skills/pbi-report-layer-pbir/references/desktop-bridge.md @@ -0,0 +1,83 @@ +# Desktop Bridge — setup, commands, errors + +Sources (read these when something here looks outdated): +- Microsoft Learn — [What is the Power BI Desktop Bridge?](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +- Microsoft Learn — [Power BI Report skill](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-report-skill-overview) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `powerbi-report-cli`, MIT) — the official agent guidance this kit follows. +- npm: [`@microsoft/powerbi-report-authoring-cli`](https://www.npmjs.com/package/@microsoft/powerbi-report-authoring-cli) + (`powerbi-report-author`), [`@microsoft/powerbi-desktop-bridge-cli`](https://www.npmjs.com/package/@microsoft/powerbi-desktop-bridge-cli) + (`powerbi-desktop`). Both MIT, Node 20+. + +## What it is +A local JSON-RPC 2.0 server inside each running Power BI Desktop window, reachable only on the +same Windows machine through the named pipe `pbi-desktop-bridge-<processId>`. One operation runs +at a time per window; a second request while one is running fails — wait and retry, never run +two Desktop commands in parallel. + +Methods (discover the current list with `bridge.manifest`): +| Method | Used for | +|---|---| +| `bridge.manifest` | Supported methods and their schemas for this Desktop build | +| `application.state.get/v1` | `currentFilePath`, `hasUnsavedChanges` | +| `report.snapshot.capture/v1` | PNG of one page (`pageId`, `scale` 1–3) | +| `file.reload/v1` | Reload the open PBIP from disk; `reloadModelDefinition` true/false | + +## Setup (once per machine) +1. Power BI Desktop for Windows, recent enough to show **File > Options and settings > Options > + Security > Desktop Bridge**. Keep *Enable external tool access to Power BI Desktop through + secure local APIs* checked (default on). Admins can force it with the registry value + `HKLM\SOFTWARE\Policies\Microsoft\Power BI Desktop\DesktopNamedPipeBridge` (`1` on, `0` off). +2. Node.js 20+, then `npm i -g @microsoft/powerbi-report-authoring-cli`. +3. Check: `powerbi-report-author doctor` and `powerbi-report-author preview --list-hosts` + (the `desktop` host must say `available: true` while a PBIP is open). +4. The report must be a **PBIP saved in PBIR format** (not a `.pbix`, not PBIR-Legacy). + +## Commands +Use `powerbi-report-author preview` for every Desktop operation; it finds the Desktop window +that has `<folder>` open, so there is no PID to pass. `<folder>` = the project folder or `.pbip`. + +| Intent | Command | +|---|---| +| Status (save gate) | `powerbi-report-author preview "<folder>" --host desktop --status` | +| Open / attach | `powerbi-report-author preview "<folder>" --host desktop` | +| Reload report | `powerbi-report-author preview "<folder>" --host desktop --reload` | +| Reload report + on-disk model | `powerbi-report-author preview "<folder>" --host desktop --reload-with-model` | +| Screenshot one page | `powerbi-report-author preview "<folder>" --host desktop --screenshot "<shots>\<page>.png" --page "<pageId or display name>"` | +| Screenshot all pages | `powerbi-report-author preview "<folder>" --host desktop --screenshot "<shots>" --all-pages` | +| Validate files | `powerbi-report-author validate "<folder>"` (`--no-schema` offline) | + +Optional `--scale 1..3` on screenshots. Every command prints a JSON envelope: +`{"data": {...}}` on success, `{"error": {"code": ..., "message": ..., "correction"?: {...}}}` on +failure. If an error carries a `correction` object, retry once changing only that value; +otherwise report the error and stop — do not improvise another command. + +The bridge CLI underneath (`powerbi-desktop status | manifest | open | reload | screenshot | +screenshot-all`) is useful for manual troubleshooting by the user (e.g. `powerbi-desktop status` +lists every open window and its unsaved state). Agents stay on `powerbi-report-author`. + +### Status before open/reload — never before a screenshot +```json +{"data":{"state":"ready","reports":[{"folder":"C:\\work\\Sales","hostInstanceId":"12345"}], + "details":{"hasUnsavedChanges":false}}} +``` +| `details.hasUnsavedChanges` | Action | +|---|---| +| `false` | Reload / open is allowed. | +| `true` | Stop. Ask the user to save (Ctrl+S) — or to discard — in Desktop, then check again. | +| missing, `null`, or the call failed | Treat as unsafe. Stop and report. | +| error `HOST_UNAVAILABLE` (no window has the report) | Open/attach is allowed; reload is not. | + +## Errors you will meet +| Code / symptom | Meaning | What to do | +|---|---|---| +| `UNSUPPORTED_OS` in `--list-hosts` | Not on Windows | Use the Desktop-closed path; `validate` still works. | +| `HOST_UNAVAILABLE` — "Several Power BI Desktop instances have this report open" | Same report in 2+ windows (`details.candidatePids`) | Ask the user to close the extras. Never close Desktop yourself. | +| `HOST_UNAVAILABLE` — no instance | Desktop closed or setting off | Ask the user to open the `.pbip`; check the Security setting. | +| `RENDER_FAILED` with "unknown page id" | Page exists on disk but not in the open window | Status → reload → capture that page again. | +| `PAGE_NOT_FOUND` | The page is not in the definition at all | Fix `pages.json` / the page id. Reload will not help. | +| `status: "partial"` + non-zero exit | Some pages captured, some failed | Keep the PNGs you got; report which pages failed and why. | +| JSON-RPC `-32601 MethodNotFound` (bridge CLI) | Desktop build lacks that method | Update Desktop; check `powerbi-desktop manifest`. | +| "operation in progress" | Another bridge call is running | Wait, then retry once. | +| Theme change invisible after reload | Desktop cached the theme by file name | Rename the theme file + both refs in `report.json`, validate, reload. | +| Change invisible after reload, no error | Edited a file Desktop does not read (e.g. outside `definition/`) or the wrong report folder | `preview-visuals <folder>` to confirm paths; check `--status` `folder`. | diff --git a/.cursor/skills/pbi-report-layer-pbir/scripts/validate_pbir.py b/.cursor/skills/pbi-report-layer-pbir/scripts/validate_pbir.py new file mode 100755 index 0000000..008c29b --- /dev/null +++ b/.cursor/skills/pbi-report-layer-pbir/scripts/validate_pbir.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Validate a PBIR report: Microsoft's `powerbi-report-author validate` when installed, +otherwise a structural check written in Python. + +Usage: + python3 validate_pbir.py <path> [--structural-only] [--offline] [--format text|json] + +<path>: a .pbip file, a <Name>.Report folder, a folder containing definition/, or a project +folder with exactly one *.Report. + +The CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`) is found through +$PBIR_AUTHOR_CLI or `powerbi-report-author` on PATH. It validates every file against the +published PBIR JSON schemas plus visual roles/types, so it is the authoritative check. +--offline passes --no-schema to it (no download of remote schemas). + +The structural fallback checks what breaks Desktop most often: JSON that does not parse, +PBIR-Legacy layout, a broken semantic-model binding, pages.json out of sync with the page +folders, page/visual `name` != folder, visuals without position/visualType, visuals outside +the page canvas, and theme/resource files that are registered but missing. It does NOT know +the schemas or visual roles — say so when reporting. + +Exit codes: 0 = no errors, 1 = errors found, 2 = usage error / the CLI could not run. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +CLI_NAME = "powerbi-report-author" +CLI_TIMEOUT_SECONDS = 300 + + +class UsageError(Exception): + pass + + +# --------------------------------------------------------------------------- +# Locate the .Report folder +# --------------------------------------------------------------------------- +def _read_json(path: Path): + return json.loads(path.read_text(encoding="utf-8-sig")) + + +def resolve_report(path: Path) -> Path: + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() != ".pbip": + raise UsageError(f"unsupported file: {path.name} (use a .pbip or a .Report folder)") + try: + artifacts = _read_json(path).get("artifacts", []) + except ValueError as exc: + raise UsageError(f"{path.name} is not valid JSON: {exc}") from exc + for artifact in artifacts: + report = artifact.get("report", {}).get("path") + if report and (path.parent / report).is_dir(): + return (path.parent / report).resolve() + sibling = path.with_name(path.stem + ".Report") + if sibling.is_dir(): + return sibling.resolve() + raise UsageError(f"{path.name} does not point to an existing .Report folder") + if (path / "definition").is_dir() or (path / "definition.pbir").is_file() or path.suffix == ".Report": + return path.resolve() + reports = sorted(p for p in path.glob("*.Report") if p.is_dir()) + if len(reports) == 1: + return reports[0].resolve() + if len(reports) > 1: + raise UsageError(f"several reports in {path}: {', '.join(p.name for p in reports)}; pass one") + raise UsageError(f"no .Report folder found under {path}") + + +# --------------------------------------------------------------------------- +# Structural fallback +# --------------------------------------------------------------------------- +class Findings: + def __init__(self, report_dir: Path): + self.report_dir = report_dir + self.items: list = [] + + def add(self, severity: str, code: str, message: str, file: Optional[Path] = None, path: str = ""): + rel = "" + if file is not None: + try: + rel = file.relative_to(self.report_dir).as_posix() + except ValueError: + rel = str(file) + self.items.append({"severity": severity, "code": code, "message": message, "file": rel, "path": path}) + + def error(self, *args, **kwargs): + self.add("error", *args, **kwargs) + + def warning(self, *args, **kwargs): + self.add("warning", *args, **kwargs) + + +def _load(findings: Findings, file: Path, required: bool = True): + if not file.is_file(): + if required: + findings.error("FILE_MISSING", f"required file is missing: {file.name}", file) + return None + try: + return _read_json(file) + except ValueError as exc: + findings.error("JSON_INVALID", f"invalid JSON: {exc}", file) + return None + + +def _is_number(value) -> bool: + return isinstance(value, (int, float)) and not isinstance(value, bool) + + +def structural_check(report_dir: Path) -> dict: + f = Findings(report_dir) + definition = report_dir / "definition" + + for file in sorted(report_dir.rglob("*.json")): + if ".pbi" in file.relative_to(report_dir).parts: + continue + try: + _read_json(file) + except ValueError as exc: + f.error("JSON_INVALID", f"invalid JSON: {exc}", file) + invalid = {i["file"] for i in f.items} + + if not definition.is_dir(): + if (report_dir / "report.json").is_file(): + f.error("PBIR_LEGACY_FORMAT", "PBIR-Legacy layout (report.json at the root); save the " + "report in PBIR format (Desktop: File > Options > Preview features > PBIR)", report_dir / "report.json") + else: + f.error("FILE_MISSING", "definition/ folder is missing", definition) + return _result(f, "structural") + + pbir = _load(f, report_dir / "definition.pbir") + if isinstance(pbir, dict): + ref = pbir.get("datasetReference", {}) + by_path = ref.get("byPath", {}).get("path") if isinstance(ref.get("byPath"), dict) else None + if by_path: + target = (report_dir / by_path).resolve() + if not target.is_dir(): + f.error("MODEL_BINDING_BROKEN", f"datasetReference.byPath points to a missing folder: {by_path}", + report_dir / "definition.pbir", "datasetReference.byPath.path") + elif not ref.get("byConnection"): + f.error("MODEL_BINDING_MISSING", "datasetReference has neither byPath nor byConnection", + report_dir / "definition.pbir", "datasetReference") + + _load(f, definition / "version.json") + report = _load(f, definition / "report.json") + if isinstance(report, dict): + _check_resources(f, report, report_dir) + + pages_dir = definition / "pages" + pages_meta = _load(f, pages_dir / "pages.json") + page_folders = sorted(p.name for p in pages_dir.iterdir() if p.is_dir()) if pages_dir.is_dir() else [] + if isinstance(pages_meta, dict): + order = pages_meta.get("pageOrder", []) + for page_id in order: + if page_id not in page_folders: + f.error("PAGE_MISSING", f"pages.json lists page '{page_id}' but its folder is missing", + pages_dir / "pages.json", "pageOrder") + for page_id in page_folders: + if page_id not in order: + f.warning("PAGE_NOT_IN_ORDER", f"page folder '{page_id}' is not listed in pages.json pageOrder", + pages_dir / page_id) + active = pages_meta.get("activePageName") + if active and active not in order: + f.error("ACTIVE_PAGE_UNKNOWN", f"activePageName '{active}' is not in pageOrder", + pages_dir / "pages.json", "activePageName") + + for page_id in page_folders: + _check_page(f, pages_dir / page_id, invalid) + + return _result(f, "structural") + + +def _check_resources(f: Findings, report: dict, report_dir: Path) -> None: + static = report_dir / "StaticResources" + for package in report.get("resourcePackages", []) or []: + folder = static / package.get("name", "") + for item in package.get("items", []) or []: + rel = item.get("path") + if rel and not (folder / rel).is_file(): + f.error("RESOURCE_MISSING", f"resource '{item.get('name')}' registered in report.json is " + f"missing on disk: {package.get('name')}/{rel}", report_dir / "definition" / "report.json", + "resourcePackages") + custom = (report.get("themeCollection") or {}).get("customTheme") + if isinstance(custom, dict) and custom.get("name"): + registered = { + item.get("name") + for package in report.get("resourcePackages", []) or [] + if package.get("type") == custom.get("type", "RegisteredResources") + for item in package.get("items", []) or [] + } + if custom["name"] not in registered: + f.error("THEME_NOT_REGISTERED", f"customTheme '{custom['name']}' has no matching resourcePackages item", + report_dir / "definition" / "report.json", "themeCollection.customTheme.name") + + +def _check_page(f: Findings, page_dir: Path, invalid: set) -> None: + page_file = page_dir / "page.json" + page = _load(f, page_file) + if not isinstance(page, dict): + return + if page.get("name") != page_dir.name: + f.error("NAME_MISMATCH", f"page.json name '{page.get('name')}' != folder '{page_dir.name}'", page_file, "name") + if not page.get("displayName"): + f.warning("PAGE_NO_DISPLAY_NAME", "page has no displayName", page_file, "displayName") + width, height = page.get("width"), page.get("height") + if not (_is_number(width) and _is_number(height)): + f.error("PAGE_SIZE_INVALID", "page width/height must be numbers", page_file) + width = height = None + + visuals_dir = page_dir / "visuals" + if not visuals_dir.is_dir(): + return + for visual_dir in sorted(p for p in visuals_dir.iterdir() if p.is_dir()): + visual_file = visual_dir / "visual.json" + rel = visual_file.relative_to(f.report_dir).as_posix() + if rel in invalid: + continue + visual = _load(f, visual_file) + if not isinstance(visual, dict): + continue + if visual.get("name") != visual_dir.name: + f.error("NAME_MISMATCH", f"visual.json name '{visual.get('name')}' != folder '{visual_dir.name}'", + visual_file, "name") + pos = visual.get("position") + if not isinstance(pos, dict) or not all(_is_number(pos.get(k)) for k in ("x", "y", "width", "height")): + f.error("POSITION_INVALID", "position needs numeric x, y, width and height", visual_file, "position") + pos = None + kind = (visual.get("visual") or {}).get("visualType") + if not kind and "visualGroup" not in visual: + f.error("VISUAL_TYPE_MISSING", "visual.visualType is missing (and this is not a visualGroup)", + visual_file, "visual.visualType") + if pos and width is not None and not visual.get("parentGroupName"): + right, bottom = pos["x"] + pos["width"], pos["y"] + pos["height"] + if pos["x"] < 0 or pos["y"] < 0 or right > width + 0.5 or bottom > height + 0.5: + f.warning("OUTSIDE_CANVAS", f"visual extends outside the {width}x{height} page " + f"(x={pos['x']}, y={pos['y']}, right={right}, bottom={bottom})", visual_file, "position") + + +def _result(f: Findings, engine: str) -> dict: + errors = sum(1 for i in f.items if i["severity"] == "error") + warnings = sum(1 for i in f.items if i["severity"] == "warning") + return { + "engine": engine, + "report": str(f.report_dir), + "result": "failed" if errors else ("succeededWithWarnings" if warnings else "succeeded"), + "errorCount": errors, + "warningCount": warnings, + "diagnostics": f.items, + } + + +# --------------------------------------------------------------------------- +# powerbi-report-author CLI +# --------------------------------------------------------------------------- +def find_cli(env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + explicit = env.get("PBIR_AUTHOR_CLI") + if explicit: + return explicit if Path(explicit).is_file() else None + return shutil.which(CLI_NAME) + + +def build_cli_command(cli: str, report_dir: Path, offline: bool) -> list: + cmd = [cli, "validate", str(report_dir)] + if offline: + cmd.append("--no-schema") + return cmd + + +def parse_cli_output(stdout: str, report_dir: Path) -> dict: + """Normalize the CLI's JSON envelope ({"data": ...} or {"error": ...}).""" + try: + envelope = json.loads(stdout) + except ValueError as exc: + raise UsageError(f"{CLI_NAME} returned non-JSON output: {stdout.strip()[:500]}") from exc + if "error" in envelope: + err = envelope["error"] + raise UsageError(f"{CLI_NAME} failed: {err.get('code')}: {err.get('message')}") + data = envelope.get("data", {}) + items = [] + for code, group in (data.get("diagnostics") or {}).items(): + for item in group.get("items", []): + file = item.get("file", "") + try: + file = Path(file).resolve().relative_to(report_dir).as_posix() if file else "" + except ValueError: + pass + items.append({ + "severity": group.get("severity", "error"), + "code": code, + "message": item.get("message", ""), + "file": file, + "path": item.get("path", ""), + }) + return { + "engine": CLI_NAME, + "report": str(report_dir), + "result": data.get("result", "unknown"), + "errorCount": int(data.get("errorCount", sum(1 for i in items if i["severity"] == "error"))), + "warningCount": int(data.get("warningCount", sum(1 for i in items if i["severity"] == "warning"))), + "diagnostics": items, + } + + +def run_cli(cli: str, report_dir: Path, offline: bool) -> dict: + cmd = build_cli_command(cli, report_dir, offline) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=CLI_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise UsageError(f"could not run {' '.join(cmd)}: {exc}") from exc + return parse_cli_output(proc.stdout, report_dir) + + +# --------------------------------------------------------------------------- +# Output +# --------------------------------------------------------------------------- +def format_text(result: dict) -> str: + if result["engine"] == "structural": + lines = [f"PBIR validation: structural fallback ({CLI_NAME} not found; schemas and visual " + "roles NOT checked — install @microsoft/powerbi-report-authoring-cli)"] + else: + lines = [f"PBIR validation: {CLI_NAME} validate"] + lines.append(f"Report: {result['report']}") + for d in sorted(result["diagnostics"], key=lambda d: (d["severity"] != "error", d["file"], d["code"])): + where = d["file"] + (f" @ {d['path']}" if d["path"] else "") + lines.append(f"{d['severity'].upper():8} [{d['code']}] {d['message']}" + (f" ({where})" if where else "")) + lines.append(f"Result: {result['result']} — {result['errorCount']} error(s), {result['warningCount']} warning(s)") + return "\n".join(lines) + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Validate a PBIR report (CLI if installed, else structural).") + parser.add_argument("path", type=Path) + parser.add_argument("--structural-only", action="store_true", help="skip the CLI even if it is installed") + parser.add_argument("--offline", action="store_true", help="pass --no-schema to the CLI") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + + try: + report_dir = resolve_report(args.path) + cli = None if args.structural_only else find_cli() + result = run_cli(cli, report_dir, args.offline) if cli else structural_check(report_dir) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + print(json.dumps(result, indent=2) if args.format == "json" else format_text(result)) + return 1 if result["errorCount"] else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.cursor/skills/pbi-semantic-layer-tmdl/SKILL.md b/.cursor/skills/pbi-semantic-layer-tmdl/SKILL.md new file mode 100644 index 0000000..da3d3bd --- /dev/null +++ b/.cursor/skills/pbi-semantic-layer-tmdl/SKILL.md @@ -0,0 +1,99 @@ +--- +name: pbi-semantic-layer-tmdl +description: Change the Power BI semantic model — measures, columns, tables, relationships, RLS, DAX — with Power BI Desktop OPEN through the Power BI Modeling (Authoring) MCP server, saving to TMDL in a PBIP project; or by editing TMDL files with Desktop closed. Includes a semantic-model review checklist. Use for any request to add or fix a measure, DAX, relationship, table, column or model metadata. +--- + +# Skill: pbi-semantic-layer-tmdl + +With Desktop open, the model lives in Desktop's memory (a local Analysis Services instance). Edit +it **there**, live, through the Power BI Modeling MCP server — then the user saves and Desktop +writes the TMDL into the PBIP. Editing the `.tmdl` files while Desktop is open does nothing +until a model reload and is overwritten by the next save. + +## Default: Desktop open + Modeling MCP +Setup and the tool list: [references/modeling-mcp.md](references/modeling-mcp.md). +1. **Connect:** "Connect to '<file name>' in Power BI Desktop". Confirm with a read: list the + tables and measures. If the MCP is not registered, stop and point the user to the setup — + do not fall back to editing TMDL silently. +2. **Inspect through the MCP**, not the `.tmdl` files: while connected, the live model is the + truth and disk may be stale. +3. **Change** in dependency order: tables/partitions → columns → relationships → measures. + Check that every referenced table/column exists and that the name is not taken. +4. **Prove it:** run a DAX query through the MCP (`EVALUATE SUMMARIZECOLUMNS(..., "x", [New Measure])` + or `EVALUATE { [New Measure] }`) and show the value. A measure that only "was created" is not done. +5. **Save:** ask the user to press Ctrl+S in Desktop. That writes the TMDL into + `<Name>.SemanticModel/definition/` — the diff the team reviews and commits. +6. **Quality:** run `pbi-quality-rules` (BPA) after significant changes — on disk, so after step 5. +7. If a visual must use the new measure, continue in `pbi-report-layer-pbir`. Report reloads do + not touch the model and are not needed for MCP changes. + +## Alternative: Desktop closed (no MCP, older Desktop, CI) +Edit the TMDL files directly: +1. Make sure Desktop is **closed** — it overwrites disk on save. +2. Edit `.tmdl` files following the syntax below; run `pbi-quality-rules`. +3. Ask the user to open the `.pbip`: field list populates, no error banner, each new measure on a + Card shows a value. On error, re-read the edited TMDL and fix the reference/syntax. + +If Desktop must stay open and the MCP is unavailable, edit TMDL, then reload model + report with +`powerbi-report-author preview <folder> --host desktop --reload-with-model` after the same +unsaved-changes gate as `pbi-report-layer-pbir`. It reloads definitions only — the user must +refresh data for new/changed imported columns. + +## `.SemanticModel` structure +``` +<Name>.SemanticModel/ + definition.pbism ← pointer (do not hand-edit) + definition/ + database.tmdl ← compatibilityLevel + model.tmdl ← model properties, `ref table` order, annotations + tables/<Table>.tmdl ← one table per file + relationships.tmdl + expressions.tmdl ← shared M expressions / parameters + roles.tmdl ← RLS + cultures/ ← translations + .pbi/ ← local cache/settings — never edit, keep out of git +``` + +## TMDL syntax +```tmdl +/// Order lines. Grain: one row per order line. +table Sales + + /// Sum of the order line amount. + measure 'Total Sales' = SUM(Sales[Amount]) + formatString: #,##0.00 + displayFolder: Revenue + + measure 'Margin %' = + DIVIDE([Total Sales] - [Total Cost], [Total Sales]) + formatString: 0.0% + displayFolder: Revenue + + column Amount + dataType: decimal + summarizeBy: sum + sourceColumn: Amount +``` +```tmdl +relationship 5f0c2a4e-9b1d-4c3a-8e2f-1a6b7c8d9e01 + fromColumn: Sales.DateKey + toColumn: Date.DateKey +``` +- Indent with tabs; properties one level under their object. +- Descriptions are `///` lines directly above the object. +- Multi-line DAX: leave the `=` line empty and indent the expression two levels deeper than the + object, or wrap it in a ```` ``` ```` block. +- Quote names with spaces or symbols: `'Margin %'`; double a quote inside: `'Customer''s'`. + +## Review checklist (run it on any model you change or are asked to review) +- **Names** are business-friendly: no `tbl_`/`dim_` prefixes or `_id` columns visible to users. +- **Every measure** has a `formatString`, a `displayFolder` and a `///` description (what it + means and its grain/filters in plain words). +- **Relationships** match reality: many-to-one from fact to dimension, single direction unless a + documented reason exists; no ambiguous paths. +- **A real Date table** (marked as date table) drives time intelligence; auto date/time is off + (`annotation __PBI_TimeIntelligenceEnabled = 0` in `model.tmdl`). +- **Technical keys hidden** (`isHidden` on `...ID`/`...Key` columns); explicit measures instead + of implicit column aggregation for anything a report shows. +- **DAX:** `DIVIDE` instead of `/` + `IFERROR`; variables for repeated sub-expressions; no + measure that just returns `BLANK()`. diff --git a/.cursor/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md b/.cursor/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md new file mode 100644 index 0000000..fdf7468 --- /dev/null +++ b/.cursor/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md @@ -0,0 +1,60 @@ +# Power BI Modeling MCP — setup and use + +Microsoft now calls it the **Power BI Authoring MCP server**; the package and server id are still +`powerbi-modeling-mcp`. Sources: +- Microsoft Learn — [Power BI Authoring MCP server](https://learn.microsoft.com/en-us/power-bi/developer/mcp/power-bi-authoring-mcp) +- [microsoft/powerbi-modeling-mcp](https://github.com/microsoft/powerbi-modeling-mcp) (README, TROUBLESHOOTING) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `semantic-model-authoring`, bundles the same server) + +## Local vs hosted +Working against a model **open in Power BI Desktop** (or PBIP files on disk) needs the **local** +server: it runs on the same Windows machine and talks to Desktop's local Analysis Services +instance over XMLA. The hosted endpoint only reaches Fabric workspaces. Register one of the two, +not both — duplicate tool sets confuse the agent. The local server does not run on macOS. + +## Register the local server +Command in every client: `npx -y @microsoft/powerbi-modeling-mcp@latest --start` +(Node 18+; or install the VS Code extension `analysis-services.powerbi-modeling-mcp`). + +| Client | File | Shape | +|---|---|---| +| Claude Code | `.mcp.json` (project) | `{"mcpServers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| GitHub Copilot (VS Code) | `.vscode/mcp.json` | `{"servers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| OpenCode | `opencode.json` | `{"mcp": {"powerbi-modeling-mcp": {"type": "local", "command": ["npx", "-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| Cursor | `.cursor/mcp.json` | same shape as Claude Code (`mcpServers`) | + +Useful flags: `--readonly` (inspection only — good for reviews), `--readwrite` (default; asks for +confirmation before the first write per database). + +## Connect and work +- Desktop: `Connect to '<file name>' in Power BI Desktop` (tool `ConnectToPowerBIDesktop`; the + PBIP must be open in Desktop). +- PBIP on disk without Desktop: `Open semantic model from PBIP folder '<path>\<Name>.SemanticModel\definition'` + (changes stay in the session until exported back to TMDL). +- First call after connecting: a read (`List the tables and measures`) to confirm the target. + +Tool groups you will use most: `measure_operations`, `column_operations`, `table_operations`, +`relationship_operations`, `dax_query_operations` (run/validate DAX, max 100,000 rows), +`transaction_operations` (begin/commit/rollback a batch), `calculation_group_operations`, +`security_role_operations`, `object_translation_operations`, `database_operations` +(import/export TMDL). Ask the agent "what can the Power BI Authoring MCP do" to list the current set. + +## Rules of thumb +- Prefer one transaction per logical change; roll back on a failed step instead of leaving + half-applied edits. +- Verify every new or changed measure with a DAX query before calling it done. +- Changes are live in Desktop but not on disk until the user saves (Ctrl+S); a PBIP under git + makes every MCP change a reviewable diff. +- The MCP edits the model only — never report pages (use `pbi-report-layer-pbir`). +- Model metadata and query results go to the LLM provider of the client. Do not query + sensitive columns just to "look at the data". + +## Troubleshooting +| Symptom | Check | +|---|---| +| Server missing from the tool list | Client in agent mode? Server registered and started? Copilot Business/Enterprise: "MCP servers in Copilot" policy enabled by an admin. | +| "No Power BI Desktop instance found" | The PBIP/PBIX is open in Desktop and the name matches the file name you gave. | +| Reads work, writes fail | `--readonly` set, or (Fabric) only Build permission / XMLA read-only. | +| Edits vanish after reopening the file | The user closed Desktop without saving. | +| TMDL on disk does not show the change | Not saved yet — Desktop writes TMDL on save. | diff --git a/.github/agents/analytics-engineer.agent.md b/.github/agents/analytics-engineer.agent.md deleted file mode 100644 index 26de9c4..0000000 --- a/.github/agents/analytics-engineer.agent.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -name: analytics-engineer -description: Expert in data modeling, dbt, dimensional modeling (Star Schema), data - warehouse architecture, and SQL transformations. Triggers on dbt, dimensional modeling, - star schema, data warehouse, transformation, partitioning, clustering. ---- - -# Analytics Engineer & Warehouse Architect - -You turn clean raw data into intuitive, performant dimensional models (Star Schemas) with dbt. -Guiding principle: *build a model so intuitive that business users query it without asking you how.* - -## Design process -1. **Grain first:** what business process, and what does one row represent? -2. **Shape:** fact or dimension? Does it need partitioning/clustering? -3. **Build & test** the dbt models, then verify grain, idempotency, and performance. - -## Dimensional modeling rules -- Build dimensions before facts. -- Every dimension has a surrogate key (`sk_`, typically a hash of the natural key). -- One unambiguous grain per fact table. -- Keep dimensions flat — snowflake only when performance demands it. - -## Performance levers -- **Partition** on the column queries consistently filter (e.g. `event_date`). -- **Cluster / Z-order** on high-cardinality join/filter keys (`user_id`, `tenant_id`). -- **Materialized views** for heavy, frequently-read aggregations. - -## dbt conventions (org context) -- CTE structure: `import` CTEs on top, `logical` in the middle, one `final` at the bottom. -- Every model: `unique` + `not_null` on its primary key; description in `schema.yml`. -- Naming: `stg_`, `dim_`, `fct_`, `mart_`. - -## Handoffs -- Raw ingestion / orchestration → `data-engineer`. -- Optimized views for import → `powerbi-developer`. -- Enforce data contracts and PII masking inline per the workspace rules. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), ad-hoc analysis & dashboards (→ `data-scientist`), -DAX tuning (→ `powerbi-developer`). diff --git a/.github/agents/data-engineer.agent.md b/.github/agents/data-engineer.agent.md deleted file mode 100644 index 8746400..0000000 --- a/.github/agents/data-engineer.agent.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -name: data-engineer -description: Expert in data engineering, ETL/ELT pipelines, Medallion architecture - (Bronze/Silver/Gold), Databricks, PySpark, and orchestration. Triggers on pipeline, - ingestion, spark, databricks, medallion, etl, elt, orchestration, airflow, dagster. ---- - -# Data Engineer - -Senior Data Engineer. You build robust, idempotent, fault-tolerant pipelines with clear -architectural boundaries. Guiding principle: *pipelines should be boring and re-runnable; -bad data is forever.* - -## Medallion architecture -- **Bronze:** append-only raw data, exactly as it arrived. No transformations. -- **Silver:** deduplicated, typed, standardized. The enterprise source of truth. -- **Gold:** aggregated, business-level tables for BI/ML consumption. - -## Pipeline patterns -- **Idempotent loads:** use `MERGE`/upsert or partition `OVERWRITE`; never `INSERT INTO` - without first scoping a delete. Re-running any window must yield identical results. -- **Incremental:** track a watermark (`last_updated_at`); process `source > target.watermark`; - handle late-arriving data with `MERGE`. -- **Backfill-ready:** parameterize `start_date`/`end_date`, defaulting to current date. -- **Write-Audit-Publish:** write to staging → run quality checks → publish only on pass. - -## Opinionated defaults (org context, not generic advice) -- Use PySpark/Polars for large transforms; reserve Pandas for small/local work. -- Fail loudly and alert — never swallow exceptions silently. -- Run lineage/downstream checks before altering any upstream schema. - -## Handoffs -- Clean Silver data → `analytics-engineer` for dimensional modeling. -- Enforce quality gates and PII masking inline (WAP, mask at Silver) per the workspace rules. - -## Out of scope -Dimensional models / dbt (→ `analytics-engineer`), dashboards, metrics & ML (→ `data-scientist`). diff --git a/.github/agents/data-scientist.agent.md b/.github/agents/data-scientist.agent.md deleted file mode 100644 index 731e0ab..0000000 --- a/.github/agents/data-scientist.agent.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -name: data-scientist -description: Expert across the analytics spectrum — from metrics, dashboards, and - ad-hoc/business analysis to statistical modeling, machine learning, forecasting, - and experimentation (A/B testing). Also reviews semantic models for business accuracy. - Triggers on analysis, dashboard, metric, KPI, insight, report, semantic model review, - machine learning, ML, prediction, forecasting, a/b testing, statistics, experiment. ---- - -# Data Scientist & Analyst - -You turn data into decisions across the full rigor spectrum: descriptive analysis and metrics at -one end, statistical inference and ML at the other. Guiding principle: *every output must lead to -an action — and start simple; a deployed logistic regression beats a notebook-bound neural net.* - -## Pick the right altitude of rigor -- **Descriptive / diagnostic** — metrics, dashboards, hypothesis-driven exploration. -- **Inferential / predictive** — statistics, experiments, ML. - -Match the method to the decision; don't bring ML to a question a well-defined metric answers. - -## Analysis & metrics -- Question the question: "show sales by region" is a data pull — ask what decision it informs. -- Hypotheses are MECE; every chart must lead to an action. -- Define each metric with: **Name**, **Definition** (plain-English), **Grain**, **Filters**, **Polarity**. -- Dashboard ideation: audience (exec vs. ops); the one 10-second question; inverted pyramid - (KPIs → trend → detail). Line for trend, bar for comparison; avoid pie. -- Semantic-model review (TMDL): friendly naming, relationships that match reality, a real Date - table with sound time-intelligence, technical keys hidden. - -## Statistics, experiments & ML -- A/B tests: fix sample size and duration *before* starting — peeking inflates false positives. -- Watch confounders; read distributions, not just averages; never leak future data into training. -- Baseline first (mean / linear / logistic); justify any added complexity. -- Imbalanced data → F1 / PR-AUC / cost-based metrics, not accuracy. -- Fit scalers/encoders on train only; ship with drift monitoring; report dollar impact, not R². - -## Handoffs -- Freshness, feature pipelines, deployment infra → `data-engineer`. -- Clean marts / new dimensions → `analytics-engineer`. -- Keep PII out of features and models (workspace privacy rules). -- DAX & TMDL/PBIR implementation → `powerbi-developer`. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), dbt models (→ `analytics-engineer`), -Power BI implementation (→ `powerbi-developer`). diff --git a/.github/agents/pbi-visual-reviewer.agent.md b/.github/agents/pbi-visual-reviewer.agent.md new file mode 100644 index 0000000..4debd45 --- /dev/null +++ b/.github/agents/pbi-visual-reviewer.agent.md @@ -0,0 +1,62 @@ +--- +name: pbi-visual-reviewer +description: Independent reviewer of rendered Power BI report pages. Reads only the + page screenshots (PNG), the request or spec, and the PBIR folder, then returns PASS + or FAIL per page and per visual against a fixed checklist. Use after every screenshot + step of the PBIR edit loop (pbi-report-layer-pbir), or whenever someone asks to + review, QA, or check a Power BI report page. Read-only; never edits files. +tools: +- read +- search +--- + +# Power BI Visual Reviewer + +You judge what a Power BI page **looks like**, not what its author meant. You are deliberately +kept away from the author's reasoning: if the caller hands you explanations ("the overlap is +intentional", "the card is fine, it just needs a refresh"), ignore them. Only the pixels, the +spec, and the PBIR files count. + +You cannot run commands or change files. If something must change, say exactly where. + +## Inputs (refuse to start without the first two) +1. **Screenshots** — paths to the PNGs (one per page), as captured by + `powerbi-report-author preview <folder> --host desktop --screenshot <dir> --all-pages`. +2. **Request / spec** — what the user asked for (the change, or the page spec). +3. **PBIR folder** — the `<Name>.Report/` path, to map what you see to files: + `definition/pages/<pageId>/page.json` (`displayName`, `width`, `height`) and + `definition/pages/<pageId>/visuals/<visualId>/visual.json` (`position`, `visual.visualType`, + titles under `visual.visualContainerObjects.title`). + +If a screenshot is missing, unreadable, or obviously stale (it does not show the requested +change at all), the verdict for that page is **BLOCKED**, not PASS. + +## Checklist (apply every item to every page) +| ID | Check | FAIL when | +|---|---|---| +| C1 | Overlap | Two visuals cover each other's content (borders touching is fine). | +| C2 | Clipped text | Titles, labels, axis text, card values or slicer items are cut, truncated with `…` where the full text matters, or wrap into unreadable stubs. | +| C3 | Empty or broken visual | Grey placeholder, "Can't display the visual", red/yellow error icon, "See details", blank chart area where data is expected. | +| C4 | Alignment | Edges that should line up are off by a visible amount; uneven gaps in a row/column of visuals; content outside the page canvas. | +| C5 | Theme consistency | Colors, fonts or backgrounds that ignore the report theme or differ between visuals of the same kind. | +| C6 | Titles | A visual without a meaningful title (unless it is a card/KPI whose label says it), duplicated titles, or titles that do not match the data shown. | +| C7 | Spec match | The requested change is not visible, or something the spec did not ask for changed. | + +Use the PBIR to confirm, not to excuse: a `position` that overlaps confirms C1; a title present in +`visual.json` but invisible in the PNG is still a C2/C6 failure. + +## Output (exact shape) +```markdown +# Visual review — <report name> +Verdict: APPROVED | REJECTED | BLOCKED + +| Page | Visual (type · visualId) | Check | Result | Evidence | Where to fix | +|---|---|---|---|---|---| +| Overview | Total Sales card · 3f2a… | C2 | FAIL | value shows "1.2…" | visuals/3f2a…/visual.json → position.width | +| Overview | — | C4 | PASS | | | +``` +- One row per failing check per visual; one PASS row per page for checks that passed overall. +- `Verdict` is APPROVED only when every page passed every check. Any FAIL → REJECTED. Any + missing/stale screenshot → BLOCKED. +- Evidence is what you see, in a few words. Do not speculate about causes you cannot see. +- No redesign proposals beyond the specific fix location. diff --git a/.github/agents/powerbi-developer.agent.md b/.github/agents/powerbi-developer.agent.md deleted file mode 100644 index 7ba0745..0000000 --- a/.github/agents/powerbi-developer.agent.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -name: powerbi-developer -description: Power BI as-code developer. Edits PBIP projects directly — semantic model - via TMDL (tables, measures, relationships) and reports via PBIR (pages, visuals, - themes). Validates with Tabular Editor 2 (free). Triggers on power bi, pbip, tmdl, - pbir, dax, measure, semantic model, report, visual, theme, bookmark. ---- - -# Power BI Developer (as-code) - -You develop Power BI by editing the **PBIP** project files directly on disk — no manual -clicking in Desktop. A PBIP project has two layers; pick the right skill for each: - -- **Semantic model** (`.SemanticModel/`, TMDL) — tables, columns, measures, relationships, - RLS. Use `pbi-semantic-layer-tmdl`. -- **Report** (`.Report/`, PBIR JSON) — pages, visuals, themes, bookmarks. Use - `pbi-report-layer-pbir`. - -This single agent owns both layers — choose the layer by the request instead of delegating. - -## Hard precondition -**Power BI Desktop must be CLOSED** before you save any TMDL or PBIR file. Desktop holds the -model in memory and overwrites disk on save, so live edits corrupt or get silently lost. -Confirm it's closed before writing. - -## Session flow -1. Confirm Desktop is closed and the PBIP folder is valid. -2. Read the relevant layer (TMDL or PBIR) to understand structure before editing. -3. Make the edits, following exact TMDL/PBIR syntax from the skills. -4. **Validate the model** with `pbi-quality-rules` → runs the real Best Practice Analyzer via - the free Tabular Editor 2 CLI. Report `error` and `warning` only. -5. Ask the user to reopen the `.pbip` in Desktop and confirm (Diagnostics clean, measures - render in a Card). Fix anything that errors. - -## Tooling -- **Tabular Editor 2 (free, OSS)** is the validation/automation backbone — BPA, batch edits, - TMDL serialization. Prefer it over reinventing checks. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (public preview) can drive bulk model edits - against the same PBIP/TMDL files. Mention it for large refactors; file editing stays the default. -- Do **not** depend on the deprecated community "Power BI MCP" servers. - -## Boundaries -- Documentation of a full dashboard → use the `/document-dashboard` workflow. -- Standalone model health check → use the `/validate-pbi` workflow. -- Upstream modeling / dbt / views → `analytics-engineer`. Metric definitions & layout intent → - `data-scientist`. - -> The PBIR JSON schema evolves with Desktop releases. Always tell the user to commit (`git add`) -> before large visual refactors and to test by opening the file. diff --git a/.github/agents/presentation-designer.agent.md b/.github/agents/presentation-designer.agent.md deleted file mode 100644 index ee11073..0000000 --- a/.github/agents/presentation-designer.agent.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -name: presentation-designer -description: Turns data work — exploratory analyses, dashboards, business plans, project - proposals — into polished, on-brand presentations as code. Builds HTML decks (reveal.js), - interactive one-page sites, and print/PDF handouts; applies a company's visual identity - from a DESIGN.md. Triggers on presentation, deck, slides, pitch, readout, report-out, - business plan, storytelling, reveal.js, slidev, marp, DESIGN.md, brand, visual identity. ---- - -# Presentation Designer - -You turn analytical content into a clear visual narrative and ship it as code — not by -clicking around in slide tools. Three skills back you up: - -- **`building-html-presentations`** — the *how*: format choice (deck / flex-deck / interactive - site / consulting PDF), ready-to-edit starters, and a chart cookbook. Default engine: **reveal.js**. -- **`generating-pptx`** — when the deliverable must be a **native, editable PowerPoint** (`.pptx`), - built with python-pptx (real charts/text, not images). -- **`applying-visual-identity`** — the *look*: read the project's `DESIGN.md` (Google Labs - spec) and apply its tokens (colors, type, spacing) so the output is on-brand. - -## How to work -1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load - `applying-visual-identity` and use its tokens. If absent, offer to scaffold one (don't - invent a brand silently) and fall back to a clean neutral theme meanwhile. -2. **Pick the format** with `building-html-presentations`: a **deck** (or flex-deck) for a live - readout, an **interactive site** for self-serve exploration, a consulting-style **PDF** for - leave-behinds, or a native **`.pptx`** (`generating-pptx`) when it must be editable in - PowerPoint. State which and why. -3. **Lead with the story, not the chart dump.** One message per slide; title = the takeaway, - not the metric name. Structure for the audience (exec vs technical). -4. **Build from the starters** in the skill; wire real numbers/charts; keep it self-contained. - -## Data storytelling rules (domain-specific) -- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". -- **One idea per slide.** Move supporting detail to an appendix section. -- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. -- **Don't fabricate data.** Use only numbers the user provides or that exist in the repo; mark - placeholders explicitly as `TODO`. - -## Boundaries -- Building the *analysis itself* (metrics, models, ML) → `data-scientist`. -- Power BI reports rendered inside Power BI → `powerbi-developer` (PBIR), not this agent. -- This agent owns the *external-facing presentation* layer — HTML/PDF artifacts for humans. diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index f85a228..86a75ff 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,4 +1,4 @@ -<!-- Auto-generated by scripts/sync_agents.py — NÃO EDITE MANUALMENTE --> +<!-- Auto-generated by scripts/sync_agents.py — DO NOT EDIT MANUALLY --> # Agentic Data Kit — Operating Rules @@ -13,17 +13,20 @@ data-specific rules — not things a capable model already does well. ## How to work -1. **Route to a specialist.** Pick the agent whose domain fits the request and briefly say - which one you're applying (e.g. "Applying @data-engineer"). For cross-domain requests, - split the work across agents and merge the results. -2. **Skills activate themselves.** Every skill declares when it applies in its `description`. - When a request matches it — "document this", "edit this Power BI model" — load that - `SKILL.md` on demand, **with or without an agent involved**. Skills do not require an owning - agent; the `skills:` field on an agent is only a hint. Don't preload every file in a folder. -3. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` +1. **Skills first.** Every skill declares when it applies in its `description`. When a request + matches one — "add a measure", "fix this visual", "document this dashboard" — load that + `SKILL.md` on demand and follow it. Don't preload every file in a folder. +2. **Power BI is developed with Desktop open.** Model changes go live through the Power BI + Modeling MCP (`pbi-semantic-layer-tmdl`); report changes follow edit → validate → save → + status → reload → screenshot → review (`pbi-report-layer-pbir`). Never reload while Desktop + has unsaved changes. +3. **Visual changes get an independent review.** After screenshots, hand the PNGs, the request + and the PBIR folder — not your reasoning — to the `pbi-visual-reviewer` agent. Where the tool + has no subagents, run its checklist in a fresh chat with only those inputs. +4. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` skill first — it decides whether to build an agent, a skill, or nothing, and keeps every installed tool folder (`.agent`, `.claude`, `.github`, `.opencode`, `.cursor`) in sync. -4. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd +5. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd likely build the wrong thing, ask. Otherwise state your assumptions and proceed. ## Data engineering principles (always apply) diff --git a/.github/prompts/document-dashboard.prompt.md b/.github/prompts/document-dashboard.prompt.md index 4aaf882..dbc451f 100644 --- a/.github/prompts/document-dashboard.prompt.md +++ b/.github/prompts/document-dashboard.prompt.md @@ -1,6 +1,6 @@ --- -description: Auto-generate complete Markdown documentation for a PBIP project. name: document-dashboard +description: Auto-generate complete Markdown documentation for a PBIP project. --- **Context:** ${selection} @@ -15,7 +15,8 @@ project root. ## Steps -1. **Inventory.** Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, +1. **Inventory.** If Power BI Desktop is open, have the user save first so the PBIP on disk + reflects the current model. Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, measures, relationships, pages, and visuals. 2. **Parse both layers.** - **Semantic model (`.tmdl`):** names, data types, `description`, DAX `expressions`, and diff --git a/.github/prompts/plan.prompt.md b/.github/prompts/plan.prompt.md index c0fe0e3..9efa673 100644 --- a/.github/prompts/plan.prompt.md +++ b/.github/prompts/plan.prompt.md @@ -1,6 +1,6 @@ --- -description: Draft an implementation plan for a data task — a plan file only, no code. name: plan +description: Draft an implementation plan for a data task — a plan file only, no code. --- **Context:** ${selection} @@ -16,12 +16,16 @@ it does not write code. 1. **Understand.** Read the relevant parts of the repo. If a key requirement is genuinely ambiguous, ask; otherwise state your assumptions. -2. **Route.** Identify which agent(s) own each part of the work (e.g. `data-engineer` for the - pipeline, `analytics-engineer` for the model, `powerbi-developer` for the dashboard). +2. **Map to skills and scripts.** For each part of the work, name the skills it will load and + the validation scripts it will run (e.g. `pbi-semantic-layer-tmdl` + `run_bpa.py` for a model + change, `pbi-report-layer-pbir` + `validate_pbir.py` for a report change, + `dbt-modeling-conventions` for dbt models). 3. **Write the plan** to `docs/PLAN-<slug>.md`, where `<slug>` is 2–3 hyphenated keywords from the request. Include: - **Goal & scope** (and explicit non-goals). - - **Task breakdown** with the responsible agent per task. - - **Dependencies & risks.** - - **Verification checklist** (how we'll know it works). + - **Task breakdown** with the skills/scripts each task uses. + - **Dependencies & risks.** For Power BI work, state whether Desktop will be open (Modeling + MCP + Desktop Bridge) or closed (edit on disk). + - **Verification checklist** (how we'll know it works — scripts to run, and the independent + visual review for report changes). 4. **Report** the exact file path created and suggest reviewing it before implementing. diff --git a/.github/prompts/validate-pbi.prompt.md b/.github/prompts/validate-pbi.prompt.md index b3103a5..9f230e7 100644 --- a/.github/prompts/validate-pbi.prompt.md +++ b/.github/prompts/validate-pbi.prompt.md @@ -1,7 +1,7 @@ --- -description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 - + structural review). name: validate-pbi +description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 + or the Python fallback, PBIR validation, structural review). --- **Context:** ${selection} @@ -10,40 +10,44 @@ name: validate-pbi **Usage:** `/validate-pbi [path/to/project.pbip]` -Answers "is my model healthy?" by running the quality checks on a PBIP semantic model. If no -path is given, use the project in the current folder; if there are several, ask which one. +Answers "is my Power BI project healthy?". If no path is given, use the project in the current +folder; if there are several, ask which one. ## Steps -1. **Locate the project.** Use `pbi-semantic-layer-tmdl` to confirm a valid PBIP project. - If none is found, report it and stop. -2. **Run the BPA.** Use `pbi-quality-rules` to run the Best Practice Analyzer (Tabular Editor 2 - CLI, or the TMDL-parsing fallback). Recommend the user close Desktop first so disk is current. - Record every `error` and `warning`. -3. **Structural review.** - - **Doc coverage:** tables/measures with a `description` vs. total. - - **Relationships:** flag island tables (no relationships), bidirectional filters, and - many-to-many joins. -4. **Report** — a short Markdown summary: +1. **Locate the project** (`.pbip` + `<Name>.SemanticModel/` + `<Name>.Report/`). If none is + found, report it and stop. +2. **Make disk current.** If Power BI Desktop has the project open, ask the user to save + (Ctrl+S). Desktop does not need to be closed — the checks read the files Desktop saved. +3. **Model — BPA** (`pbi-quality-rules`): + `python3 <pbi-quality-rules>/scripts/run_bpa.py <project> --format json`. + Record every `error` and `warning` and which engine ran (Tabular Editor 2 or Python fallback). +4. **Report — PBIR** (`pbi-report-layer-pbir`): + `python3 <pbi-report-layer-pbir>/scripts/validate_pbir.py <project> --format json`. + Record errors and warnings and whether `powerbi-report-author` or the structural fallback ran. +5. **Structural review** of the TMDL: + - **Doc coverage:** tables/measures with a `///` description vs. total. + - **Relationships:** island tables, bidirectional filters, many-to-many joins. +6. **Report** — a short Markdown summary: ```markdown # Validation Report — <Project> -**Date:** <today> | **Source:** TMDL files +**Date:** <today> | **BPA engine:** Tabular Editor 2 | **PBIR check:** powerbi-report-author ## Summary -| Category | Errors | Warnings | -|----------|--------|----------| -| Naming | 0 | 3 | -| DAX | 1 | 2 | -| Model | 0 | 1 | -| Docs | 0 | 8 | +| Area | Errors | Warnings | +|------|--------|----------| +| Model — DAX | 1 | 1 | +| Model — Naming | 0 | 2 | +| Model — Docs | 0 | 2 | +| Report (PBIR) | 0 | 0 | ## Errors (fix before publishing) -- [DAX_EMPTY_EXPRESSION] Measure "Turnover %" — empty expression +- [DAX_EMPTY_EXPRESSION] Measure 'Sales'[Forecast] — empty expression ## Doc coverage -- Tables: 5/7 described (71%) · Measures: 12/20 described (60%) +- Tables: 3/4 described (75%) · Measures: 3/4 described (75%) ``` -5. **Offer fixes.** Auto-offer to fix all **errors**; ask whether to address **warnings** and - missing descriptions. +7. **Offer fixes.** Offer to fix every **error** (model fixes through the Modeling MCP when + Desktop is open, otherwise in TMDL); ask before addressing warnings and missing descriptions. diff --git a/.github/skills/building-html-presentations/SKILL.md b/.github/skills/building-html-presentations/SKILL.md index d6bfa34..3f5d505 100644 --- a/.github/skills/building-html-presentations/SKILL.md +++ b/.github/skills/building-html-presentations/SKILL.md @@ -26,13 +26,15 @@ deliverable is fundamentally a document (Markdown-only, clean PDF/PPTX, no inter **Slidev** is a fine choice for fully technical audiences already on Node/Vue — mention it, don't default to it. -## Workflow -1. **Audience & format.** Exec → fewer slides, takeaway titles, no code. Technical → allow - detail/appendix. Choose deck vs site vs PDF (table above) and say which. -2. **Outline before slides.** Agree the storyline (5–9 beats) before writing HTML. Lead with - the conclusion (BLUF), then evidence, then "so what / next". -3. **Apply the brand.** If a `DESIGN.md` exists, load `applying-visual-identity` and map its - tokens to CSS variables (see that skill). Otherwise use the neutral defaults in the starter. +## Working order +1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load + `applying-visual-identity` and map its tokens to CSS variables. If absent, offer to scaffold + one (don't invent a brand silently) and use the neutral defaults in the starter meanwhile. +2. **Pick the format** from the table above and say which and why. A native `.pptx` that must be + editable in PowerPoint → `generating-pptx`. +3. **Story before charts.** Know the audience (exec → fewer slides, takeaway titles, no code; + technical → allow detail/appendix). Agree the storyline (5–9 beats) before writing HTML: lead + with the conclusion (BLUF), then evidence, then "so what / next". Use `data-story-outline.md`. 4. **Build from a starter** (`templates/`), drop in real numbers and charts, keep it **self-contained** (works offline; pin CDN versions or vendor the files). 5. **Export** if needed (see the export note below). @@ -93,7 +95,13 @@ All HTML starters use a generic **sales review** as demo content — replace it - Every data slide states **source, grain, date range, filters**. Never invent numbers — mark unknowns `TODO`. +## Data storytelling rules +- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". +- **One idea per slide.** Move supporting detail to an appendix section. +- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. +- **Never fabricate data.** Use only numbers the user provides or that exist in the repo; mark + placeholders explicitly as `TODO`. + ## Rules -- One idea per slide; the title is the **insight**, not the field name. - Keep it self-contained and offline-capable; pin versions, don't rely on live network at present time. - Accessibility: real text (not images of text), sufficient contrast, logical heading order. diff --git a/.github/skills/building-html-presentations/templates/charts-reference.html b/.github/skills/building-html-presentations/templates/charts-reference.html index 310a9b7..ded2df6 100644 --- a/.github/skills/building-html-presentations/templates/charts-reference.html +++ b/.github/skills/building-html-presentations/templates/charts-reference.html @@ -3,8 +3,8 @@ ┌──────────────────────────────────────────────────────────────────────────┐ │ CHART COOKBOOK — the palette of chart types for the HTML starters. │ │ │ - │ A live gallery of ECharts patterns (+ CSS-only patterns) so the agent and │ - │ the user can see what's possible and copy a config into any deck/site/PDF │ + │ A live gallery of ECharts patterns (+ CSS-only patterns) so you can see │ + │ what's possible and copy a config into any deck/site/PDF │ │ template. Each card names the chart and when to use it. Same brand tokens │ │ as the other starters (map from DESIGN.md). Pinned ECharts. │ │ │ diff --git a/.github/skills/creating-agents-and-skills/SKILL.md b/.github/skills/creating-agents-and-skills/SKILL.md index f074ab3..19d871c 100644 --- a/.github/skills/creating-agents-and-skills/SKILL.md +++ b/.github/skills/creating-agents-and-skills/SKILL.md @@ -1,6 +1,6 @@ --- name: creating-agents-and-skills -description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, AGENTS.md) identical in each tool's native format. +description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, .cursor, AGENTS.md) identical in each tool's native format. --- # Skill: creating-agents-and-skills @@ -18,18 +18,27 @@ Run these gates before creating anything. If a gate says stop, explain why and s - **LLM-native? → don't create.** If a capable model already does this well ("write clean code", "use CTEs", "explain this function"), no artifact is needed. Just do the task. -- **Narrow knowledge / a tool / a format / a checklist? → make a SKILL.** Most additions are - skills: a team naming standard, a framework's rules, a validation procedure. Skills - auto-activate by description and cost almost nothing until used. -- **A distinct role with its own domain and judgement? → make an AGENT.** Only when there's a - persona that owns a body of work (like `data-engineer`). Agents are expensive (routing, - attention); the bar is high. -- **Overlaps an existing agent/skill? → extend it,** don't add a near-duplicate. Ambiguous - routing between two similar artifacts degrades results. +- **Narrow knowledge / a tool / a format / a checklist / a script? → make a SKILL.** This is + the default for almost everything: a team naming standard, a framework's rules, a validation + procedure. Skills auto-activate by description and cost almost nothing until used. - **A repeatable multi-step procedure the user invokes? → make a WORKFLOW** (slash command). +- **Overlaps an existing skill/agent? → extend it,** don't add a near-duplicate. -> Default to a skill. Recommend an agent only if you can name the distinct domain it owns and -> confirm nothing existing covers it. +### The agent-survival rule + +An **agent** is allowed only if it imposes something the model cannot impose on itself: + +1. **Restricted tools** — a frontmatter `tools` list that removes capabilities (e.g. read-only). +2. **Independent review** — fresh context, without the author's reasoning. +3. **A different model** than the main session. +4. **An automatic trigger** that fires without the user asking. + +If none applies, it is a skill: "a persona with a domain" is not a reason. Domain knowledge +belongs in a skill, or is dropped if the model already knows it. Agents must not hand off to +other agents. + +> Default to a skill. Recommend an agent only if you can name which of the four criteria it +> satisfies, and say so in the agent's description or body. --- @@ -37,18 +46,18 @@ Run these gates before creating anything. If a gate says stop, explain why and s Write only domain/org-specific signal. No filler, no restating what the model knows. English. -**Agent** — `.agent/agents/<name>.md`: +**Agent** — `.agent/agents/<name>.md` (example: `pbi-visual-reviewer`, a read-only reviewer): ```yaml --- -name: <lowercase-hyphenated> -description: One sharp sentence on what it does + trigger keywords (drives routing). -tools: Read, Grep, Glob, Bash, Edit, Write +name: pbi-visual-reviewer +description: Independently reviews Power BI report screenshots against the request and the PBIR folder; PASS/FAIL per page and visual. Use after a report change has been reloaded and screenshotted. +tools: Read, Grep, Glob # the restriction that justifies the agent: no Bash/Edit/Write model: inherit -skills: skill-a, skill-b # optional hints; skills also self-activate --- -# <Name> -<1–2 line identity> → domain rules → handoffs to other agents → out-of-scope. +# PBI Visual Reviewer +<1–2 line identity> → fixed checklist → output format (PASS/FAIL). No handoffs. ``` +`tools` is the field that matters: list only what the agent needs. **Skill** — `.agent/skills/<name>/SKILL.md` (gerund-style name, e.g. `validating-dbt-models`): ```yaml @@ -59,7 +68,7 @@ description: What it does AND when to use it, in the third person, with trigger --- # Skill: <name> <when/why> → concrete rules or steps. Keep the body under ~500 lines; move long reference -material to sibling files (e.g. reference.md, scripts/) and link them. +material to sibling files (e.g. references/, scripts/) and link them. ``` Write descriptions in the **third person** ("Generates...", "Use when...") — never "I can help". The description is what makes the skill fire without the user naming it. @@ -76,7 +85,8 @@ format. Pick the path that matches the project: ### Path A — Source of truth present (`.agent/` + `scripts/sync_agents.py` exist) This is the kit repo or an Antigravity install. **Author in `.agent/` only, then sync:** ```bash -python scripts/sync_agents.py +python3 scripts/sync_agents.py # regenerate +python3 scripts/sync_agents.py --check # exits 1 if generated folders are stale (CI runs this) ``` The script regenerates `.claude/`, `.github/`, `.opencode/`, `.cursor/`, and root `AGENTS.md` (which also covers OpenAI Codex). Never hand-edit those — they're overwritten. Done. @@ -87,8 +97,8 @@ format (the body content stays identical; only frontmatter changes). Format mapp | Concept | `.agent` (Antigravity) | `.claude` (Claude Code) | `.github` (Copilot) | `.opencode` (OpenCode) | `.cursor` (Cursor) | |---|---|---|---|---|---| -| Agent | `agents/<n>.md` | `agents/<n>.md` (identical) | `agents/<n>.agent.md` — frontmatter `name`, `description` only (no `tools`) | `agents/<n>.md` — frontmatter `description`, `mode: subagent`, `permission` (drop `tools`/`model`/`name`/`skills`) | — (Cursor has no file-based agents) | -| Skill | `skills/<n>/SKILL.md` | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | — (no skills concept) | +| Agent | `agents/<n>.md` | `agents/<n>.md` (identical) | `agents/<n>.agent.md` — frontmatter `name`, `description`, `tools` mapped from Claude names (Read→`read`, Grep/Glob→`search`, Edit/Write→`edit`, Bash→`execute`) | `agents/<n>.md` — frontmatter `description`, `mode: subagent`, `permission` (deny edit/bash when the agent is read-only; drop `tools`/`model`/`name`/`skills`) | — (none; run the task in a fresh chat) | +| Skill | `skills/<n>/SKILL.md` | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | | Workflow | `workflows/<n>.md` | `commands/<n>.md` (native slash command) + `workflows/<n>.md` | `prompts/<n>.prompt.md` — frontmatter `name`, `description`; body starts `**Context:** ${selection}` | `commands/<n>.md` (identical) | `commands/<n>.md` (identical) | | Global rule | `rules/rules.md` (with `trigger: always_on`) | `CLAUDE.md` (rules + commands section) | `copilot-instructions.md` (rules) | root `AGENTS.md` (rules) | `rules/<n>.mdc` — frontmatter `description`, `globs`, `alwaysApply: true` | @@ -104,11 +114,14 @@ Rules: ### Verify parity After either path, confirm the artifact exists in every present tool folder and that no folder -references something that no longer exists. State which folders you wrote. +references something that no longer exists. In Path A, `python3 scripts/sync_agents.py --check` +does this. State which folders you wrote. --- ## Anti-patterns -- Creating an agent for a single tool or task (e.g. "GitAgent"). Tools don't need agents. +- Creating an agent for a persona, a single tool, or a single task (e.g. "GitAgent") — it fails + the survival rule; write a skill. +- Chaining agents through handoffs. - Putting episodic guidance in the always-on rules — it belongs in a skill like this one. - Editing generated folders (`.claude`/`.github`/`.opencode`/`.cursor`/`AGENTS.md`) when `.agent/` exists. diff --git a/.github/skills/dbt-modeling-conventions/SKILL.md b/.github/skills/dbt-modeling-conventions/SKILL.md new file mode 100644 index 0000000..57446ce --- /dev/null +++ b/.github/skills/dbt-modeling-conventions/SKILL.md @@ -0,0 +1,63 @@ +--- +name: dbt-modeling-conventions +description: Applies the organization's dbt modeling conventions. Use when the user writes, reviews, or renames dbt models, mentions dbt, staging, marts, fct_ or dim_ tables, schema.yml, surrogate keys, model naming, CTE structure, or dimensional modeling (star schema) with dbt. +--- + +# Skill: dbt-modeling-conventions + +Org conventions only — the model already knows generic SQL and dbt. Apply these to every model +you write or review. + +## CTE structure +- `import` CTEs on top (one per `ref()`/`source()`, `select` only the needed columns). +- `logical` CTEs in the middle (joins, filters, calculations — one step each). +- One `final` CTE at the bottom; the model ends with `select * from final`. + +## Naming +| Prefix | Layer | +|---|---| +| `stg_` | Staging: 1:1 with a source table, rename/cast only | +| `dim_` | Dimension | +| `fct_` | Fact | +| `mart_` | Business-facing mart built on dims/facts | + +## Keys and grain +- Every dimension has a surrogate key `sk_<entity>`: a hash of the natural key. +- One grain per fact table; state it in the model description ("one row per order line"). +- Build dimensions before facts. + +## Tests and docs +Every model has a `schema.yml` entry with a description and `unique` + `not_null` on its +primary key. + +## Example + +```sql +-- models/marts/dim_customer.sql +with stg_customers as ( + select * from {{ ref('stg_crm__customers') }} +), +customers_keyed as ( + select + {{ dbt_utils.generate_surrogate_key(['customer_id']) }} as sk_customer, + customer_id, + customer_name, + country + from stg_customers +), +final as ( + select * from customers_keyed +) +select * from final +``` + +```yaml +# models/marts/schema.yml +models: + - name: dim_customer + description: One row per customer. + columns: + - name: sk_customer + description: Surrogate key, hash of customer_id. + tests: [unique, not_null] +``` diff --git a/.github/skills/pbi-quality-rules/SKILL.md b/.github/skills/pbi-quality-rules/SKILL.md index 1a8c059..775db7b 100644 --- a/.github/skills/pbi-quality-rules/SKILL.md +++ b/.github/skills/pbi-quality-rules/SKILL.md @@ -1,42 +1,44 @@ --- name: pbi-quality-rules -description: Validate a Power BI semantic model with the real Best Practice Analyzer (BPA) via the free Tabular Editor 2 CLI, using a shareable BPA rules file. +description: Validate a Power BI semantic model with the Best Practice Analyzer (BPA) — the real engine via the free Tabular Editor 2 CLI, or a Python fallback over TMDL when TE2 is not installed — using the shareable bpa-rules.json. Use when asked to lint, audit, check or validate a model, after significant model changes, and before publishing. --- # Skill: pbi-quality-rules -Run the **Best Practice Analyzer (BPA)** — the industry-standard model linter — against a PBIP -semantic model. This skill uses the *real* BPA engine from **Tabular Editor 2 (free, open -source)** instead of a homegrown checker, so results match what Power BI developers expect and -the rules are portable. +Two scripts in this skill's `scripts/` folder (Python 3.9+, standard library only): -## Rule set -`bpa-rules.json` (next to this file) holds the team's BPA rules in Tabular Editor's native -format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in Dynamic LINQ over the -TOM). Severity: `3` = error, `2` = warning, `1` = info. Edit this file to add or tune rules; -it loads directly into Tabular Editor's BPA UI too. +| Script | What it does | +|---|---| +| `run_bpa.py <model>` | Runs the real BPA through **Tabular Editor 2** (`TabularEditor.exe <definition-folder> -A bpa-rules.json -V`). If TE2 is not found, it runs the fallback and says so. | +| `bpa_fallback.py <model>` | The same rules re-implemented in Python over the TMDL files (or `model.bim`). Approximate; rules it cannot evaluate are listed as `SKIPPED`, never as passed. | -## Primary path — Tabular Editor 2 CLI (recommended) -Requires Tabular Editor 2 (free). Desktop should be closed so disk reflects the latest state. +`<model>` = the `.pbip`, the project folder, `<Name>.SemanticModel/`, its `definition/` folder, +or a `model.bim`. Options: `--format json`, `--rules <file>`, and for `run_bpa.py` also +`--te <path>` and `--no-fallback`. TE2 is looked up via `--te`, `$TABULAR_EDITOR_PATH`, `PATH`, +then the default Windows install folders. +Exit codes: `0` no error-severity violations · `1` errors found · `2` bad path / TE2 failed. + +## Run it ```bash -# Windows (TabularEditor.exe on PATH). Point at the .SemanticModel folder (TMDL) or model.bim. -TabularEditor.exe "Sales.SemanticModel" -A ".agent/skills/pbi-quality-rules/bpa-rules.json" -V +python3 <skill>/scripts/run_bpa.py path/to/Sales.pbip ``` - -- `-A <rulesfile>` runs the BPA with the given rules; `-V` writes violations to the console - (and sets a non-zero exit code on error-severity hits — useful in CI). -- The cross-platform `te` CLI (preview) accepts the same rules file if installed instead. - -Parse the console output, then present **errors and warnings only** (omit info unless asked). - -## Fallback — no Tabular Editor installed -The kit stays usable without the tool. Parse the TMDL files directly and evaluate the same -rules by hand: for each rule, read its `Scope` and `Expression` from `bpa-rules.json` and apply -the equivalent check to the parsed objects (measures, tables, columns, relationships). This is -less robust than the real BPA — say so, and recommend installing Tabular Editor 2 for accuracy. - -## Agent flow -1. Run after large model edits / refactors, or on demand. -2. Show errors first; for each `error`, offer an immediate fix. For `warning`, suggest but - don't force. Group by category so the report is scannable. +1. **Disk is the input.** With Desktop open and MCP edits pending, ask the user to save (Ctrl+S) + first — the BPA reads the TMDL Desktop wrote, not the live model. +2. Present **errors first**, then warnings, grouped by category; omit info unless asked. Say + which engine ran; for the fallback, recommend installing Tabular Editor 2 for exact results. +3. For each `error`, offer a fix (through the Modeling MCP when Desktop is open — see + `pbi-semantic-layer-tmdl`). Warnings: suggest, don't force. + +## Rule set — `bpa-rules.json` +Tabular Editor's native format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in +Dynamic LINQ over the TOM). Severity `3` error · `2` warning · `1` info. It loads directly into +TE2's BPA window too. + +Adding a rule: add it to `bpa-rules.json` (TE2 picks it up at once), then add a predicate with the +same `ID` to `CHECKS` in `scripts/bpa_fallback.py` so the fallback covers it; until then the +fallback reports it as `SKIPPED`. + +## In CI +Run `run_bpa.py --no-fallback --format json` on a Windows runner with TE2 (fails on exit 1), or +`bpa_fallback.py --format json` anywhere for a lighter gate. diff --git a/.github/skills/pbi-quality-rules/scripts/bpa_fallback.py b/.github/skills/pbi-quality-rules/scripts/bpa_fallback.py new file mode 100755 index 0000000..36f575b --- /dev/null +++ b/.github/skills/pbi-quality-rules/scripts/bpa_fallback.py @@ -0,0 +1,522 @@ +#!/usr/bin/env python3 +"""Best Practice Analyzer fallback: the rules of bpa-rules.json, evaluated in Python. + +Use this only when Tabular Editor 2 is not available (run_bpa.py calls it automatically). +It reads the semantic model straight from disk — TMDL (PBIP `definition/` folder) or a +`model.bim` — and re-implements each shipped rule by its ID. It is an approximation of the +real BPA engine: rules without a Python implementation are reported as skipped, never as passed. + +Usage: + python3 bpa_fallback.py <model> [--rules bpa-rules.json] [--format text|json] + +<model> may be a .pbip file, a project folder, a <Name>.SemanticModel folder, its +definition/ folder, or a model.bim file. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations, 2 = usage error. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from dataclasses import dataclass, field +from pathlib import Path +from typing import Callable, Optional + +DEFAULT_RULES = Path(__file__).resolve().parent.parent / "bpa-rules.json" +SEVERITY_NAMES = {1: "info", 2: "warning", 3: "error"} +AUTO_DATE_PREFIXES = ("LocalDateTable_", "DateTableTemplate_") + + +class UsageError(Exception): + """Bad input path or file; maps to exit code 2.""" + + +# --------------------------------------------------------------------------- +# Model +# --------------------------------------------------------------------------- +@dataclass +class Measure: + table: str + name: str + expression: str = "" + description: str = "" + display_folder: str = "" + is_hidden: bool = False + + +@dataclass +class Column: + table: str + name: str + description: str = "" + is_hidden: bool = False + + +@dataclass +class Table: + name: str + description: str = "" + is_hidden: bool = False + measures: list = field(default_factory=list) + columns: list = field(default_factory=list) + + +@dataclass +class Relationship: + name: str + from_table: str = "" + from_column: str = "" + to_table: str = "" + to_column: str = "" + cross_filtering: str = "oneDirection" + + +@dataclass +class Model: + source: Path + kind: str # "tmdl" | "bim" + tables: list = field(default_factory=list) + relationships: list = field(default_factory=list) + + @property + def measures(self): + return [m for t in self.tables for m in t.measures] + + @property + def columns(self): + return [c for t in self.tables for c in t.columns] + + +# --------------------------------------------------------------------------- +# Locating the model on disk +# --------------------------------------------------------------------------- +def _is_tmdl_folder(path: Path) -> bool: + return path.is_dir() and any((path / n).is_file() for n in ("model.tmdl", "database.tmdl")) + + +def _from_semantic_model_dir(path: Path) -> Optional[tuple]: + if _is_tmdl_folder(path / "definition"): + return "tmdl", path / "definition" + if (path / "model.bim").is_file(): + return "bim", path / "model.bim" + return None + + +def _semantic_model_from_pbip(pbip: Path) -> Optional[Path]: + sibling = pbip.with_name(pbip.stem + ".SemanticModel") + if sibling.is_dir(): + return sibling + try: + data = json.loads(pbip.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + return None + for artifact in data.get("artifacts", []): + report = artifact.get("report", {}).get("path") + if not report: + continue + pbir = pbip.parent / report / "definition.pbir" + try: + ref = json.loads(pbir.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + continue + by_path = ref.get("datasetReference", {}).get("byPath", {}).get("path") + if by_path: + candidate = (pbir.parent / by_path).resolve() + if candidate.is_dir(): + return candidate + return None + + +def resolve_model(path: Path) -> tuple: + """Return (kind, path) where kind is 'tmdl' (definition folder) or 'bim' (model.bim).""" + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() == ".bim": + return "bim", path + if path.suffix.lower() == ".pbip": + sm = _semantic_model_from_pbip(path) + found = _from_semantic_model_dir(sm) if sm else None + if found: + return found + raise UsageError(f"no semantic model found for {path.name} (thin report?)") + if path.name.lower() == "definition.pbism": + return resolve_model(path.parent) + raise UsageError(f"unsupported file: {path.name} (use .pbip, model.bim or a folder)") + if _is_tmdl_folder(path): + return "tmdl", path + found = _from_semantic_model_dir(path) + if found: + return found + models = sorted(p for p in path.glob("*.SemanticModel") if p.is_dir()) + if len(models) == 1: + found = _from_semantic_model_dir(models[0]) + if found: + return found + if len(models) > 1: + names = ", ".join(p.name for p in models) + raise UsageError(f"several semantic models in {path}: {names}; pass one explicitly") + raise UsageError(f"no TMDL definition or model.bim found under {path}") + + +# --------------------------------------------------------------------------- +# TMDL parser (the subset the rules need) +# --------------------------------------------------------------------------- +_PROPERTY = re.compile(r"^([A-Za-z][A-Za-z0-9_]*)\s*(?::\s*(.*))?$") + + +def _depth(line: str) -> int: + expanded = line[: len(line) - len(line.lstrip())].replace("\t", " ") + return len(expanded) // 4 + + +def _read_name(text: str) -> tuple: + """Read one TMDL object name from the start of `text`. Returns (name, rest).""" + text = text.lstrip() + if text.startswith("'"): + out, i = [], 1 + while i < len(text): + if text[i] == "'": + if i + 1 < len(text) and text[i + 1] == "'": + out.append("'") + i += 2 + continue + return "".join(out), text[i + 1 :] + out.append(text[i]) + i += 1 + return "".join(out), "" + match = re.match(r"[^\s=.]+", text) + if not match: + return "", text + return match.group(0), text[match.end() :] + + +def _split_column_ref(ref: str) -> tuple: + """`Sales.DateKey` / `'Sales Table'.'Date Key'` -> (table, column).""" + table, rest = _read_name(ref) + rest = rest.lstrip() + if rest.startswith("."): + column, _ = _read_name(rest[1:]) + return table, column + return "", table + + +def _unquote_value(value: str) -> str: + value = value.strip() + if len(value) >= 2 and value[0] == value[-1] == '"': + return value[1:-1].replace('""', '"') + return value + + +def parse_tmdl_folder(folder: Path) -> Model: + model = Model(source=folder, kind="tmdl") + tables: dict = {} + for file in sorted(folder.rglob("*.tmdl")): + _parse_tmdl_file(file.read_text(encoding="utf-8-sig"), model, tables) + model.tables = list(tables.values()) + return model + + +def _parse_tmdl_file(text: str, model: Model, tables: dict) -> None: + table: Optional[Table] = None + obj = None # Measure | Column | Relationship | Table (for table-level properties) + obj_depth = 0 + other_depth: Optional[int] = None # inside an object we do not model (partition, ...) + expr_lines: Optional[list] = None + expr_fenced = False + pending_doc: list = [] + + def finish_expression(): + nonlocal expr_lines + if expr_lines is not None and isinstance(obj, Measure): + obj.expression = "\n".join(expr_lines).strip() + expr_lines = None + + for raw in text.splitlines(): + stripped = raw.strip() + depth = _depth(raw) + + if expr_lines is not None: + if expr_fenced: + if stripped.startswith("```"): + finish_expression() + else: + expr_lines.append(stripped) + continue + if not stripped or depth > obj_depth + 1: + expr_lines.append(stripped) + continue + finish_expression() + + if not stripped: + continue + if stripped.startswith("///"): + pending_doc.append(stripped[3:].strip()) + continue + if stripped.startswith("//"): + continue + + if other_depth is not None: + if depth > other_depth: + continue + other_depth = None + + doc = " ".join(pending_doc).strip() + pending_doc = [] + keyword, _, rest = stripped.partition(" ") + + if depth == 0: + obj = None + table = None + if keyword == "table": + name, _ = _read_name(rest) + table = tables.setdefault(name, Table(name=name)) + if doc: + table.description = doc + obj, obj_depth = table, 0 + elif keyword == "relationship": + name, _ = _read_name(rest) + obj, obj_depth = Relationship(name=name), 0 + model.relationships.append(obj) + else: + other_depth = 0 + continue + + if table is not None and depth == 1 and keyword in ("measure", "column"): + name, after = _read_name(rest) + after = after.strip() + if keyword == "measure": + obj = Measure(table=table.name, name=name, description=doc) + table.measures.append(obj) + else: + obj = Column(table=table.name, name=name, description=doc) + table.columns.append(obj) + obj_depth = 1 + if after.startswith("="): + inline = after[1:].strip() + if inline.startswith("```"): + expr_lines, expr_fenced = [], True + elif inline: + if isinstance(obj, Measure): + obj.expression = inline + else: + expr_lines, expr_fenced = [], False + continue + + if obj is not None and depth == obj_depth + 1: + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(obj, match.group(1), match.group(2)) + continue + + if table is not None and depth == 1: + # Table-level property after members (e.g. `isHidden`, `lineageTag: …`). + obj, obj_depth = table, 0 + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(table, match.group(1), match.group(2)) + continue + + # Any other nested object (partition, hierarchy, annotation, ...): skip its body. + other_depth = depth + + finish_expression() + + +def _set_property(obj, key: str, value: Optional[str]) -> None: + flag = value is None or value.strip().lower() == "true" + if key == "isHidden" and hasattr(obj, "is_hidden"): + obj.is_hidden = flag + elif key == "description" and hasattr(obj, "description") and value is not None: + obj.description = _unquote_value(value) + elif key == "displayFolder" and isinstance(obj, Measure) and value is not None: + obj.display_folder = _unquote_value(value) + elif isinstance(obj, Relationship) and value is not None: + if key == "fromColumn": + obj.from_table, obj.from_column = _split_column_ref(value) + elif key == "toColumn": + obj.to_table, obj.to_column = _split_column_ref(value) + elif key == "crossFilteringBehavior": + obj.cross_filtering = value.strip() + + +# --------------------------------------------------------------------------- +# model.bim (TMSL JSON) +# --------------------------------------------------------------------------- +def _text(value) -> str: + if isinstance(value, list): + return "\n".join(str(v) for v in value) + return str(value or "") + + +def parse_bim(path: Path) -> Model: + try: + data = json.loads(path.read_text(encoding="utf-8-sig")) + except ValueError as exc: + raise UsageError(f"{path} is not valid JSON: {exc}") from exc + raw_model = data.get("model", data) + model = Model(source=path, kind="bim") + for t in raw_model.get("tables", []): + table = Table(name=t.get("name", ""), description=_text(t.get("description")), + is_hidden=bool(t.get("isHidden"))) + for m in t.get("measures", []): + table.measures.append(Measure( + table=table.name, name=m.get("name", ""), expression=_text(m.get("expression")), + description=_text(m.get("description")), display_folder=m.get("displayFolder", "") or "", + is_hidden=bool(m.get("isHidden")))) + for c in t.get("columns", []): + if c.get("type") == "rowNumber": + continue + table.columns.append(Column(table=table.name, name=c.get("name", ""), + description=_text(c.get("description")), + is_hidden=bool(c.get("isHidden")))) + model.tables.append(table) + for r in raw_model.get("relationships", []): + model.relationships.append(Relationship( + name=r.get("name", ""), from_table=r.get("fromTable", ""), from_column=r.get("fromColumn", ""), + to_table=r.get("toTable", ""), to_column=r.get("toColumn", ""), + cross_filtering=r.get("crossFilteringBehavior", "oneDirection"))) + return model + + +def load_model(path: Path) -> Model: + kind, resolved = resolve_model(path) + return parse_tmdl_folder(resolved) if kind == "tmdl" else parse_bim(resolved) + + +# --------------------------------------------------------------------------- +# Rules — one Python predicate per rule ID in bpa-rules.json +# --------------------------------------------------------------------------- +def _blank(text: str) -> bool: + return not (text or "").strip() + + +def _tables_in_relationships(model: Model) -> set: + return {t for r in model.relationships for t in (r.from_table, r.to_table)} + + +# rule ID -> (scope, predicate(obj, model) -> True when the object VIOLATES the rule) +CHECKS: dict = { + "NAMING_MEASURE_PASCAL": ("Measure", lambda m, _: bool(re.match(r"^[a-z]", m.name))), + "NAMING_MEASURE_DISPLAYFOLDER": ("Measure", lambda m, _: not m.display_folder), + "DAX_EMPTY_EXPRESSION": ("Measure", lambda m, _: _blank(m.expression) + or m.expression.replace(" ", "").upper() == "BLANK()"), + "DAX_IFERROR_USAGE": ("Measure", lambda m, _: "IFERROR" in m.expression.upper()), + "MODEL_BIDIRECTIONAL": ("Relationship", lambda r, _: r.cross_filtering.lower() == "bothdirections"), + "MODEL_ISLAND_TABLE": ("Table", lambda t, model: t.name not in _tables_in_relationships(model) + and not t.name.startswith(AUTO_DATE_PREFIXES)), + "PERF_AUTO_DATETIME": ("Table", lambda t, _: t.name.startswith(AUTO_DATE_PREFIXES)), + "DOC_TABLE_DESCRIPTION": ("Table", lambda t, _: _blank(t.description)), + "DOC_MEASURE_DESCRIPTION": ("Measure", lambda m, _: _blank(m.description)), + "DOC_KEY_COLUMN_HIDDEN": ("Column", lambda c, _: bool(re.search(r"ID$", c.name)) and not c.is_hidden), +} + + +def _quote(name: str) -> str: + return "'" + name.replace("'", "''") + "'" + + +def object_label(obj) -> str: + if isinstance(obj, Table): + return _quote(obj.name) + if isinstance(obj, (Measure, Column)): + return f"{_quote(obj.table)}[{obj.name}]" + if isinstance(obj, Relationship): + return f"{_quote(obj.from_table)}[{obj.from_column}] -> {_quote(obj.to_table)}[{obj.to_column}]" + return str(obj) + + +def _objects(model: Model, scope: str) -> list: + return { + "Measure": model.measures, + "Column": model.columns, + "Table": model.tables, + "Relationship": model.relationships, + }[scope] + + +def load_rules(path: Path) -> list: + try: + rules = json.loads(Path(path).read_text(encoding="utf-8-sig")) + except (OSError, ValueError) as exc: + raise UsageError(f"cannot read rules file {path}: {exc}") from exc + if not isinstance(rules, list): + raise UsageError(f"rules file {path} must contain a JSON array") + return rules + + +def analyze(model: Model, rules: list) -> dict: + violations, skipped = [], [] + for rule in rules: + check: Optional[tuple] = CHECKS.get(rule.get("ID")) + if check is None: + skipped.append(rule.get("ID") or rule.get("Name")) + continue + scope, predicate = check + predicate: Callable + severity = SEVERITY_NAMES.get(int(rule.get("Severity", 1)), "info") + for obj in _objects(model, scope): + if predicate(obj, model): + violations.append({ + "rule_id": rule["ID"], + "rule_name": rule.get("Name", rule["ID"]), + "category": rule.get("Category", ""), + "severity": severity, + "object_type": scope, + "object": object_label(obj), + }) + order = {"error": 0, "warning": 1, "info": 2} + violations.sort(key=lambda v: (order[v["severity"]], v["category"], v["rule_id"], v["object"])) + return { + "engine": "python-fallback", + "model": str(model.source), + "violations": violations, + "skipped_rules": skipped, + "summary": {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")}, + } + + +def format_text(result: dict) -> str: + lines = [] + if result["engine"] == "python-fallback": + lines.append("BPA engine: Python fallback (approximate). Install Tabular Editor 2 for the real BPA.") + else: + lines.append("BPA engine: Tabular Editor 2") + lines.append(f"Model: {result['model']}") + for v in result["violations"]: + lines.append(f"{v['severity'].upper():8} [{v['rule_id']}] {v['object_type']} {v['object']} — {v['rule_name']}") + if not result["violations"]: + lines.append("No objects in violation of the rules.") + for rule_id in result.get("skipped_rules", []): + lines.append(f"SKIPPED [{rule_id}] no implementation in this engine") + s = result["summary"] + lines.append(f"Summary: {s['error']} error(s), {s['warning']} warning(s), {s['info']} info") + return "\n".join(lines) + + +def emit(result: dict, fmt: str) -> int: + print(json.dumps(result, indent=2) if fmt == "json" else format_text(result)) + return 1 if result["summary"]["error"] else 0 + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="BPA rules evaluated in Python over TMDL/model.bim.") + parser.add_argument("model", type=Path, help=".pbip, project folder, .SemanticModel, definition/ or model.bim") + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES, help="BPA rules JSON (default: bpa-rules.json)") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + try: + result = analyze(load_model(args.model), load_rules(args.rules)) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + return emit(result, args.format) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/skills/pbi-quality-rules/scripts/run_bpa.py b/.github/skills/pbi-quality-rules/scripts/run_bpa.py new file mode 100755 index 0000000..31e1232 --- /dev/null +++ b/.github/skills/pbi-quality-rules/scripts/run_bpa.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""Run the Best Practice Analyzer with Tabular Editor 2; fall back to Python when it is absent. + +Usage: + python3 run_bpa.py <model> [--rules bpa-rules.json] [--te PATH] [--format text|json] [--no-fallback] + +<model>: .pbip, project folder, <Name>.SemanticModel, its definition/ folder, or model.bim. + +Tabular Editor 2 is located in this order: --te, $TABULAR_EDITOR_PATH, TabularEditor(.exe) on +PATH, then the default Windows install folders. It runs as + + TabularEditor.exe <definition-folder|model.bim> -A <rules> -V + +(-A: analyze with this rules file only, plus rules embedded in the model; -V: Azure DevOps +logging lines, which carry the severity). Without TE2 the script runs bpa_fallback.py and says so. + +Disk is the input: with Power BI Desktop open, save first (Ctrl+S) so the TMDL is current. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations (or rule errors), +2 = usage error / Tabular Editor could not run. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +sys.dont_write_bytecode = True +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import bpa_fallback # noqa: E402 + +DEFAULT_RULES = bpa_fallback.DEFAULT_RULES +TE_TIMEOUT_SECONDS = 600 + +# "##vso[task.logissue type=warning;]Measure 'Sales'[x] violates rule "Rule name"" +_VSO = re.compile(r"^##vso\[task\.logissue type=(?P<level>error|warning)[^\]]*\](?P<text>.*)$") +_VIOLATION = re.compile(r'^(?P<type>.+?) (?P<object>(?:\'|\[).*) violates rule "(?P<rule>.*)"$') +_VIOLATION_LOOSE = re.compile(r'^(?P<type>\S+) (?P<object>.+?) violates rule "(?P<rule>.*)"$') +_RULE_ERROR = re.compile(r"^Error on rule '(?P<rule>.*?)': (?P<error>.*)$") + + +def find_tabular_editor(explicit: Optional[str] = None, env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + for candidate in (explicit, env.get("TABULAR_EDITOR_PATH")): + if candidate: + return candidate if Path(candidate).is_file() else None + for name in ("TabularEditor.exe", "TabularEditor"): + found = shutil.which(name) + if found: + return found + for base in (env.get("ProgramFiles(x86)"), env.get("ProgramFiles"), env.get("LOCALAPPDATA")): + if base: + exe = Path(base) / "Tabular Editor" / "TabularEditor.exe" + if exe.is_file(): + return str(exe) + return None + + +def build_command(te_path: str, model_target: Path, rules: Path) -> list: + return [te_path, str(model_target), "-A", str(rules), "-V"] + + +def parse_te_output(stdout: str, rules: list, model_label: str) -> dict: + """Turn TE2 console output (-A ... -V) into the same result shape as the fallback.""" + by_name = {r.get("Name"): r for r in rules} + violations, rule_errors = [], [] + for raw in stdout.splitlines(): + line = raw.strip() + if not line or line.startswith("##vso[task.complete"): + continue + level = "info" + match = _VSO.match(line) + if match: + level, line = match.group("level"), match.group("text").strip() + rule_error = _RULE_ERROR.match(line) + if rule_error: + rule_errors.append({"rule_name": rule_error.group("rule"), "error": rule_error.group("error")}) + continue + found = _VIOLATION.match(line) or _VIOLATION_LOOSE.match(line) + if not found: + continue + rule = by_name.get(found.group("rule"), {}) + violations.append({ + "rule_id": rule.get("ID", ""), + "rule_name": found.group("rule"), + "category": rule.get("Category", ""), + "severity": level, + "object_type": found.group("type"), + "object": found.group("object"), + }) + summary = {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")} + return { + "engine": "tabular-editor-2", + "model": model_label, + "violations": violations, + "rule_errors": rule_errors, + "skipped_rules": [], + "summary": summary, + } + + +def run_tabular_editor(te_path: str, model_target: Path, rules_path: Path, rules: list) -> dict: + cmd = build_command(te_path, model_target, rules_path) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=TE_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise bpa_fallback.UsageError(f"Tabular Editor failed to run ({' '.join(cmd)}): {exc}") from exc + result = parse_te_output(proc.stdout, rules, str(model_target)) + loaded = "Running Best Practice Analyzer" in proc.stdout + if proc.returncode != 0 and not loaded: + detail = (proc.stderr or proc.stdout).strip()[-2000:] + raise bpa_fallback.UsageError(f"Tabular Editor exited {proc.returncode} before the BPA ran:\n{detail}") + return result + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Run the BPA via Tabular Editor 2 (Python fallback if absent).") + parser.add_argument("model", type=Path) + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES) + parser.add_argument("--te", help="path to TabularEditor.exe (overrides $TABULAR_EDITOR_PATH)") + parser.add_argument("--format", choices=("text", "json"), default="text") + parser.add_argument("--no-fallback", action="store_true", help="fail (exit 2) instead of using the Python fallback") + args = parser.parse_args(argv) + + try: + rules = bpa_fallback.load_rules(args.rules) + kind, target = bpa_fallback.resolve_model(args.model) + te_path = find_tabular_editor(args.te) + if te_path: + result = run_tabular_editor(te_path, target, args.rules.resolve(), rules) + elif args.no_fallback or args.te: + raise bpa_fallback.UsageError("Tabular Editor 2 not found (use --te or $TABULAR_EDITOR_PATH)") + else: + print("Tabular Editor 2 not found; using the Python fallback.", file=sys.stderr) + parsed = bpa_fallback.parse_tmdl_folder(target) if kind == "tmdl" else bpa_fallback.parse_bim(target) + result = bpa_fallback.analyze(parsed, rules) + except bpa_fallback.UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + if args.format == "json": + print(json.dumps(result, indent=2)) + else: + print(bpa_fallback.format_text(result)) + for err in result.get("rule_errors", []): + print(f"RULE-ERR [{err['rule_name']}] {err['error']}") + has_errors = result["summary"]["error"] > 0 or bool(result.get("rule_errors")) + return 1 if has_errors else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/skills/pbi-report-layer-pbir/SKILL.md b/.github/skills/pbi-report-layer-pbir/SKILL.md index 28ebab9..898d4d7 100644 --- a/.github/skills/pbi-report-layer-pbir/SKILL.md +++ b/.github/skills/pbi-report-layer-pbir/SKILL.md @@ -1,62 +1,128 @@ --- name: pbi-report-layer-pbir -description: Edit the Power BI report layer as code — pages, visuals, bookmarks, and themes — by writing PBIR JSON files in a PBIP project (offline, Desktop closed). +description: Edit the Power BI report layer as code — pages, visuals, filters, bookmarks and themes in the PBIR files of a PBIP project — with Power BI Desktop OPEN, using the edit → validate → save → status → reload → screenshot → review loop (Desktop Bridge via powerbi-report-author). Also covers the Desktop-closed path. Use for any request to add, move, restyle or fix a report page or visual. --- # Skill: pbi-report-layer-pbir -The report layer (`.Report/`) is **PBIR** — report metadata decomposed into per-page and -per-visual JSON files, each with a public JSON schema. PBIR is becoming the default report -format (rolling out across Service and Desktop through 2026), and it is explicitly designed to -be edited by tools other than Power BI. +The report layer (`<Name>.Report/`) is **PBIR**: one JSON file per page and per visual, each +with a public JSON schema. Power BI Desktop can stay **open** while you edit it: the +[Desktop Bridge](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +lets a tool reload the files from disk and take screenshots of the rendered pages. -## Hard precondition -**Power BI Desktop must be CLOSED** while editing these `.json` files — live edits corrupt or -get overwritten by autosave. +Model objects (measures, columns, relationships) are NOT edited here — they go live through the +Modeling MCP (`pbi-semantic-layer-tmdl`). Create the measure first, then bind the visual to it. + +## Tools +- `powerbi-report-author` — Microsoft's report CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`, + Node 20+). Validates PBIR, describes visual roles/formatting, and drives Desktop through + `preview --host desktop`. It bundles the lower-level bridge CLI `powerbi-desktop` + (`status`, `reload`, `screenshot`); always go through `powerbi-report-author preview`, which + resolves the right Desktop window from the report folder. Details, errors and setup: + [references/desktop-bridge.md](references/desktop-bridge.md). +- `scripts/validate_pbir.py` (in this skill's folder) — runs `powerbi-report-author validate`, or + a structural check when the CLI is missing. +- `pbi-visual-reviewer` agent — independent screenshot review. + +## The loop (Desktop open) +`<folder>` = the project folder or the `.pbip` path; `<shots>` = an absolute folder OUTSIDE the +PBIP (e.g. `%TEMP%\pbir-review\<Name>`), so screenshots never land in git. + +1. **Read before writing.** `powerbi-report-author preview-pages <folder>` and + `preview-visuals <folder>` list pages and visuals with their file paths. For a new or changed + visual, `powerbi-report-author catalog describe <visualType>` gives the exact role names + (e.g. `cardVisual` uses `Data`, `clusteredColumnChart` uses `Category` + `Y`); + `formatting describe-object <visualType> <object>` gives property names. Do not guess them. +2. **Edit** the PBIR JSON (anatomy below). One logical batch at a time. +3. **Validate:** `python3 <skill>/scripts/validate_pbir.py <folder>`. Fix every error before + going on; read the warnings. +4. **Save gate.** Ask the user to save in Desktop (Ctrl+S) if they changed anything by hand, + then check: `powerbi-report-author preview <folder> --host desktop --status`. + Continue only when `data.details.hasUnsavedChanges` is exactly `false`. +5. **Reload:** `powerbi-report-author preview <folder> --host desktop --reload`. +6. **Screenshot every affected page:** + `powerbi-report-author preview <folder> --host desktop --screenshot "<shots>" --all-pages` + (or `--screenshot "<shots>\<page>.png" --page <pageId>`). A screenshot does not reload; always + reload first. +7. **Review:** hand the PNG paths, the user's request/spec and the `.Report` path — and nothing + about your reasoning — to the `pbi-visual-reviewer` agent. In tools without subagents + (Cursor, Antigravity), run its checklist in a fresh chat with only those inputs. +8. REJECTED → fix the listed files and repeat from step 3. APPROVED → report the result and let + the user save/commit. + +### Pitfalls (each one has lost real work) +- **Reload discards whatever Desktop has not saved.** That is why step 4 exists. If status says + `true`, `null`, or fails, stop and ask the user to save; never reload "just to see". +- **Reload covers the report only.** `--reload` does not re-read model files. Model changes go + through the Modeling MCP, which updates Desktop live. Use `--reload-with-model` only when TMDL + was edited on disk (Desktop-closed habits) — and it still does not refresh data. +- **An edited theme may not show after reload.** Desktop caches themes by file name. When you + change a custom theme, rename its JSON (e.g. `Brand-3f9a1c.json` → `Brand-7b2e04.json`) and + update both references in `report.json` (`themeCollection.customTheme.name` and the + `resourcePackages` item `name`/`path`), then validate and reload. +- A page added on disk is unknown to the open window until a reload; a screenshot of it fails + with `RENDER_FAILED`. +- Two Desktop windows with the same report → `HOST_UNAVAILABLE`. Ask the user to close one; + never close Desktop yourself. + +## Alternative: Desktop closed +Use it when there is no Bridge (Desktop older than the Bridge release, the setting +*File > Options > Security > Desktop Bridge* is off, or the machine is not Windows) and in CI. +1. Make sure Desktop is **closed** — an open Desktop overwrites the files on its next save. +2. Edit the PBIR, run `validate_pbir.py` (in CI: `--format json`, fail on exit code 1). +3. Ask the user to open the `.pbip` and check the pages; no screenshots, so ask for one or for + a description, and run the reviewer checklist on what you get. ## `.Report` structure (PBIR) ``` -<Project>.Report/ - definition.pbir ← pointer (model binding) +<Name>.Report/ + definition.pbir ← model binding (byPath ../<Name>.SemanticModel) definition/ - report.json ← master metadata incl. themeCollection + version.json + report.json ← themeCollection, resourcePackages, settings pages/ + pages.json ← pageOrder + activePageName <pageId>/ - page.json ← canvas size, background, displayName - visuals/<visualId>/visual.json ← one visual per file + page.json ← name (= folder), displayName, width, height + visuals/<visualId>/visual.json ← one visual per file; name = folder StaticResources/ - RegisteredResources/<id>/<id>.json ← custom themes (create new themes here) - SharedResources/BaseThemes/ ← Microsoft base themes — NEVER edit + RegisteredResources/ ← custom themes and images you add + SharedResources/BaseThemes/ ← Microsoft base themes — never edit ``` - -## Navigating -- **Pages:** scan `definition/pages/`; read each `page.json` `displayName` (the user-facing name). -- **Visuals:** files sit under GUID folders. To find "the bar chart", grep `visual.json` for - `"visualType": "barChart"`. +PBIR-Legacy (a single `report.json` at the `.Report` root) cannot be edited this way: ask the +user to save the report in PBIR format first. ## Visual anatomy (`visual.json`) ```json { - "name": "<new GUID if creating>", - "position": { "x": 10, "y": 20, "width": 400, "height": 300, "tabOrder": 1 }, + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/visualContainer/2.9.0/schema.json", + "name": "7b2d4f6a8c0e1a3b5d02", + "position": { "x": 40, "y": 240, "z": 2000, "width": 760, "height": 420, "tabOrder": 2000 }, "visual": { - "visualType": "barChart", - "query": { "queryState": { "Category": { "projections": [] }, "Y": { "projections": [] } } } + "visualType": "clusteredColumnChart", + "query": { "queryState": { + "Category": { "projections": [ { "field": { "Column": { "Expression": { "SourceRef": { "Entity": "Product" } }, "Property": "Category" } }, "queryRef": "Product.Category", "nativeQueryRef": "Category", "active": true } ] }, + "Y": { "projections": [ { "field": { "Measure": { "Expression": { "SourceRef": { "Entity": "Sales" } }, "Property": "Total Sales" } }, "queryRef": "Sales.Total Sales", "nativeQueryRef": "Total Sales" } ] } + } }, + "visualContainerObjects": { "title": [ { "properties": { + "show": { "expr": { "Literal": { "Value": "true" } } }, + "text": { "expr": { "Literal": { "Value": "'Sales by category'" } } } } } ] } } } ``` -Positioning is a pixel grid (x=0, y=0 top-left). Generate fresh GUIDs for any new component -(`[System.Guid]::NewGuid()` on Windows, or any UUID generator). +- New page/visual: fresh 20-hex-char `name`, folder with the same name; add new pages to + `pages.json` `pageOrder`. +- Position is in pixels from the page's top-left; keep `x + width ≤ page width` and + `y + height ≤ page height`. Align edges to a shared grid (e.g. multiples of 8) and keep equal gaps. +- Literal encoding: text `'quoted'`, numbers `12D`, booleans `true`. When unsure, + `powerbi-report-author expr encode --kind <type> <value>`. +- Avoid visuals Microsoft is retiring (Q&A, Bing Map, filled map). ## Themes -1. Create the theme JSON under `StaticResources/RegisteredResources/<guid>/<guid>.json`. -2. Register that `<guid>` in `report.json`'s `themeCollection`. -3. Define ≥ 8 `dataColors`. To standardize fonts globally, use the wildcard `visualStyles`: - ```json - "visualStyles": { "*": { "*": { "fontSize": [{ "value": 11 }], "fontFamily": [{ "value": "Segoe UI" }] } } } - ``` - -## Cautions -- The visual schema evolves with Desktop releases; old keys may be ignored on newer versions. -- Validate JSON against the PBIR schema where possible. Always have the user commit (`git add`) - before large UI refactors and test by opening the file in Desktop. +Create custom themes under `StaticResources/RegisteredResources/`, register them in +`report.json` (`themeCollection.customTheme` + a `resourcePackages` item of type +`RegisteredResources`), define ≥ 8 `dataColors`, and standardize fonts with the wildcard +`visualStyles` (`"*": { "*": { ... } }`). Rename the file on every edit (pitfall above). + +## Before large changes +Ask the user to commit the PBIP first so every agent edit is a reviewable diff. diff --git a/.github/skills/pbi-report-layer-pbir/references/desktop-bridge.md b/.github/skills/pbi-report-layer-pbir/references/desktop-bridge.md new file mode 100644 index 0000000..ef02d9a --- /dev/null +++ b/.github/skills/pbi-report-layer-pbir/references/desktop-bridge.md @@ -0,0 +1,83 @@ +# Desktop Bridge — setup, commands, errors + +Sources (read these when something here looks outdated): +- Microsoft Learn — [What is the Power BI Desktop Bridge?](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +- Microsoft Learn — [Power BI Report skill](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-report-skill-overview) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `powerbi-report-cli`, MIT) — the official agent guidance this kit follows. +- npm: [`@microsoft/powerbi-report-authoring-cli`](https://www.npmjs.com/package/@microsoft/powerbi-report-authoring-cli) + (`powerbi-report-author`), [`@microsoft/powerbi-desktop-bridge-cli`](https://www.npmjs.com/package/@microsoft/powerbi-desktop-bridge-cli) + (`powerbi-desktop`). Both MIT, Node 20+. + +## What it is +A local JSON-RPC 2.0 server inside each running Power BI Desktop window, reachable only on the +same Windows machine through the named pipe `pbi-desktop-bridge-<processId>`. One operation runs +at a time per window; a second request while one is running fails — wait and retry, never run +two Desktop commands in parallel. + +Methods (discover the current list with `bridge.manifest`): +| Method | Used for | +|---|---| +| `bridge.manifest` | Supported methods and their schemas for this Desktop build | +| `application.state.get/v1` | `currentFilePath`, `hasUnsavedChanges` | +| `report.snapshot.capture/v1` | PNG of one page (`pageId`, `scale` 1–3) | +| `file.reload/v1` | Reload the open PBIP from disk; `reloadModelDefinition` true/false | + +## Setup (once per machine) +1. Power BI Desktop for Windows, recent enough to show **File > Options and settings > Options > + Security > Desktop Bridge**. Keep *Enable external tool access to Power BI Desktop through + secure local APIs* checked (default on). Admins can force it with the registry value + `HKLM\SOFTWARE\Policies\Microsoft\Power BI Desktop\DesktopNamedPipeBridge` (`1` on, `0` off). +2. Node.js 20+, then `npm i -g @microsoft/powerbi-report-authoring-cli`. +3. Check: `powerbi-report-author doctor` and `powerbi-report-author preview --list-hosts` + (the `desktop` host must say `available: true` while a PBIP is open). +4. The report must be a **PBIP saved in PBIR format** (not a `.pbix`, not PBIR-Legacy). + +## Commands +Use `powerbi-report-author preview` for every Desktop operation; it finds the Desktop window +that has `<folder>` open, so there is no PID to pass. `<folder>` = the project folder or `.pbip`. + +| Intent | Command | +|---|---| +| Status (save gate) | `powerbi-report-author preview "<folder>" --host desktop --status` | +| Open / attach | `powerbi-report-author preview "<folder>" --host desktop` | +| Reload report | `powerbi-report-author preview "<folder>" --host desktop --reload` | +| Reload report + on-disk model | `powerbi-report-author preview "<folder>" --host desktop --reload-with-model` | +| Screenshot one page | `powerbi-report-author preview "<folder>" --host desktop --screenshot "<shots>\<page>.png" --page "<pageId or display name>"` | +| Screenshot all pages | `powerbi-report-author preview "<folder>" --host desktop --screenshot "<shots>" --all-pages` | +| Validate files | `powerbi-report-author validate "<folder>"` (`--no-schema` offline) | + +Optional `--scale 1..3` on screenshots. Every command prints a JSON envelope: +`{"data": {...}}` on success, `{"error": {"code": ..., "message": ..., "correction"?: {...}}}` on +failure. If an error carries a `correction` object, retry once changing only that value; +otherwise report the error and stop — do not improvise another command. + +The bridge CLI underneath (`powerbi-desktop status | manifest | open | reload | screenshot | +screenshot-all`) is useful for manual troubleshooting by the user (e.g. `powerbi-desktop status` +lists every open window and its unsaved state). Agents stay on `powerbi-report-author`. + +### Status before open/reload — never before a screenshot +```json +{"data":{"state":"ready","reports":[{"folder":"C:\\work\\Sales","hostInstanceId":"12345"}], + "details":{"hasUnsavedChanges":false}}} +``` +| `details.hasUnsavedChanges` | Action | +|---|---| +| `false` | Reload / open is allowed. | +| `true` | Stop. Ask the user to save (Ctrl+S) — or to discard — in Desktop, then check again. | +| missing, `null`, or the call failed | Treat as unsafe. Stop and report. | +| error `HOST_UNAVAILABLE` (no window has the report) | Open/attach is allowed; reload is not. | + +## Errors you will meet +| Code / symptom | Meaning | What to do | +|---|---|---| +| `UNSUPPORTED_OS` in `--list-hosts` | Not on Windows | Use the Desktop-closed path; `validate` still works. | +| `HOST_UNAVAILABLE` — "Several Power BI Desktop instances have this report open" | Same report in 2+ windows (`details.candidatePids`) | Ask the user to close the extras. Never close Desktop yourself. | +| `HOST_UNAVAILABLE` — no instance | Desktop closed or setting off | Ask the user to open the `.pbip`; check the Security setting. | +| `RENDER_FAILED` with "unknown page id" | Page exists on disk but not in the open window | Status → reload → capture that page again. | +| `PAGE_NOT_FOUND` | The page is not in the definition at all | Fix `pages.json` / the page id. Reload will not help. | +| `status: "partial"` + non-zero exit | Some pages captured, some failed | Keep the PNGs you got; report which pages failed and why. | +| JSON-RPC `-32601 MethodNotFound` (bridge CLI) | Desktop build lacks that method | Update Desktop; check `powerbi-desktop manifest`. | +| "operation in progress" | Another bridge call is running | Wait, then retry once. | +| Theme change invisible after reload | Desktop cached the theme by file name | Rename the theme file + both refs in `report.json`, validate, reload. | +| Change invisible after reload, no error | Edited a file Desktop does not read (e.g. outside `definition/`) or the wrong report folder | `preview-visuals <folder>` to confirm paths; check `--status` `folder`. | diff --git a/.github/skills/pbi-report-layer-pbir/scripts/validate_pbir.py b/.github/skills/pbi-report-layer-pbir/scripts/validate_pbir.py new file mode 100755 index 0000000..008c29b --- /dev/null +++ b/.github/skills/pbi-report-layer-pbir/scripts/validate_pbir.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Validate a PBIR report: Microsoft's `powerbi-report-author validate` when installed, +otherwise a structural check written in Python. + +Usage: + python3 validate_pbir.py <path> [--structural-only] [--offline] [--format text|json] + +<path>: a .pbip file, a <Name>.Report folder, a folder containing definition/, or a project +folder with exactly one *.Report. + +The CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`) is found through +$PBIR_AUTHOR_CLI or `powerbi-report-author` on PATH. It validates every file against the +published PBIR JSON schemas plus visual roles/types, so it is the authoritative check. +--offline passes --no-schema to it (no download of remote schemas). + +The structural fallback checks what breaks Desktop most often: JSON that does not parse, +PBIR-Legacy layout, a broken semantic-model binding, pages.json out of sync with the page +folders, page/visual `name` != folder, visuals without position/visualType, visuals outside +the page canvas, and theme/resource files that are registered but missing. It does NOT know +the schemas or visual roles — say so when reporting. + +Exit codes: 0 = no errors, 1 = errors found, 2 = usage error / the CLI could not run. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +CLI_NAME = "powerbi-report-author" +CLI_TIMEOUT_SECONDS = 300 + + +class UsageError(Exception): + pass + + +# --------------------------------------------------------------------------- +# Locate the .Report folder +# --------------------------------------------------------------------------- +def _read_json(path: Path): + return json.loads(path.read_text(encoding="utf-8-sig")) + + +def resolve_report(path: Path) -> Path: + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() != ".pbip": + raise UsageError(f"unsupported file: {path.name} (use a .pbip or a .Report folder)") + try: + artifacts = _read_json(path).get("artifacts", []) + except ValueError as exc: + raise UsageError(f"{path.name} is not valid JSON: {exc}") from exc + for artifact in artifacts: + report = artifact.get("report", {}).get("path") + if report and (path.parent / report).is_dir(): + return (path.parent / report).resolve() + sibling = path.with_name(path.stem + ".Report") + if sibling.is_dir(): + return sibling.resolve() + raise UsageError(f"{path.name} does not point to an existing .Report folder") + if (path / "definition").is_dir() or (path / "definition.pbir").is_file() or path.suffix == ".Report": + return path.resolve() + reports = sorted(p for p in path.glob("*.Report") if p.is_dir()) + if len(reports) == 1: + return reports[0].resolve() + if len(reports) > 1: + raise UsageError(f"several reports in {path}: {', '.join(p.name for p in reports)}; pass one") + raise UsageError(f"no .Report folder found under {path}") + + +# --------------------------------------------------------------------------- +# Structural fallback +# --------------------------------------------------------------------------- +class Findings: + def __init__(self, report_dir: Path): + self.report_dir = report_dir + self.items: list = [] + + def add(self, severity: str, code: str, message: str, file: Optional[Path] = None, path: str = ""): + rel = "" + if file is not None: + try: + rel = file.relative_to(self.report_dir).as_posix() + except ValueError: + rel = str(file) + self.items.append({"severity": severity, "code": code, "message": message, "file": rel, "path": path}) + + def error(self, *args, **kwargs): + self.add("error", *args, **kwargs) + + def warning(self, *args, **kwargs): + self.add("warning", *args, **kwargs) + + +def _load(findings: Findings, file: Path, required: bool = True): + if not file.is_file(): + if required: + findings.error("FILE_MISSING", f"required file is missing: {file.name}", file) + return None + try: + return _read_json(file) + except ValueError as exc: + findings.error("JSON_INVALID", f"invalid JSON: {exc}", file) + return None + + +def _is_number(value) -> bool: + return isinstance(value, (int, float)) and not isinstance(value, bool) + + +def structural_check(report_dir: Path) -> dict: + f = Findings(report_dir) + definition = report_dir / "definition" + + for file in sorted(report_dir.rglob("*.json")): + if ".pbi" in file.relative_to(report_dir).parts: + continue + try: + _read_json(file) + except ValueError as exc: + f.error("JSON_INVALID", f"invalid JSON: {exc}", file) + invalid = {i["file"] for i in f.items} + + if not definition.is_dir(): + if (report_dir / "report.json").is_file(): + f.error("PBIR_LEGACY_FORMAT", "PBIR-Legacy layout (report.json at the root); save the " + "report in PBIR format (Desktop: File > Options > Preview features > PBIR)", report_dir / "report.json") + else: + f.error("FILE_MISSING", "definition/ folder is missing", definition) + return _result(f, "structural") + + pbir = _load(f, report_dir / "definition.pbir") + if isinstance(pbir, dict): + ref = pbir.get("datasetReference", {}) + by_path = ref.get("byPath", {}).get("path") if isinstance(ref.get("byPath"), dict) else None + if by_path: + target = (report_dir / by_path).resolve() + if not target.is_dir(): + f.error("MODEL_BINDING_BROKEN", f"datasetReference.byPath points to a missing folder: {by_path}", + report_dir / "definition.pbir", "datasetReference.byPath.path") + elif not ref.get("byConnection"): + f.error("MODEL_BINDING_MISSING", "datasetReference has neither byPath nor byConnection", + report_dir / "definition.pbir", "datasetReference") + + _load(f, definition / "version.json") + report = _load(f, definition / "report.json") + if isinstance(report, dict): + _check_resources(f, report, report_dir) + + pages_dir = definition / "pages" + pages_meta = _load(f, pages_dir / "pages.json") + page_folders = sorted(p.name for p in pages_dir.iterdir() if p.is_dir()) if pages_dir.is_dir() else [] + if isinstance(pages_meta, dict): + order = pages_meta.get("pageOrder", []) + for page_id in order: + if page_id not in page_folders: + f.error("PAGE_MISSING", f"pages.json lists page '{page_id}' but its folder is missing", + pages_dir / "pages.json", "pageOrder") + for page_id in page_folders: + if page_id not in order: + f.warning("PAGE_NOT_IN_ORDER", f"page folder '{page_id}' is not listed in pages.json pageOrder", + pages_dir / page_id) + active = pages_meta.get("activePageName") + if active and active not in order: + f.error("ACTIVE_PAGE_UNKNOWN", f"activePageName '{active}' is not in pageOrder", + pages_dir / "pages.json", "activePageName") + + for page_id in page_folders: + _check_page(f, pages_dir / page_id, invalid) + + return _result(f, "structural") + + +def _check_resources(f: Findings, report: dict, report_dir: Path) -> None: + static = report_dir / "StaticResources" + for package in report.get("resourcePackages", []) or []: + folder = static / package.get("name", "") + for item in package.get("items", []) or []: + rel = item.get("path") + if rel and not (folder / rel).is_file(): + f.error("RESOURCE_MISSING", f"resource '{item.get('name')}' registered in report.json is " + f"missing on disk: {package.get('name')}/{rel}", report_dir / "definition" / "report.json", + "resourcePackages") + custom = (report.get("themeCollection") or {}).get("customTheme") + if isinstance(custom, dict) and custom.get("name"): + registered = { + item.get("name") + for package in report.get("resourcePackages", []) or [] + if package.get("type") == custom.get("type", "RegisteredResources") + for item in package.get("items", []) or [] + } + if custom["name"] not in registered: + f.error("THEME_NOT_REGISTERED", f"customTheme '{custom['name']}' has no matching resourcePackages item", + report_dir / "definition" / "report.json", "themeCollection.customTheme.name") + + +def _check_page(f: Findings, page_dir: Path, invalid: set) -> None: + page_file = page_dir / "page.json" + page = _load(f, page_file) + if not isinstance(page, dict): + return + if page.get("name") != page_dir.name: + f.error("NAME_MISMATCH", f"page.json name '{page.get('name')}' != folder '{page_dir.name}'", page_file, "name") + if not page.get("displayName"): + f.warning("PAGE_NO_DISPLAY_NAME", "page has no displayName", page_file, "displayName") + width, height = page.get("width"), page.get("height") + if not (_is_number(width) and _is_number(height)): + f.error("PAGE_SIZE_INVALID", "page width/height must be numbers", page_file) + width = height = None + + visuals_dir = page_dir / "visuals" + if not visuals_dir.is_dir(): + return + for visual_dir in sorted(p for p in visuals_dir.iterdir() if p.is_dir()): + visual_file = visual_dir / "visual.json" + rel = visual_file.relative_to(f.report_dir).as_posix() + if rel in invalid: + continue + visual = _load(f, visual_file) + if not isinstance(visual, dict): + continue + if visual.get("name") != visual_dir.name: + f.error("NAME_MISMATCH", f"visual.json name '{visual.get('name')}' != folder '{visual_dir.name}'", + visual_file, "name") + pos = visual.get("position") + if not isinstance(pos, dict) or not all(_is_number(pos.get(k)) for k in ("x", "y", "width", "height")): + f.error("POSITION_INVALID", "position needs numeric x, y, width and height", visual_file, "position") + pos = None + kind = (visual.get("visual") or {}).get("visualType") + if not kind and "visualGroup" not in visual: + f.error("VISUAL_TYPE_MISSING", "visual.visualType is missing (and this is not a visualGroup)", + visual_file, "visual.visualType") + if pos and width is not None and not visual.get("parentGroupName"): + right, bottom = pos["x"] + pos["width"], pos["y"] + pos["height"] + if pos["x"] < 0 or pos["y"] < 0 or right > width + 0.5 or bottom > height + 0.5: + f.warning("OUTSIDE_CANVAS", f"visual extends outside the {width}x{height} page " + f"(x={pos['x']}, y={pos['y']}, right={right}, bottom={bottom})", visual_file, "position") + + +def _result(f: Findings, engine: str) -> dict: + errors = sum(1 for i in f.items if i["severity"] == "error") + warnings = sum(1 for i in f.items if i["severity"] == "warning") + return { + "engine": engine, + "report": str(f.report_dir), + "result": "failed" if errors else ("succeededWithWarnings" if warnings else "succeeded"), + "errorCount": errors, + "warningCount": warnings, + "diagnostics": f.items, + } + + +# --------------------------------------------------------------------------- +# powerbi-report-author CLI +# --------------------------------------------------------------------------- +def find_cli(env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + explicit = env.get("PBIR_AUTHOR_CLI") + if explicit: + return explicit if Path(explicit).is_file() else None + return shutil.which(CLI_NAME) + + +def build_cli_command(cli: str, report_dir: Path, offline: bool) -> list: + cmd = [cli, "validate", str(report_dir)] + if offline: + cmd.append("--no-schema") + return cmd + + +def parse_cli_output(stdout: str, report_dir: Path) -> dict: + """Normalize the CLI's JSON envelope ({"data": ...} or {"error": ...}).""" + try: + envelope = json.loads(stdout) + except ValueError as exc: + raise UsageError(f"{CLI_NAME} returned non-JSON output: {stdout.strip()[:500]}") from exc + if "error" in envelope: + err = envelope["error"] + raise UsageError(f"{CLI_NAME} failed: {err.get('code')}: {err.get('message')}") + data = envelope.get("data", {}) + items = [] + for code, group in (data.get("diagnostics") or {}).items(): + for item in group.get("items", []): + file = item.get("file", "") + try: + file = Path(file).resolve().relative_to(report_dir).as_posix() if file else "" + except ValueError: + pass + items.append({ + "severity": group.get("severity", "error"), + "code": code, + "message": item.get("message", ""), + "file": file, + "path": item.get("path", ""), + }) + return { + "engine": CLI_NAME, + "report": str(report_dir), + "result": data.get("result", "unknown"), + "errorCount": int(data.get("errorCount", sum(1 for i in items if i["severity"] == "error"))), + "warningCount": int(data.get("warningCount", sum(1 for i in items if i["severity"] == "warning"))), + "diagnostics": items, + } + + +def run_cli(cli: str, report_dir: Path, offline: bool) -> dict: + cmd = build_cli_command(cli, report_dir, offline) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=CLI_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise UsageError(f"could not run {' '.join(cmd)}: {exc}") from exc + return parse_cli_output(proc.stdout, report_dir) + + +# --------------------------------------------------------------------------- +# Output +# --------------------------------------------------------------------------- +def format_text(result: dict) -> str: + if result["engine"] == "structural": + lines = [f"PBIR validation: structural fallback ({CLI_NAME} not found; schemas and visual " + "roles NOT checked — install @microsoft/powerbi-report-authoring-cli)"] + else: + lines = [f"PBIR validation: {CLI_NAME} validate"] + lines.append(f"Report: {result['report']}") + for d in sorted(result["diagnostics"], key=lambda d: (d["severity"] != "error", d["file"], d["code"])): + where = d["file"] + (f" @ {d['path']}" if d["path"] else "") + lines.append(f"{d['severity'].upper():8} [{d['code']}] {d['message']}" + (f" ({where})" if where else "")) + lines.append(f"Result: {result['result']} — {result['errorCount']} error(s), {result['warningCount']} warning(s)") + return "\n".join(lines) + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Validate a PBIR report (CLI if installed, else structural).") + parser.add_argument("path", type=Path) + parser.add_argument("--structural-only", action="store_true", help="skip the CLI even if it is installed") + parser.add_argument("--offline", action="store_true", help="pass --no-schema to the CLI") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + + try: + report_dir = resolve_report(args.path) + cli = None if args.structural_only else find_cli() + result = run_cli(cli, report_dir, args.offline) if cli else structural_check(report_dir) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + print(json.dumps(result, indent=2) if args.format == "json" else format_text(result)) + return 1 if result["errorCount"] else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/skills/pbi-semantic-layer-tmdl/SKILL.md b/.github/skills/pbi-semantic-layer-tmdl/SKILL.md index dc1f3c2..da3d3bd 100644 --- a/.github/skills/pbi-semantic-layer-tmdl/SKILL.md +++ b/.github/skills/pbi-semantic-layer-tmdl/SKILL.md @@ -1,69 +1,99 @@ --- name: pbi-semantic-layer-tmdl -description: Edit the Power BI semantic model as code by authoring TMDL files in a PBIP project (offline, Desktop closed). Covers structure, syntax, and post-edit verification. +description: Change the Power BI semantic model — measures, columns, tables, relationships, RLS, DAX — with Power BI Desktop OPEN through the Power BI Modeling (Authoring) MCP server, saving to TMDL in a PBIP project; or by editing TMDL files with Desktop closed. Includes a semantic-model review checklist. Use for any request to add or fix a measure, DAX, relationship, table, column or model metadata. --- # Skill: pbi-semantic-layer-tmdl -Author and edit the semantic model (tables, columns, measures, relationships, RLS) by writing -**TMDL** files directly. TMDL is the GA, human-readable text format for tabular models — great -diffs, one file per table, fully scriptable. +With Desktop open, the model lives in Desktop's memory (a local Analysis Services instance). Edit +it **there**, live, through the Power BI Modeling MCP server — then the user saves and Desktop +writes the TMDL into the PBIP. Editing the `.tmdl` files while Desktop is open does nothing +until a model reload and is overwritten by the next save. -## Hard precondition -**Power BI Desktop must be CLOSED** before saving. Desktop keeps the model in memory and -overwrites disk on save, so editing live corrupts files or loses work. +## Default: Desktop open + Modeling MCP +Setup and the tool list: [references/modeling-mcp.md](references/modeling-mcp.md). +1. **Connect:** "Connect to '<file name>' in Power BI Desktop". Confirm with a read: list the + tables and measures. If the MCP is not registered, stop and point the user to the setup — + do not fall back to editing TMDL silently. +2. **Inspect through the MCP**, not the `.tmdl` files: while connected, the live model is the + truth and disk may be stale. +3. **Change** in dependency order: tables/partitions → columns → relationships → measures. + Check that every referenced table/column exists and that the name is not taken. +4. **Prove it:** run a DAX query through the MCP (`EVALUATE SUMMARIZECOLUMNS(..., "x", [New Measure])` + or `EVALUATE { [New Measure] }`) and show the value. A measure that only "was created" is not done. +5. **Save:** ask the user to press Ctrl+S in Desktop. That writes the TMDL into + `<Name>.SemanticModel/definition/` — the diff the team reviews and commits. +6. **Quality:** run `pbi-quality-rules` (BPA) after significant changes — on disk, so after step 5. +7. If a visual must use the new measure, continue in `pbi-report-layer-pbir`. Report reloads do + not touch the model and are not needed for MCP changes. + +## Alternative: Desktop closed (no MCP, older Desktop, CI) +Edit the TMDL files directly: +1. Make sure Desktop is **closed** — it overwrites disk on save. +2. Edit `.tmdl` files following the syntax below; run `pbi-quality-rules`. +3. Ask the user to open the `.pbip`: field list populates, no error banner, each new measure on a + Card shows a value. On error, re-read the edited TMDL and fix the reference/syntax. + +If Desktop must stay open and the MCP is unavailable, edit TMDL, then reload model + report with +`powerbi-report-author preview <folder> --host desktop --reload-with-model` after the same +unsaved-changes gate as `pbi-report-layer-pbir`. It reloads definitions only — the user must +refresh data for new/changed imported columns. ## `.SemanticModel` structure ``` -<Project>.SemanticModel/ +<Name>.SemanticModel/ definition.pbism ← pointer (do not hand-edit) definition/ - model.tmdl ← model-level definition, lists tables - database.tmdl ← compatibility level / annotations - tables/<Table>.tmdl ← ONE table per file (golden rule) - relationships.tmdl ← all relationships + database.tmdl ← compatibilityLevel + model.tmdl ← model properties, `ref table` order, annotations + tables/<Table>.tmdl ← one table per file + relationships.tmdl expressions.tmdl ← shared M expressions / parameters - cultures/ ← translations roles.tmdl ← RLS - .pbi/ ← local cache/settings — NEVER edit (localSettings.json, cache.abf) + cultures/ ← translations + .pbi/ ← local cache/settings — never edit, keep out of git ``` -Edit `.tmdl` files only. One table per file — never define every table inside `model.tmdl`. -## TMDL syntax reference +## TMDL syntax ```tmdl -table 'Sales' - description: "Primary sales fact." +/// Order lines. Grain: one row per order line. +table Sales - measure 'Total Sales' = SUM(Sales[Amount]) - formatString: "#,##0.00" - displayFolder: "Financials" - description: "Sum of sales amount." + /// Sum of the order line amount. + measure 'Total Sales' = SUM(Sales[Amount]) + formatString: #,##0.00 + displayFolder: Revenue - column 'Amount' - dataType: decimal - sourceColumn: BaseAmount + measure 'Margin %' = + DIVIDE([Total Sales] - [Total Cost], [Total Sales]) + formatString: 0.0% + displayFolder: Revenue + + column Amount + dataType: decimal + summarizeBy: sum + sourceColumn: Amount ``` ```tmdl -relationship Sales_Date - fromColumn: Sales.DateId - toColumn: Date.DateId - fromCardinality: many - toCardinality: one +relationship 5f0c2a4e-9b1d-4c3a-8e2f-1a6b7c8d9e01 + fromColumn: Sales.DateKey + toColumn: Date.DateKey ``` +- Indent with tabs; properties one level under their object. +- Descriptions are `///` lines directly above the object. +- Multi-line DAX: leave the `=` line empty and indent the expression two levels deeper than the + object, or wrap it in a ```` ``` ```` block. +- Quote names with spaces or symbols: `'Margin %'`; double a quote inside: `'Customer''s'`. -**Serialization rules:** keep functional properties (`dataType`, `sourceColumn`) before -cosmetic ones; match the file's existing indentation; for multi-line DAX, break with `\` at the -end of the line. - -## Tooling -- **Tabular Editor 2 (free)** can open the `.SemanticModel` folder, validate, batch-edit, and - re-serialize clean TMDL. Use it to sanity-check non-trivial edits. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (preview) edits the same PBIP/TMDL files and - is handy for bulk operations. File editing remains the default, accessible path. - -## Post-edit verification -1. Ask the user to open the `.pbip` in Desktop and wait for the field list to populate. -2. **View → Diagnostics** — report any error/warning shown. -3. For each new/changed measure, drop it on a **Card** and confirm the value (no red error icon). -4. On error: re-read the edited TMDL, find the syntax/reference issue, propose the fix. -5. Run the `pbi-quality-rules` skill (BPA) after significant changes. +## Review checklist (run it on any model you change or are asked to review) +- **Names** are business-friendly: no `tbl_`/`dim_` prefixes or `_id` columns visible to users. +- **Every measure** has a `formatString`, a `displayFolder` and a `///` description (what it + means and its grain/filters in plain words). +- **Relationships** match reality: many-to-one from fact to dimension, single direction unless a + documented reason exists; no ambiguous paths. +- **A real Date table** (marked as date table) drives time intelligence; auto date/time is off + (`annotation __PBI_TimeIntelligenceEnabled = 0` in `model.tmdl`). +- **Technical keys hidden** (`isHidden` on `...ID`/`...Key` columns); explicit measures instead + of implicit column aggregation for anything a report shows. +- **DAX:** `DIVIDE` instead of `/` + `IFERROR`; variables for repeated sub-expressions; no + measure that just returns `BLANK()`. diff --git a/.github/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md b/.github/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md new file mode 100644 index 0000000..fdf7468 --- /dev/null +++ b/.github/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md @@ -0,0 +1,60 @@ +# Power BI Modeling MCP — setup and use + +Microsoft now calls it the **Power BI Authoring MCP server**; the package and server id are still +`powerbi-modeling-mcp`. Sources: +- Microsoft Learn — [Power BI Authoring MCP server](https://learn.microsoft.com/en-us/power-bi/developer/mcp/power-bi-authoring-mcp) +- [microsoft/powerbi-modeling-mcp](https://github.com/microsoft/powerbi-modeling-mcp) (README, TROUBLESHOOTING) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `semantic-model-authoring`, bundles the same server) + +## Local vs hosted +Working against a model **open in Power BI Desktop** (or PBIP files on disk) needs the **local** +server: it runs on the same Windows machine and talks to Desktop's local Analysis Services +instance over XMLA. The hosted endpoint only reaches Fabric workspaces. Register one of the two, +not both — duplicate tool sets confuse the agent. The local server does not run on macOS. + +## Register the local server +Command in every client: `npx -y @microsoft/powerbi-modeling-mcp@latest --start` +(Node 18+; or install the VS Code extension `analysis-services.powerbi-modeling-mcp`). + +| Client | File | Shape | +|---|---|---| +| Claude Code | `.mcp.json` (project) | `{"mcpServers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| GitHub Copilot (VS Code) | `.vscode/mcp.json` | `{"servers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| OpenCode | `opencode.json` | `{"mcp": {"powerbi-modeling-mcp": {"type": "local", "command": ["npx", "-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| Cursor | `.cursor/mcp.json` | same shape as Claude Code (`mcpServers`) | + +Useful flags: `--readonly` (inspection only — good for reviews), `--readwrite` (default; asks for +confirmation before the first write per database). + +## Connect and work +- Desktop: `Connect to '<file name>' in Power BI Desktop` (tool `ConnectToPowerBIDesktop`; the + PBIP must be open in Desktop). +- PBIP on disk without Desktop: `Open semantic model from PBIP folder '<path>\<Name>.SemanticModel\definition'` + (changes stay in the session until exported back to TMDL). +- First call after connecting: a read (`List the tables and measures`) to confirm the target. + +Tool groups you will use most: `measure_operations`, `column_operations`, `table_operations`, +`relationship_operations`, `dax_query_operations` (run/validate DAX, max 100,000 rows), +`transaction_operations` (begin/commit/rollback a batch), `calculation_group_operations`, +`security_role_operations`, `object_translation_operations`, `database_operations` +(import/export TMDL). Ask the agent "what can the Power BI Authoring MCP do" to list the current set. + +## Rules of thumb +- Prefer one transaction per logical change; roll back on a failed step instead of leaving + half-applied edits. +- Verify every new or changed measure with a DAX query before calling it done. +- Changes are live in Desktop but not on disk until the user saves (Ctrl+S); a PBIP under git + makes every MCP change a reviewable diff. +- The MCP edits the model only — never report pages (use `pbi-report-layer-pbir`). +- Model metadata and query results go to the LLM provider of the client. Do not query + sensitive columns just to "look at the data". + +## Troubleshooting +| Symptom | Check | +|---|---| +| Server missing from the tool list | Client in agent mode? Server registered and started? Copilot Business/Enterprise: "MCP servers in Copilot" policy enabled by an admin. | +| "No Power BI Desktop instance found" | The PBIP/PBIX is open in Desktop and the name matches the file name you gave. | +| Reads work, writes fail | `--readonly` set, or (Fabric) only Build permission / XMLA read-only. | +| Edits vanish after reopening the file | The user closed Desktop without saving. | +| TMDL on disk does not show the change | Not saved yet — Desktop writes TMDL on save. | diff --git a/.github/workflows/sync-check.yml b/.github/workflows/sync-check.yml new file mode 100644 index 0000000..6f6f89a --- /dev/null +++ b/.github/workflows/sync-check.yml @@ -0,0 +1,25 @@ +name: sync-check + +on: + push: + pull_request: + +jobs: + check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - uses: actions/setup-node@v4 + with: + node-version: "20" + - name: Install dependencies + run: | + pip install pyyaml + npm install --no-audit --no-fund + - name: Generated folders match .agent/ + run: python3 scripts/sync_agents.py --check + - name: Tests + run: npm test diff --git a/.gitignore b/.gitignore index 2fb0c03..050c15a 100644 --- a/.gitignore +++ b/.gitignore @@ -9,6 +9,13 @@ lerna-debug.log* # Dependencies node_modules/ +# Python +__pycache__/ +*.pyc + +# Power BI Desktop local state (written when a PBIP is opened) +.pbi/ + # OS metadata .DS_Store Thumbs.db diff --git a/.opencode/agents/analytics-engineer.md b/.opencode/agents/analytics-engineer.md deleted file mode 100644 index e4965aa..0000000 --- a/.opencode/agents/analytics-engineer.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: Expert in data modeling, dbt, dimensional modeling (Star Schema), data - warehouse architecture, and SQL transformations. Triggers on dbt, dimensional modeling, - star schema, data warehouse, transformation, partitioning, clustering. -mode: subagent -permission: - edit: allow - bash: allow ---- - -# Analytics Engineer & Warehouse Architect - -You turn clean raw data into intuitive, performant dimensional models (Star Schemas) with dbt. -Guiding principle: *build a model so intuitive that business users query it without asking you how.* - -## Design process -1. **Grain first:** what business process, and what does one row represent? -2. **Shape:** fact or dimension? Does it need partitioning/clustering? -3. **Build & test** the dbt models, then verify grain, idempotency, and performance. - -## Dimensional modeling rules -- Build dimensions before facts. -- Every dimension has a surrogate key (`sk_`, typically a hash of the natural key). -- One unambiguous grain per fact table. -- Keep dimensions flat — snowflake only when performance demands it. - -## Performance levers -- **Partition** on the column queries consistently filter (e.g. `event_date`). -- **Cluster / Z-order** on high-cardinality join/filter keys (`user_id`, `tenant_id`). -- **Materialized views** for heavy, frequently-read aggregations. - -## dbt conventions (org context) -- CTE structure: `import` CTEs on top, `logical` in the middle, one `final` at the bottom. -- Every model: `unique` + `not_null` on its primary key; description in `schema.yml`. -- Naming: `stg_`, `dim_`, `fct_`, `mart_`. - -## Handoffs -- Raw ingestion / orchestration → `data-engineer`. -- Optimized views for import → `powerbi-developer`. -- Enforce data contracts and PII masking inline per the workspace rules. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), ad-hoc analysis & dashboards (→ `data-scientist`), -DAX tuning (→ `powerbi-developer`). diff --git a/.opencode/agents/data-engineer.md b/.opencode/agents/data-engineer.md deleted file mode 100644 index 9ada6c2..0000000 --- a/.opencode/agents/data-engineer.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: Expert in data engineering, ETL/ELT pipelines, Medallion architecture - (Bronze/Silver/Gold), Databricks, PySpark, and orchestration. Triggers on pipeline, - ingestion, spark, databricks, medallion, etl, elt, orchestration, airflow, dagster. -mode: subagent -permission: - edit: allow - bash: allow ---- - -# Data Engineer - -Senior Data Engineer. You build robust, idempotent, fault-tolerant pipelines with clear -architectural boundaries. Guiding principle: *pipelines should be boring and re-runnable; -bad data is forever.* - -## Medallion architecture -- **Bronze:** append-only raw data, exactly as it arrived. No transformations. -- **Silver:** deduplicated, typed, standardized. The enterprise source of truth. -- **Gold:** aggregated, business-level tables for BI/ML consumption. - -## Pipeline patterns -- **Idempotent loads:** use `MERGE`/upsert or partition `OVERWRITE`; never `INSERT INTO` - without first scoping a delete. Re-running any window must yield identical results. -- **Incremental:** track a watermark (`last_updated_at`); process `source > target.watermark`; - handle late-arriving data with `MERGE`. -- **Backfill-ready:** parameterize `start_date`/`end_date`, defaulting to current date. -- **Write-Audit-Publish:** write to staging → run quality checks → publish only on pass. - -## Opinionated defaults (org context, not generic advice) -- Use PySpark/Polars for large transforms; reserve Pandas for small/local work. -- Fail loudly and alert — never swallow exceptions silently. -- Run lineage/downstream checks before altering any upstream schema. - -## Handoffs -- Clean Silver data → `analytics-engineer` for dimensional modeling. -- Enforce quality gates and PII masking inline (WAP, mask at Silver) per the workspace rules. - -## Out of scope -Dimensional models / dbt (→ `analytics-engineer`), dashboards, metrics & ML (→ `data-scientist`). diff --git a/.opencode/agents/data-scientist.md b/.opencode/agents/data-scientist.md deleted file mode 100644 index 0254531..0000000 --- a/.opencode/agents/data-scientist.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: Expert across the analytics spectrum — from metrics, dashboards, and - ad-hoc/business analysis to statistical modeling, machine learning, forecasting, - and experimentation (A/B testing). Also reviews semantic models for business accuracy. - Triggers on analysis, dashboard, metric, KPI, insight, report, semantic model review, - machine learning, ML, prediction, forecasting, a/b testing, statistics, experiment. -mode: subagent -permission: - edit: allow - bash: allow ---- - -# Data Scientist & Analyst - -You turn data into decisions across the full rigor spectrum: descriptive analysis and metrics at -one end, statistical inference and ML at the other. Guiding principle: *every output must lead to -an action — and start simple; a deployed logistic regression beats a notebook-bound neural net.* - -## Pick the right altitude of rigor -- **Descriptive / diagnostic** — metrics, dashboards, hypothesis-driven exploration. -- **Inferential / predictive** — statistics, experiments, ML. - -Match the method to the decision; don't bring ML to a question a well-defined metric answers. - -## Analysis & metrics -- Question the question: "show sales by region" is a data pull — ask what decision it informs. -- Hypotheses are MECE; every chart must lead to an action. -- Define each metric with: **Name**, **Definition** (plain-English), **Grain**, **Filters**, **Polarity**. -- Dashboard ideation: audience (exec vs. ops); the one 10-second question; inverted pyramid - (KPIs → trend → detail). Line for trend, bar for comparison; avoid pie. -- Semantic-model review (TMDL): friendly naming, relationships that match reality, a real Date - table with sound time-intelligence, technical keys hidden. - -## Statistics, experiments & ML -- A/B tests: fix sample size and duration *before* starting — peeking inflates false positives. -- Watch confounders; read distributions, not just averages; never leak future data into training. -- Baseline first (mean / linear / logistic); justify any added complexity. -- Imbalanced data → F1 / PR-AUC / cost-based metrics, not accuracy. -- Fit scalers/encoders on train only; ship with drift monitoring; report dollar impact, not R². - -## Handoffs -- Freshness, feature pipelines, deployment infra → `data-engineer`. -- Clean marts / new dimensions → `analytics-engineer`. -- Keep PII out of features and models (workspace privacy rules). -- DAX & TMDL/PBIR implementation → `powerbi-developer`. - -## Out of scope -Ingestion pipelines (→ `data-engineer`), dbt models (→ `analytics-engineer`), -Power BI implementation (→ `powerbi-developer`). diff --git a/.opencode/agents/pbi-visual-reviewer.md b/.opencode/agents/pbi-visual-reviewer.md new file mode 100644 index 0000000..72d6807 --- /dev/null +++ b/.opencode/agents/pbi-visual-reviewer.md @@ -0,0 +1,62 @@ +--- +description: Independent reviewer of rendered Power BI report pages. Reads only the + page screenshots (PNG), the request or spec, and the PBIR folder, then returns PASS + or FAIL per page and per visual against a fixed checklist. Use after every screenshot + step of the PBIR edit loop (pbi-report-layer-pbir), or whenever someone asks to + review, QA, or check a Power BI report page. Read-only; never edits files. +mode: subagent +permission: + edit: deny + bash: deny +--- + +# Power BI Visual Reviewer + +You judge what a Power BI page **looks like**, not what its author meant. You are deliberately +kept away from the author's reasoning: if the caller hands you explanations ("the overlap is +intentional", "the card is fine, it just needs a refresh"), ignore them. Only the pixels, the +spec, and the PBIR files count. + +You cannot run commands or change files. If something must change, say exactly where. + +## Inputs (refuse to start without the first two) +1. **Screenshots** — paths to the PNGs (one per page), as captured by + `powerbi-report-author preview <folder> --host desktop --screenshot <dir> --all-pages`. +2. **Request / spec** — what the user asked for (the change, or the page spec). +3. **PBIR folder** — the `<Name>.Report/` path, to map what you see to files: + `definition/pages/<pageId>/page.json` (`displayName`, `width`, `height`) and + `definition/pages/<pageId>/visuals/<visualId>/visual.json` (`position`, `visual.visualType`, + titles under `visual.visualContainerObjects.title`). + +If a screenshot is missing, unreadable, or obviously stale (it does not show the requested +change at all), the verdict for that page is **BLOCKED**, not PASS. + +## Checklist (apply every item to every page) +| ID | Check | FAIL when | +|---|---|---| +| C1 | Overlap | Two visuals cover each other's content (borders touching is fine). | +| C2 | Clipped text | Titles, labels, axis text, card values or slicer items are cut, truncated with `…` where the full text matters, or wrap into unreadable stubs. | +| C3 | Empty or broken visual | Grey placeholder, "Can't display the visual", red/yellow error icon, "See details", blank chart area where data is expected. | +| C4 | Alignment | Edges that should line up are off by a visible amount; uneven gaps in a row/column of visuals; content outside the page canvas. | +| C5 | Theme consistency | Colors, fonts or backgrounds that ignore the report theme or differ between visuals of the same kind. | +| C6 | Titles | A visual without a meaningful title (unless it is a card/KPI whose label says it), duplicated titles, or titles that do not match the data shown. | +| C7 | Spec match | The requested change is not visible, or something the spec did not ask for changed. | + +Use the PBIR to confirm, not to excuse: a `position` that overlaps confirms C1; a title present in +`visual.json` but invisible in the PNG is still a C2/C6 failure. + +## Output (exact shape) +```markdown +# Visual review — <report name> +Verdict: APPROVED | REJECTED | BLOCKED + +| Page | Visual (type · visualId) | Check | Result | Evidence | Where to fix | +|---|---|---|---|---|---| +| Overview | Total Sales card · 3f2a… | C2 | FAIL | value shows "1.2…" | visuals/3f2a…/visual.json → position.width | +| Overview | — | C4 | PASS | | | +``` +- One row per failing check per visual; one PASS row per page for checks that passed overall. +- `Verdict` is APPROVED only when every page passed every check. Any FAIL → REJECTED. Any + missing/stale screenshot → BLOCKED. +- Evidence is what you see, in a few words. Do not speculate about causes you cannot see. +- No redesign proposals beyond the specific fix location. diff --git a/.opencode/agents/powerbi-developer.md b/.opencode/agents/powerbi-developer.md deleted file mode 100644 index 2b9e528..0000000 --- a/.opencode/agents/powerbi-developer.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: Power BI as-code developer. Edits PBIP projects directly — semantic model - via TMDL (tables, measures, relationships) and reports via PBIR (pages, visuals, - themes). Validates with Tabular Editor 2 (free). Triggers on power bi, pbip, tmdl, - pbir, dax, measure, semantic model, report, visual, theme, bookmark. -mode: subagent -permission: - edit: allow - bash: allow ---- - -# Power BI Developer (as-code) - -You develop Power BI by editing the **PBIP** project files directly on disk — no manual -clicking in Desktop. A PBIP project has two layers; pick the right skill for each: - -- **Semantic model** (`.SemanticModel/`, TMDL) — tables, columns, measures, relationships, - RLS. Use `pbi-semantic-layer-tmdl`. -- **Report** (`.Report/`, PBIR JSON) — pages, visuals, themes, bookmarks. Use - `pbi-report-layer-pbir`. - -This single agent owns both layers — choose the layer by the request instead of delegating. - -## Hard precondition -**Power BI Desktop must be CLOSED** before you save any TMDL or PBIR file. Desktop holds the -model in memory and overwrites disk on save, so live edits corrupt or get silently lost. -Confirm it's closed before writing. - -## Session flow -1. Confirm Desktop is closed and the PBIP folder is valid. -2. Read the relevant layer (TMDL or PBIR) to understand structure before editing. -3. Make the edits, following exact TMDL/PBIR syntax from the skills. -4. **Validate the model** with `pbi-quality-rules` → runs the real Best Practice Analyzer via - the free Tabular Editor 2 CLI. Report `error` and `warning` only. -5. Ask the user to reopen the `.pbip` in Desktop and confirm (Diagnostics clean, measures - render in a Card). Fix anything that errors. - -## Tooling -- **Tabular Editor 2 (free, OSS)** is the validation/automation backbone — BPA, batch edits, - TMDL serialization. Prefer it over reinventing checks. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (public preview) can drive bulk model edits - against the same PBIP/TMDL files. Mention it for large refactors; file editing stays the default. -- Do **not** depend on the deprecated community "Power BI MCP" servers. - -## Boundaries -- Documentation of a full dashboard → use the `/document-dashboard` workflow. -- Standalone model health check → use the `/validate-pbi` workflow. -- Upstream modeling / dbt / views → `analytics-engineer`. Metric definitions & layout intent → - `data-scientist`. - -> The PBIR JSON schema evolves with Desktop releases. Always tell the user to commit (`git add`) -> before large visual refactors and to test by opening the file. diff --git a/.opencode/agents/presentation-designer.md b/.opencode/agents/presentation-designer.md deleted file mode 100644 index 371a480..0000000 --- a/.opencode/agents/presentation-designer.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: Turns data work — exploratory analyses, dashboards, business plans, project - proposals — into polished, on-brand presentations as code. Builds HTML decks (reveal.js), - interactive one-page sites, and print/PDF handouts; applies a company's visual identity - from a DESIGN.md. Triggers on presentation, deck, slides, pitch, readout, report-out, - business plan, storytelling, reveal.js, slidev, marp, DESIGN.md, brand, visual identity. -mode: subagent -permission: - edit: allow - bash: allow ---- - -# Presentation Designer - -You turn analytical content into a clear visual narrative and ship it as code — not by -clicking around in slide tools. Three skills back you up: - -- **`building-html-presentations`** — the *how*: format choice (deck / flex-deck / interactive - site / consulting PDF), ready-to-edit starters, and a chart cookbook. Default engine: **reveal.js**. -- **`generating-pptx`** — when the deliverable must be a **native, editable PowerPoint** (`.pptx`), - built with python-pptx (real charts/text, not images). -- **`applying-visual-identity`** — the *look*: read the project's `DESIGN.md` (Google Labs - spec) and apply its tokens (colors, type, spacing) so the output is on-brand. - -## How to work -1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load - `applying-visual-identity` and use its tokens. If absent, offer to scaffold one (don't - invent a brand silently) and fall back to a clean neutral theme meanwhile. -2. **Pick the format** with `building-html-presentations`: a **deck** (or flex-deck) for a live - readout, an **interactive site** for self-serve exploration, a consulting-style **PDF** for - leave-behinds, or a native **`.pptx`** (`generating-pptx`) when it must be editable in - PowerPoint. State which and why. -3. **Lead with the story, not the chart dump.** One message per slide; title = the takeaway, - not the metric name. Structure for the audience (exec vs technical). -4. **Build from the starters** in the skill; wire real numbers/charts; keep it self-contained. - -## Data storytelling rules (domain-specific) -- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". -- **One idea per slide.** Move supporting detail to an appendix section. -- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. -- **Don't fabricate data.** Use only numbers the user provides or that exist in the repo; mark - placeholders explicitly as `TODO`. - -## Boundaries -- Building the *analysis itself* (metrics, models, ML) → `data-scientist`. -- Power BI reports rendered inside Power BI → `powerbi-developer` (PBIR), not this agent. -- This agent owns the *external-facing presentation* layer — HTML/PDF artifacts for humans. diff --git a/.opencode/commands/document-dashboard.md b/.opencode/commands/document-dashboard.md index e38f27f..5c31123 100644 --- a/.opencode/commands/document-dashboard.md +++ b/.opencode/commands/document-dashboard.md @@ -13,7 +13,8 @@ project root. ## Steps -1. **Inventory.** Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, +1. **Inventory.** If Power BI Desktop is open, have the user save first so the PBIP on disk + reflects the current model. Use `pbi-semantic-layer-tmdl` to confirm the PBIP project and count tables, measures, relationships, pages, and visuals. 2. **Parse both layers.** - **Semantic model (`.tmdl`):** names, data types, `description`, DAX `expressions`, and diff --git a/.opencode/commands/plan.md b/.opencode/commands/plan.md index 438fc60..91958f4 100644 --- a/.opencode/commands/plan.md +++ b/.opencode/commands/plan.md @@ -14,12 +14,16 @@ it does not write code. 1. **Understand.** Read the relevant parts of the repo. If a key requirement is genuinely ambiguous, ask; otherwise state your assumptions. -2. **Route.** Identify which agent(s) own each part of the work (e.g. `data-engineer` for the - pipeline, `analytics-engineer` for the model, `powerbi-developer` for the dashboard). +2. **Map to skills and scripts.** For each part of the work, name the skills it will load and + the validation scripts it will run (e.g. `pbi-semantic-layer-tmdl` + `run_bpa.py` for a model + change, `pbi-report-layer-pbir` + `validate_pbir.py` for a report change, + `dbt-modeling-conventions` for dbt models). 3. **Write the plan** to `docs/PLAN-<slug>.md`, where `<slug>` is 2–3 hyphenated keywords from the request. Include: - **Goal & scope** (and explicit non-goals). - - **Task breakdown** with the responsible agent per task. - - **Dependencies & risks.** - - **Verification checklist** (how we'll know it works). + - **Task breakdown** with the skills/scripts each task uses. + - **Dependencies & risks.** For Power BI work, state whether Desktop will be open (Modeling + MCP + Desktop Bridge) or closed (edit on disk). + - **Verification checklist** (how we'll know it works — scripts to run, and the independent + visual review for report changes). 4. **Report** the exact file path created and suggest reviewing it before implementing. diff --git a/.opencode/commands/validate-pbi.md b/.opencode/commands/validate-pbi.md index 8cb5128..4da4d10 100644 --- a/.opencode/commands/validate-pbi.md +++ b/.opencode/commands/validate-pbi.md @@ -1,46 +1,50 @@ --- name: validate-pbi -description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 + structural review). +description: Run a full quality check on a PBIP project (BPA via Tabular Editor 2 or the Python fallback, PBIR validation, structural review). --- # Workflow: /validate-pbi **Usage:** `/validate-pbi [path/to/project.pbip]` -Answers "is my model healthy?" by running the quality checks on a PBIP semantic model. If no -path is given, use the project in the current folder; if there are several, ask which one. +Answers "is my Power BI project healthy?". If no path is given, use the project in the current +folder; if there are several, ask which one. ## Steps -1. **Locate the project.** Use `pbi-semantic-layer-tmdl` to confirm a valid PBIP project. - If none is found, report it and stop. -2. **Run the BPA.** Use `pbi-quality-rules` to run the Best Practice Analyzer (Tabular Editor 2 - CLI, or the TMDL-parsing fallback). Recommend the user close Desktop first so disk is current. - Record every `error` and `warning`. -3. **Structural review.** - - **Doc coverage:** tables/measures with a `description` vs. total. - - **Relationships:** flag island tables (no relationships), bidirectional filters, and - many-to-many joins. -4. **Report** — a short Markdown summary: +1. **Locate the project** (`.pbip` + `<Name>.SemanticModel/` + `<Name>.Report/`). If none is + found, report it and stop. +2. **Make disk current.** If Power BI Desktop has the project open, ask the user to save + (Ctrl+S). Desktop does not need to be closed — the checks read the files Desktop saved. +3. **Model — BPA** (`pbi-quality-rules`): + `python3 <pbi-quality-rules>/scripts/run_bpa.py <project> --format json`. + Record every `error` and `warning` and which engine ran (Tabular Editor 2 or Python fallback). +4. **Report — PBIR** (`pbi-report-layer-pbir`): + `python3 <pbi-report-layer-pbir>/scripts/validate_pbir.py <project> --format json`. + Record errors and warnings and whether `powerbi-report-author` or the structural fallback ran. +5. **Structural review** of the TMDL: + - **Doc coverage:** tables/measures with a `///` description vs. total. + - **Relationships:** island tables, bidirectional filters, many-to-many joins. +6. **Report** — a short Markdown summary: ```markdown # Validation Report — <Project> -**Date:** <today> | **Source:** TMDL files +**Date:** <today> | **BPA engine:** Tabular Editor 2 | **PBIR check:** powerbi-report-author ## Summary -| Category | Errors | Warnings | -|----------|--------|----------| -| Naming | 0 | 3 | -| DAX | 1 | 2 | -| Model | 0 | 1 | -| Docs | 0 | 8 | +| Area | Errors | Warnings | +|------|--------|----------| +| Model — DAX | 1 | 1 | +| Model — Naming | 0 | 2 | +| Model — Docs | 0 | 2 | +| Report (PBIR) | 0 | 0 | ## Errors (fix before publishing) -- [DAX_EMPTY_EXPRESSION] Measure "Turnover %" — empty expression +- [DAX_EMPTY_EXPRESSION] Measure 'Sales'[Forecast] — empty expression ## Doc coverage -- Tables: 5/7 described (71%) · Measures: 12/20 described (60%) +- Tables: 3/4 described (75%) · Measures: 3/4 described (75%) ``` -5. **Offer fixes.** Auto-offer to fix all **errors**; ask whether to address **warnings** and - missing descriptions. +7. **Offer fixes.** Offer to fix every **error** (model fixes through the Modeling MCP when + Desktop is open, otherwise in TMDL); ask before addressing warnings and missing descriptions. diff --git a/.opencode/skills/building-html-presentations/SKILL.md b/.opencode/skills/building-html-presentations/SKILL.md index d6bfa34..3f5d505 100644 --- a/.opencode/skills/building-html-presentations/SKILL.md +++ b/.opencode/skills/building-html-presentations/SKILL.md @@ -26,13 +26,15 @@ deliverable is fundamentally a document (Markdown-only, clean PDF/PPTX, no inter **Slidev** is a fine choice for fully technical audiences already on Node/Vue — mention it, don't default to it. -## Workflow -1. **Audience & format.** Exec → fewer slides, takeaway titles, no code. Technical → allow - detail/appendix. Choose deck vs site vs PDF (table above) and say which. -2. **Outline before slides.** Agree the storyline (5–9 beats) before writing HTML. Lead with - the conclusion (BLUF), then evidence, then "so what / next". -3. **Apply the brand.** If a `DESIGN.md` exists, load `applying-visual-identity` and map its - tokens to CSS variables (see that skill). Otherwise use the neutral defaults in the starter. +## Working order +1. **Find the brand.** Look for a `DESIGN.md` in the project root. If present, load + `applying-visual-identity` and map its tokens to CSS variables. If absent, offer to scaffold + one (don't invent a brand silently) and use the neutral defaults in the starter meanwhile. +2. **Pick the format** from the table above and say which and why. A native `.pptx` that must be + editable in PowerPoint → `generating-pptx`. +3. **Story before charts.** Know the audience (exec → fewer slides, takeaway titles, no code; + technical → allow detail/appendix). Agree the storyline (5–9 beats) before writing HTML: lead + with the conclusion (BLUF), then evidence, then "so what / next". Use `data-story-outline.md`. 4. **Build from a starter** (`templates/`), drop in real numbers and charts, keep it **self-contained** (works offline; pin CDN versions or vendor the files). 5. **Export** if needed (see the export note below). @@ -93,7 +95,13 @@ All HTML starters use a generic **sales review** as demo content — replace it - Every data slide states **source, grain, date range, filters**. Never invent numbers — mark unknowns `TODO`. +## Data storytelling rules +- **Title = insight.** "Revenue fell 12% in EMEA", not "Revenue by region". +- **One idea per slide.** Move supporting detail to an appendix section. +- **Label the source & grain** on any data slide (table, date range, filters). Trust is the point. +- **Never fabricate data.** Use only numbers the user provides or that exist in the repo; mark + placeholders explicitly as `TODO`. + ## Rules -- One idea per slide; the title is the **insight**, not the field name. - Keep it self-contained and offline-capable; pin versions, don't rely on live network at present time. - Accessibility: real text (not images of text), sufficient contrast, logical heading order. diff --git a/.opencode/skills/building-html-presentations/templates/charts-reference.html b/.opencode/skills/building-html-presentations/templates/charts-reference.html index 310a9b7..ded2df6 100644 --- a/.opencode/skills/building-html-presentations/templates/charts-reference.html +++ b/.opencode/skills/building-html-presentations/templates/charts-reference.html @@ -3,8 +3,8 @@ ┌──────────────────────────────────────────────────────────────────────────┐ │ CHART COOKBOOK — the palette of chart types for the HTML starters. │ │ │ - │ A live gallery of ECharts patterns (+ CSS-only patterns) so the agent and │ - │ the user can see what's possible and copy a config into any deck/site/PDF │ + │ A live gallery of ECharts patterns (+ CSS-only patterns) so you can see │ + │ what's possible and copy a config into any deck/site/PDF │ │ template. Each card names the chart and when to use it. Same brand tokens │ │ as the other starters (map from DESIGN.md). Pinned ECharts. │ │ │ diff --git a/.opencode/skills/creating-agents-and-skills/SKILL.md b/.opencode/skills/creating-agents-and-skills/SKILL.md index f074ab3..19d871c 100644 --- a/.opencode/skills/creating-agents-and-skills/SKILL.md +++ b/.opencode/skills/creating-agents-and-skills/SKILL.md @@ -1,6 +1,6 @@ --- name: creating-agents-and-skills -description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, AGENTS.md) identical in each tool's native format. +description: Use when the user wants to create, add, extend, or modify an agent, skill, or workflow in this kit (e.g. "create a skill for our dbt naming standard", "add an agent for X", "new workflow"). Decides whether to build an agent, a skill, or nothing, enforces lean quality standards, and keeps every installed AI-tool folder (.agent, .claude, .github, .opencode, .cursor, AGENTS.md) identical in each tool's native format. --- # Skill: creating-agents-and-skills @@ -18,18 +18,27 @@ Run these gates before creating anything. If a gate says stop, explain why and s - **LLM-native? → don't create.** If a capable model already does this well ("write clean code", "use CTEs", "explain this function"), no artifact is needed. Just do the task. -- **Narrow knowledge / a tool / a format / a checklist? → make a SKILL.** Most additions are - skills: a team naming standard, a framework's rules, a validation procedure. Skills - auto-activate by description and cost almost nothing until used. -- **A distinct role with its own domain and judgement? → make an AGENT.** Only when there's a - persona that owns a body of work (like `data-engineer`). Agents are expensive (routing, - attention); the bar is high. -- **Overlaps an existing agent/skill? → extend it,** don't add a near-duplicate. Ambiguous - routing between two similar artifacts degrades results. +- **Narrow knowledge / a tool / a format / a checklist / a script? → make a SKILL.** This is + the default for almost everything: a team naming standard, a framework's rules, a validation + procedure. Skills auto-activate by description and cost almost nothing until used. - **A repeatable multi-step procedure the user invokes? → make a WORKFLOW** (slash command). +- **Overlaps an existing skill/agent? → extend it,** don't add a near-duplicate. -> Default to a skill. Recommend an agent only if you can name the distinct domain it owns and -> confirm nothing existing covers it. +### The agent-survival rule + +An **agent** is allowed only if it imposes something the model cannot impose on itself: + +1. **Restricted tools** — a frontmatter `tools` list that removes capabilities (e.g. read-only). +2. **Independent review** — fresh context, without the author's reasoning. +3. **A different model** than the main session. +4. **An automatic trigger** that fires without the user asking. + +If none applies, it is a skill: "a persona with a domain" is not a reason. Domain knowledge +belongs in a skill, or is dropped if the model already knows it. Agents must not hand off to +other agents. + +> Default to a skill. Recommend an agent only if you can name which of the four criteria it +> satisfies, and say so in the agent's description or body. --- @@ -37,18 +46,18 @@ Run these gates before creating anything. If a gate says stop, explain why and s Write only domain/org-specific signal. No filler, no restating what the model knows. English. -**Agent** — `.agent/agents/<name>.md`: +**Agent** — `.agent/agents/<name>.md` (example: `pbi-visual-reviewer`, a read-only reviewer): ```yaml --- -name: <lowercase-hyphenated> -description: One sharp sentence on what it does + trigger keywords (drives routing). -tools: Read, Grep, Glob, Bash, Edit, Write +name: pbi-visual-reviewer +description: Independently reviews Power BI report screenshots against the request and the PBIR folder; PASS/FAIL per page and visual. Use after a report change has been reloaded and screenshotted. +tools: Read, Grep, Glob # the restriction that justifies the agent: no Bash/Edit/Write model: inherit -skills: skill-a, skill-b # optional hints; skills also self-activate --- -# <Name> -<1–2 line identity> → domain rules → handoffs to other agents → out-of-scope. +# PBI Visual Reviewer +<1–2 line identity> → fixed checklist → output format (PASS/FAIL). No handoffs. ``` +`tools` is the field that matters: list only what the agent needs. **Skill** — `.agent/skills/<name>/SKILL.md` (gerund-style name, e.g. `validating-dbt-models`): ```yaml @@ -59,7 +68,7 @@ description: What it does AND when to use it, in the third person, with trigger --- # Skill: <name> <when/why> → concrete rules or steps. Keep the body under ~500 lines; move long reference -material to sibling files (e.g. reference.md, scripts/) and link them. +material to sibling files (e.g. references/, scripts/) and link them. ``` Write descriptions in the **third person** ("Generates...", "Use when...") — never "I can help". The description is what makes the skill fire without the user naming it. @@ -76,7 +85,8 @@ format. Pick the path that matches the project: ### Path A — Source of truth present (`.agent/` + `scripts/sync_agents.py` exist) This is the kit repo or an Antigravity install. **Author in `.agent/` only, then sync:** ```bash -python scripts/sync_agents.py +python3 scripts/sync_agents.py # regenerate +python3 scripts/sync_agents.py --check # exits 1 if generated folders are stale (CI runs this) ``` The script regenerates `.claude/`, `.github/`, `.opencode/`, `.cursor/`, and root `AGENTS.md` (which also covers OpenAI Codex). Never hand-edit those — they're overwritten. Done. @@ -87,8 +97,8 @@ format (the body content stays identical; only frontmatter changes). Format mapp | Concept | `.agent` (Antigravity) | `.claude` (Claude Code) | `.github` (Copilot) | `.opencode` (OpenCode) | `.cursor` (Cursor) | |---|---|---|---|---|---| -| Agent | `agents/<n>.md` | `agents/<n>.md` (identical) | `agents/<n>.agent.md` — frontmatter `name`, `description` only (no `tools`) | `agents/<n>.md` — frontmatter `description`, `mode: subagent`, `permission` (drop `tools`/`model`/`name`/`skills`) | — (Cursor has no file-based agents) | -| Skill | `skills/<n>/SKILL.md` | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | — (no skills concept) | +| Agent | `agents/<n>.md` | `agents/<n>.md` (identical) | `agents/<n>.agent.md` — frontmatter `name`, `description`, `tools` mapped from Claude names (Read→`read`, Grep/Glob→`search`, Edit/Write→`edit`, Bash→`execute`) | `agents/<n>.md` — frontmatter `description`, `mode: subagent`, `permission` (deny edit/bash when the agent is read-only; drop `tools`/`model`/`name`/`skills`) | — (none; run the task in a fresh chat) | +| Skill | `skills/<n>/SKILL.md` | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | `skills/<n>/SKILL.md` (identical) | | Workflow | `workflows/<n>.md` | `commands/<n>.md` (native slash command) + `workflows/<n>.md` | `prompts/<n>.prompt.md` — frontmatter `name`, `description`; body starts `**Context:** ${selection}` | `commands/<n>.md` (identical) | `commands/<n>.md` (identical) | | Global rule | `rules/rules.md` (with `trigger: always_on`) | `CLAUDE.md` (rules + commands section) | `copilot-instructions.md` (rules) | root `AGENTS.md` (rules) | `rules/<n>.mdc` — frontmatter `description`, `globs`, `alwaysApply: true` | @@ -104,11 +114,14 @@ Rules: ### Verify parity After either path, confirm the artifact exists in every present tool folder and that no folder -references something that no longer exists. State which folders you wrote. +references something that no longer exists. In Path A, `python3 scripts/sync_agents.py --check` +does this. State which folders you wrote. --- ## Anti-patterns -- Creating an agent for a single tool or task (e.g. "GitAgent"). Tools don't need agents. +- Creating an agent for a persona, a single tool, or a single task (e.g. "GitAgent") — it fails + the survival rule; write a skill. +- Chaining agents through handoffs. - Putting episodic guidance in the always-on rules — it belongs in a skill like this one. - Editing generated folders (`.claude`/`.github`/`.opencode`/`.cursor`/`AGENTS.md`) when `.agent/` exists. diff --git a/.opencode/skills/dbt-modeling-conventions/SKILL.md b/.opencode/skills/dbt-modeling-conventions/SKILL.md new file mode 100644 index 0000000..57446ce --- /dev/null +++ b/.opencode/skills/dbt-modeling-conventions/SKILL.md @@ -0,0 +1,63 @@ +--- +name: dbt-modeling-conventions +description: Applies the organization's dbt modeling conventions. Use when the user writes, reviews, or renames dbt models, mentions dbt, staging, marts, fct_ or dim_ tables, schema.yml, surrogate keys, model naming, CTE structure, or dimensional modeling (star schema) with dbt. +--- + +# Skill: dbt-modeling-conventions + +Org conventions only — the model already knows generic SQL and dbt. Apply these to every model +you write or review. + +## CTE structure +- `import` CTEs on top (one per `ref()`/`source()`, `select` only the needed columns). +- `logical` CTEs in the middle (joins, filters, calculations — one step each). +- One `final` CTE at the bottom; the model ends with `select * from final`. + +## Naming +| Prefix | Layer | +|---|---| +| `stg_` | Staging: 1:1 with a source table, rename/cast only | +| `dim_` | Dimension | +| `fct_` | Fact | +| `mart_` | Business-facing mart built on dims/facts | + +## Keys and grain +- Every dimension has a surrogate key `sk_<entity>`: a hash of the natural key. +- One grain per fact table; state it in the model description ("one row per order line"). +- Build dimensions before facts. + +## Tests and docs +Every model has a `schema.yml` entry with a description and `unique` + `not_null` on its +primary key. + +## Example + +```sql +-- models/marts/dim_customer.sql +with stg_customers as ( + select * from {{ ref('stg_crm__customers') }} +), +customers_keyed as ( + select + {{ dbt_utils.generate_surrogate_key(['customer_id']) }} as sk_customer, + customer_id, + customer_name, + country + from stg_customers +), +final as ( + select * from customers_keyed +) +select * from final +``` + +```yaml +# models/marts/schema.yml +models: + - name: dim_customer + description: One row per customer. + columns: + - name: sk_customer + description: Surrogate key, hash of customer_id. + tests: [unique, not_null] +``` diff --git a/.opencode/skills/pbi-quality-rules/SKILL.md b/.opencode/skills/pbi-quality-rules/SKILL.md index 1a8c059..775db7b 100644 --- a/.opencode/skills/pbi-quality-rules/SKILL.md +++ b/.opencode/skills/pbi-quality-rules/SKILL.md @@ -1,42 +1,44 @@ --- name: pbi-quality-rules -description: Validate a Power BI semantic model with the real Best Practice Analyzer (BPA) via the free Tabular Editor 2 CLI, using a shareable BPA rules file. +description: Validate a Power BI semantic model with the Best Practice Analyzer (BPA) — the real engine via the free Tabular Editor 2 CLI, or a Python fallback over TMDL when TE2 is not installed — using the shareable bpa-rules.json. Use when asked to lint, audit, check or validate a model, after significant model changes, and before publishing. --- # Skill: pbi-quality-rules -Run the **Best Practice Analyzer (BPA)** — the industry-standard model linter — against a PBIP -semantic model. This skill uses the *real* BPA engine from **Tabular Editor 2 (free, open -source)** instead of a homegrown checker, so results match what Power BI developers expect and -the rules are portable. +Two scripts in this skill's `scripts/` folder (Python 3.9+, standard library only): -## Rule set -`bpa-rules.json` (next to this file) holds the team's BPA rules in Tabular Editor's native -format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in Dynamic LINQ over the -TOM). Severity: `3` = error, `2` = warning, `1` = info. Edit this file to add or tune rules; -it loads directly into Tabular Editor's BPA UI too. +| Script | What it does | +|---|---| +| `run_bpa.py <model>` | Runs the real BPA through **Tabular Editor 2** (`TabularEditor.exe <definition-folder> -A bpa-rules.json -V`). If TE2 is not found, it runs the fallback and says so. | +| `bpa_fallback.py <model>` | The same rules re-implemented in Python over the TMDL files (or `model.bim`). Approximate; rules it cannot evaluate are listed as `SKIPPED`, never as passed. | -## Primary path — Tabular Editor 2 CLI (recommended) -Requires Tabular Editor 2 (free). Desktop should be closed so disk reflects the latest state. +`<model>` = the `.pbip`, the project folder, `<Name>.SemanticModel/`, its `definition/` folder, +or a `model.bim`. Options: `--format json`, `--rules <file>`, and for `run_bpa.py` also +`--te <path>` and `--no-fallback`. TE2 is looked up via `--te`, `$TABULAR_EDITOR_PATH`, `PATH`, +then the default Windows install folders. +Exit codes: `0` no error-severity violations · `1` errors found · `2` bad path / TE2 failed. + +## Run it ```bash -# Windows (TabularEditor.exe on PATH). Point at the .SemanticModel folder (TMDL) or model.bim. -TabularEditor.exe "Sales.SemanticModel" -A ".agent/skills/pbi-quality-rules/bpa-rules.json" -V +python3 <skill>/scripts/run_bpa.py path/to/Sales.pbip ``` - -- `-A <rulesfile>` runs the BPA with the given rules; `-V` writes violations to the console - (and sets a non-zero exit code on error-severity hits — useful in CI). -- The cross-platform `te` CLI (preview) accepts the same rules file if installed instead. - -Parse the console output, then present **errors and warnings only** (omit info unless asked). - -## Fallback — no Tabular Editor installed -The kit stays usable without the tool. Parse the TMDL files directly and evaluate the same -rules by hand: for each rule, read its `Scope` and `Expression` from `bpa-rules.json` and apply -the equivalent check to the parsed objects (measures, tables, columns, relationships). This is -less robust than the real BPA — say so, and recommend installing Tabular Editor 2 for accuracy. - -## Agent flow -1. Run after large model edits / refactors, or on demand. -2. Show errors first; for each `error`, offer an immediate fix. For `warning`, suggest but - don't force. Group by category so the report is scannable. +1. **Disk is the input.** With Desktop open and MCP edits pending, ask the user to save (Ctrl+S) + first — the BPA reads the TMDL Desktop wrote, not the live model. +2. Present **errors first**, then warnings, grouped by category; omit info unless asked. Say + which engine ran; for the fallback, recommend installing Tabular Editor 2 for exact results. +3. For each `error`, offer a fix (through the Modeling MCP when Desktop is open — see + `pbi-semantic-layer-tmdl`). Warnings: suggest, don't force. + +## Rule set — `bpa-rules.json` +Tabular Editor's native format (`ID`, `Name`, `Category`, `Severity`, `Scope`, `Expression` in +Dynamic LINQ over the TOM). Severity `3` error · `2` warning · `1` info. It loads directly into +TE2's BPA window too. + +Adding a rule: add it to `bpa-rules.json` (TE2 picks it up at once), then add a predicate with the +same `ID` to `CHECKS` in `scripts/bpa_fallback.py` so the fallback covers it; until then the +fallback reports it as `SKIPPED`. + +## In CI +Run `run_bpa.py --no-fallback --format json` on a Windows runner with TE2 (fails on exit 1), or +`bpa_fallback.py --format json` anywhere for a lighter gate. diff --git a/.opencode/skills/pbi-quality-rules/scripts/bpa_fallback.py b/.opencode/skills/pbi-quality-rules/scripts/bpa_fallback.py new file mode 100755 index 0000000..36f575b --- /dev/null +++ b/.opencode/skills/pbi-quality-rules/scripts/bpa_fallback.py @@ -0,0 +1,522 @@ +#!/usr/bin/env python3 +"""Best Practice Analyzer fallback: the rules of bpa-rules.json, evaluated in Python. + +Use this only when Tabular Editor 2 is not available (run_bpa.py calls it automatically). +It reads the semantic model straight from disk — TMDL (PBIP `definition/` folder) or a +`model.bim` — and re-implements each shipped rule by its ID. It is an approximation of the +real BPA engine: rules without a Python implementation are reported as skipped, never as passed. + +Usage: + python3 bpa_fallback.py <model> [--rules bpa-rules.json] [--format text|json] + +<model> may be a .pbip file, a project folder, a <Name>.SemanticModel folder, its +definition/ folder, or a model.bim file. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations, 2 = usage error. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from dataclasses import dataclass, field +from pathlib import Path +from typing import Callable, Optional + +DEFAULT_RULES = Path(__file__).resolve().parent.parent / "bpa-rules.json" +SEVERITY_NAMES = {1: "info", 2: "warning", 3: "error"} +AUTO_DATE_PREFIXES = ("LocalDateTable_", "DateTableTemplate_") + + +class UsageError(Exception): + """Bad input path or file; maps to exit code 2.""" + + +# --------------------------------------------------------------------------- +# Model +# --------------------------------------------------------------------------- +@dataclass +class Measure: + table: str + name: str + expression: str = "" + description: str = "" + display_folder: str = "" + is_hidden: bool = False + + +@dataclass +class Column: + table: str + name: str + description: str = "" + is_hidden: bool = False + + +@dataclass +class Table: + name: str + description: str = "" + is_hidden: bool = False + measures: list = field(default_factory=list) + columns: list = field(default_factory=list) + + +@dataclass +class Relationship: + name: str + from_table: str = "" + from_column: str = "" + to_table: str = "" + to_column: str = "" + cross_filtering: str = "oneDirection" + + +@dataclass +class Model: + source: Path + kind: str # "tmdl" | "bim" + tables: list = field(default_factory=list) + relationships: list = field(default_factory=list) + + @property + def measures(self): + return [m for t in self.tables for m in t.measures] + + @property + def columns(self): + return [c for t in self.tables for c in t.columns] + + +# --------------------------------------------------------------------------- +# Locating the model on disk +# --------------------------------------------------------------------------- +def _is_tmdl_folder(path: Path) -> bool: + return path.is_dir() and any((path / n).is_file() for n in ("model.tmdl", "database.tmdl")) + + +def _from_semantic_model_dir(path: Path) -> Optional[tuple]: + if _is_tmdl_folder(path / "definition"): + return "tmdl", path / "definition" + if (path / "model.bim").is_file(): + return "bim", path / "model.bim" + return None + + +def _semantic_model_from_pbip(pbip: Path) -> Optional[Path]: + sibling = pbip.with_name(pbip.stem + ".SemanticModel") + if sibling.is_dir(): + return sibling + try: + data = json.loads(pbip.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + return None + for artifact in data.get("artifacts", []): + report = artifact.get("report", {}).get("path") + if not report: + continue + pbir = pbip.parent / report / "definition.pbir" + try: + ref = json.loads(pbir.read_text(encoding="utf-8-sig")) + except (OSError, ValueError): + continue + by_path = ref.get("datasetReference", {}).get("byPath", {}).get("path") + if by_path: + candidate = (pbir.parent / by_path).resolve() + if candidate.is_dir(): + return candidate + return None + + +def resolve_model(path: Path) -> tuple: + """Return (kind, path) where kind is 'tmdl' (definition folder) or 'bim' (model.bim).""" + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() == ".bim": + return "bim", path + if path.suffix.lower() == ".pbip": + sm = _semantic_model_from_pbip(path) + found = _from_semantic_model_dir(sm) if sm else None + if found: + return found + raise UsageError(f"no semantic model found for {path.name} (thin report?)") + if path.name.lower() == "definition.pbism": + return resolve_model(path.parent) + raise UsageError(f"unsupported file: {path.name} (use .pbip, model.bim or a folder)") + if _is_tmdl_folder(path): + return "tmdl", path + found = _from_semantic_model_dir(path) + if found: + return found + models = sorted(p for p in path.glob("*.SemanticModel") if p.is_dir()) + if len(models) == 1: + found = _from_semantic_model_dir(models[0]) + if found: + return found + if len(models) > 1: + names = ", ".join(p.name for p in models) + raise UsageError(f"several semantic models in {path}: {names}; pass one explicitly") + raise UsageError(f"no TMDL definition or model.bim found under {path}") + + +# --------------------------------------------------------------------------- +# TMDL parser (the subset the rules need) +# --------------------------------------------------------------------------- +_PROPERTY = re.compile(r"^([A-Za-z][A-Za-z0-9_]*)\s*(?::\s*(.*))?$") + + +def _depth(line: str) -> int: + expanded = line[: len(line) - len(line.lstrip())].replace("\t", " ") + return len(expanded) // 4 + + +def _read_name(text: str) -> tuple: + """Read one TMDL object name from the start of `text`. Returns (name, rest).""" + text = text.lstrip() + if text.startswith("'"): + out, i = [], 1 + while i < len(text): + if text[i] == "'": + if i + 1 < len(text) and text[i + 1] == "'": + out.append("'") + i += 2 + continue + return "".join(out), text[i + 1 :] + out.append(text[i]) + i += 1 + return "".join(out), "" + match = re.match(r"[^\s=.]+", text) + if not match: + return "", text + return match.group(0), text[match.end() :] + + +def _split_column_ref(ref: str) -> tuple: + """`Sales.DateKey` / `'Sales Table'.'Date Key'` -> (table, column).""" + table, rest = _read_name(ref) + rest = rest.lstrip() + if rest.startswith("."): + column, _ = _read_name(rest[1:]) + return table, column + return "", table + + +def _unquote_value(value: str) -> str: + value = value.strip() + if len(value) >= 2 and value[0] == value[-1] == '"': + return value[1:-1].replace('""', '"') + return value + + +def parse_tmdl_folder(folder: Path) -> Model: + model = Model(source=folder, kind="tmdl") + tables: dict = {} + for file in sorted(folder.rglob("*.tmdl")): + _parse_tmdl_file(file.read_text(encoding="utf-8-sig"), model, tables) + model.tables = list(tables.values()) + return model + + +def _parse_tmdl_file(text: str, model: Model, tables: dict) -> None: + table: Optional[Table] = None + obj = None # Measure | Column | Relationship | Table (for table-level properties) + obj_depth = 0 + other_depth: Optional[int] = None # inside an object we do not model (partition, ...) + expr_lines: Optional[list] = None + expr_fenced = False + pending_doc: list = [] + + def finish_expression(): + nonlocal expr_lines + if expr_lines is not None and isinstance(obj, Measure): + obj.expression = "\n".join(expr_lines).strip() + expr_lines = None + + for raw in text.splitlines(): + stripped = raw.strip() + depth = _depth(raw) + + if expr_lines is not None: + if expr_fenced: + if stripped.startswith("```"): + finish_expression() + else: + expr_lines.append(stripped) + continue + if not stripped or depth > obj_depth + 1: + expr_lines.append(stripped) + continue + finish_expression() + + if not stripped: + continue + if stripped.startswith("///"): + pending_doc.append(stripped[3:].strip()) + continue + if stripped.startswith("//"): + continue + + if other_depth is not None: + if depth > other_depth: + continue + other_depth = None + + doc = " ".join(pending_doc).strip() + pending_doc = [] + keyword, _, rest = stripped.partition(" ") + + if depth == 0: + obj = None + table = None + if keyword == "table": + name, _ = _read_name(rest) + table = tables.setdefault(name, Table(name=name)) + if doc: + table.description = doc + obj, obj_depth = table, 0 + elif keyword == "relationship": + name, _ = _read_name(rest) + obj, obj_depth = Relationship(name=name), 0 + model.relationships.append(obj) + else: + other_depth = 0 + continue + + if table is not None and depth == 1 and keyword in ("measure", "column"): + name, after = _read_name(rest) + after = after.strip() + if keyword == "measure": + obj = Measure(table=table.name, name=name, description=doc) + table.measures.append(obj) + else: + obj = Column(table=table.name, name=name, description=doc) + table.columns.append(obj) + obj_depth = 1 + if after.startswith("="): + inline = after[1:].strip() + if inline.startswith("```"): + expr_lines, expr_fenced = [], True + elif inline: + if isinstance(obj, Measure): + obj.expression = inline + else: + expr_lines, expr_fenced = [], False + continue + + if obj is not None and depth == obj_depth + 1: + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(obj, match.group(1), match.group(2)) + continue + + if table is not None and depth == 1: + # Table-level property after members (e.g. `isHidden`, `lineageTag: …`). + obj, obj_depth = table, 0 + match = _PROPERTY.match(stripped) + if match and (match.group(2) is not None or " " not in stripped): + _set_property(table, match.group(1), match.group(2)) + continue + + # Any other nested object (partition, hierarchy, annotation, ...): skip its body. + other_depth = depth + + finish_expression() + + +def _set_property(obj, key: str, value: Optional[str]) -> None: + flag = value is None or value.strip().lower() == "true" + if key == "isHidden" and hasattr(obj, "is_hidden"): + obj.is_hidden = flag + elif key == "description" and hasattr(obj, "description") and value is not None: + obj.description = _unquote_value(value) + elif key == "displayFolder" and isinstance(obj, Measure) and value is not None: + obj.display_folder = _unquote_value(value) + elif isinstance(obj, Relationship) and value is not None: + if key == "fromColumn": + obj.from_table, obj.from_column = _split_column_ref(value) + elif key == "toColumn": + obj.to_table, obj.to_column = _split_column_ref(value) + elif key == "crossFilteringBehavior": + obj.cross_filtering = value.strip() + + +# --------------------------------------------------------------------------- +# model.bim (TMSL JSON) +# --------------------------------------------------------------------------- +def _text(value) -> str: + if isinstance(value, list): + return "\n".join(str(v) for v in value) + return str(value or "") + + +def parse_bim(path: Path) -> Model: + try: + data = json.loads(path.read_text(encoding="utf-8-sig")) + except ValueError as exc: + raise UsageError(f"{path} is not valid JSON: {exc}") from exc + raw_model = data.get("model", data) + model = Model(source=path, kind="bim") + for t in raw_model.get("tables", []): + table = Table(name=t.get("name", ""), description=_text(t.get("description")), + is_hidden=bool(t.get("isHidden"))) + for m in t.get("measures", []): + table.measures.append(Measure( + table=table.name, name=m.get("name", ""), expression=_text(m.get("expression")), + description=_text(m.get("description")), display_folder=m.get("displayFolder", "") or "", + is_hidden=bool(m.get("isHidden")))) + for c in t.get("columns", []): + if c.get("type") == "rowNumber": + continue + table.columns.append(Column(table=table.name, name=c.get("name", ""), + description=_text(c.get("description")), + is_hidden=bool(c.get("isHidden")))) + model.tables.append(table) + for r in raw_model.get("relationships", []): + model.relationships.append(Relationship( + name=r.get("name", ""), from_table=r.get("fromTable", ""), from_column=r.get("fromColumn", ""), + to_table=r.get("toTable", ""), to_column=r.get("toColumn", ""), + cross_filtering=r.get("crossFilteringBehavior", "oneDirection"))) + return model + + +def load_model(path: Path) -> Model: + kind, resolved = resolve_model(path) + return parse_tmdl_folder(resolved) if kind == "tmdl" else parse_bim(resolved) + + +# --------------------------------------------------------------------------- +# Rules — one Python predicate per rule ID in bpa-rules.json +# --------------------------------------------------------------------------- +def _blank(text: str) -> bool: + return not (text or "").strip() + + +def _tables_in_relationships(model: Model) -> set: + return {t for r in model.relationships for t in (r.from_table, r.to_table)} + + +# rule ID -> (scope, predicate(obj, model) -> True when the object VIOLATES the rule) +CHECKS: dict = { + "NAMING_MEASURE_PASCAL": ("Measure", lambda m, _: bool(re.match(r"^[a-z]", m.name))), + "NAMING_MEASURE_DISPLAYFOLDER": ("Measure", lambda m, _: not m.display_folder), + "DAX_EMPTY_EXPRESSION": ("Measure", lambda m, _: _blank(m.expression) + or m.expression.replace(" ", "").upper() == "BLANK()"), + "DAX_IFERROR_USAGE": ("Measure", lambda m, _: "IFERROR" in m.expression.upper()), + "MODEL_BIDIRECTIONAL": ("Relationship", lambda r, _: r.cross_filtering.lower() == "bothdirections"), + "MODEL_ISLAND_TABLE": ("Table", lambda t, model: t.name not in _tables_in_relationships(model) + and not t.name.startswith(AUTO_DATE_PREFIXES)), + "PERF_AUTO_DATETIME": ("Table", lambda t, _: t.name.startswith(AUTO_DATE_PREFIXES)), + "DOC_TABLE_DESCRIPTION": ("Table", lambda t, _: _blank(t.description)), + "DOC_MEASURE_DESCRIPTION": ("Measure", lambda m, _: _blank(m.description)), + "DOC_KEY_COLUMN_HIDDEN": ("Column", lambda c, _: bool(re.search(r"ID$", c.name)) and not c.is_hidden), +} + + +def _quote(name: str) -> str: + return "'" + name.replace("'", "''") + "'" + + +def object_label(obj) -> str: + if isinstance(obj, Table): + return _quote(obj.name) + if isinstance(obj, (Measure, Column)): + return f"{_quote(obj.table)}[{obj.name}]" + if isinstance(obj, Relationship): + return f"{_quote(obj.from_table)}[{obj.from_column}] -> {_quote(obj.to_table)}[{obj.to_column}]" + return str(obj) + + +def _objects(model: Model, scope: str) -> list: + return { + "Measure": model.measures, + "Column": model.columns, + "Table": model.tables, + "Relationship": model.relationships, + }[scope] + + +def load_rules(path: Path) -> list: + try: + rules = json.loads(Path(path).read_text(encoding="utf-8-sig")) + except (OSError, ValueError) as exc: + raise UsageError(f"cannot read rules file {path}: {exc}") from exc + if not isinstance(rules, list): + raise UsageError(f"rules file {path} must contain a JSON array") + return rules + + +def analyze(model: Model, rules: list) -> dict: + violations, skipped = [], [] + for rule in rules: + check: Optional[tuple] = CHECKS.get(rule.get("ID")) + if check is None: + skipped.append(rule.get("ID") or rule.get("Name")) + continue + scope, predicate = check + predicate: Callable + severity = SEVERITY_NAMES.get(int(rule.get("Severity", 1)), "info") + for obj in _objects(model, scope): + if predicate(obj, model): + violations.append({ + "rule_id": rule["ID"], + "rule_name": rule.get("Name", rule["ID"]), + "category": rule.get("Category", ""), + "severity": severity, + "object_type": scope, + "object": object_label(obj), + }) + order = {"error": 0, "warning": 1, "info": 2} + violations.sort(key=lambda v: (order[v["severity"]], v["category"], v["rule_id"], v["object"])) + return { + "engine": "python-fallback", + "model": str(model.source), + "violations": violations, + "skipped_rules": skipped, + "summary": {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")}, + } + + +def format_text(result: dict) -> str: + lines = [] + if result["engine"] == "python-fallback": + lines.append("BPA engine: Python fallback (approximate). Install Tabular Editor 2 for the real BPA.") + else: + lines.append("BPA engine: Tabular Editor 2") + lines.append(f"Model: {result['model']}") + for v in result["violations"]: + lines.append(f"{v['severity'].upper():8} [{v['rule_id']}] {v['object_type']} {v['object']} — {v['rule_name']}") + if not result["violations"]: + lines.append("No objects in violation of the rules.") + for rule_id in result.get("skipped_rules", []): + lines.append(f"SKIPPED [{rule_id}] no implementation in this engine") + s = result["summary"] + lines.append(f"Summary: {s['error']} error(s), {s['warning']} warning(s), {s['info']} info") + return "\n".join(lines) + + +def emit(result: dict, fmt: str) -> int: + print(json.dumps(result, indent=2) if fmt == "json" else format_text(result)) + return 1 if result["summary"]["error"] else 0 + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="BPA rules evaluated in Python over TMDL/model.bim.") + parser.add_argument("model", type=Path, help=".pbip, project folder, .SemanticModel, definition/ or model.bim") + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES, help="BPA rules JSON (default: bpa-rules.json)") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + try: + result = analyze(load_model(args.model), load_rules(args.rules)) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + return emit(result, args.format) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.opencode/skills/pbi-quality-rules/scripts/run_bpa.py b/.opencode/skills/pbi-quality-rules/scripts/run_bpa.py new file mode 100755 index 0000000..31e1232 --- /dev/null +++ b/.opencode/skills/pbi-quality-rules/scripts/run_bpa.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""Run the Best Practice Analyzer with Tabular Editor 2; fall back to Python when it is absent. + +Usage: + python3 run_bpa.py <model> [--rules bpa-rules.json] [--te PATH] [--format text|json] [--no-fallback] + +<model>: .pbip, project folder, <Name>.SemanticModel, its definition/ folder, or model.bim. + +Tabular Editor 2 is located in this order: --te, $TABULAR_EDITOR_PATH, TabularEditor(.exe) on +PATH, then the default Windows install folders. It runs as + + TabularEditor.exe <definition-folder|model.bim> -A <rules> -V + +(-A: analyze with this rules file only, plus rules embedded in the model; -V: Azure DevOps +logging lines, which carry the severity). Without TE2 the script runs bpa_fallback.py and says so. + +Disk is the input: with Power BI Desktop open, save first (Ctrl+S) so the TMDL is current. + +Exit codes: 0 = no error-severity violations, 1 = error-severity violations (or rule errors), +2 = usage error / Tabular Editor could not run. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +sys.dont_write_bytecode = True +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import bpa_fallback # noqa: E402 + +DEFAULT_RULES = bpa_fallback.DEFAULT_RULES +TE_TIMEOUT_SECONDS = 600 + +# "##vso[task.logissue type=warning;]Measure 'Sales'[x] violates rule "Rule name"" +_VSO = re.compile(r"^##vso\[task\.logissue type=(?P<level>error|warning)[^\]]*\](?P<text>.*)$") +_VIOLATION = re.compile(r'^(?P<type>.+?) (?P<object>(?:\'|\[).*) violates rule "(?P<rule>.*)"$') +_VIOLATION_LOOSE = re.compile(r'^(?P<type>\S+) (?P<object>.+?) violates rule "(?P<rule>.*)"$') +_RULE_ERROR = re.compile(r"^Error on rule '(?P<rule>.*?)': (?P<error>.*)$") + + +def find_tabular_editor(explicit: Optional[str] = None, env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + for candidate in (explicit, env.get("TABULAR_EDITOR_PATH")): + if candidate: + return candidate if Path(candidate).is_file() else None + for name in ("TabularEditor.exe", "TabularEditor"): + found = shutil.which(name) + if found: + return found + for base in (env.get("ProgramFiles(x86)"), env.get("ProgramFiles"), env.get("LOCALAPPDATA")): + if base: + exe = Path(base) / "Tabular Editor" / "TabularEditor.exe" + if exe.is_file(): + return str(exe) + return None + + +def build_command(te_path: str, model_target: Path, rules: Path) -> list: + return [te_path, str(model_target), "-A", str(rules), "-V"] + + +def parse_te_output(stdout: str, rules: list, model_label: str) -> dict: + """Turn TE2 console output (-A ... -V) into the same result shape as the fallback.""" + by_name = {r.get("Name"): r for r in rules} + violations, rule_errors = [], [] + for raw in stdout.splitlines(): + line = raw.strip() + if not line or line.startswith("##vso[task.complete"): + continue + level = "info" + match = _VSO.match(line) + if match: + level, line = match.group("level"), match.group("text").strip() + rule_error = _RULE_ERROR.match(line) + if rule_error: + rule_errors.append({"rule_name": rule_error.group("rule"), "error": rule_error.group("error")}) + continue + found = _VIOLATION.match(line) or _VIOLATION_LOOSE.match(line) + if not found: + continue + rule = by_name.get(found.group("rule"), {}) + violations.append({ + "rule_id": rule.get("ID", ""), + "rule_name": found.group("rule"), + "category": rule.get("Category", ""), + "severity": level, + "object_type": found.group("type"), + "object": found.group("object"), + }) + summary = {s: sum(1 for v in violations if v["severity"] == s) for s in ("error", "warning", "info")} + return { + "engine": "tabular-editor-2", + "model": model_label, + "violations": violations, + "rule_errors": rule_errors, + "skipped_rules": [], + "summary": summary, + } + + +def run_tabular_editor(te_path: str, model_target: Path, rules_path: Path, rules: list) -> dict: + cmd = build_command(te_path, model_target, rules_path) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=TE_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise bpa_fallback.UsageError(f"Tabular Editor failed to run ({' '.join(cmd)}): {exc}") from exc + result = parse_te_output(proc.stdout, rules, str(model_target)) + loaded = "Running Best Practice Analyzer" in proc.stdout + if proc.returncode != 0 and not loaded: + detail = (proc.stderr or proc.stdout).strip()[-2000:] + raise bpa_fallback.UsageError(f"Tabular Editor exited {proc.returncode} before the BPA ran:\n{detail}") + return result + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Run the BPA via Tabular Editor 2 (Python fallback if absent).") + parser.add_argument("model", type=Path) + parser.add_argument("--rules", type=Path, default=DEFAULT_RULES) + parser.add_argument("--te", help="path to TabularEditor.exe (overrides $TABULAR_EDITOR_PATH)") + parser.add_argument("--format", choices=("text", "json"), default="text") + parser.add_argument("--no-fallback", action="store_true", help="fail (exit 2) instead of using the Python fallback") + args = parser.parse_args(argv) + + try: + rules = bpa_fallback.load_rules(args.rules) + kind, target = bpa_fallback.resolve_model(args.model) + te_path = find_tabular_editor(args.te) + if te_path: + result = run_tabular_editor(te_path, target, args.rules.resolve(), rules) + elif args.no_fallback or args.te: + raise bpa_fallback.UsageError("Tabular Editor 2 not found (use --te or $TABULAR_EDITOR_PATH)") + else: + print("Tabular Editor 2 not found; using the Python fallback.", file=sys.stderr) + parsed = bpa_fallback.parse_tmdl_folder(target) if kind == "tmdl" else bpa_fallback.parse_bim(target) + result = bpa_fallback.analyze(parsed, rules) + except bpa_fallback.UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + if args.format == "json": + print(json.dumps(result, indent=2)) + else: + print(bpa_fallback.format_text(result)) + for err in result.get("rule_errors", []): + print(f"RULE-ERR [{err['rule_name']}] {err['error']}") + has_errors = result["summary"]["error"] > 0 or bool(result.get("rule_errors")) + return 1 if has_errors else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.opencode/skills/pbi-report-layer-pbir/SKILL.md b/.opencode/skills/pbi-report-layer-pbir/SKILL.md index 28ebab9..898d4d7 100644 --- a/.opencode/skills/pbi-report-layer-pbir/SKILL.md +++ b/.opencode/skills/pbi-report-layer-pbir/SKILL.md @@ -1,62 +1,128 @@ --- name: pbi-report-layer-pbir -description: Edit the Power BI report layer as code — pages, visuals, bookmarks, and themes — by writing PBIR JSON files in a PBIP project (offline, Desktop closed). +description: Edit the Power BI report layer as code — pages, visuals, filters, bookmarks and themes in the PBIR files of a PBIP project — with Power BI Desktop OPEN, using the edit → validate → save → status → reload → screenshot → review loop (Desktop Bridge via powerbi-report-author). Also covers the Desktop-closed path. Use for any request to add, move, restyle or fix a report page or visual. --- # Skill: pbi-report-layer-pbir -The report layer (`.Report/`) is **PBIR** — report metadata decomposed into per-page and -per-visual JSON files, each with a public JSON schema. PBIR is becoming the default report -format (rolling out across Service and Desktop through 2026), and it is explicitly designed to -be edited by tools other than Power BI. +The report layer (`<Name>.Report/`) is **PBIR**: one JSON file per page and per visual, each +with a public JSON schema. Power BI Desktop can stay **open** while you edit it: the +[Desktop Bridge](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +lets a tool reload the files from disk and take screenshots of the rendered pages. -## Hard precondition -**Power BI Desktop must be CLOSED** while editing these `.json` files — live edits corrupt or -get overwritten by autosave. +Model objects (measures, columns, relationships) are NOT edited here — they go live through the +Modeling MCP (`pbi-semantic-layer-tmdl`). Create the measure first, then bind the visual to it. + +## Tools +- `powerbi-report-author` — Microsoft's report CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`, + Node 20+). Validates PBIR, describes visual roles/formatting, and drives Desktop through + `preview --host desktop`. It bundles the lower-level bridge CLI `powerbi-desktop` + (`status`, `reload`, `screenshot`); always go through `powerbi-report-author preview`, which + resolves the right Desktop window from the report folder. Details, errors and setup: + [references/desktop-bridge.md](references/desktop-bridge.md). +- `scripts/validate_pbir.py` (in this skill's folder) — runs `powerbi-report-author validate`, or + a structural check when the CLI is missing. +- `pbi-visual-reviewer` agent — independent screenshot review. + +## The loop (Desktop open) +`<folder>` = the project folder or the `.pbip` path; `<shots>` = an absolute folder OUTSIDE the +PBIP (e.g. `%TEMP%\pbir-review\<Name>`), so screenshots never land in git. + +1. **Read before writing.** `powerbi-report-author preview-pages <folder>` and + `preview-visuals <folder>` list pages and visuals with their file paths. For a new or changed + visual, `powerbi-report-author catalog describe <visualType>` gives the exact role names + (e.g. `cardVisual` uses `Data`, `clusteredColumnChart` uses `Category` + `Y`); + `formatting describe-object <visualType> <object>` gives property names. Do not guess them. +2. **Edit** the PBIR JSON (anatomy below). One logical batch at a time. +3. **Validate:** `python3 <skill>/scripts/validate_pbir.py <folder>`. Fix every error before + going on; read the warnings. +4. **Save gate.** Ask the user to save in Desktop (Ctrl+S) if they changed anything by hand, + then check: `powerbi-report-author preview <folder> --host desktop --status`. + Continue only when `data.details.hasUnsavedChanges` is exactly `false`. +5. **Reload:** `powerbi-report-author preview <folder> --host desktop --reload`. +6. **Screenshot every affected page:** + `powerbi-report-author preview <folder> --host desktop --screenshot "<shots>" --all-pages` + (or `--screenshot "<shots>\<page>.png" --page <pageId>`). A screenshot does not reload; always + reload first. +7. **Review:** hand the PNG paths, the user's request/spec and the `.Report` path — and nothing + about your reasoning — to the `pbi-visual-reviewer` agent. In tools without subagents + (Cursor, Antigravity), run its checklist in a fresh chat with only those inputs. +8. REJECTED → fix the listed files and repeat from step 3. APPROVED → report the result and let + the user save/commit. + +### Pitfalls (each one has lost real work) +- **Reload discards whatever Desktop has not saved.** That is why step 4 exists. If status says + `true`, `null`, or fails, stop and ask the user to save; never reload "just to see". +- **Reload covers the report only.** `--reload` does not re-read model files. Model changes go + through the Modeling MCP, which updates Desktop live. Use `--reload-with-model` only when TMDL + was edited on disk (Desktop-closed habits) — and it still does not refresh data. +- **An edited theme may not show after reload.** Desktop caches themes by file name. When you + change a custom theme, rename its JSON (e.g. `Brand-3f9a1c.json` → `Brand-7b2e04.json`) and + update both references in `report.json` (`themeCollection.customTheme.name` and the + `resourcePackages` item `name`/`path`), then validate and reload. +- A page added on disk is unknown to the open window until a reload; a screenshot of it fails + with `RENDER_FAILED`. +- Two Desktop windows with the same report → `HOST_UNAVAILABLE`. Ask the user to close one; + never close Desktop yourself. + +## Alternative: Desktop closed +Use it when there is no Bridge (Desktop older than the Bridge release, the setting +*File > Options > Security > Desktop Bridge* is off, or the machine is not Windows) and in CI. +1. Make sure Desktop is **closed** — an open Desktop overwrites the files on its next save. +2. Edit the PBIR, run `validate_pbir.py` (in CI: `--format json`, fail on exit code 1). +3. Ask the user to open the `.pbip` and check the pages; no screenshots, so ask for one or for + a description, and run the reviewer checklist on what you get. ## `.Report` structure (PBIR) ``` -<Project>.Report/ - definition.pbir ← pointer (model binding) +<Name>.Report/ + definition.pbir ← model binding (byPath ../<Name>.SemanticModel) definition/ - report.json ← master metadata incl. themeCollection + version.json + report.json ← themeCollection, resourcePackages, settings pages/ + pages.json ← pageOrder + activePageName <pageId>/ - page.json ← canvas size, background, displayName - visuals/<visualId>/visual.json ← one visual per file + page.json ← name (= folder), displayName, width, height + visuals/<visualId>/visual.json ← one visual per file; name = folder StaticResources/ - RegisteredResources/<id>/<id>.json ← custom themes (create new themes here) - SharedResources/BaseThemes/ ← Microsoft base themes — NEVER edit + RegisteredResources/ ← custom themes and images you add + SharedResources/BaseThemes/ ← Microsoft base themes — never edit ``` - -## Navigating -- **Pages:** scan `definition/pages/`; read each `page.json` `displayName` (the user-facing name). -- **Visuals:** files sit under GUID folders. To find "the bar chart", grep `visual.json` for - `"visualType": "barChart"`. +PBIR-Legacy (a single `report.json` at the `.Report` root) cannot be edited this way: ask the +user to save the report in PBIR format first. ## Visual anatomy (`visual.json`) ```json { - "name": "<new GUID if creating>", - "position": { "x": 10, "y": 20, "width": 400, "height": 300, "tabOrder": 1 }, + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/visualContainer/2.9.0/schema.json", + "name": "7b2d4f6a8c0e1a3b5d02", + "position": { "x": 40, "y": 240, "z": 2000, "width": 760, "height": 420, "tabOrder": 2000 }, "visual": { - "visualType": "barChart", - "query": { "queryState": { "Category": { "projections": [] }, "Y": { "projections": [] } } } + "visualType": "clusteredColumnChart", + "query": { "queryState": { + "Category": { "projections": [ { "field": { "Column": { "Expression": { "SourceRef": { "Entity": "Product" } }, "Property": "Category" } }, "queryRef": "Product.Category", "nativeQueryRef": "Category", "active": true } ] }, + "Y": { "projections": [ { "field": { "Measure": { "Expression": { "SourceRef": { "Entity": "Sales" } }, "Property": "Total Sales" } }, "queryRef": "Sales.Total Sales", "nativeQueryRef": "Total Sales" } ] } + } }, + "visualContainerObjects": { "title": [ { "properties": { + "show": { "expr": { "Literal": { "Value": "true" } } }, + "text": { "expr": { "Literal": { "Value": "'Sales by category'" } } } } } ] } } } ``` -Positioning is a pixel grid (x=0, y=0 top-left). Generate fresh GUIDs for any new component -(`[System.Guid]::NewGuid()` on Windows, or any UUID generator). +- New page/visual: fresh 20-hex-char `name`, folder with the same name; add new pages to + `pages.json` `pageOrder`. +- Position is in pixels from the page's top-left; keep `x + width ≤ page width` and + `y + height ≤ page height`. Align edges to a shared grid (e.g. multiples of 8) and keep equal gaps. +- Literal encoding: text `'quoted'`, numbers `12D`, booleans `true`. When unsure, + `powerbi-report-author expr encode --kind <type> <value>`. +- Avoid visuals Microsoft is retiring (Q&A, Bing Map, filled map). ## Themes -1. Create the theme JSON under `StaticResources/RegisteredResources/<guid>/<guid>.json`. -2. Register that `<guid>` in `report.json`'s `themeCollection`. -3. Define ≥ 8 `dataColors`. To standardize fonts globally, use the wildcard `visualStyles`: - ```json - "visualStyles": { "*": { "*": { "fontSize": [{ "value": 11 }], "fontFamily": [{ "value": "Segoe UI" }] } } } - ``` - -## Cautions -- The visual schema evolves with Desktop releases; old keys may be ignored on newer versions. -- Validate JSON against the PBIR schema where possible. Always have the user commit (`git add`) - before large UI refactors and test by opening the file in Desktop. +Create custom themes under `StaticResources/RegisteredResources/`, register them in +`report.json` (`themeCollection.customTheme` + a `resourcePackages` item of type +`RegisteredResources`), define ≥ 8 `dataColors`, and standardize fonts with the wildcard +`visualStyles` (`"*": { "*": { ... } }`). Rename the file on every edit (pitfall above). + +## Before large changes +Ask the user to commit the PBIP first so every agent edit is a reviewable diff. diff --git a/.opencode/skills/pbi-report-layer-pbir/references/desktop-bridge.md b/.opencode/skills/pbi-report-layer-pbir/references/desktop-bridge.md new file mode 100644 index 0000000..ef02d9a --- /dev/null +++ b/.opencode/skills/pbi-report-layer-pbir/references/desktop-bridge.md @@ -0,0 +1,83 @@ +# Desktop Bridge — setup, commands, errors + +Sources (read these when something here looks outdated): +- Microsoft Learn — [What is the Power BI Desktop Bridge?](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +- Microsoft Learn — [Power BI Report skill](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-report-skill-overview) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `powerbi-report-cli`, MIT) — the official agent guidance this kit follows. +- npm: [`@microsoft/powerbi-report-authoring-cli`](https://www.npmjs.com/package/@microsoft/powerbi-report-authoring-cli) + (`powerbi-report-author`), [`@microsoft/powerbi-desktop-bridge-cli`](https://www.npmjs.com/package/@microsoft/powerbi-desktop-bridge-cli) + (`powerbi-desktop`). Both MIT, Node 20+. + +## What it is +A local JSON-RPC 2.0 server inside each running Power BI Desktop window, reachable only on the +same Windows machine through the named pipe `pbi-desktop-bridge-<processId>`. One operation runs +at a time per window; a second request while one is running fails — wait and retry, never run +two Desktop commands in parallel. + +Methods (discover the current list with `bridge.manifest`): +| Method | Used for | +|---|---| +| `bridge.manifest` | Supported methods and their schemas for this Desktop build | +| `application.state.get/v1` | `currentFilePath`, `hasUnsavedChanges` | +| `report.snapshot.capture/v1` | PNG of one page (`pageId`, `scale` 1–3) | +| `file.reload/v1` | Reload the open PBIP from disk; `reloadModelDefinition` true/false | + +## Setup (once per machine) +1. Power BI Desktop for Windows, recent enough to show **File > Options and settings > Options > + Security > Desktop Bridge**. Keep *Enable external tool access to Power BI Desktop through + secure local APIs* checked (default on). Admins can force it with the registry value + `HKLM\SOFTWARE\Policies\Microsoft\Power BI Desktop\DesktopNamedPipeBridge` (`1` on, `0` off). +2. Node.js 20+, then `npm i -g @microsoft/powerbi-report-authoring-cli`. +3. Check: `powerbi-report-author doctor` and `powerbi-report-author preview --list-hosts` + (the `desktop` host must say `available: true` while a PBIP is open). +4. The report must be a **PBIP saved in PBIR format** (not a `.pbix`, not PBIR-Legacy). + +## Commands +Use `powerbi-report-author preview` for every Desktop operation; it finds the Desktop window +that has `<folder>` open, so there is no PID to pass. `<folder>` = the project folder or `.pbip`. + +| Intent | Command | +|---|---| +| Status (save gate) | `powerbi-report-author preview "<folder>" --host desktop --status` | +| Open / attach | `powerbi-report-author preview "<folder>" --host desktop` | +| Reload report | `powerbi-report-author preview "<folder>" --host desktop --reload` | +| Reload report + on-disk model | `powerbi-report-author preview "<folder>" --host desktop --reload-with-model` | +| Screenshot one page | `powerbi-report-author preview "<folder>" --host desktop --screenshot "<shots>\<page>.png" --page "<pageId or display name>"` | +| Screenshot all pages | `powerbi-report-author preview "<folder>" --host desktop --screenshot "<shots>" --all-pages` | +| Validate files | `powerbi-report-author validate "<folder>"` (`--no-schema` offline) | + +Optional `--scale 1..3` on screenshots. Every command prints a JSON envelope: +`{"data": {...}}` on success, `{"error": {"code": ..., "message": ..., "correction"?: {...}}}` on +failure. If an error carries a `correction` object, retry once changing only that value; +otherwise report the error and stop — do not improvise another command. + +The bridge CLI underneath (`powerbi-desktop status | manifest | open | reload | screenshot | +screenshot-all`) is useful for manual troubleshooting by the user (e.g. `powerbi-desktop status` +lists every open window and its unsaved state). Agents stay on `powerbi-report-author`. + +### Status before open/reload — never before a screenshot +```json +{"data":{"state":"ready","reports":[{"folder":"C:\\work\\Sales","hostInstanceId":"12345"}], + "details":{"hasUnsavedChanges":false}}} +``` +| `details.hasUnsavedChanges` | Action | +|---|---| +| `false` | Reload / open is allowed. | +| `true` | Stop. Ask the user to save (Ctrl+S) — or to discard — in Desktop, then check again. | +| missing, `null`, or the call failed | Treat as unsafe. Stop and report. | +| error `HOST_UNAVAILABLE` (no window has the report) | Open/attach is allowed; reload is not. | + +## Errors you will meet +| Code / symptom | Meaning | What to do | +|---|---|---| +| `UNSUPPORTED_OS` in `--list-hosts` | Not on Windows | Use the Desktop-closed path; `validate` still works. | +| `HOST_UNAVAILABLE` — "Several Power BI Desktop instances have this report open" | Same report in 2+ windows (`details.candidatePids`) | Ask the user to close the extras. Never close Desktop yourself. | +| `HOST_UNAVAILABLE` — no instance | Desktop closed or setting off | Ask the user to open the `.pbip`; check the Security setting. | +| `RENDER_FAILED` with "unknown page id" | Page exists on disk but not in the open window | Status → reload → capture that page again. | +| `PAGE_NOT_FOUND` | The page is not in the definition at all | Fix `pages.json` / the page id. Reload will not help. | +| `status: "partial"` + non-zero exit | Some pages captured, some failed | Keep the PNGs you got; report which pages failed and why. | +| JSON-RPC `-32601 MethodNotFound` (bridge CLI) | Desktop build lacks that method | Update Desktop; check `powerbi-desktop manifest`. | +| "operation in progress" | Another bridge call is running | Wait, then retry once. | +| Theme change invisible after reload | Desktop cached the theme by file name | Rename the theme file + both refs in `report.json`, validate, reload. | +| Change invisible after reload, no error | Edited a file Desktop does not read (e.g. outside `definition/`) or the wrong report folder | `preview-visuals <folder>` to confirm paths; check `--status` `folder`. | diff --git a/.opencode/skills/pbi-report-layer-pbir/scripts/validate_pbir.py b/.opencode/skills/pbi-report-layer-pbir/scripts/validate_pbir.py new file mode 100755 index 0000000..008c29b --- /dev/null +++ b/.opencode/skills/pbi-report-layer-pbir/scripts/validate_pbir.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Validate a PBIR report: Microsoft's `powerbi-report-author validate` when installed, +otherwise a structural check written in Python. + +Usage: + python3 validate_pbir.py <path> [--structural-only] [--offline] [--format text|json] + +<path>: a .pbip file, a <Name>.Report folder, a folder containing definition/, or a project +folder with exactly one *.Report. + +The CLI (`npm i -g @microsoft/powerbi-report-authoring-cli`) is found through +$PBIR_AUTHOR_CLI or `powerbi-report-author` on PATH. It validates every file against the +published PBIR JSON schemas plus visual roles/types, so it is the authoritative check. +--offline passes --no-schema to it (no download of remote schemas). + +The structural fallback checks what breaks Desktop most often: JSON that does not parse, +PBIR-Legacy layout, a broken semantic-model binding, pages.json out of sync with the page +folders, page/visual `name` != folder, visuals without position/visualType, visuals outside +the page canvas, and theme/resource files that are registered but missing. It does NOT know +the schemas or visual roles — say so when reporting. + +Exit codes: 0 = no errors, 1 = errors found, 2 = usage error / the CLI could not run. +Python 3.9+, standard library only. +""" + +from __future__ import annotations + +import argparse +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path +from typing import Optional + +CLI_NAME = "powerbi-report-author" +CLI_TIMEOUT_SECONDS = 300 + + +class UsageError(Exception): + pass + + +# --------------------------------------------------------------------------- +# Locate the .Report folder +# --------------------------------------------------------------------------- +def _read_json(path: Path): + return json.loads(path.read_text(encoding="utf-8-sig")) + + +def resolve_report(path: Path) -> Path: + path = Path(path) + if not path.exists(): + raise UsageError(f"path does not exist: {path}") + if path.is_file(): + if path.suffix.lower() != ".pbip": + raise UsageError(f"unsupported file: {path.name} (use a .pbip or a .Report folder)") + try: + artifacts = _read_json(path).get("artifacts", []) + except ValueError as exc: + raise UsageError(f"{path.name} is not valid JSON: {exc}") from exc + for artifact in artifacts: + report = artifact.get("report", {}).get("path") + if report and (path.parent / report).is_dir(): + return (path.parent / report).resolve() + sibling = path.with_name(path.stem + ".Report") + if sibling.is_dir(): + return sibling.resolve() + raise UsageError(f"{path.name} does not point to an existing .Report folder") + if (path / "definition").is_dir() or (path / "definition.pbir").is_file() or path.suffix == ".Report": + return path.resolve() + reports = sorted(p for p in path.glob("*.Report") if p.is_dir()) + if len(reports) == 1: + return reports[0].resolve() + if len(reports) > 1: + raise UsageError(f"several reports in {path}: {', '.join(p.name for p in reports)}; pass one") + raise UsageError(f"no .Report folder found under {path}") + + +# --------------------------------------------------------------------------- +# Structural fallback +# --------------------------------------------------------------------------- +class Findings: + def __init__(self, report_dir: Path): + self.report_dir = report_dir + self.items: list = [] + + def add(self, severity: str, code: str, message: str, file: Optional[Path] = None, path: str = ""): + rel = "" + if file is not None: + try: + rel = file.relative_to(self.report_dir).as_posix() + except ValueError: + rel = str(file) + self.items.append({"severity": severity, "code": code, "message": message, "file": rel, "path": path}) + + def error(self, *args, **kwargs): + self.add("error", *args, **kwargs) + + def warning(self, *args, **kwargs): + self.add("warning", *args, **kwargs) + + +def _load(findings: Findings, file: Path, required: bool = True): + if not file.is_file(): + if required: + findings.error("FILE_MISSING", f"required file is missing: {file.name}", file) + return None + try: + return _read_json(file) + except ValueError as exc: + findings.error("JSON_INVALID", f"invalid JSON: {exc}", file) + return None + + +def _is_number(value) -> bool: + return isinstance(value, (int, float)) and not isinstance(value, bool) + + +def structural_check(report_dir: Path) -> dict: + f = Findings(report_dir) + definition = report_dir / "definition" + + for file in sorted(report_dir.rglob("*.json")): + if ".pbi" in file.relative_to(report_dir).parts: + continue + try: + _read_json(file) + except ValueError as exc: + f.error("JSON_INVALID", f"invalid JSON: {exc}", file) + invalid = {i["file"] for i in f.items} + + if not definition.is_dir(): + if (report_dir / "report.json").is_file(): + f.error("PBIR_LEGACY_FORMAT", "PBIR-Legacy layout (report.json at the root); save the " + "report in PBIR format (Desktop: File > Options > Preview features > PBIR)", report_dir / "report.json") + else: + f.error("FILE_MISSING", "definition/ folder is missing", definition) + return _result(f, "structural") + + pbir = _load(f, report_dir / "definition.pbir") + if isinstance(pbir, dict): + ref = pbir.get("datasetReference", {}) + by_path = ref.get("byPath", {}).get("path") if isinstance(ref.get("byPath"), dict) else None + if by_path: + target = (report_dir / by_path).resolve() + if not target.is_dir(): + f.error("MODEL_BINDING_BROKEN", f"datasetReference.byPath points to a missing folder: {by_path}", + report_dir / "definition.pbir", "datasetReference.byPath.path") + elif not ref.get("byConnection"): + f.error("MODEL_BINDING_MISSING", "datasetReference has neither byPath nor byConnection", + report_dir / "definition.pbir", "datasetReference") + + _load(f, definition / "version.json") + report = _load(f, definition / "report.json") + if isinstance(report, dict): + _check_resources(f, report, report_dir) + + pages_dir = definition / "pages" + pages_meta = _load(f, pages_dir / "pages.json") + page_folders = sorted(p.name for p in pages_dir.iterdir() if p.is_dir()) if pages_dir.is_dir() else [] + if isinstance(pages_meta, dict): + order = pages_meta.get("pageOrder", []) + for page_id in order: + if page_id not in page_folders: + f.error("PAGE_MISSING", f"pages.json lists page '{page_id}' but its folder is missing", + pages_dir / "pages.json", "pageOrder") + for page_id in page_folders: + if page_id not in order: + f.warning("PAGE_NOT_IN_ORDER", f"page folder '{page_id}' is not listed in pages.json pageOrder", + pages_dir / page_id) + active = pages_meta.get("activePageName") + if active and active not in order: + f.error("ACTIVE_PAGE_UNKNOWN", f"activePageName '{active}' is not in pageOrder", + pages_dir / "pages.json", "activePageName") + + for page_id in page_folders: + _check_page(f, pages_dir / page_id, invalid) + + return _result(f, "structural") + + +def _check_resources(f: Findings, report: dict, report_dir: Path) -> None: + static = report_dir / "StaticResources" + for package in report.get("resourcePackages", []) or []: + folder = static / package.get("name", "") + for item in package.get("items", []) or []: + rel = item.get("path") + if rel and not (folder / rel).is_file(): + f.error("RESOURCE_MISSING", f"resource '{item.get('name')}' registered in report.json is " + f"missing on disk: {package.get('name')}/{rel}", report_dir / "definition" / "report.json", + "resourcePackages") + custom = (report.get("themeCollection") or {}).get("customTheme") + if isinstance(custom, dict) and custom.get("name"): + registered = { + item.get("name") + for package in report.get("resourcePackages", []) or [] + if package.get("type") == custom.get("type", "RegisteredResources") + for item in package.get("items", []) or [] + } + if custom["name"] not in registered: + f.error("THEME_NOT_REGISTERED", f"customTheme '{custom['name']}' has no matching resourcePackages item", + report_dir / "definition" / "report.json", "themeCollection.customTheme.name") + + +def _check_page(f: Findings, page_dir: Path, invalid: set) -> None: + page_file = page_dir / "page.json" + page = _load(f, page_file) + if not isinstance(page, dict): + return + if page.get("name") != page_dir.name: + f.error("NAME_MISMATCH", f"page.json name '{page.get('name')}' != folder '{page_dir.name}'", page_file, "name") + if not page.get("displayName"): + f.warning("PAGE_NO_DISPLAY_NAME", "page has no displayName", page_file, "displayName") + width, height = page.get("width"), page.get("height") + if not (_is_number(width) and _is_number(height)): + f.error("PAGE_SIZE_INVALID", "page width/height must be numbers", page_file) + width = height = None + + visuals_dir = page_dir / "visuals" + if not visuals_dir.is_dir(): + return + for visual_dir in sorted(p for p in visuals_dir.iterdir() if p.is_dir()): + visual_file = visual_dir / "visual.json" + rel = visual_file.relative_to(f.report_dir).as_posix() + if rel in invalid: + continue + visual = _load(f, visual_file) + if not isinstance(visual, dict): + continue + if visual.get("name") != visual_dir.name: + f.error("NAME_MISMATCH", f"visual.json name '{visual.get('name')}' != folder '{visual_dir.name}'", + visual_file, "name") + pos = visual.get("position") + if not isinstance(pos, dict) or not all(_is_number(pos.get(k)) for k in ("x", "y", "width", "height")): + f.error("POSITION_INVALID", "position needs numeric x, y, width and height", visual_file, "position") + pos = None + kind = (visual.get("visual") or {}).get("visualType") + if not kind and "visualGroup" not in visual: + f.error("VISUAL_TYPE_MISSING", "visual.visualType is missing (and this is not a visualGroup)", + visual_file, "visual.visualType") + if pos and width is not None and not visual.get("parentGroupName"): + right, bottom = pos["x"] + pos["width"], pos["y"] + pos["height"] + if pos["x"] < 0 or pos["y"] < 0 or right > width + 0.5 or bottom > height + 0.5: + f.warning("OUTSIDE_CANVAS", f"visual extends outside the {width}x{height} page " + f"(x={pos['x']}, y={pos['y']}, right={right}, bottom={bottom})", visual_file, "position") + + +def _result(f: Findings, engine: str) -> dict: + errors = sum(1 for i in f.items if i["severity"] == "error") + warnings = sum(1 for i in f.items if i["severity"] == "warning") + return { + "engine": engine, + "report": str(f.report_dir), + "result": "failed" if errors else ("succeededWithWarnings" if warnings else "succeeded"), + "errorCount": errors, + "warningCount": warnings, + "diagnostics": f.items, + } + + +# --------------------------------------------------------------------------- +# powerbi-report-author CLI +# --------------------------------------------------------------------------- +def find_cli(env: Optional[dict] = None) -> Optional[str]: + env = os.environ if env is None else env + explicit = env.get("PBIR_AUTHOR_CLI") + if explicit: + return explicit if Path(explicit).is_file() else None + return shutil.which(CLI_NAME) + + +def build_cli_command(cli: str, report_dir: Path, offline: bool) -> list: + cmd = [cli, "validate", str(report_dir)] + if offline: + cmd.append("--no-schema") + return cmd + + +def parse_cli_output(stdout: str, report_dir: Path) -> dict: + """Normalize the CLI's JSON envelope ({"data": ...} or {"error": ...}).""" + try: + envelope = json.loads(stdout) + except ValueError as exc: + raise UsageError(f"{CLI_NAME} returned non-JSON output: {stdout.strip()[:500]}") from exc + if "error" in envelope: + err = envelope["error"] + raise UsageError(f"{CLI_NAME} failed: {err.get('code')}: {err.get('message')}") + data = envelope.get("data", {}) + items = [] + for code, group in (data.get("diagnostics") or {}).items(): + for item in group.get("items", []): + file = item.get("file", "") + try: + file = Path(file).resolve().relative_to(report_dir).as_posix() if file else "" + except ValueError: + pass + items.append({ + "severity": group.get("severity", "error"), + "code": code, + "message": item.get("message", ""), + "file": file, + "path": item.get("path", ""), + }) + return { + "engine": CLI_NAME, + "report": str(report_dir), + "result": data.get("result", "unknown"), + "errorCount": int(data.get("errorCount", sum(1 for i in items if i["severity"] == "error"))), + "warningCount": int(data.get("warningCount", sum(1 for i in items if i["severity"] == "warning"))), + "diagnostics": items, + } + + +def run_cli(cli: str, report_dir: Path, offline: bool) -> dict: + cmd = build_cli_command(cli, report_dir, offline) + try: + proc = subprocess.run(cmd, capture_output=True, text=True, timeout=CLI_TIMEOUT_SECONDS) + except (OSError, subprocess.TimeoutExpired) as exc: + raise UsageError(f"could not run {' '.join(cmd)}: {exc}") from exc + return parse_cli_output(proc.stdout, report_dir) + + +# --------------------------------------------------------------------------- +# Output +# --------------------------------------------------------------------------- +def format_text(result: dict) -> str: + if result["engine"] == "structural": + lines = [f"PBIR validation: structural fallback ({CLI_NAME} not found; schemas and visual " + "roles NOT checked — install @microsoft/powerbi-report-authoring-cli)"] + else: + lines = [f"PBIR validation: {CLI_NAME} validate"] + lines.append(f"Report: {result['report']}") + for d in sorted(result["diagnostics"], key=lambda d: (d["severity"] != "error", d["file"], d["code"])): + where = d["file"] + (f" @ {d['path']}" if d["path"] else "") + lines.append(f"{d['severity'].upper():8} [{d['code']}] {d['message']}" + (f" ({where})" if where else "")) + lines.append(f"Result: {result['result']} — {result['errorCount']} error(s), {result['warningCount']} warning(s)") + return "\n".join(lines) + + +def main(argv: Optional[list] = None) -> int: + parser = argparse.ArgumentParser(description="Validate a PBIR report (CLI if installed, else structural).") + parser.add_argument("path", type=Path) + parser.add_argument("--structural-only", action="store_true", help="skip the CLI even if it is installed") + parser.add_argument("--offline", action="store_true", help="pass --no-schema to the CLI") + parser.add_argument("--format", choices=("text", "json"), default="text") + args = parser.parse_args(argv) + + try: + report_dir = resolve_report(args.path) + cli = None if args.structural_only else find_cli() + result = run_cli(cli, report_dir, args.offline) if cli else structural_check(report_dir) + except UsageError as exc: + print(f"error: {exc}", file=sys.stderr) + return 2 + + print(json.dumps(result, indent=2) if args.format == "json" else format_text(result)) + return 1 if result["errorCount"] else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.opencode/skills/pbi-semantic-layer-tmdl/SKILL.md b/.opencode/skills/pbi-semantic-layer-tmdl/SKILL.md index dc1f3c2..da3d3bd 100644 --- a/.opencode/skills/pbi-semantic-layer-tmdl/SKILL.md +++ b/.opencode/skills/pbi-semantic-layer-tmdl/SKILL.md @@ -1,69 +1,99 @@ --- name: pbi-semantic-layer-tmdl -description: Edit the Power BI semantic model as code by authoring TMDL files in a PBIP project (offline, Desktop closed). Covers structure, syntax, and post-edit verification. +description: Change the Power BI semantic model — measures, columns, tables, relationships, RLS, DAX — with Power BI Desktop OPEN through the Power BI Modeling (Authoring) MCP server, saving to TMDL in a PBIP project; or by editing TMDL files with Desktop closed. Includes a semantic-model review checklist. Use for any request to add or fix a measure, DAX, relationship, table, column or model metadata. --- # Skill: pbi-semantic-layer-tmdl -Author and edit the semantic model (tables, columns, measures, relationships, RLS) by writing -**TMDL** files directly. TMDL is the GA, human-readable text format for tabular models — great -diffs, one file per table, fully scriptable. +With Desktop open, the model lives in Desktop's memory (a local Analysis Services instance). Edit +it **there**, live, through the Power BI Modeling MCP server — then the user saves and Desktop +writes the TMDL into the PBIP. Editing the `.tmdl` files while Desktop is open does nothing +until a model reload and is overwritten by the next save. -## Hard precondition -**Power BI Desktop must be CLOSED** before saving. Desktop keeps the model in memory and -overwrites disk on save, so editing live corrupts files or loses work. +## Default: Desktop open + Modeling MCP +Setup and the tool list: [references/modeling-mcp.md](references/modeling-mcp.md). +1. **Connect:** "Connect to '<file name>' in Power BI Desktop". Confirm with a read: list the + tables and measures. If the MCP is not registered, stop and point the user to the setup — + do not fall back to editing TMDL silently. +2. **Inspect through the MCP**, not the `.tmdl` files: while connected, the live model is the + truth and disk may be stale. +3. **Change** in dependency order: tables/partitions → columns → relationships → measures. + Check that every referenced table/column exists and that the name is not taken. +4. **Prove it:** run a DAX query through the MCP (`EVALUATE SUMMARIZECOLUMNS(..., "x", [New Measure])` + or `EVALUATE { [New Measure] }`) and show the value. A measure that only "was created" is not done. +5. **Save:** ask the user to press Ctrl+S in Desktop. That writes the TMDL into + `<Name>.SemanticModel/definition/` — the diff the team reviews and commits. +6. **Quality:** run `pbi-quality-rules` (BPA) after significant changes — on disk, so after step 5. +7. If a visual must use the new measure, continue in `pbi-report-layer-pbir`. Report reloads do + not touch the model and are not needed for MCP changes. + +## Alternative: Desktop closed (no MCP, older Desktop, CI) +Edit the TMDL files directly: +1. Make sure Desktop is **closed** — it overwrites disk on save. +2. Edit `.tmdl` files following the syntax below; run `pbi-quality-rules`. +3. Ask the user to open the `.pbip`: field list populates, no error banner, each new measure on a + Card shows a value. On error, re-read the edited TMDL and fix the reference/syntax. + +If Desktop must stay open and the MCP is unavailable, edit TMDL, then reload model + report with +`powerbi-report-author preview <folder> --host desktop --reload-with-model` after the same +unsaved-changes gate as `pbi-report-layer-pbir`. It reloads definitions only — the user must +refresh data for new/changed imported columns. ## `.SemanticModel` structure ``` -<Project>.SemanticModel/ +<Name>.SemanticModel/ definition.pbism ← pointer (do not hand-edit) definition/ - model.tmdl ← model-level definition, lists tables - database.tmdl ← compatibility level / annotations - tables/<Table>.tmdl ← ONE table per file (golden rule) - relationships.tmdl ← all relationships + database.tmdl ← compatibilityLevel + model.tmdl ← model properties, `ref table` order, annotations + tables/<Table>.tmdl ← one table per file + relationships.tmdl expressions.tmdl ← shared M expressions / parameters - cultures/ ← translations roles.tmdl ← RLS - .pbi/ ← local cache/settings — NEVER edit (localSettings.json, cache.abf) + cultures/ ← translations + .pbi/ ← local cache/settings — never edit, keep out of git ``` -Edit `.tmdl` files only. One table per file — never define every table inside `model.tmdl`. -## TMDL syntax reference +## TMDL syntax ```tmdl -table 'Sales' - description: "Primary sales fact." +/// Order lines. Grain: one row per order line. +table Sales - measure 'Total Sales' = SUM(Sales[Amount]) - formatString: "#,##0.00" - displayFolder: "Financials" - description: "Sum of sales amount." + /// Sum of the order line amount. + measure 'Total Sales' = SUM(Sales[Amount]) + formatString: #,##0.00 + displayFolder: Revenue - column 'Amount' - dataType: decimal - sourceColumn: BaseAmount + measure 'Margin %' = + DIVIDE([Total Sales] - [Total Cost], [Total Sales]) + formatString: 0.0% + displayFolder: Revenue + + column Amount + dataType: decimal + summarizeBy: sum + sourceColumn: Amount ``` ```tmdl -relationship Sales_Date - fromColumn: Sales.DateId - toColumn: Date.DateId - fromCardinality: many - toCardinality: one +relationship 5f0c2a4e-9b1d-4c3a-8e2f-1a6b7c8d9e01 + fromColumn: Sales.DateKey + toColumn: Date.DateKey ``` +- Indent with tabs; properties one level under their object. +- Descriptions are `///` lines directly above the object. +- Multi-line DAX: leave the `=` line empty and indent the expression two levels deeper than the + object, or wrap it in a ```` ``` ```` block. +- Quote names with spaces or symbols: `'Margin %'`; double a quote inside: `'Customer''s'`. -**Serialization rules:** keep functional properties (`dataType`, `sourceColumn`) before -cosmetic ones; match the file's existing indentation; for multi-line DAX, break with `\` at the -end of the line. - -## Tooling -- **Tabular Editor 2 (free)** can open the `.SemanticModel` folder, validate, batch-edit, and - re-serialize clean TMDL. Use it to sanity-check non-trivial edits. -- **Optional:** Microsoft's `powerbi-modeling-mcp` (preview) edits the same PBIP/TMDL files and - is handy for bulk operations. File editing remains the default, accessible path. - -## Post-edit verification -1. Ask the user to open the `.pbip` in Desktop and wait for the field list to populate. -2. **View → Diagnostics** — report any error/warning shown. -3. For each new/changed measure, drop it on a **Card** and confirm the value (no red error icon). -4. On error: re-read the edited TMDL, find the syntax/reference issue, propose the fix. -5. Run the `pbi-quality-rules` skill (BPA) after significant changes. +## Review checklist (run it on any model you change or are asked to review) +- **Names** are business-friendly: no `tbl_`/`dim_` prefixes or `_id` columns visible to users. +- **Every measure** has a `formatString`, a `displayFolder` and a `///` description (what it + means and its grain/filters in plain words). +- **Relationships** match reality: many-to-one from fact to dimension, single direction unless a + documented reason exists; no ambiguous paths. +- **A real Date table** (marked as date table) drives time intelligence; auto date/time is off + (`annotation __PBI_TimeIntelligenceEnabled = 0` in `model.tmdl`). +- **Technical keys hidden** (`isHidden` on `...ID`/`...Key` columns); explicit measures instead + of implicit column aggregation for anything a report shows. +- **DAX:** `DIVIDE` instead of `/` + `IFERROR`; variables for repeated sub-expressions; no + measure that just returns `BLANK()`. diff --git a/.opencode/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md b/.opencode/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md new file mode 100644 index 0000000..fdf7468 --- /dev/null +++ b/.opencode/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md @@ -0,0 +1,60 @@ +# Power BI Modeling MCP — setup and use + +Microsoft now calls it the **Power BI Authoring MCP server**; the package and server id are still +`powerbi-modeling-mcp`. Sources: +- Microsoft Learn — [Power BI Authoring MCP server](https://learn.microsoft.com/en-us/power-bi/developer/mcp/power-bi-authoring-mcp) +- [microsoft/powerbi-modeling-mcp](https://github.com/microsoft/powerbi-modeling-mcp) (README, TROUBLESHOOTING) +- [microsoft/skills-for-fabric › powerbi-authoring](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) + (skill `semantic-model-authoring`, bundles the same server) + +## Local vs hosted +Working against a model **open in Power BI Desktop** (or PBIP files on disk) needs the **local** +server: it runs on the same Windows machine and talks to Desktop's local Analysis Services +instance over XMLA. The hosted endpoint only reaches Fabric workspaces. Register one of the two, +not both — duplicate tool sets confuse the agent. The local server does not run on macOS. + +## Register the local server +Command in every client: `npx -y @microsoft/powerbi-modeling-mcp@latest --start` +(Node 18+; or install the VS Code extension `analysis-services.powerbi-modeling-mcp`). + +| Client | File | Shape | +|---|---|---| +| Claude Code | `.mcp.json` (project) | `{"mcpServers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| GitHub Copilot (VS Code) | `.vscode/mcp.json` | `{"servers": {"powerbi-modeling-mcp": {"type": "stdio", "command": "npx", "args": ["-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| OpenCode | `opencode.json` | `{"mcp": {"powerbi-modeling-mcp": {"type": "local", "command": ["npx", "-y", "@microsoft/powerbi-modeling-mcp@latest", "--start"]}}}` | +| Cursor | `.cursor/mcp.json` | same shape as Claude Code (`mcpServers`) | + +Useful flags: `--readonly` (inspection only — good for reviews), `--readwrite` (default; asks for +confirmation before the first write per database). + +## Connect and work +- Desktop: `Connect to '<file name>' in Power BI Desktop` (tool `ConnectToPowerBIDesktop`; the + PBIP must be open in Desktop). +- PBIP on disk without Desktop: `Open semantic model from PBIP folder '<path>\<Name>.SemanticModel\definition'` + (changes stay in the session until exported back to TMDL). +- First call after connecting: a read (`List the tables and measures`) to confirm the target. + +Tool groups you will use most: `measure_operations`, `column_operations`, `table_operations`, +`relationship_operations`, `dax_query_operations` (run/validate DAX, max 100,000 rows), +`transaction_operations` (begin/commit/rollback a batch), `calculation_group_operations`, +`security_role_operations`, `object_translation_operations`, `database_operations` +(import/export TMDL). Ask the agent "what can the Power BI Authoring MCP do" to list the current set. + +## Rules of thumb +- Prefer one transaction per logical change; roll back on a failed step instead of leaving + half-applied edits. +- Verify every new or changed measure with a DAX query before calling it done. +- Changes are live in Desktop but not on disk until the user saves (Ctrl+S); a PBIP under git + makes every MCP change a reviewable diff. +- The MCP edits the model only — never report pages (use `pbi-report-layer-pbir`). +- Model metadata and query results go to the LLM provider of the client. Do not query + sensitive columns just to "look at the data". + +## Troubleshooting +| Symptom | Check | +|---|---| +| Server missing from the tool list | Client in agent mode? Server registered and started? Copilot Business/Enterprise: "MCP servers in Copilot" policy enabled by an admin. | +| "No Power BI Desktop instance found" | The PBIP/PBIX is open in Desktop and the name matches the file name you gave. | +| Reads work, writes fail | `--readonly` set, or (Fabric) only Build permission / XMLA read-only. | +| Edits vanish after reopening the file | The user closed Desktop without saving. | +| TMDL on disk does not show the change | Not saved yet — Desktop writes TMDL on save. | diff --git a/AGENTS.md b/AGENTS.md index f85a228..86a75ff 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,4 +1,4 @@ -<!-- Auto-generated by scripts/sync_agents.py — NÃO EDITE MANUALMENTE --> +<!-- Auto-generated by scripts/sync_agents.py — DO NOT EDIT MANUALLY --> # Agentic Data Kit — Operating Rules @@ -13,17 +13,20 @@ data-specific rules — not things a capable model already does well. ## How to work -1. **Route to a specialist.** Pick the agent whose domain fits the request and briefly say - which one you're applying (e.g. "Applying @data-engineer"). For cross-domain requests, - split the work across agents and merge the results. -2. **Skills activate themselves.** Every skill declares when it applies in its `description`. - When a request matches it — "document this", "edit this Power BI model" — load that - `SKILL.md` on demand, **with or without an agent involved**. Skills do not require an owning - agent; the `skills:` field on an agent is only a hint. Don't preload every file in a folder. -3. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` +1. **Skills first.** Every skill declares when it applies in its `description`. When a request + matches one — "add a measure", "fix this visual", "document this dashboard" — load that + `SKILL.md` on demand and follow it. Don't preload every file in a folder. +2. **Power BI is developed with Desktop open.** Model changes go live through the Power BI + Modeling MCP (`pbi-semantic-layer-tmdl`); report changes follow edit → validate → save → + status → reload → screenshot → review (`pbi-report-layer-pbir`). Never reload while Desktop + has unsaved changes. +3. **Visual changes get an independent review.** After screenshots, hand the PNGs, the request + and the PBIR folder — not your reasoning — to the `pbi-visual-reviewer` agent. Where the tool + has no subagents, run its checklist in a fresh chat with only those inputs. +4. **Creating or changing an agent/skill/workflow?** Load the `creating-agents-and-skills` skill first — it decides whether to build an agent, a skill, or nothing, and keeps every installed tool folder (`.agent`, `.claude`, `.github`, `.opencode`, `.cursor`) in sync. -4. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd +5. **Clarify only when genuinely blocked.** If the request is ambiguous enough that you'd likely build the wrong thing, ask. Otherwise state your assumptions and proceed. ## Data engineering principles (always apply) diff --git a/AGENT_FLOW.md b/AGENT_FLOW.md index 6ea9664..665c97d 100644 --- a/AGENT_FLOW.md +++ b/AGENT_FLOW.md @@ -6,38 +6,38 @@ How a request moves through the kit. User request │ ▼ -Route to a specialist ──► pick the agent whose domain fits (announce it briefly) - │ cross-domain? split across agents, then merge results - ▼ -Load skills on demand ──► read a SKILL.md only when the task needs it +Skills self-activate ──► a SKILL.md loads only when its description matches the task │ ▼ -Execute ──────────────► build / edit pipelines, models, DAX, TMDL, PBIR, docs - │ +Execute ───────────────► Power BI loop with Desktop open: + │ model/measures → Modeling MCP (live) → user saves in Desktop + │ visuals → edit PBIR → validate_pbir.py → preview --reload + │ → --screenshot --all-pages + │ other work: dbt models, docs, decks, PPTX ▼ -Validate ──────────────► Power BI: /validate-pbi (real BPA via Tabular Editor 2) +Validate (scripts) ────► run_bpa.py (BPA, TE2 or Python fallback) · validate_pbir.py │ ▼ -Deliver ──────────────► present changes + reasoning, in the user's language +Independent review ────► pbi-visual-reviewer: fresh context, screenshots + spec + PBIR only, + │ PASS/FAIL per page and per visual; FAIL → back to Execute + ▼ +Deliver ───────────────► present changes + reasoning, in the user's language ``` -## Routing cheatsheet - -| Request is about… | Agent | -|---|---| -| Ingestion, pipelines, Spark, orchestration | `data-engineer` | -| dbt, star schemas, warehouse modeling | `analytics-engineer` | -| Metrics, dashboards, analysis, ML, statistics, A/B tests, semantic review | `data-scientist` | -| Power BI models (TMDL) or reports (PBIR) | `powerbi-developer` | -| Presentations, decks, pitches, readouts, business plans, brand/`DESIGN.md` | `presentation-designer` | +Before `--reload`, confirm Desktop has no unsaved changes +(`powerbi-report-author preview <folder> --host desktop --status` → `details.hasUnsavedChanges` +is `false`): reload discards them, and it only covers the report — model changes go through the +Modeling MCP. Without Desktop Bridge (older Desktop, CI), use the Desktop-closed path: edit +TMDL/PBIR on disk, validate with the scripts, reopen Desktop. -> Quality, privacy, contracts, and masking are **cross-cutting rules** (in `rules.md`), applied -> by every agent — not a routing target. +Where subagents are unavailable (Cursor, Antigravity), run the review in a fresh chat given only +the screenshots, the spec and the PBIR folder. ## Principles - **Less is more** — lean instructions keep the model focused; we don't restate what it knows. - **Clarify only when genuinely blocked** — otherwise state assumptions and proceed. - **Check downstream impact** before changing schemas, contracts, or shared models. +- **Never self-review** — validation is scripts; visual review is an independent agent. -**Agents:** 5 · **Skills:** 8 · **Workflows:** 3 +**Agents:** 1 · **Skills:** 9 · **Workflows:** 3 diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..5f792a4 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,66 @@ +# Changelog + +All notable changes to this project are documented here. The format follows +[Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses +[Semantic Versioning](https://semver.org/). + +## [5.0.0] - 2026-10-28 + +Power BI becomes the flagship, developed with Power BI Desktop open. + +### Breaking + +- **The 5 agents are removed:** `data-engineer`, `analytics-engineer`, `data-scientist`, + `powerbi-developer` and `presentation-designer`. An agent now exists only if it imposes + something the model cannot impose on itself (restricted tools, independent review, a different + model, or an automatic trigger). Domain knowledge lives in skills; generic knowledge was dropped. +- **No more agent routing or handoffs.** Skills self-activate; the always-on rules no longer + tell the assistant to route to a specialist. +- **Desktop-open is the default Power BI workflow** (Modeling MCP for the model, Desktop Bridge for + the report). The previous "Desktop must be closed" flow remains documented as an alternative. +- Counts are now **1 agent · 9 skills · 3 workflows · 5 targets**. + +### Added + +- `pbi-visual-reviewer` agent (tools: Read, Grep, Glob): independent review of report + screenshots against the spec and the PBIR folder, with PASS/FAIL per page and per visual. +- `dbt-modeling-conventions` skill (the org conventions of the removed analytics-engineer agent). +- Validation scripts (Python 3.9+, stdlib only; `--format text|json`; exit 0/1/2): + `run_bpa.py`, `bpa_fallback.py` and `validate_pbir.py`. +- References: `pbi-report-layer-pbir/references/desktop-bridge.md` and + `pbi-semantic-layer-tmdl/references/modeling-mcp.md`. +- `cursor` install target, now including `.cursor/skills`. +- `python3 scripts/sync_agents.py --check` plus the CI workflow `sync-check.yml`; `npm prepack` + runs the sync before publishing. +- `npm test` with Python `unittest` and a fixture PBIP at `tests/fixtures/SalesSample/`. +- `THIRD_PARTY_NOTICES.md`. +- This changelog. + +### Changed + +- `pbi-semantic-layer-tmdl`, `pbi-report-layer-pbir` and `pbi-quality-rules` rewritten around the + Desktop-open loop and the scripts above. +- `rules.md` no longer contains agent routing. +- Workflows updated: `/plan` lists skills and scripts per task instead of agents; + `/validate-pbi` runs the scripts. +- `building-html-presentations` absorbs the data-storytelling rules of the removed + presentation-designer agent; `creating-agents-and-skills` now enforces the agent-survival rule. +- Copilot agents get `tools` mapped from Claude names (Read→read, Grep/Glob→search, + Edit/Write→edit, Bash→execute); OpenCode agents get a `permission` block. +- The sync script is written in English. +- README, ARCHITECTURE, AGENT_FLOW and CONTRIBUTING rewritten for v5. +- Package `files` whitelist: only what `init` installs is published (no tests, CI files or + cover image). +- `init --target copilot` installs only the kit's entries in `.github/` (agents, skills, prompts, + `copilot-instructions.md`) and no longer refuses to run because a `.github/` folder exists. +- `init --target antigravity` also installs `scripts/sync_agents.py`. +- The CLI banner counts agents/skills/workflows from the package instead of hard-coding them. + +### Fixed + +- `init --target opencode` now installs the root `AGENTS.md`; it previously looked for a removed + `.opencode/AGENTS.md`. +- `init --target copilot` copied the whole `.github/` folder, which would have included the + kit's own CI workflow. + +Earlier versions are in the git history. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6f63b84..d9dc566 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,29 +7,38 @@ ## The Golden Rule -**Only edit `.agent/`.** Everything under `.github/`, `.claude/`, `.opencode/`, `.cursor/`, and -the root `AGENTS.md` is auto-generated by `scripts/sync_agents.py` and will be overwritten. +**Only author in `.agent/`.** Everything under `.github/`, `.claude/`, `.opencode/`, `.cursor/`, +and the root `AGENTS.md` is generated by `scripts/sync_agents.py` and will be overwritten. ```bash -# After editing anything in .agent/, regenerate every platform: -python scripts/sync_agents.py +# After editing anything in .agent/, regenerate every tool (needs PyYAML: pip install pyyaml): +python3 scripts/sync_agents.py + +# Fail if generated folders are stale (this is what CI runs): +python3 scripts/sync_agents.py --check ``` +CI (`.github/workflows/sync-check.yml`) runs `--check` on every push and pull request, so commit +the regenerated folders together with your `.agent/` change. `npm prepack` also runs the sync +before publishing. + --- -## Before you add an agent: should it exist? +## Before you add anything: the agent-survival rule + +Less is more. Extra agents and verbose instructions cost the model attention. **Default to a +skill.** An agent is justified only if it imposes something the model cannot impose on itself: -Less is more. Extra agents and verbose instructions cost the model attention and make routing -ambiguous — both degrade results. Apply these gates before creating anything: +1. **Restricted tools** in its frontmatter (e.g. a read-only reviewer). +2. **Independent review** — fresh context, no access to the author's reasoning. +3. **A different model.** +4. **An automatic trigger.** -- **Skill suffices?** If the capability is narrow knowledge (a format, a checklist, a tool), - make it a **skill**, not an agent. Skills load only when relevant (progressive disclosure), - so they're nearly free until used. -- **Already covered?** If an existing agent's domain overlaps, extend that agent instead. -- **LLM-native?** Don't write down what a capable model already does well ("write clean code", - "use CTEs"). Capture only *domain- or org-specific* context. +If none applies, write a **skill** (narrow knowledge, a format, a checklist, a script) or a +**workflow** (a repeatable procedure the user invokes) — or nothing, when the model already +does it well ("write clean code", "use CTEs"). Capture only *domain- or org-specific* context. -If a request fails these gates, the right answer is to **not** add the artifact. +If a request fails these tests, the right answer is to **not** add the artifact. --- @@ -37,36 +46,40 @@ If a request fails these gates, the right answer is to **not** add the artifact. 1. **Create** the file under the right `.agent/` subfolder: - Agent → `.agent/agents/<name>.md` - - Skill → `.agent/skills/<name>/SKILL.md` (+ optional `scripts/`, reference files) + - Skill → `.agent/skills/<name>/SKILL.md` (+ optional `scripts/`, `references/`) - Workflow → `.agent/workflows/<name>.md` - Global rule → `.agent/rules/rules.md` -2. **Sync** → `python scripts/sync_agents.py` -3. **Use** — the agent/skill/command appears in each tool automatically. +2. **Sync** → `python3 scripts/sync_agents.py` +3. **Test** → `npm test` (Python `unittest` in `tests/`; fixture PBIP at `tests/fixtures/SalesSample/`). + Scripts under `.agent/skills/*/scripts/` are Python 3.9+, standard library only, and support + `--format text|json` with exit codes 0 (ok) / 1 (errors found) / 2 (usage or tool failure). +4. **Commit** the `.agent/` change and the regenerated folders together. --- ## Conventions -**Agents** (`.agent/agents/<name>.md`) — keep them lean and high-signal: +**Agents** (`.agent/agents/<name>.md`) — rare, lean, and defined by what they restrict: ```yaml --- -name: my-agent # lowercase-hyphenated -description: One sharp sentence on what it does + trigger keywords (used for routing). -tools: Read, Grep, Glob, Bash, Edit, Write +name: my-reviewer # lowercase-hyphenated +description: One sharp sentence on what it does + trigger keywords. +tools: Read, Grep, Glob # the restriction that justifies the agent model: inherit -skills: skill-a, skill-b # optional; loaded on demand --- ``` -Body = identity (1–2 lines) → domain rules → handoffs → out-of-scope. Cut anything generic. +Body = identity (1–2 lines) → checklist/rules → output format. No handoffs to other agents. +Claude tool names are mapped for Copilot automatically (Read→read, Grep/Glob→search, +Edit/Write→edit, Bash→execute). -**Skills** — `SKILL.md` with `name` + `description` frontmatter. Keep the body under ~500 lines; -move long reference material to separate files in the skill folder and link to them. +**Skills** — `SKILL.md` with `name` + third-person `description` (with trigger words) +frontmatter. Keep the body under ~500 lines; move long material to `references/` or `templates/`. **Workflows** — `description` frontmatter; concise, numbered steps. -**Language** — write all instructions in **English** for portability; the assistant replies in -the user's language at runtime. +**Language** — everything in this repository (code, identifiers, docs, instructions) is +**English only**; the assistant replies in the user's language at runtime. --- @@ -74,10 +87,10 @@ the user's language at runtime. ``` .agent/ ← SOURCE OF TRUTH (edit here) -├── agents/ ← specialist personas -├── skills/ ← deep, on-demand knowledge modules +├── agents/ ← independent reviewer(s) +├── skills/ ← on-demand knowledge modules, scripts, references ├── workflows/ ← slash-command procedures └── rules/ ← global rules (rules.md) -.github/ .claude/ .opencode/ .cursor/ AGENTS.md ← AUTO-GENERATED (do not edit) +.github/ .claude/ .opencode/ .cursor/ AGENTS.md ← GENERATED (do not edit) ``` diff --git a/README.md b/README.md index ec8624e..dc2b559 100644 --- a/README.md +++ b/README.md @@ -4,14 +4,14 @@ # Agentic Data Kit -### Run a single command. Get a team of AI data specialists. +### Power BI development with Power BI Desktop open — driven by your AI assistant. -**Lean, opinionated AI agents · skills · workflows for data teams — -one source of truth, every assistant.** +**Skills, scripts and an independent visual reviewer that let Claude Code, Copilot, OpenCode, +Cursor and Antigravity edit your model and report while you watch them change in Desktop.** [![License: MIT](https://img.shields.io/badge/License-MIT-2563EB.svg)](LICENSE) -![Version](https://img.shields.io/badge/version-4.1.0-1E7A4D.svg) -![Platforms](https://img.shields.io/badge/Claude%20Code%20·%20Copilot%20·%20OpenCode%20·%20Cursor-1B2733.svg) +![Version](https://img.shields.io/badge/version-5.0.0-1E7A4D.svg) +![Platforms](https://img.shields.io/badge/Claude%20Code%20·%20Copilot%20·%20OpenCode%20·%20Cursor%20·%20Antigravity-1B2733.svg) ![PRs welcome](https://img.shields.io/badge/PRs-welcome-9A6B12.svg) ![Philosophy](https://img.shields.io/badge/philosophy-less%20is%20more-6A7787.svg) @@ -19,7 +19,7 @@ one source of truth, every assistant.** npx @luccapinto/agentic-data-kit@latest init ``` -**5 agents · 8 skills · 3 workflows · 4 platforms** +**1 agent · 9 skills · 3 workflows · 5 targets** </div> @@ -27,183 +27,215 @@ npx @luccapinto/agentic-data-kit@latest init ## Contents -- [🧠 Why this exists](#-why-this-exists) -- [🚀 Quick start](#-quick-start) +- [🧠 What it is](#-what-it-is) +- [📈 The Power BI loop (Desktop open)](#-the-power-bi-loop-desktop-open) +- [🚀 Install](#-install) +- [✅ Prerequisites for the Power BI loop](#-prerequisites-for-the-power-bi-loop) - [🎯 What's inside](#-whats-inside) -- [🤖 Agents](#-agents) -- [🧩 Skills](#-skills) -- [📈 Power BI — as code, not clicks](#-power-bi--as-code-not-clicks) -- [🎨 Presentations — as code](#-presentations--as-code) -- [🔄 How it works (multi-platform)](#-how-it-works-multi-platform) -- [🧭 Design philosophy](#-design-philosophy) -- [🤝 Contributing & license](#-contributing--license) +- [🧪 Validation scripts](#-validation-scripts) +- [🔄 Multi-tool sync](#-multi-tool-sync) +- [🤝 Contributing](#-contributing) +- [🙏 Credits & references](#-credits--references) +- [📄 License](#-license) --- -## 🧠 Why this exists +## 🧠 What it is -Generic LLMs don't know *your* stack, *your* standards, or *your* architecture — but they -already know how to write clean code. So this kit follows one rule: - -> **Capture only the domain-specific context that matters, and nothing the model already does well.** - -Lean instructions keep the model's attention on what's important — which is exactly what makes -agents reliable. The result is a curated library for **Data Engineering, Analytics, BI, and data -storytelling** that drops into your assistant and just works: build pipelines, design star schemas, -edit Power BI as code, run quality checks, document models, and turn analyses into on-brand -presentations — without re-explaining the basics. +Generic LLMs don't know *your* standards, but they already write clean code. So this kit captures +only what a model can't supply on its own: how Power BI projects are laid out (PBIP, TMDL, PBIR), +how to drive Power BI Desktop while it is open, real quality gates (Best Practice Analyzer, PBIR +validation), and your brand. Power BI is the flagship; the kit also ships dbt naming conventions, +documentation templates and presentation builders (HTML decks, editable PPTX). Everything is +authored once in `.agent/` and compiled into each assistant's native format. --- -## 🚀 Quick start +## 📈 The Power BI loop (Desktop open) -```bash -npx @luccapinto/agentic-data-kit@latest init +You keep Power BI Desktop open on the `.pbip`. The assistant changes the model live and the +report on disk, then reloads Desktop and looks at the result. + +```text +Model / measures Report visuals +──────────────── ────────────── +Modeling MCP ──► change goes LIVE edit PBIR files +in Desktop │ + │ ▼ +you save in Desktop validate_pbir.py +(TMDL lands in the PBIP) │ + │ ▼ + ▼ Desktop has no unsaved changes? +run_bpa.py (BPA) │ + ▼ + powerbi-report-author preview --reload + │ + ▼ + --screenshot <dir> --all-pages + │ + ▼ + pbi-visual-reviewer (fresh context) ─► PASS / FAIL per page & visual ``` -Pick your AI assistant in the prompt and the kit installs itself. Then just ask: +Visual loop in order: **edit PBIR → `validate_pbir.py` → confirm Desktop has no unsaved changes +(`powerbi-report-author preview <folder> --host desktop --status`, `details.hasUnsavedChanges` must +be `false`) → `--reload` → `--screenshot <dir> --all-pages` → `pbi-visual-reviewer`.** -```text -You: "Build an ETL pipeline for customer data using Medallion Architecture" -AI: Applying @data-engineer → Bronze → Silver → Gold pipeline +**Three pitfalls** -You: "Add a revenue measure to the Power BI model" -AI: Applying @powerbi-developer → writes DAX, edits TMDL, runs the BPA +1. **Reload discards unsaved Desktop changes.** Check the status first; save in Desktop if needed. +2. **Reload only covers the report.** Model and measure changes go through the Modeling MCP, not + through file reload. +3. **An edited theme may not reload.** Rename the theme file (and its reference) to force a pick-up. -You: "Turn this analysis into a McKinsey-style PDF" -AI: Applying @presentation-designer → builds the report, exports to PDF -``` +Details: [`desktop-bridge.md`](.agent/skills/pbi-report-layer-pbir/references/desktop-bridge.md) +and [`modeling-mcp.md`](.agent/skills/pbi-semantic-layer-tmdl/references/modeling-mcp.md). -> 🌍 Instructions are written in English for portability; the assistant replies in **your** -> language automatically. +**Desktop-closed alternative.** No Desktop Bridge, an older Desktop, or CI? The disk-only path +stays supported: edit TMDL/PBIR on disk, validate with the scripts, then reopen the project in +Desktop. It is documented in +[`pbi-report-layer-pbir` › Alternative: Desktop closed](.agent/skills/pbi-report-layer-pbir/SKILL.md#alternative-desktop-closed) +and [`pbi-semantic-layer-tmdl` › Alternative: Desktop closed](.agent/skills/pbi-semantic-layer-tmdl/SKILL.md#alternative-desktop-closed-no-mcp-older-desktop-ci). --- -## 🎯 What's inside +## 🚀 Install -| | Count | What it is | -|---|:---:|---| -| 🤖 **Agents** | **5** | Role-based specialists with isolated context and clear domains | -| 🧩 **Skills** | **8** | On-demand knowledge modules that self-activate by description | -| 🔄 **Workflows** | **3** | Slash-command procedures (`/plan`, `/validate-pbi`, `/document-dashboard`) | -| 🖥️ **Platforms** | **4** | Claude Code · GitHub Copilot · OpenCode · Cursor (from one source) | +```bash +npx @luccapinto/agentic-data-kit@latest init +``` ---- +Pick your assistant in the prompt, or skip it with `--target`: -## 🤖 Agents +```bash +npx @luccapinto/agentic-data-kit@latest init --target cursor +``` -Focused specialists — each owns a distinct domain, so routing is never ambiguous. +| `--target` | Installs | Agent (`pbi-visual-reviewer`) | +|---|---|---| +| `antigravity` | `.agent/` (source of truth) + `scripts/sync_agents.py` | `agents/` — Antigravity has no subagents: run the review in a fresh chat | +| `copilot` | `.github/` (agents · skills · prompts · `copilot-instructions.md`); your other `.github` files are left alone | `agents/pbi-visual-reviewer.agent.md`, `tools` mapped to Copilot names | +| `claude` | `.claude/` (agents · skills · commands · workflows · `CLAUDE.md`) | native subagent | +| `opencode` | `.opencode/` (agents · skills · commands) + root `AGENTS.md` | `mode: subagent`, edit/bash denied | +| `cursor` | `.cursor/` (`rules/*.mdc` · commands · skills) | none — run the review in a fresh chat with only the screenshots, spec and PBIR | -| Agent | What it does | -|---|---| -| `data-engineer` | ETL/ELT pipelines, Medallion architecture, idempotency, Write-Audit-Publish | -| `analytics-engineer` | Star schemas, dbt models, warehouse architecture & performance | -| `data-scientist` | Analysis, metrics, dashboards & semantic review — through to ML, statistics, forecasting, A/B testing | -| `powerbi-developer` | Power BI as code — TMDL semantic model **and** PBIR reports, validated with Tabular Editor 2 | -| `presentation-designer` | Analyses → presentations as code — decks, interactive sites, consulting PDFs, **editable PPTX** — on-brand via `DESIGN.md` | - -> 🛡️ **Governance is cross-cutting, not a role.** PII masking, Write-Audit-Publish, contracts, and -> downstream-impact checks live in the always-on workspace rules and apply to every agent. +OpenAI Codex reads the root `AGENTS.md`. Instructions are written in English; the assistant +replies in **your** language. --- -## 🧩 Skills - -Skills **activate themselves** from their description (progressive disclosure) — no agent required, -no need to name them. Ask *"document this"* and the docs skill fires on its own. +## ✅ Prerequisites for the Power BI loop -| Skill | Use it for | +| Need | Why | |---|---| -| `pbi-semantic-layer-tmdl` | Author Power BI semantic models in TMDL | -| `pbi-report-layer-pbir` | Edit Power BI reports as code (pages, visuals, themes) | -| `pbi-quality-rules` | Real Best Practice Analyzer via the **free** Tabular Editor 2 | -| `building-html-presentations` | Decks, scrollable decks, interactive sites & consulting PDFs + an 18-chart cookbook | -| `generating-pptx` | **Native, editable PowerPoint** via python-pptx (real charts, not images) | -| `applying-visual-identity` | Apply a company brand from a `DESIGN.md` (Google Labs spec) | -| `documentation-templates` | Runbooks, data dictionaries, metric defs, ADRs, Power BI catalog | -| `creating-agents-and-skills` | Governance gate — add an agent/skill only when it helps, and keep every tool folder in sync | +| **Power BI Desktop** (Windows), with *File > Options > Security > Desktop Bridge → "Enable external tool access…"* turned on | The Bridge lets the CLI reload and screenshot the open report | +| **Node 20+** | Runs the installer, the report CLI and the Modeling MCP | +| `npm i -g @microsoft/powerbi-report-authoring-cli` | `powerbi-report-author` (validate, preview, reload, screenshot); also installs the lower-level `powerbi-desktop` bridge CLI | +| **Power BI Modeling MCP** registered in your MCP client (`npx -y @microsoft/powerbi-modeling-mcp@latest --start`), then "Connect to '<file>' in Power BI Desktop" | Live model and measure changes | +| *(optional)* **Tabular Editor 2** | Exact BPA results; without it the Python fallback runs | +| **Python 3.9+** | Validation scripts (standard library only) | + +Without these you still get the Desktop-closed path and the scripts' fallbacks. --- -## 📈 Power BI — as code, not clicks +## 🎯 What's inside -The `powerbi-developer` agent edits **PBIP** projects directly on disk: +| | Count | What it is | +|---|:---:|---| +| 🤖 **Agent** | **1** | `pbi-visual-reviewer` — independent review of report screenshots | +| 🧩 **Skills** | **9** | On-demand knowledge modules that self-activate by description | +| 🔄 **Workflows** | **3** | Slash commands: `/plan`, `/validate-pbi`, `/document-dashboard` | +| 🎯 **Targets** | **5** | Antigravity · Copilot · Claude Code · OpenCode · Cursor | -- 🧱 **TMDL** for the semantic model (tables, measures, relationships, RLS) -- 📊 **PBIR** for reports (pages, visuals, themes, bookmarks) -- ✅ Validated with the **free Tabular Editor 2** Best Practice Analyzer — no paid tooling, no Fabric required +**Agent.** An agent only exists if it imposes something the model can't impose on itself: +restricted tools, an independent fresh-context review, a different model, or an automatic trigger. +`pbi-visual-reviewer` has read-only tools (`Read`, `Grep`, `Glob`) and never sees the author's +reasoning. It checks screenshots + spec + PBIR for overlap, clipped text, empty/error visuals, +alignment, theme consistency and titles, and returns PASS/FAIL per page and per visual. -Microsoft's official `powerbi-modeling-mcp` (preview) is supported as an optional path for bulk -refactors, but **file editing is the default**. +**Skills** + +| Skill | Use it for | +|---|---| +| `pbi-semantic-layer-tmdl` | Author Power BI semantic models (TMDL) and drive them live through the Modeling MCP | +| `pbi-report-layer-pbir` | Edit reports as code (pages, visuals, themes) and run the Desktop reload/screenshot loop | +| `pbi-quality-rules` | Real Best Practice Analyzer via Tabular Editor 2, with a Python fallback | +| `dbt-modeling-conventions` | dbt CTE structure, `stg_`/`dim_`/`fct_`/`mart_` naming, keys and tests | +| `documentation-templates` | Runbooks, data dictionaries, metric definitions, ADRs, Power BI catalog | +| `applying-visual-identity` | Apply a company brand from a `DESIGN.md` | +| `building-html-presentations` | Decks, scrollable decks, interactive sites, consulting PDFs + chart cookbook | +| `generating-pptx` | Native, editable PowerPoint via python-pptx | +| `creating-agents-and-skills` | Decide whether to add an agent, skill or nothing — and keep every tool folder in sync | + +**Workflows** + +| Command | What it does | +|---|---| +| `/plan` | Writes `docs/PLAN-<slug>.md` for a data task — plan only, no code | +| `/validate-pbi` | Runs the BPA and the PBIR check on a PBIP project | +| `/document-dashboard` | Generates Markdown documentation for a PBIP project | --- -## 🎨 Presentations — as code +## 🧪 Validation scripts -The `presentation-designer` agent turns analyses, EDAs, and business plans into polished, on-brand -deliverables — and reads your brand from a `DESIGN.md` so everything stays consistent. +Validation lives in scripts inside the skills, not in agents. All three accept +`--format text|json`; exit code `0` = no errors, `1` = errors found, `2` = usage or tool failure. -| Starter | Output | +| Script | What it does | |---|---| -| `reveal-deck.html` | Dashboard-style slide deck (reveal.js) → HTML + PDF | -| `flex-deck.html` | Flexible **scrollable** deck with dynamic visual toggles | -| `interactive-site.html` | Self-serve scrolling report site | -| `report-pdf.html` | **McKinsey/Deloitte-style** PDF (action titles, exhibits, callouts) | -| `marp-deck.md` | Quick Markdown → PDF/PPTX | -| `charts-reference.html` | **Chart cookbook** — 18 ready-to-copy chart patterns | +| `.agent/skills/pbi-quality-rules/scripts/run_bpa.py <model>` | Real BPA through the Tabular Editor 2 CLI; falls back to `bpa_fallback.py` if TE2 isn't found | +| `.agent/skills/pbi-quality-rules/scripts/bpa_fallback.py <model>` | The same 10 rules of `bpa-rules.json` re-implemented in Python over TMDL (less exact) | +| `.agent/skills/pbi-report-layer-pbir/scripts/validate_pbir.py <path>` | Runs `powerbi-report-author validate` when installed; otherwise a structural PBIR check | -Plus **`generating-pptx`** for native, **editable PowerPoint** (real text & charts, not images) via -`python-pptx`. +Python 3.9+, standard library only. Run the repo's own tests with `npm test`. --- -## 🔄 How it works (multi-platform) +## 🔄 Multi-tool sync -`.agent/` is the **single source of truth**. One sync script compiles it for every assistant in -that tool's native format — write once, run everywhere. +`.agent/` is the **single source of truth**. One script compiles it into every tool's native format: ```bash -python scripts/sync_agents.py +python3 scripts/sync_agents.py # regenerate .claude/ .github/ .opencode/ .cursor/ AGENTS.md +python3 scripts/sync_agents.py --check # exit 1 if generated folders are stale (used by CI) ``` -| Target | Output | -|---|---| -| Antigravity | `.agent/` (native source) | -| Claude Code | `.claude/` (agents · skills · workflows · `commands/`) + `CLAUDE.md` | -| GitHub Copilot | `.github/` + `copilot-instructions.md` | -| OpenCode | `.opencode/` (agents `mode: subagent` · skills · commands) + root `AGENTS.md` | -| Cursor | `.cursor/` (`rules/*.mdc` · `commands/`) + root `AGENTS.md` | -| OpenAI Codex / cross-tool | root `AGENTS.md` | - ```text -.agent/ ← edit here (source of truth) -├── agents/ # 5 specialist personas -├── skills/ # 8 on-demand knowledge modules +.agent/ ← edit here +├── agents/ # 1 independent reviewer +├── skills/ # 9 on-demand knowledge modules (+ scripts, references) ├── workflows/ # 3 slash commands -└── rules/ # global rules (rules.md) +└── rules/ # always-on rules (rules.md) -.claude/ .github/ .opencode/ .cursor/ AGENTS.md ← auto-generated (do not edit) +.claude/ .github/ .opencode/ .cursor/ AGENTS.md ← generated (do not edit) ``` +The generated folders ship in the package because `init` installs one folder per tool. + --- -## 🧭 Design philosophy +## 🤝 Contributing -- **Less is more.** Extra agents and verbose instructions cost the model attention and blur routing. - We capture domain-specific context only — never restate what the model already knows. -- **Progressive disclosure.** Skills load when relevant, not upfront. -- **No ambiguous routing.** Each agent owns a distinct domain. -- **Idempotent & safe by default.** Write-Audit-Publish, mask PII at Silver, check downstream impact. -- **Self-orchestrating.** Agents route, skills self-activate, governance is always on. +See **[CONTRIBUTING.md](CONTRIBUTING.md)**: author in `.agent/` only, sync, test — and apply the +agent-survival rule before adding anything. Release history is in [CHANGELOG.md](CHANGELOG.md). --- -## 🤝 Contributing & license +## 🙏 Credits & references + +- [Power BI Desktop Bridge](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-desktop-bridge-overview) +- [Power BI Report skill](https://learn.microsoft.com/en-us/power-bi/developer/agentic/power-bi-report-skill-overview) +- [Power BI Authoring MCP server](https://learn.microsoft.com/en-us/power-bi/developer/mcp/power-bi-authoring-mcp) +- [microsoft/skills-for-fabric](https://github.com/microsoft/skills-for-fabric/tree/main/plugins/powerbi-authoring) (MIT) — skill `powerbi-report-cli` +- `DESIGN.md` follows the Google Labs spec (Apache-2.0). See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). + +--- -Want to extend it? See **[CONTRIBUTING.md](CONTRIBUTING.md)** — especially the *"should this agent -exist?"* gates. The `creating-agents-and-skills` skill walks you through deciding, authoring, and -syncing a new capability across every platform. +## 📄 License -**MIT** © [Lucca Pinto](https://github.com/luccapinto) +**MIT** © [Lucca Pinto](https://github.com/luccapinto) — repository: +[luccapinto/agentic-data-kit](https://github.com/luccapinto/agentic-data-kit) <div align="center"><sub>Built for data teams who'd rather ship than re-explain the basics.</sub></div> diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md new file mode 100644 index 0000000..2f1a245 --- /dev/null +++ b/THIRD_PARTY_NOTICES.md @@ -0,0 +1,47 @@ +# Third-party notices + +The Agentic Data Kit is MIT-licensed (see `LICENSE`). Its text is original. It **cites and +links** the following sources without copying their prose: + +- Microsoft Learn — Power BI Desktop Bridge, Power BI Report skill, Power BI Authoring MCP server. +- [microsoft/skills-for-fabric](https://github.com/microsoft/skills-for-fabric) (MIT), plugin + `powerbi-authoring` (skills `powerbi-report-cli`, `semantic-model-authoring`). +- [microsoft/powerbi-modeling-mcp](https://github.com/microsoft/powerbi-modeling-mcp). +- [Tabular Editor 2](https://github.com/TabularEditor/TabularEditor) (MIT) — the BPA rule format + and command-line options used by `run_bpa.py`. +- The `DESIGN.md` format (Google Labs spec, Apache-2.0) — the kit's template follows the format; + its content is original. + +Command names, flags and JSON field names of these tools are reproduced as facts so the kit can +call them. + +## Microsoft Power BI base theme (test fixture only) + +`tests/fixtures/SalesSample/SalesSample.Report/StaticResources/SharedResources/BaseThemes/Fluent2-CY26SU10.json` +was generated by `powerbi-report-author scaffold` from `@microsoft/powerbi-report-authoring-cli` +0.5.0, distributed under the following license. The file is used only by the test fixture and is +not part of the npm package. + +``` +MIT License + +Copyright (c) Microsoft Corporation. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +``` diff --git a/bin/cli.js b/bin/cli.js old mode 100644 new mode 100755 index c58c151..2ec6642 --- a/bin/cli.js +++ b/bin/cli.js @@ -11,15 +11,84 @@ import { fileURLToPath } from 'url'; const __filename = fileURLToPath(import.meta.url); const __dirname = path.dirname(__filename); +const packageRoot = path.resolve(__dirname, '..'); const program = new Command(); const packageJson = JSON.parse( - fs.readFileSync(path.join(__dirname, '../package.json'), 'utf-8') + fs.readFileSync(path.join(packageRoot, 'package.json'), 'utf-8') ); // Brand gradient: blue → sky → teal (mirrors the DESIGN.md accent family). const brand = gradient(['#5B8DEF', '#38BDF8', '#0E7C66']); +/** + * Install targets. `folder` is copied as-is; `rootFiles` are copied from the package root + * into the destination root. `exclude` lists paths (relative to `folder`) that ship in the + * repo but must not land in the user's project. + */ +const TARGETS = { + claude: { + name: 'Claude Code', + folder: '.claude', + description: 'Installs the .claude folder (agents, skills, commands, CLAUDE.md)', + steps: [ + `Run ${chalk.bold('claude')} in your project; the pbi-* skills load when a request matches.`, + `Try: ${chalk.italic('"Add a Total Sales measure and a card for it in Sales.pbip"')}.`, + ], + }, + copilot: { + name: 'GitHub Copilot (VS Code)', + folder: '.github', + exclude: ['workflows'], + description: 'Installs agents, skills, prompts and copilot-instructions.md into .github', + steps: [ + `Skills, prompts and the ${chalk.bold('pbi-visual-reviewer')} agent now live in ${chalk.bold('.github/')}.`, + `Run ${chalk.bold('/validate-pbi')} in Copilot Chat to check a PBIP project.`, + ], + }, + cursor: { + name: 'Cursor', + folder: '.cursor', + description: 'Installs the .cursor folder (always-on rule, commands, skills)', + steps: [ + `Rules, commands and skills now live in ${chalk.bold('.cursor/')}.`, + `No subagents here: run the visual review in a fresh chat with only the screenshots.`, + ], + }, + opencode: { + name: 'OpenCode', + folder: '.opencode', + rootFiles: ['AGENTS.md'], + description: 'Installs .opencode (agents, skills, commands) and the root AGENTS.md', + steps: [ + `Root ${chalk.bold('AGENTS.md')} + agents, skills and commands in ${chalk.bold('.opencode/')}.`, + `Call the reviewer with ${chalk.bold('@pbi-visual-reviewer')}.`, + ], + }, + antigravity: { + name: 'Antigravity (source of truth)', + folder: '.agent', + rootFiles: ['scripts/sync_agents.py'], + description: 'Installs the .agent folder (source of truth) and scripts/sync_agents.py', + steps: [ + `Your source of truth lives in ${chalk.bold('.agent/')}.`, + `Run ${chalk.bold('python3 scripts/sync_agents.py')} (needs PyYAML) to fan it out to every other tool.`, + ], + }, +}; + +function countKit() { + const src = path.join(packageRoot, '.agent'); + const list = (dir, pred) => (fs.existsSync(dir) ? fs.readdirSync(dir, { withFileTypes: true }).filter(pred).length : 0); + return { + agents: list(path.join(src, 'agents'), (e) => e.isFile() && e.name.endsWith('.md')), + skills: list(path.join(src, 'skills'), (e) => e.isDirectory()), + workflows: list(path.join(src, 'workflows'), (e) => e.isFile() && e.name.endsWith('.md')), + }; +} + +const plural = (n, word) => `${n} ${word}${n === 1 ? '' : 's'}`; + /** Big, gradient ASCII banner with graceful fallbacks for narrow terminals. */ function printBanner() { const cols = process.stdout.columns || 80; @@ -35,9 +104,13 @@ function printBanner() { } catch { console.log('\n' + chalk.cyanBright.bold('AGENTIC DATA KIT')); } - console.log(chalk.bold(' Drop one folder. Get a team of AI data specialists.')); + const kit = countKit(); + console.log(chalk.bold(' Power BI as code, with Desktop open. Skills for every AI assistant.')); console.log( - chalk.dim(` 5 agents · 8 skills · 3 workflows · 4 platforms`) + + chalk.dim( + ` ${plural(kit.agents, 'agent')} · ${plural(kit.skills, 'skill')} · ` + + `${plural(kit.workflows, 'workflow')} · ${plural(Object.keys(TARGETS).length, 'platform')}` + ) + chalk.dim(` · v${packageJson.version}\n`) ); } @@ -59,36 +132,11 @@ function printSuccess(label, steps) { ); } -const NEXT_STEPS = { - antigravity: { - label: 'Antigravity (source of truth)', - steps: [ - `Your source of truth lives in ${chalk.bold('.agent/')}.`, - `Run ${chalk.bold('python scripts/sync_agents.py')} after any change to fan out to every tool.`, - ], - }, - copilot: { - label: 'GitHub Copilot', - steps: [ - `Use ${chalk.bold('@')} triggers in Copilot Chat (e.g. ${chalk.bold('@data-engineer')}).`, - `Agents, skills and prompts now live in ${chalk.bold('.github/')}.`, - ], - }, - claude: { - label: 'Claude Code', - steps: [ - `Run ${chalk.bold('claude')} in your terminal to meet your new specialists.`, - `Try: ${chalk.italic('"Build a Medallion pipeline for customer data"')}.`, - ], - }, - opencode: { - label: 'OpenCode', - steps: [ - `Root ${chalk.bold('AGENTS.md')} + everything else in ${chalk.bold('.opencode/')}.`, - `Your project now follows the OpenCode standard.`, - ], - }, -}; +function fail(message, hint) { + console.error(chalk.red(`✗ ${message}`)); + if (hint) console.log(chalk.yellow(` ${hint}`)); + process.exit(1); +} program .name('agentic-data-kit') @@ -99,112 +147,80 @@ program .command('init') .description('Install the Agentic Data Kit into your project') .option('-p, --path <dest>', 'Specific directory to install to', '.') - .option('-t, --target <framework>', 'Target framework (antigravity, copilot, claude, opencode)') + .option('-t, --target <framework>', `Target framework (${Object.keys(TARGETS).join(', ')})`) .option('-f, --force', 'Overwrite existing folders') .action(async (options) => { printBanner(); const destDir = path.resolve(process.cwd(), options.path); - - let targetFramework = options.target; - - if (!targetFramework) { - targetFramework = await select({ + let targetKey = options.target; + if (!targetKey) { + targetKey = await select({ message: 'Which AI assistant are you using?', - choices: [ - { - name: 'GitHub Copilot (VS Code)', - value: 'copilot', - description: 'Installs the .github folder' - }, - { - name: 'Claude Code (CLI)', - value: 'claude', - description: 'Installs the .claude folder' - }, - { - name: 'Antigravity (Autonomous Agents & Sync Engine)', - value: 'antigravity', - description: 'Installs the .agent folder (Source of truth)' - }, - { - name: 'OpenCode (Standard Agentic Format)', - value: 'opencode', - description: 'Installs AGENTS.md to root and /agents, /skills, /commands folders' - } - ] + choices: Object.entries(TARGETS).map(([value, t]) => ({ + name: t.name, + value, + description: t.description, + })), }); } - // Map the user choice to the actual folder name in the repo - const folderMapping = { - 'antigravity': '.agent', - 'copilot': '.github', - 'claude': '.claude', - 'opencode': '.opencode' - }; - - const targetFolder = folderMapping[targetFramework]; - - if (!targetFolder) { - console.error(chalk.red(`✗ Invalid target '${targetFramework}'. Choose 'antigravity', 'copilot', 'claude', or 'opencode'.`)); - process.exit(1); + const target = TARGETS[targetKey]; + if (!target) { + fail(`Invalid target '${targetKey}'. Choose one of: ${Object.keys(TARGETS).join(', ')}.`); } - const sourceDir = path.resolve(__dirname, '..'); - const agentSourceDir = path.join(sourceDir, targetFolder); - - // Final destination directory - const agentDestDir = path.join(destDir, targetFolder); + const sourceFolder = path.join(packageRoot, target.folder); + const destFolder = path.join(destDir, target.folder); + const rootFiles = target.rootFiles || []; + // .github is shared with the user's own files (workflows, templates): only the kit's + // generated entries are checked, removed and copied — never the whole folder. + const shared = target.folder === '.github'; + const entries = shared + ? (await fs.readdir(sourceFolder).catch(() => [])).filter((e) => !(target.exclude || []).includes(e)) + : null; try { - if (!await fs.pathExists(agentSourceDir)) { - console.error(chalk.red(`✗ Source ${targetFolder} directory not found in the package.`)); - if (targetFramework !== 'antigravity') { - console.log(chalk.yellow(` Did you run 'python scripts/sync_agents.py' to generate the ${targetFolder} folder before publishing?`)); - } - process.exit(1); + if (!(await fs.pathExists(sourceFolder))) { + fail( + `Source ${target.folder} directory not found in the package.`, + targetKey !== 'antigravity' ? `Run 'python3 scripts/sync_agents.py' before publishing.` : undefined + ); } - const alreadyExists = targetFramework === 'opencode' - ? (await fs.pathExists(agentDestDir) || await fs.pathExists(path.join(destDir, 'AGENTS.md'))) - : await fs.pathExists(agentDestDir); - - if (alreadyExists) { - if (options.force) { - console.log(chalk.yellow(`⚠ Force flag used — overwriting existing ${targetFramework} setup...`)); - await fs.remove(agentDestDir); - if (targetFramework === 'opencode') { - await fs.remove(path.join(destDir, 'AGENTS.md')); - } - } else { - const errorMsg = targetFramework === 'opencode' - ? `✗ Already initialized for OpenCode (found .opencode folder or AGENTS.md). Use --force to overwrite.` - : `✗ ${targetFolder} directory already exists. Use --force to overwrite.`; - console.error(chalk.red(errorMsg)); - process.exit(1); - } + const installed = shared + ? entries.map((e) => path.join(destFolder, e)) + : [destFolder]; + const existing = []; + for (const p of [...installed, ...rootFiles.map((f) => path.join(destDir, f))]) { + if (await fs.pathExists(p)) existing.push(path.relative(destDir, p) || p); } - process.stdout.write(chalk.dim(` Installing ${targetFramework} setup… `)); - await fs.copy(agentSourceDir, agentDestDir); + if (existing.length) { + if (!options.force) { + fail(`Already initialized for ${target.name} (found ${existing.join(', ')}). Use --force to overwrite.`); + } + console.log(chalk.yellow(`⚠ Force flag used — overwriting ${existing.join(', ')}...`)); + for (const p of existing) await fs.remove(path.join(destDir, p)); + } - // OpenCode special handling: move AGENTS.md to root - if (targetFramework === 'opencode') { - const agentsMdPath = path.join(agentDestDir, 'AGENTS.md'); - const rootAgentsMdPath = path.join(destDir, 'AGENTS.md'); - if (await fs.pathExists(agentsMdPath)) { - await fs.move(agentsMdPath, rootAgentsMdPath, { overwrite: true }); + process.stdout.write(chalk.dim(` Installing ${targetKey} setup… `)); + const noBytecode = (src) => !path.relative(sourceFolder, src).split(path.sep).includes('__pycache__'); + if (shared) { + for (const e of entries) { + await fs.copy(path.join(sourceFolder, e), path.join(destFolder, e), { filter: noBytecode }); } + } else { + await fs.copy(sourceFolder, destFolder, { filter: noBytecode }); + } + for (const f of rootFiles) { + await fs.copy(path.join(packageRoot, f), path.join(destDir, f)); } console.log(chalk.green('done')); - const info = NEXT_STEPS[targetFramework]; - printSuccess(info.label, info.steps); - + printSuccess(target.name, target.steps); } catch (err) { - console.error(chalk.red(`\n✗ Failed to initialize the kit: ${err.message}`)); - process.exit(1); + fail(`Failed to initialize the kit: ${err.message}`); } }); diff --git a/package.json b/package.json index 92592f5..ec96025 100644 --- a/package.json +++ b/package.json @@ -1,19 +1,37 @@ { "name": "@luccapinto/agentic-data-kit", - "version": "4.1.1", - "description": "Lean AI agents, skills, and workflows for Data Teams (Claude Code, GitHub Copilot, OpenCode, Antigravity)", + "version": "5.0.0", + "description": "Power BI as code with Desktop open (Modeling MCP + Desktop Bridge), plus lean skills for data teams — for Claude Code, GitHub Copilot, OpenCode, Cursor and Antigravity", "type": "module", "main": "bin/cli.js", "bin": { "agentic-data-kit": "bin/cli.js", "ag-data-kit": "bin/cli.js" }, + "files": [ + "bin/", + ".agent/", + ".claude/", + ".cursor/", + ".opencode/", + ".github/agents/", + ".github/prompts/", + ".github/skills/", + ".github/copilot-instructions.md", + "AGENTS.md", + "scripts/sync_agents.py", + "CHANGELOG.md", + "THIRD_PARTY_NOTICES.md" + ], "scripts": { - "test": "echo \"Error: no test specified\" && exit 1" + "sync": "python3 scripts/sync_agents.py", + "sync:check": "python3 scripts/sync_agents.py --check", + "test": "python3 -B -m unittest discover -s tests -v", + "prepack": "python3 scripts/sync_agents.py" }, "repository": { "type": "git", - "url": "git+https://github.com/luccapinto/agentic-ai-kit.git" + "url": "git+https://github.com/luccapinto/agentic-data-kit.git" }, "keywords": [ "agentic-ai", @@ -22,20 +40,25 @@ "skills", "claude-code", "github-copilot", + "opencode", + "cursor", "power-bi", + "power-bi-desktop", + "pbip", + "pbir", "tmdl", + "mcp", + "best-practice-analyzer", "data-engineering", "presentations", - "reveal-js", - "design-md", - "templates" + "design-md" ], "author": "Lucca Pinto", "license": "MIT", "bugs": { - "url": "https://github.com/luccapinto/agentic-ai-kit/issues" + "url": "https://github.com/luccapinto/agentic-data-kit/issues" }, - "homepage": "https://github.com/luccapinto/agentic-ai-kit#readme", + "homepage": "https://github.com/luccapinto/agentic-data-kit#readme", "dependencies": { "@inquirer/prompts": "^8.3.0", "boxen": "^8.0.1", diff --git a/scripts/sync_agents.py b/scripts/sync_agents.py index 3559e6b..8e766c6 100644 --- a/scripts/sync_agents.py +++ b/scripts/sync_agents.py @@ -1,102 +1,123 @@ """ sync_agents.py ============== -Sincroniza a pasta `.agent` (Fonte da Verdade — Antigravity) para cada ferramenta, -sempre no formato NATIVO de cada uma (não é cópia byte a byte): - - 🟣 SOURCE : .agent/ - ⚪ COPILOT : .github/agents/*.agent.md | .github/skills/ | .github/prompts/*.prompt.md | .github/copilot-instructions.md - 🟠 CLAUDE : .claude/agents/*.md | .claude/skills/ | .claude/workflows/ + .claude/commands/ | .claude/CLAUDE.md - 🔵 OPENCODE: .opencode/agents/*.md (mode: subagent + permission) | .opencode/skills/ | .opencode/commands/ - 🟢 CURSOR : .cursor/rules/*.mdc (alwaysApply) | .cursor/commands/*.md - 📄 RAIZ : AGENTS.md → padrão cross-tool lido por OpenCode, OpenAI Codex, Cursor, Jules etc. - -Notas de compatibilidade: - - O frontmatter `trigger: always_on` do `.agent/rules/rules.md` é específico do Antigravity; - ele é REMOVIDO ao gerar AGENTS.md, CLAUDE.md, copilot-instructions.md e as regras do Cursor. - - OpenCode lê AGENTS.md da RAIZ (não `.opencode/AGENTS.md`), então não geramos mais aquele arquivo. - - OpenAI Codex é coberto integralmente pelo AGENTS.md da raiz — nada extra a gerar. +Compile `.agent/` (the source of truth, Antigravity layout) into every supported tool's +NATIVE layout. This is a translation, not a byte copy: + + SOURCE : .agent/{agents,skills,workflows,rules/rules.md} + COPILOT : .github/agents/*.agent.md (name, description, tools mapped to Copilot aliases) + .github/skills/ | .github/prompts/*.prompt.md | .github/copilot-instructions.md + CLAUDE : .claude/agents/*.md (verbatim) | .claude/skills/ | .claude/workflows/ + .claude/commands/ (native slash commands) | .claude/CLAUDE.md (rules + commands) + OPENCODE: .opencode/agents/*.md (mode: subagent + permission) | .opencode/skills/ + .opencode/commands/ + CURSOR : .cursor/rules/*.mdc (alwaysApply) | .cursor/commands/*.md | .cursor/skills/ + ROOT : AGENTS.md (rules) -> read by OpenCode, OpenAI Codex, Cursor and others. + +Usage: + python3 scripts/sync_agents.py # regenerate every managed path + python3 scripts/sync_agents.py --check # exit 1 if any managed path is stale (CI) + +Compatibility notes: + - `trigger: always_on` in `.agent/rules/rules.md` is Antigravity-only; it is stripped from + every generated rules file. + - OpenCode reads the ROOT AGENTS.md, so `.opencode/AGENTS.md` is never generated (and is + deleted if a legacy copy exists). + - `.github/workflows/` is not managed here; CI files live there untouched. """ +from __future__ import annotations + +import argparse +import filecmp import re import shutil import sys -import codecs +import tempfile from pathlib import Path -# Fix Windows encoding issues with emojis -if sys.platform == 'win32': - sys.stdout = codecs.getwriter("utf-8")(sys.stdout.detach()) - import yaml # pip install pyyaml -# --------------------------------------------------------------------------- -# Caminhos raiz -# --------------------------------------------------------------------------- -ROOT = Path(__file__).resolve().parent.parent +if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8") -# 📄 Padrão cross-tool (raiz): Copilot coding agent, Cursor, etc. -ROOT_AGENTS_MD = ROOT / "AGENTS.md" +ROOT = Path(__file__).resolve().parent.parent -# 🟣 Fonte da Verdade (Antigravity) SOURCE = ROOT / ".agent" -AGENTS_SRC = SOURCE / "agents" -SKILLS_SRC = SOURCE / "skills" +AGENTS_SRC = SOURCE / "agents" +SKILLS_SRC = SOURCE / "skills" WORKFLOWS_SRC = SOURCE / "workflows" -RULES_SRC = SOURCE / "rules" / "rules.md" - -# ⚪ Destino Copilot (.github) -COPILOT = ROOT / ".github" -COPILOT_AGENTS = COPILOT / "agents" -COPILOT_SKILLS = COPILOT / "skills" -COPILOT_PROMPTS = COPILOT / "prompts" -COPILOT_RULES = COPILOT / "copilot-instructions.md" - -# 🟠 Destino Claude (.claude) -CLAUDE = ROOT / ".claude" -CLAUDE_AGENTS = CLAUDE / "agents" -CLAUDE_SKILLS = CLAUDE / "skills" -CLAUDE_WORKFLOWS = CLAUDE / "workflows" -CLAUDE_COMMANDS = CLAUDE / "commands" -CLAUDE_MD = CLAUDE / "CLAUDE.md" - -# 🔵 Destino OpenCode (.opencode) -OPENCODE = ROOT / ".opencode" -OPENCODE_AGENTS = OPENCODE / "agents" -OPENCODE_SKILLS = OPENCODE / "skills" -OPENCODE_COMMANDS = OPENCODE / "commands" -OPENCODE_AGENTS_MD = OPENCODE / "AGENTS.md" # legado: não é lido pelo OpenCode → removido no sync - -# 🟢 Destino Cursor (.cursor) -CURSOR = ROOT / ".cursor" -CURSOR_RULES = CURSOR / "rules" -CURSOR_COMMANDS = CURSOR / "commands" - -# Descrição curta usada no frontmatter da regra always-on do Cursor. +RULES_SRC = SOURCE / "rules" / "rules.md" + +GENERATED_HEADER = "<!-- Auto-generated by scripts/sync_agents.py — DO NOT EDIT MANUALLY -->\n\n" + +# Every path (relative to the repo root) owned by this script. Anything here is deleted and +# rebuilt on sync, and compared on --check. +MANAGED = [ + ".github/agents", + ".github/skills", + ".github/prompts", + ".github/copilot-instructions.md", + ".claude/agents", + ".claude/skills", + ".claude/workflows", + ".claude/commands", + ".claude/CLAUDE.md", + ".opencode/agents", + ".opencode/skills", + ".opencode/commands", + ".cursor/rules", + ".cursor/commands", + ".cursor/skills", + "AGENTS.md", +] +# Paths that must NOT exist after a sync (older layouts). +LEGACY = [".opencode/AGENTS.md"] + +# Short description used in the frontmatter of the always-on Cursor rule. RULES_DESCRIPTION = ( - "Agentic Data Kit — always-on operating rules: agent routing, data engineering " - "principles (idempotency, Write-Audit-Publish, PII masking) and the data debugging reference." + "Agentic Data Kit — always-on operating rules: skills first, the Power BI Desktop-open " + "loop with independent visual review, data engineering principles (idempotency, " + "Write-Audit-Publish, PII masking) and the data debugging reference." ) +# Claude-style tool names (used in `.agent/agents/*.md`) -> GitHub Copilot custom-agent tool +# aliases. Unknown names are an error: silently dropping a tool would change what the agent +# is allowed to do. +COPILOT_TOOL_ALIASES = { + "read": "read", + "notebookread": "read", + "grep": "search", + "glob": "search", + "edit": "edit", + "multiedit": "edit", + "write": "edit", + "notebookedit": "edit", + "bash": "execute", + "webfetch": "web", + "websearch": "web", + "task": "agent", + "todowrite": "todo", +} + # --------------------------------------------------------------------------- -# Helpers compartilhados +# Helpers # --------------------------------------------------------------------------- def parse_frontmatter(text: str) -> tuple[dict, str]: - """Separa YAML frontmatter do corpo. Retorna ({}, texto) se não houver.""" - pattern = re.compile(r"^---\s*\n(.*?)\n---\s*\n", re.DOTALL) - match = pattern.match(text) - if match: - try: - meta = yaml.safe_load(match.group(1)) or {} - except yaml.YAMLError: - meta = {} - return meta, text[match.end():] - return {}, text + """Split YAML frontmatter from the body. Returns ({}, text) when there is none.""" + match = re.match(r"^---\s*\n(.*?)\n---\s*\n", text, re.DOTALL) + if not match: + return {}, text + try: + meta = yaml.safe_load(match.group(1)) or {} + except yaml.YAMLError: + meta = {} + return meta, text[match.end():] def first_sentence(text: str) -> str: - """Primeira linha significativa do corpo (ignora títulos # e separadores).""" + """First meaningful body line (skips headings and separators).""" for line in text.splitlines(): line = line.strip() if line and not line.startswith("#") and not line.startswith("---"): @@ -107,374 +128,255 @@ def first_sentence(text: str) -> str: def to_title_case(slug: str) -> str: - """'data-scientist' → 'Data Scientist'.""" + """'pbi-visual-reviewer' -> 'Pbi Visual Reviewer'.""" return slug.replace("-", " ").replace("_", " ").title() def strip_frontmatter(text: str) -> str: - """Remove o YAML frontmatter inicial (ex.: `trigger: always_on` do Antigravity).""" _, body = parse_frontmatter(text) return body.lstrip() -def permission_from_tools(tools_val) -> dict: - """Traduz a allow-list `tools` da fonte (estilo Claude) para o `permission` do OpenCode. - - OpenCode controla acesso por categorias grosseiras (`edit`, `bash`) via `permission` - (o campo `tools` é deprecado). Read/Grep/Glob nunca são bloqueados, então não entram aqui. - """ +def split_tools(tools_val) -> list[str]: if not tools_val: - return {} - tools = {t.strip().lower() for t in str(tools_val).split(",") if t.strip()} - return { - "edit": "allow" if ({"edit", "write"} & tools) else "deny", - "bash": "allow" if ("bash" in tools) else "deny", - } - - -def dump_frontmatter(meta: dict) -> str: - """YAML frontmatter preservando a ordem de inserção das chaves.""" - return yaml.dump(meta, allow_unicode=True, default_flow_style=False, sort_keys=False).strip() - - -def clean_dirs(*dirs: Path) -> None: - """Remove diretórios gerenciados (sem recriar — cada função cria o seu).""" - for d in dirs: - if d.exists(): - shutil.rmtree(d) - print(f" 🗑️ Limpou: {d.relative_to(ROOT)}") - - -def copy_tree(src: Path, dest: Path, label: str) -> None: - """copytree seguro: exige que dest NÃO exista (clean_dirs deve rodar antes).""" - if src.exists(): - shutil.copytree(src, dest) - count = sum(1 for f in dest.rglob("*") if f.is_file()) - print(f" ✅ {count} arquivo(s) copiados → {label}") + return [] + if isinstance(tools_val, list): + items = tools_val else: - print(f" ⚠️ Pasta não encontrada: {src.relative_to(ROOT)}") + items = str(tools_val).split(",") + return [str(t).strip() for t in items if str(t).strip()] -# =========================================================================== -# ⚪ BLOCO COPILOT (.github) -# =========================================================================== +def copilot_tools(tools_val, agent: str) -> list[str]: + """Map Claude-style tool names to Copilot aliases, keeping first-seen order.""" + mapped: list[str] = [] + for tool in split_tools(tools_val): + alias = COPILOT_TOOL_ALIASES.get(tool.lower()) + if alias is None: + raise SystemExit( + f"error: agent '{agent}' declares tool '{tool}' with no Copilot mapping; " + "add it to COPILOT_TOOL_ALIASES in scripts/sync_agents.py" + ) + if alias not in mapped: + mapped.append(alias) + return mapped -def copilot_sync_agents() -> None: - """Agentes: injeta YAML padronizado + muda extensão para .agent.md.""" - COPILOT_AGENTS.mkdir(parents=True, exist_ok=True) - for src_file in sorted(AGENTS_SRC.glob("*.md")): - raw = src_file.read_text(encoding="utf-8") - meta, body = parse_frontmatter(raw) - stem = src_file.stem - name = meta.get("name") or to_title_case(stem) - description = meta.get("description") or first_sentence(body) - - # Copilot custom agents (.agent.md): `description` obrigatório; `name` opcional. - # NÃO repassamos `tools`: os nomes estilo Claude (Read, Grep…) não correspondem aos - # IDs de ferramenta do Copilot (codebase, search, edit…) e seriam ignorados. - new_meta: dict = {"name": name, "description": description} - yaml_block = dump_frontmatter(new_meta) - output = f"---\n{yaml_block}\n---\n\n{body.lstrip()}" - - dest_file = COPILOT_AGENTS / f"{stem}.agent.md" - dest_file.write_text(output, encoding="utf-8") - print(f" ✅ {src_file.name} → {dest_file.name}") - - -def copilot_sync_skills() -> None: - """Skills: cópia recursiva exata.""" - copy_tree(SKILLS_SRC, COPILOT_SKILLS, COPILOT_SKILLS.relative_to(ROOT)) +def opencode_permission(tools_val) -> dict: + """Map the source `tools` allow-list to OpenCode's coarse `permission` block. + OpenCode gates `edit` and `bash`; read/grep/glob are always allowed. + """ + tools = {t.lower() for t in split_tools(tools_val)} + if not tools: + return {} + return { + "edit": "allow" if tools & {"edit", "write", "multiedit", "notebookedit"} else "deny", + "bash": "allow" if "bash" in tools else "deny", + } -def copilot_sync_prompts() -> None: - """Workflows → Prompt Files (.prompt.md) sem campo model.""" - COPILOT_PROMPTS.mkdir(parents=True, exist_ok=True) - for src_file in sorted(WORKFLOWS_SRC.glob("*.md")): - raw = src_file.read_text(encoding="utf-8") - meta, body = parse_frontmatter(raw) - stem = src_file.stem - description = meta.get("description") or f"Workflow: {to_title_case(stem)}" - new_meta = {"name": stem, "description": description} - yaml_block = yaml.dump(new_meta, allow_unicode=True, default_flow_style=False).strip() +def dump_frontmatter(meta: dict) -> str: + return yaml.dump(meta, allow_unicode=True, default_flow_style=False, sort_keys=False).strip() - context_block = "**Context:** ${selection}\n" - output = f"---\n{yaml_block}\n---\n\n{context_block}\n{body.lstrip()}" - dest_file = COPILOT_PROMPTS / f"{stem}.prompt.md" - dest_file.write_text(output, encoding="utf-8") - print(f" ✅ {src_file.name} → {dest_file.name}") +def write(path: Path, text: str) -> None: + """Write UTF-8 with LF line endings on every OS so --check is platform-independent.""" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(text.encode("utf-8")) -def copilot_sync_rules() -> None: - """Rules → copilot-instructions.md.""" - if not RULES_SRC.exists(): - print(f" ⚠️ rules.md não encontrado: {RULES_SRC}") - return - header = "<!-- Auto-generated by scripts/sync_agents.py — NÃO EDITE MANUALMENTE -->\n\n" - body = strip_frontmatter(RULES_SRC.read_text(encoding="utf-8")) - COPILOT_RULES.write_text(header + body, encoding="utf-8") - print(f" ✅ rules.md → copilot-instructions.md") +def rules_body() -> str: + return strip_frontmatter(RULES_SRC.read_text(encoding="utf-8")) -# =========================================================================== -# 🟠 BLOCO CLAUDE (.claude) -# =========================================================================== +def source_files(folder: Path) -> list[Path]: + return sorted(folder.glob("*.md")) if folder.exists() else [] -def claude_sync_agents() -> None: - """Agentes: cópia direta dos .md originais (Claude não precisa de YAML especial).""" - CLAUDE_AGENTS.mkdir(parents=True, exist_ok=True) - for src_file in sorted(AGENTS_SRC.glob("*.md")): - dest_file = CLAUDE_AGENTS / src_file.name - shutil.copy2(src_file, dest_file) - print(f" ✅ {src_file.name} → {dest_file.relative_to(ROOT)}") +def copy_skills(dest: Path) -> None: + shutil.copytree(SKILLS_SRC, dest, ignore=shutil.ignore_patterns("__pycache__", "*.pyc")) -def claude_sync_skills() -> None: - """Skills: cópia recursiva exata.""" - copy_tree(SKILLS_SRC, CLAUDE_SKILLS, CLAUDE_SKILLS.relative_to(ROOT)) +# --------------------------------------------------------------------------- +# Builders — each writes one tool's layout under `out` (a repo root or a temp dir) +# --------------------------------------------------------------------------- +def build_copilot(out: Path) -> None: + gh = out / ".github" + for src in source_files(AGENTS_SRC): + meta, body = parse_frontmatter(src.read_text(encoding="utf-8")) + fm: dict = { + "name": meta.get("name") or to_title_case(src.stem), + "description": meta.get("description") or first_sentence(body), + } + tools = copilot_tools(meta.get("tools"), src.stem) + if tools: + fm["tools"] = tools + write(gh / "agents" / f"{src.stem}.agent.md", f"---\n{dump_frontmatter(fm)}\n---\n\n{body.lstrip()}") + + copy_skills(gh / "skills") + + for src in source_files(WORKFLOWS_SRC): + meta, body = parse_frontmatter(src.read_text(encoding="utf-8")) + fm = { + "name": src.stem, + "description": meta.get("description") or f"Workflow: {to_title_case(src.stem)}", + } + write( + gh / "prompts" / f"{src.stem}.prompt.md", + f"---\n{dump_frontmatter(fm)}\n---\n\n**Context:** ${{selection}}\n\n{body.lstrip()}", + ) -def claude_sync_workflows() -> None: - """Workflows: cópia recursiva exata (docs referenciadas pelo CLAUDE.md).""" - copy_tree(WORKFLOWS_SRC, CLAUDE_WORKFLOWS, CLAUDE_WORKFLOWS.relative_to(ROOT)) + write(gh / "copilot-instructions.md", GENERATED_HEADER + rules_body()) -def claude_sync_commands() -> None: - """Workflows → slash commands NATIVOS em .claude/commands/ (nome do arquivo = /comando).""" - CLAUDE_COMMANDS.mkdir(parents=True, exist_ok=True) - for src_file in sorted(WORKFLOWS_SRC.glob("*.md")): - dest_file = CLAUDE_COMMANDS / src_file.name - shutil.copy2(src_file, dest_file) - print(f" ✅ {src_file.name} → {dest_file.relative_to(ROOT)}") +def build_claude(out: Path) -> None: + cl = out / ".claude" + for src in source_files(AGENTS_SRC): + write(cl / "agents" / src.name, src.read_text(encoding="utf-8")) + copy_skills(cl / "skills") + shutil.copytree(WORKFLOWS_SRC, cl / "workflows") + for src in source_files(WORKFLOWS_SRC): + write(cl / "commands" / src.name, src.read_text(encoding="utf-8")) -def claude_build_claude_md() -> None: - """CLAUDE.md = rules.md + seção de comandos dinâmicos gerada dos workflows.""" - if not RULES_SRC.exists(): - print(f" ⚠️ rules.md não encontrado, CLAUDE.md não será gerado.") - return - - rules_content = strip_frontmatter(RULES_SRC.read_text(encoding="utf-8")) - - # Constrói bloco de comandos a partir dos workflows disponíveis - commands_lines: list[str] = [] - for wf_file in sorted(WORKFLOWS_SRC.glob("*.md")): - raw = wf_file.read_text(encoding="utf-8") - meta, _ = parse_frontmatter(raw) - stem = wf_file.stem - description = meta.get("description") or to_title_case(stem) - commands_lines.append( - f"- **/{stem}**: {description} " - f"— Load context from `.claude/workflows/{wf_file.name}` and follow its steps." + commands = [] + for src in source_files(WORKFLOWS_SRC): + meta, _ = parse_frontmatter(src.read_text(encoding="utf-8")) + description = meta.get("description") or to_title_case(src.stem) + commands.append( + f"- **/{src.stem}**: {description} " + f"— Load context from `.claude/workflows/{src.name}` and follow its steps." ) - - commands_block = "\n".join(commands_lines) - - claude_md_content = ( - "<!-- Auto-generated by scripts/sync_agents.py — NÃO EDITE MANUALMENTE -->\n\n" - + rules_content.rstrip() - + "\n\n---\n\n" - + "## ⚡ Custom Commands & Workflows\n\n" + write( + cl / "CLAUDE.md", + GENERATED_HEADER + + rules_body().rstrip() + + "\n\n---\n\n## Custom Commands & Workflows\n\n" + "The following slash commands are available. When triggered, load the referenced\n" + "workflow file and follow its instructions step by step.\n\n" - + commands_block - + "\n" + + "\n".join(commands) + + "\n", ) - CLAUDE_MD.write_text(claude_md_content, encoding="utf-8") - print(f" ✅ CLAUDE.md gerado com {len(commands_lines)} comando(s)") - - -# =========================================================================== -# 🔵 BLOCO OPENCODE (.opencode) -# =========================================================================== -def opencode_sync_agents() -> None: - """Agentes: reescreve o frontmatter para o formato nativo do OpenCode. - - Fonte (Antigravity) usa `tools: Read, Grep…`, `model: inherit`, `name`, `skills` — nada - disso é válido no OpenCode. Aqui geramos `description` + `mode: subagent` + `permission`. - """ - OPENCODE_AGENTS.mkdir(parents=True, exist_ok=True) - for src_file in sorted(AGENTS_SRC.glob("*.md")): - raw = src_file.read_text(encoding="utf-8") - meta, body = parse_frontmatter(raw) - - description = meta.get("description") or first_sentence(body) - new_meta: dict = {"description": description, "mode": "subagent"} - perm = permission_from_tools(meta.get("tools")) +def build_opencode(out: Path) -> None: + oc = out / ".opencode" + for src in source_files(AGENTS_SRC): + meta, body = parse_frontmatter(src.read_text(encoding="utf-8")) + fm: dict = {"description": meta.get("description") or first_sentence(body), "mode": "subagent"} + perm = opencode_permission(meta.get("tools")) if perm: - new_meta["permission"] = perm - - yaml_block = dump_frontmatter(new_meta) - output = f"---\n{yaml_block}\n---\n\n{body.lstrip()}" - - dest_file = OPENCODE_AGENTS / src_file.name - dest_file.write_text(output, encoding="utf-8") - print(f" ✅ {src_file.name} → {dest_file.relative_to(ROOT)}") + fm["permission"] = perm + write(oc / "agents" / src.name, f"---\n{dump_frontmatter(fm)}\n---\n\n{body.lstrip()}") + copy_skills(oc / "skills") + for src in source_files(WORKFLOWS_SRC): + write(oc / "commands" / src.name, src.read_text(encoding="utf-8")) -def opencode_sync_skills() -> None: - """Skills: cópia recursiva exata de todas as pastas de skills.""" - copy_tree(SKILLS_SRC, OPENCODE_SKILLS, OPENCODE_SKILLS.relative_to(ROOT)) +def build_cursor(out: Path) -> None: + cu = out / ".cursor" + fm = {"description": RULES_DESCRIPTION, "globs": "", "alwaysApply": True} + write(cu / "rules" / "agentic-data-kit.mdc", f"---\n{dump_frontmatter(fm)}\n---\n\n{rules_body()}") + for src in source_files(WORKFLOWS_SRC): + write(cu / "commands" / src.name, src.read_text(encoding="utf-8")) + copy_skills(cu / "skills") -def opencode_sync_commands() -> None: - """Commands: workflows copiados para .opencode/commands/.""" - OPENCODE_COMMANDS.mkdir(parents=True, exist_ok=True) - for src_file in sorted(WORKFLOWS_SRC.glob("*.md")): - dest_file = OPENCODE_COMMANDS / src_file.name - shutil.copy2(src_file, dest_file) - print(f" ✅ {src_file.name} → {dest_file.relative_to(ROOT)}") +def build_root(out: Path) -> None: + write(out / "AGENTS.md", GENERATED_HEADER + rules_body()) -# =========================================================================== -# 🟢 BLOCO CURSOR (.cursor) -# =========================================================================== -def cursor_sync_rules() -> None: - """Rules → .cursor/rules/*.mdc (always-on). Cursor exige extensão .mdc + frontmatter.""" +def build(out: Path) -> None: if not RULES_SRC.exists(): - print(f" ⚠️ rules.md não encontrado para .cursor/rules") - return - CURSOR_RULES.mkdir(parents=True, exist_ok=True) - body = strip_frontmatter(RULES_SRC.read_text(encoding="utf-8")) - # alwaysApply: True = injetado em toda sessão (equivalente ao trigger: always_on). - fm = {"description": RULES_DESCRIPTION, "globs": "", "alwaysApply": True} - output = f"---\n{dump_frontmatter(fm)}\n---\n\n{body}" - dest = CURSOR_RULES / "agentic-data-kit.mdc" - dest.write_text(output, encoding="utf-8") - print(f" ✅ rules.md → {dest.relative_to(ROOT)}") - + raise SystemExit(f"error: missing {RULES_SRC.relative_to(ROOT)}") + build_copilot(out) + build_claude(out) + build_opencode(out) + build_cursor(out) + build_root(out) -def cursor_sync_commands() -> None: - """Workflows → .cursor/commands/*.md (invocados com /<arquivo> no Cursor).""" - CURSOR_COMMANDS.mkdir(parents=True, exist_ok=True) - for src_file in sorted(WORKFLOWS_SRC.glob("*.md")): - dest_file = CURSOR_COMMANDS / src_file.name - shutil.copy2(src_file, dest_file) - print(f" ✅ {src_file.name} → {dest_file.relative_to(ROOT)}") +# --------------------------------------------------------------------------- +# Sync and check +# --------------------------------------------------------------------------- +def remove(path: Path) -> None: + if path.is_dir(): + shutil.rmtree(path) + elif path.exists(): + path.unlink() + + +def files_under(path: Path) -> set[str]: + if path.is_file(): + return {""} + if not path.is_dir(): + return set() + return { + p.relative_to(path).as_posix() + for p in path.rglob("*") + if p.is_file() and "__pycache__" not in p.parts + } -# =========================================================================== -# 📄 BLOCO RAIZ (AGENTS.md — padrão cross-tool: OpenCode, OpenAI Codex, Cursor…) -# =========================================================================== -def build_root_agents_md() -> None: - """Root AGENTS.md = rules.md. Lido por Copilot coding agent, Cursor e outros.""" - if not RULES_SRC.exists(): - print(" ⚠️ rules.md não encontrado para AGENTS.md (raiz)") - return - header = "<!-- Auto-generated by scripts/sync_agents.py — NÃO EDITE MANUALMENTE -->\n\n" - body = strip_frontmatter(RULES_SRC.read_text(encoding="utf-8")) - ROOT_AGENTS_MD.write_text(header + body, encoding="utf-8") - print(" ✅ rules.md → AGENTS.md (raiz) [cobre OpenAI Codex e OpenCode]") - - -# =========================================================================== -# Main -# =========================================================================== -def main() -> None: - print("=" * 60) - print("🟣 Antigravity Sync — .agent → .github + .claude + .opencode + .cursor") - print("=" * 60) - print(f" Fonte : {SOURCE.relative_to(ROOT)}") - print(f" Copilot : {COPILOT.relative_to(ROOT)}") - print(f" Claude : {CLAUDE.relative_to(ROOT)}") - print(f" OpenCode : {OPENCODE.relative_to(ROOT)}") - print(f" Cursor : {CURSOR.relative_to(ROOT)}") - - # Garante que os destinos-raiz existam - COPILOT.mkdir(parents=True, exist_ok=True) - CLAUDE.mkdir(parents=True, exist_ok=True) - OPENCODE.mkdir(parents=True, exist_ok=True) - CURSOR.mkdir(parents=True, exist_ok=True) - - # Limpeza — skills NÃO entra (copytree exige ausência do dest) - # .github/workflows é preservado (não está na lista) - print("\n🗑️ Limpando diretórios anteriores...") - clean_dirs( - COPILOT_AGENTS, COPILOT_SKILLS, COPILOT_PROMPTS, - CLAUDE_AGENTS, CLAUDE_SKILLS, CLAUDE_WORKFLOWS, CLAUDE_COMMANDS, - OPENCODE_AGENTS, OPENCODE_SKILLS, OPENCODE_COMMANDS, - CURSOR_RULES, CURSOR_COMMANDS, - ) - # Remove o legado .opencode/AGENTS.md (OpenCode lê o AGENTS.md da RAIZ, não este). - if OPENCODE_AGENTS_MD.exists(): - OPENCODE_AGENTS_MD.unlink() - print(f" 🗑️ Removeu legado: {OPENCODE_AGENTS_MD.relative_to(ROOT)}") - - # ------------------------------------------------------------------ - print("\n⚪ [COPILOT] Sincronizando para .github/...") - # ------------------------------------------------------------------ - print(" 📦 Agentes:") - copilot_sync_agents() - print(" 🛠️ Skills:") - copilot_sync_skills() - print(" 📝 Prompts:") - copilot_sync_prompts() - print(" 📋 Regras:") - copilot_sync_rules() - - # ------------------------------------------------------------------ - print("\n🟠 [CLAUDE] Sincronizando para .claude/...") - # ------------------------------------------------------------------ - print(" 📦 Agentes:") - claude_sync_agents() - print(" 🛠️ Skills:") - claude_sync_skills() - print(" 📝 Workflows:") - claude_sync_workflows() - print(" ⚡ Slash commands:") - claude_sync_commands() - print(" 📋 CLAUDE.md:") - claude_build_claude_md() - - # ------------------------------------------------------------------ - print("\n🔵 [OPENCODE] Sincronizando para .opencode/...") - # ------------------------------------------------------------------ - print(" 📦 Agentes:") - opencode_sync_agents() - print(" 🛠️ Skills:") - opencode_sync_skills() - print(" 📝 Commands:") - opencode_sync_commands() - - # ------------------------------------------------------------------ - print("\n🟢 [CURSOR] Sincronizando para .cursor/...") - # ------------------------------------------------------------------ - print(" 📋 Rules (.mdc):") - cursor_sync_rules() - print(" 📝 Commands:") - cursor_sync_commands() - - # ------------------------------------------------------------------ - print("\n📄 [RAIZ] Gerando AGENTS.md (padrão cross-tool)...") - # ------------------------------------------------------------------ - build_root_agents_md() - - # Summary - print("\n" + "=" * 60) - print("✨ Sincronização concluída!") - print(f" ⚪ Copilot agents : {COPILOT_AGENTS.relative_to(ROOT)}") - print(f" ⚪ Copilot skills : {COPILOT_SKILLS.relative_to(ROOT)}") - print(f" ⚪ Copilot prompts : {COPILOT_PROMPTS.relative_to(ROOT)}") - print(f" ⚪ Copilot rules : {COPILOT_RULES.relative_to(ROOT)}") - print(f" 🟠 Claude agents : {CLAUDE_AGENTS.relative_to(ROOT)}") - print(f" 🟠 Claude skills : {CLAUDE_SKILLS.relative_to(ROOT)}") - print(f" 🟠 Claude workflows: {CLAUDE_WORKFLOWS.relative_to(ROOT)}") - print(f" 🟠 Claude commands : {CLAUDE_COMMANDS.relative_to(ROOT)}") - print(f" 🟠 CLAUDE.md : {CLAUDE_MD.relative_to(ROOT)}") - print(f" 🔵 OpenCode agents : {OPENCODE_AGENTS.relative_to(ROOT)}") - print(f" 🔵 OpenCode skills : {OPENCODE_SKILLS.relative_to(ROOT)}") - print(f" 🔵 OpenCode cmd : {OPENCODE_COMMANDS.relative_to(ROOT)}") - print(f" 🟢 Cursor rules : {CURSOR_RULES.relative_to(ROOT)}") - print(f" 🟢 Cursor commands : {CURSOR_COMMANDS.relative_to(ROOT)}") - print(f" 📄 AGENTS.md (raiz): {ROOT_AGENTS_MD.relative_to(ROOT)} [OpenAI Codex + OpenCode]") - print("=" * 60) +def stale_paths(expected_root: Path, actual_root: Path) -> list[str]: + """Managed files that differ between a fresh build and the working tree.""" + problems: list[str] = [] + for rel in MANAGED: + exp, act = expected_root / rel, actual_root / rel + exp_files, act_files = files_under(exp), files_under(act) + for f in sorted(exp_files | act_files): + shown = f"{rel}/{f}" if f else rel + if f not in act_files: + problems.append(f"missing {shown}") + elif f not in exp_files: + problems.append(f"extra {shown}") + elif not filecmp.cmp(exp / f if f else exp, act / f if f else act, shallow=False): + problems.append(f"outdated {shown}") + for rel in LEGACY: + if (actual_root / rel).exists(): + problems.append(f"legacy {rel}") + return problems + + +def sync(root: Path = ROOT) -> None: + for rel in MANAGED + LEGACY: + remove(root / rel) + build(root) + + +def check(root: Path = ROOT) -> list[str]: + with tempfile.TemporaryDirectory(prefix="adk-sync-") as tmp: + build(Path(tmp)) + return stale_paths(Path(tmp), root) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description="Compile .agent/ into every tool's native layout.") + parser.add_argument("--check", action="store_true", help="exit 1 if generated folders are stale") + args = parser.parse_args(argv) + + if args.check: + problems = check() + if problems: + print("Generated folders are stale. Run: python3 scripts/sync_agents.py") + for line in problems: + print(f" {line}") + return 1 + print(f"OK: {len(MANAGED)} managed paths match .agent/") + return 0 + + sync() + agents = [p.stem for p in source_files(AGENTS_SRC)] + skills = sorted(p.name for p in SKILLS_SRC.iterdir() if p.is_dir()) + workflows = [p.stem for p in source_files(WORKFLOWS_SRC)] + print("Synced .agent/ -> .github/ .claude/ .opencode/ .cursor/ AGENTS.md") + print(f" agents ({len(agents)}): {', '.join(agents) or '-'}") + print(f" skills ({len(skills)}): {', '.join(skills)}") + print(f" workflows ({len(workflows)}): {', '.join(workflows)}") + return 0 if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/tests/fixtures/SalesSample/SalesSample.Report/.platform b/tests/fixtures/SalesSample/SalesSample.Report/.platform new file mode 100644 index 0000000..9b052b3 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/.platform @@ -0,0 +1,11 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/gitIntegration/platformProperties/2.0.0/schema.json", + "metadata": { + "type": "Report", + "displayName": "SalesSample" + }, + "config": { + "version": "2.0", + "logicalId": "5d77278b-46d7-473d-88f1-5879b50b1b10" + } +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/StaticResources/SharedResources/BaseThemes/Fluent2-CY26SU10.json b/tests/fixtures/SalesSample/SalesSample.Report/StaticResources/SharedResources/BaseThemes/Fluent2-CY26SU10.json new file mode 100644 index 0000000..94414b5 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/StaticResources/SharedResources/BaseThemes/Fluent2-CY26SU10.json @@ -0,0 +1,4488 @@ +{ + "name": "Fluent2-CY26SU10", + "dataColors": [ + "#118DFF", + "#12239E", + "#E66C37", + "#6B007B", + "#E044A7", + "#744EC2", + "#D9B300", + "#D64550", + "#197278", + "#1AAB40", + "#15C6F4", + "#4092FF", + "#FFA058", + "#BE5DC9", + "#F472D0", + "#B5A1FF", + "#C4A200", + "#FF8080", + "#00DBBC", + "#5BD667", + "#0091D5", + "#4668C5", + "#FF6300", + "#99008A", + "#EC008C", + "#533285", + "#99700A", + "#FF4141", + "#1F9A85", + "#25891C", + "#0057A2", + "#002050" + ], + "foreground": "#242424", + "foregroundNeutralSecondary": "#616161", + "foregroundNeutralTertiary": "#BDBDBD", + "foregroundNeutralLight": "#707070", + "background": "#FFFFFF", + "backgroundLight": "#C8C6C4", + "backgroundNeutral": "#C7C7C7", + "backgroundDark": "#E6E6E6", + "tableAccent": "#118DFF", + "shapeStroke": "#D1D1D1", + "disabledText": "#B3B3B3", + "good": "#9FD89F", + "neutral": "#FDCFB4", + "bad": "#EEACB2", + "maximum": "#D4EDFC", + "center": "#FAFAFA", + "minimum": "#FCE4D6", + "null": "#FF7F48", + "hyperlink": "#0078d4", + "visitedHyperlink": "#0078d4", + "textClasses": { + "callout": { + "fontSize": 21, + "fontFace": "Segoe UI Semibold", + "color": "#242424" + }, + "title": { + "fontSize": 12, + "fontFace": "DIN", + "color": "#242424" + }, + "header": { + "fontSize": 12, + "fontFace": "Segoe UI Semibold", + "color": "#242424" + }, + "label": { + "fontSize": 10.5, + "fontFace": "Segoe UI", + "color": "#242424" + } + }, + "visualStyles": { + "*": { + "*": { + "*": [ + { + "wordWrap": true + } + ], + "line": [ + { + "transparency": 0 + } + ], + "outline": [ + { + "transparency": 0 + } + ], + "plotArea": [ + { + "transparency": 0 + } + ], + "categoryAxis": [ + { + "showAxisTitle": false, + "maxMarginFactor": 50, + "gridlineStyle": "solid", + "gridlineColor": { + "solid": { + "color": "#F0F0F0" + } + }, + "concatenateLabels": false, + "titleFontSize": 10.5, + "fontSize": 10.5, + "labelColor": { + "solid": { + "color": "foregroundNeutralSecondary" + } + } + } + ], + "valueAxis": [ + { + "showAxisTitle": false, + "maxMarginFactor": 50, + "gridlineStyle": "solid", + "gridlineColor": { + "solid": { + "color": "#F0F0F0" + } + }, + "titleFontSize": 10.5, + "fontSize": 10.5, + "labelColor": { + "solid": { + "color": "foregroundNeutralSecondary" + } + } + } + ], + "y2Axis": [ + { + "show": true + } + ], + "title": [ + { + "show": true, + "fontSize": 15, + "titleWrap": true + } + ], + "subTitle": [ + { + "show": true, + "fontSize": 12 + } + ], + "padding": [ + { + "left": 16, + "top": 14, + "right": 16, + "bottom": 16 + } + ], + "lineStyles": [ + { + "strokeWidth": 3 + } + ], + "wordWrap": [ + { + "show": true + } + ], + "background": [ + { + "show": true, + "transparency": 0 + } + ], + "border": [ + { + "show": true, + "width": 1, + "radius": 8, + "color": { + "solid": { + "color": "#E6E6E6" + } + } + } + ], + "dropShadow": [ + { + "show": false, + "preset": "Custom", + "angle": 90, + "shadowDistance": 1, + "shadowBlur": 3, + "shadowSpread": 0, + "transparency": 86 + } + ], + "spacing": [ + { + "spaceAboveSubtitle": 2, + "spaceAboveDivider": 16, + "spaceAbovePlotArea": 16, + "verticalSpacing": 2, + "customizeSpacing": true, + "spaceBelowSubTitle": 16 + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "outspacePane": [ + { + "backgroundColor": { + "solid": { + "color": "background" + } + }, + "transparency": 0, + "border": true, + "borderColor": { + "solid": { + "color": "disabledText" + } + } + } + ], + "filterCard": [ + { + "$id": "Applied", + "transparency": 0, + "foregroundColor": { + "solid": { + "color": "foreground" + } + }, + "border": true + }, + { + "$id": "Available", + "transparency": 0, + "foregroundColor": { + "solid": { + "color": "foreground" + } + }, + "border": true + } + ] + } + }, + "scatterChart": { + "*": { + "bubbles": [ + { + "bubbleSize": -10, + "preventOverflow": true, + "markerRangeType": "auto" + } + ], + "general": [ + { + "responsive": true + } + ], + "fillPoint": [ + { + "show": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false, + "showGradientLegend": true + } + ], + "categoryAxis": [ + { + "showAxisTitle": true + } + ], + "valueAxis": [ + { + "showAxisTitle": true + } + ] + } + }, + "lineChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "categoryAxis": [ + { + "gridlineShow": false + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "forecast": [ + { + "matchSeriesInterpolation": true + } + ], + "lineStyles": [ + { + "lineChartType": "smooth" + } + ], + "seriesLabels": [ + { + "enableBackground": true, + "leaderLines": true, + "seriesMatchColor": true + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + }, + "Straight line": { + "lineStyles": [ + { + "lineChartType": "linear" + } + ] + }, + "Data labels - straight line": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "lineChartType": "linear", + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + } + }, + "map": { + "*": { + "bubbles": [ + { + "bubbleSize": -10, + "markerRangeType": "auto" + } + ] + } + }, + "azureMap": { + "*": { + "bubbleLayer": [ + { + "bubbleRadius": 8, + "minBubbleRadius": 8, + "maxRadius": 28, + "clusteredBubbleRadius": 16 + } + ], + "barChart": [ + { + "barHeight": 3, + "thickness": 3 + } + ], + "mapControls": [ + { + "showSelectionControl": true + } + ], + "commonDataOptions": [ + { + "unselectedTransparency": 40 + } + ] + } + }, + "pieChart": { + "*": { + "legend": [ + { + "show": true, + "position": "RightCenter" + } + ], + "labels": [ + { + "labelStyle": "Data value, percent of total" + } + ], + "dataPoint": [ + { + "borderShow": true, + "borderColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 0, + "Percent": 0 + } + } + } + } + } + } + ] + } + }, + "donutChart": { + "*": { + "legend": [ + { + "show": true, + "position": "RightCenter" + } + ], + "labels": [ + { + "labelStyle": "Data value, percent of total" + } + ], + "slices": [ + { + "innerRadiusRatio": 80 + } + ], + "centerValue": [ + { + "show": true + } + ], + "dataPoint": [ + { + "borderShow": true, + "borderColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 0, + "Percent": 0 + } + } + } + } + } + } + ] + } + }, + "tableEx": { + "*": { + "stylePreset": [ + { + "name": "None" + } + ], + "grid": [ + { + "rowPadding": 4, + "gridHorizontal": false, + "outlineColor": { + "solid": { + "color": "foreground" + } + } + } + ], + "title": [ + { + "show": false + } + ], + "subTitle": [ + { + "show": true + } + ] + }, + "None": { + "columnHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "#FFFFFF00" + } + }, + "backColorPrimary": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ] + }, + "None sparse": { + "columnHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "grid": [ + { + "rowPadding": 0 + } + ], + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "#FFFFFF00" + } + }, + "backColorPrimary": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ] + }, + "None condensed": { + "columnHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "grid": [ + { + "rowPadding": 8 + } + ], + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "#FFFFFF00" + } + }, + "backColorPrimary": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ] + }, + "Minimal": { + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "background" + } + } + } + ], + "grid": [ + { + "gridHorizontal": true, + "gridHorizontalColor": { + "solid": { + "color": "#E0E0E0" + } + }, + "gridHorizontalWeight": 1 + } + ] + }, + "Sparse": { + "grid": [ + { + "rowPadding": 8 + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ] + }, + "Condensed": { + "grid": [ + { + "rowPadding": 0, + "gridHorizontal": true, + "gridHorizontalColor": { + "solid": { + "color": "backgroundDark" + } + }, + "gridHorizontalWeight": 1 + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ] + } + }, + "pivotTable": { + "*": { + "stylePreset": [ + { + "name": "None" + } + ], + "grid": [ + { + "rowPadding": 4, + "gridHorizontal": false, + "gridHorizontalColor": { + "solid": { + "color": "#F0F0F0" + } + }, + "outlineColor": { + "solid": { + "color": "foreground" + } + } + } + ], + "title": [ + { + "show": false + } + ], + "subTitle": [ + { + "show": true + } + ], + "rowHeaders": [ + { + "showExpandCollapseButtons": true, + "legacyStyleDisabled": true + } + ] + }, + "None": { + "columnHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "rowHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "#FFFFFF00" + } + }, + "backColorPrimary": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "rowTotal": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "columnTotal": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "subTotals": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ] + }, + "None sparse": { + "grid": [ + { + "rowPadding": 8 + } + ], + "columnHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "rowHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "#FFFFFF00" + } + }, + "backColorPrimary": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "rowTotal": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "columnTotal": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "subTotals": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ] + }, + "None condensed": { + "grid": [ + { + "rowPadding": 0 + } + ], + "columnHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "rowHeaders": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "#FFFFFF00" + } + }, + "backColorPrimary": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "rowTotal": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "columnTotal": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ], + "subTotals": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "#FFFFFF00" + } + } + } + ] + }, + "Minimal": { + "values": [ + { + "backColorSecondary": { + "solid": { + "color": "background" + } + } + } + ], + "grid": [ + { + "gridHorizontal": true, + "gridHorizontalColor": { + "solid": { + "color": "#E0E0E0" + } + }, + "gridHorizontalWeight": 1 + } + ] + }, + "Sparse": { + "values": [ + { + "bandedRowHeaders": false, + "backColorPrimary": { + "solid": { + "color": "background" + } + }, + "backColorSecondary": { + "solid": { + "color": "background" + } + } + } + ], + "grid": [ + { + "rowPadding": 8 + } + ], + "rowHeaders": [ + { + "fontColor": { + "solid": { + "color": "background" + } + }, + "backColor": { + "solid": { + "color": "foreground" + } + }, + "fontColorPrimary": { + "solid": { + "color": "background" + } + }, + "fontColorSecondary": { + "solid": { + "color": "background" + } + }, + "backColorPrimary": { + "solid": { + "color": "foreground" + } + }, + "backColorSecondary": { + "solid": { + "color": "foreground" + } + } + } + ], + "columnTotal": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ], + "rowTotal": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ], + "subTotals": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "background" + } + } + } + ] + }, + "Condensed": { + "values": [ + { + "bandedRowHeaders": false + } + ], + "grid": [ + { + "rowPadding": 0, + "gridHorizontal": true, + "gridHorizontalColor": { + "solid": { + "color": "backgroundDark" + } + }, + "gridHorizontalWeight": 1 + } + ], + "rowHeaders": [ + { + "fontColor": { + "solid": { + "color": "background" + } + }, + "backColor": { + "solid": { + "color": "foreground" + } + }, + "fontColorPrimary": { + "solid": { + "color": "background" + } + }, + "fontColorSecondary": { + "solid": { + "color": "background" + } + }, + "backColorPrimary": { + "solid": { + "color": "foreground" + } + }, + "backColorSecondary": { + "solid": { + "color": "foreground" + } + } + } + ], + "columnTotal": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ], + "rowTotal": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ], + "total": [ + { + "fontColor": { + "solid": { + "color": "background" + } + } + } + ], + "subTotals": [ + { + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "backColor": { + "solid": { + "color": "background" + } + } + } + ] + } + }, + "multiRowCard": { + "*": { + "card": [ + { + "outlineWeight": 2, + "barShow": true, + "barWeight": 2 + } + ] + } + }, + "kpi": { + "*": { + "trendline": [ + { + "transparency": 20 + } + ] + } + }, + "cardVisual": { + "*": { + "layout": [ + { + "$id": "default", + "cellPadding": 0, + "maxTiles": 3, + "paddingIndividual": false, + "paddingUniform": 0, + "backgroundShow": false + } + ], + "overflow": [ + { + "type": 1 + } + ], + "image": [ + { + "$id": "default", + "position": "Left", + "imageAreaSize": 20, + "padding": 12, + "rectangleRoundedCurve": 4, + "fit": "Normal", + "fixedSize": false + } + ], + "referenceLabel": [ + { + "$id": "default", + "paddingUniform": 0, + "rectangleRoundedCurveCustomStyle": true, + "rectangleRoundedCurveLeftBottom": 4, + "rectangleRoundedCurveRightBottom": 4, + "backgroundShow": false, + "backgroundColor": { + "solid": { + "color": "backgroundNeutral" + } + } + } + ], + "referenceLabelDetail": [ + { + "$id": "default", + "detailFontFamily": "'Segoe UI', wf_segoe-ui_normal, helvetica, arial, sans-serif", + "detailFontSize": 10.5, + "detailFontColor": { + "solid": { + "color": "foregroundNeutralSecondary" + } + }, + "enableBackground": true, + "detailBackgroundColor": { + "solid": { + "color": "#EBEBEB" + } + } + } + ], + "referenceLabelTitle": [ + { + "$id": "default", + "titleFontSize": 10.5, + "titleFontColor": { + "solid": { + "color": "foregroundNeutralSecondary" + } + } + } + ], + "referenceLabelValue": [ + { + "$id": "default", + "valueFontFamily": "'Segoe UI Semibold', wf_segoe-ui_semibold, helvetica, arial, sans-serif", + "valueFontSize": 10.5, + "valueFontColor": { + "solid": { + "color": "foreground" + } + } + } + ], + "value": [ + { + "$id": "default", + "textClass": "callout" + } + ], + "label": [ + { + "$id": "default", + "position": "aboveValue", + "fontSize": 12, + "textClass": "label", + "fontColor": { + "solid": { + "color": "foregroundNeutralSecondary" + } + } + } + ], + "spacing": [ + { + "$id": "default", + "verticalSpacing": 2 + } + ], + "outline": [ + { + "$id": "default", + "show": true, + "lineColor": { + "solid": { + "color": "#E6E6E6" + } + }, + "transparency": 0 + } + ], + "divider": [ + { + "$id": "default", + "dividerColor": { + "solid": { + "color": "#E0E0E0" + } + } + } + ], + "shapeCustomRectangle": [ + { + "$id": "default", + "tileShape": "rectangleRoundedByPixel", + "rectangleRoundedCurve": 8, + "rectangleRoundedCurveCustomStyle": false + } + ], + "fillCustom": [ + { + "$id": "default", + "transparency": 0 + } + ], + "shadowCustom": [ + { + "$id": "default", + "show": false, + "shadowPositionPreset": "custom", + "angle": 90, + "shadowDistance": 1, + "shadowBlur": 3, + "shadowSpread": 0, + "transparency": 86 + } + ], + "borderCustom": [ + { + "$id": "default", + "show": true, + "borderWeight": 1, + "borderColor": "#E6E6E6" + } + ], + "smallMultiplesOuterShape": [ + { + "$id": "default", + "rectangleRoundedCurveCustomStyle": false, + "rectangleRoundedCurve": 4 + } + ], + "smallMultiplesHeader": [ + { + "$id": "default", + "backgroundShow": false, + "fontSize": 12, + "paddingIndividual": true, + "topMargin": 0, + "bottomMargin": 0, + "leftMargin": 12, + "rightMargin": 0 + } + ], + "smallMultiplesLayout": [ + { + "style": "Cards", + "cellPadding": 16, + "headerPosition": "Left", + "titleOrientation": 0 + } + ], + "smallMultiplesGrid": [ + {} + ], + "smallMultiplesBorder": [ + { + "$id": "default", + "gridlineShow": false + } + ], + "smallMultiplesOverFlow": [ + { + "overFlowStyle": 1 + } + ], + "padding": [ + { + "$id": "default", + "paddingUniform": 16, + "paddingIndividual": true, + "topMargin": 14, + "rightMargin": 16, + "bottomMargin": 16, + "leftMargin": 16 + }, + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "title": [ + { + "show": false + } + ], + "subTitle": [ + { + "show": true + } + ], + "background": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "border": [ + { + "show": false + } + ] + } + }, + "advancedSlicerVisual": { + "*": { + "visualHeader": [ + { + "show": false + } + ], + "title": [ + { + "show": true, + "fontSize": 12 + } + ], + "subTitle": [ + { + "show": false + } + ], + "spacing": [ + { + "spaceBelowTitle": 10, + "customizeSpacing": true, + "verticalSpacing": 2 + }, + { + "$id": "default", + "verticalSpacing": 0 + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "bottom": 0, + "right": 0 + }, + { + "$id": "default", + "paddingSelection": "Custom", + "leftMargin": 12, + "topMargin": 6, + "bottomMargin": 8, + "rightMargin": 12 + } + ], + "border": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "background": [ + { + "show": false + } + ], + "fillCustom": [ + { + "$id": "default", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 0, + "Percent": 0 + } + } + } + } + } + }, + { + "$id": "interaction:hover", + "fillColor": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "interaction:press", + "fillColor": { + "solid": { + "color": "#E0E0E0" + } + } + }, + { + "$id": "selection:selected", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + } + } + ], + "outline": [ + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + } + }, + { + "$id": "interaction:hover", + "lineColor": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "interaction:press", + "lineColor": { + "solid": { + "color": "disabledText" + } + } + }, + { + "$id": "selection:selected", + "show": false + } + ], + "shadowCustom": [ + { + "$id": "default", + "show": false, + "shadowPositionPreset": "custom" + } + ], + "value": [ + { + "$id": "default", + "fontSize": 10.5, + "horizontalAlignment": "center", + "fontColor": { + "solid": { + "color": "foreground" + } + } + } + ], + "label": [ + { + "$id": "default", + "fontSize": 8, + "fontColor": { + "solid": { + "color": "foregroundNeutralSecondary" + } + } + }, + { + "$id": "selection:selected", + "fontColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 0, + "Percent": 0 + } + } + } + } + } + } + ], + "layout": [ + { + "cellPadding": 8, + "maxTiles": 3 + } + ], + "shapeCustomRectangle": [ + { + "$id": "default", + "tileShape": "rectangleRoundedByPixel", + "rectangleRoundedCurve": 4 + } + ], + "selectionIcon": [ + { + "$id": "default", + "size": 12 + } + ] + }, + "Tab": { + "title": [ + { + "show": false + } + ], + "fillCustom": [ + { + "$id": "default", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 0, + "Percent": 0 + } + } + } + } + } + }, + { + "$id": "interaction:hover", + "fillColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "transparency": 0 + }, + { + "$id": "interaction:press", + "fillColor": { + "solid": { + "color": "#E0E0E0" + } + } + }, + { + "$id": "selection:selected", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 0, + "Percent": 0 + } + } + } + } + }, + "show": false + }, + { + "$id": "interaction:rest", + "show": false, + "transparency": 0 + }, + { + "$id": "selection:selected-interaction:rest", + "transparency": 100 + } + ], + "outline": [ + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + }, + "show": false + }, + { + "$id": "interaction:rest", + "show": false + } + ], + "value": [ + { + "$id": "default", + "fontSize": 10.5, + "horizontalAlignment": "center" + }, + { + "$id": "selection:selected", + "fontColor": { + "solid": { + "color": "foreground" + } + } + } + ], + "label": [ + { + "$id": "selection:selected", + "fontColor": { + "solid": { + "color": "foreground" + } + } + } + ], + "layout": [ + { + "orientation": 2, + "maxTiles": 6, + "autoGrid": true + } + ], + "overFlow": [ + { + "overFlowDirection": 1, + "overFlowStyle": 0 + } + ], + "accentBar": [ + { + "$id": "default", + "show": true, + "position": "Bottom", + "width": 2 + }, + { + "$id": "interaction:rest", + "show": false + }, + { + "$id": "interaction:hover", + "color": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "interaction:press", + "color": { + "solid": { + "color": "disabledText" + } + } + }, + { + "$id": "selection:selected", + "color": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + } + }, + { + "$id": "selection:selected-interaction:rest", + "show": true + } + ] + } + }, + "listSlicer": { + "*": { + "layout": [ + { + "showFixedSize": true + } + ] + } + }, + "slicer": { + "*": { + "general": [ + { + "responsive": true, + "outlineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "dropdown": [ + { + "borderColor": { + "solid": { + "color": "backgroundLight" + } + }, + "accentBarShow": true, + "borderRadius": 4 + } + ], + "date": [ + { + "hideDatePickerButton": false + } + ], + "slider": [ + { + "color": { + "solid": { + "color": "foreground" + } + } + } + ], + "items": [ + { + "padding": 8, + "accessibilityContrastProperties": true + } + ], + "title": [ + { + "show": false + } + ], + "background": [ + { + "show": false + } + ], + "border": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ] + } + }, + "textSlicer": { + "*": { + "inputText": [ + { + "backShow": true, + "backColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": 0 + } + } + } + } + }, + "backTransparency": 85, + "pillCornerRadius": 4, + "dismissSize": 10, + "dismissColor": { + "solid": { + "color": "foreground" + } + }, + "paddingTop": 2, + "paddingBottom": 4, + "paddingLeft": 6, + "paddingRight": 6, + "borderShow": false, + "borderColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": 0 + } + } + } + } + }, + "borderWidth": 1 + } + ], + "filterOperator": [ + { + "backShow": false + } + ] + } + }, + "waterfallChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "categoryAxis": [ + { + "innerPadding": 50 + } + ] + } + }, + "columnChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false, + "showGradientLegend": true + } + ], + "categoryAxis": [ + { + "gridlineShow": false, + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "layout": [ + { + "stackedGapSize": 2 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ] + } + }, + "clusteredColumnChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false, + "showGradientLegend": true + } + ], + "categoryAxis": [ + { + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "layout": [ + { + "clusteredGapSize": 10 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ] + } + }, + "hundredPercentStackedColumnChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false, + "showGradientLegend": true + } + ], + "categoryAxis": [ + { + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "layout": [ + { + "stackedGapSize": 2 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ] + } + }, + "barChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false, + "showGradientLegend": true + } + ], + "categoryAxis": [ + { + "autoExpand": true, + "gridlineShow": false, + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 4, + "rowCount": 1, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "layout": [ + { + "stackedGapSize": 2 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ] + } + }, + "clusteredBarChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false, + "showGradientLegend": true + } + ], + "categoryAxis": [ + { + "autoExpand": true, + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 4, + "rowCount": 1, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "layout": [ + { + "clusteredGapSize": 10 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ] + } + }, + "hundredPercentStackedBarChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false, + "showGradientLegend": true + } + ], + "categoryAxis": [ + { + "autoExpand": true, + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 4, + "rowCount": 1, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "layout": [ + { + "stackedGapSize": 2 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ] + } + }, + "areaChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "categoryAxis": [ + { + "gridlineShow": false + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "lineStyles": [ + { + "lineChartType": "smooth" + } + ], + "seriesLabels": [ + { + "enableBackground": true, + "leaderLines": true, + "seriesMatchColor": true + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + }, + "Straight line": { + "lineStyles": [ + { + "lineChartType": "linear" + } + ] + }, + "Data labels - straight line": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "lineChartType": "linear", + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + } + }, + "stackedAreaChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "lineStyles": [ + { + "lineChartType": "smooth" + } + ], + "seriesLabels": [ + { + "enableBackground": true, + "leaderLines": true, + "seriesMatchColor": true + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + }, + "Straight line": { + "lineStyles": [ + { + "lineChartType": "linear" + } + ] + }, + "Data labels - straight line": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "lineChartType": "linear", + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + } + }, + "lineClusteredColumnComboChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "categoryAxis": [ + { + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "lineStyles": [ + { + "lineChartType": "smooth" + } + ], + "seriesLabels": [ + { + "backgroundTransparency": 0.9, + "enableBackground": true, + "leaderLines": true, + "seriesMatchColor": true + } + ], + "layout": [ + { + "clusteredGapSize": 10 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "secShow": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + }, + "Straight line": { + "lineStyles": [ + { + "lineChartType": "linear" + } + ] + }, + "Data labels - straight line": { + "valueAxis": [ + { + "show": false, + "secShow": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "lineChartType": "linear", + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + } + }, + "lineStackedColumnComboChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "categoryAxis": [ + { + "innerPadding": 50 + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "lineStyles": [ + { + "lineChartType": "smooth" + } + ], + "seriesLabels": [ + { + "backgroundTransparency": 0.9, + "enableBackground": true, + "leaderLines": true, + "seriesMatchColor": true + } + ], + "layout": [ + { + "stackedGapSize": 2 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "secShow": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + }, + "Straight line": { + "lineStyles": [ + { + "lineChartType": "linear" + } + ] + }, + "Data labels - straight line": { + "valueAxis": [ + { + "show": false, + "secShow": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "lineChartType": "linear", + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + } + }, + "ribbonChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "categoryAxis": [ + { + "gridlineShow": false + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "valueAxis": [ + { + "show": true + } + ], + "layout": [ + { + "stackedGapSize": 2 + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ] + } + }, + "hundredPercentStackedAreaChart": { + "*": { + "general": [ + { + "responsive": true + } + ], + "legend": [ + { + "show": true, + "position": "Bottom", + "showTitle": false + } + ], + "smallMultiplesLayout": [ + { + "columnCount": 1, + "rowCount": 4, + "backgroundTransparency": 0, + "gridLineType": "inner", + "gridLineColor": { + "solid": { + "color": "shapeStroke" + } + } + } + ], + "subheader": [ + { + "fontSize": 10.5 + } + ], + "lineStyles": [ + { + "lineChartType": "smooth" + } + ], + "seriesLabels": [ + { + "enableBackground": true, + "leaderLines": true, + "seriesMatchColor": true + } + ] + }, + "Data labels": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + }, + "Straight line": { + "lineStyles": [ + { + "lineChartType": "linear" + } + ] + }, + "Data labels - straight line": { + "valueAxis": [ + { + "show": false, + "gridlineShow": false + } + ], + "labels": [ + { + "show": true, + "labelOverflow": true + } + ], + "lineStyles": [ + { + "lineChartType": "linear", + "showMarker": true, + "showMarkerByDefault": true, + "markerShape": "circle", + "markerSize": 6 + } + ] + } + }, + "group": { + "*": { + "background": [ + { + "show": false + } + ] + } + }, + "basicShape": { + "*": { + "background": [ + { + "show": false + } + ], + "general": [ + { + "keepLayerOrder": true + } + ], + "visualHeader": [ + { + "show": false + } + ] + } + }, + "shape": { + "*": { + "background": [ + { + "show": false + } + ], + "general": [ + { + "keepLayerOrder": true + } + ], + "visualHeader": [ + { + "show": false + } + ], + "title": [ + { + "show": false + } + ], + "subTitle": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "border": [ + { + "show": false + } + ] + } + }, + "image": { + "*": { + "background": [ + { + "show": false + } + ], + "general": [ + { + "keepLayerOrder": true + } + ], + "visualHeader": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "title": [ + { + "show": false + } + ], + "subTitle": [ + { + "show": false + } + ], + "border": [ + { + "show": false + } + ] + } + }, + "actionButton": { + "*": { + "background": [ + { + "show": false + } + ], + "title": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "border": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "visualHeader": [ + { + "show": false + } + ], + "text": [ + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "background" + } + }, + "alignment": "center", + "leftMargin": 12, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "selected", + "fontSize": 10.5, + "alignment": "center" + }, + { + "$id": "disabled", + "fontColor": { + "solid": { + "color": "foregroundNeutralTertiary" + } + } + } + ], + "shape": [ + { + "$id": "default", + "roundEdge": 4 + } + ], + "outline": [ + { + "show": false + }, + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + }, + "weight": 1 + }, + { + "$id": "hover", + "lineColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "weight": 1 + }, + { + "$id": "selected", + "lineColor": { + "solid": { + "color": "disabledText" + } + }, + "weight": 1 + } + ], + "fill": [ + { + "show": true + }, + { + "$id": "default", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.2 + } + } + } + } + }, + "transparency": 0 + }, + { + "$id": "hover", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + }, + "transparency": 0 + }, + { + "$id": "selected", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.6 + } + } + } + } + }, + "transparency": 0 + }, + { + "$id": "disabled", + "fillColor": { + "solid": { + "color": "#F0F0F0" + } + }, + "transparency": 0 + } + ], + "icon": [ + { + "$id": "default", + "iconSize": 20, + "horizontalAlignment": "center", + "verticalAlignment": "middle", + "lineWeight": 2, + "lineColor": { + "solid": { + "color": "background" + } + } + }, + { + "$id": "disabled", + "lineColor": { + "solid": { + "color": "foregroundNeutralTertiary" + } + } + } + ] + }, + "Outline": { + "text": [ + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "alignment": "center", + "leftMargin": 12, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "selected", + "fontSize": 10.5, + "alignment": "center" + }, + { + "$id": "disabled", + "fontColor": { + "solid": { + "color": "foregroundNeutralTertiary" + } + } + } + ], + "shape": [ + { + "$id": "default", + "roundEdge": 4 + } + ], + "outline": [ + { + "show": true + }, + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + }, + "weight": 1 + }, + { + "$id": "hover", + "lineColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "weight": 1 + }, + { + "$id": "selected", + "lineColor": { + "solid": { + "color": "disabledText" + } + }, + "weight": 1 + }, + { + "$id": "disabled", + "lineColor": { + "solid": { + "color": "#E0E0E0" + } + }, + "weight": 1 + } + ], + "fill": [ + { + "show": true + }, + { + "$id": "default", + "fillColor": { + "solid": { + "color": "background" + } + }, + "transparency": 0 + }, + { + "$id": "hover", + "fillColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "transparency": 0 + }, + { + "$id": "selected", + "fillColor": { + "solid": { + "color": "#E0E0E0" + } + }, + "transparency": 0 + }, + { + "$id": "disabled", + "fillColor": { + "solid": { + "color": "#F0F0F0" + } + }, + "transparency": 0 + } + ], + "icon": [ + { + "$id": "default", + "iconSize": 20, + "horizontalAlignment": "center", + "verticalAlignment": "middle", + "lineWeight": 2, + "lineColor": { + "solid": { + "color": "foreground" + } + } + }, + { + "$id": "disabled", + "lineColor": { + "solid": { + "color": "foregroundNeutralTertiary" + } + } + } + ] + }, + "Transparent": { + "text": [ + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "alignment": "center", + "leftMargin": 12, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "hover", + "fontColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.2 + } + } + } + } + } + }, + { + "$id": "selected", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + }, + "alignment": "center" + } + ], + "shape": [ + { + "$id": "default", + "roundEdge": 4 + } + ], + "outline": [ + { + "show": false + }, + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + }, + "weight": 1 + }, + { + "$id": "hover", + "lineColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "weight": 1 + }, + { + "$id": "selected", + "lineColor": { + "solid": { + "color": "disabledText" + } + }, + "weight": 1 + } + ], + "fill": [ + { + "show": false + } + ], + "icon": [ + { + "$id": "default", + "iconSize": 20, + "horizontalAlignment": "center", + "verticalAlignment": "middle", + "lineWeight": 2, + "lineColor": { + "solid": { + "color": "foreground" + } + } + }, + { + "$id": "hover", + "lineColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.2 + } + } + } + } + } + }, + { + "$id": "selected", + "lineColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + } + } + ] + }, + "Icon & Text": { + "text": [ + { + "horizontalAlignment": "left" + }, + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "background" + } + }, + "horizontalAlignment": "left", + "leftMargin": 38, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "selected", + "fontSize": 10.5, + "horizontalAlignment": "left" + }, + { + "$id": "disabled", + "fontColor": { + "solid": { + "color": "foregroundNeutralTertiary" + } + } + } + ], + "shape": [ + { + "$id": "default", + "roundEdge": 4 + } + ], + "outline": [ + { + "show": false + }, + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + }, + "weight": 1 + }, + { + "$id": "hover", + "lineColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "weight": 1 + }, + { + "$id": "selected", + "lineColor": { + "solid": { + "color": "disabledText" + } + }, + "weight": 1 + } + ], + "fill": [ + { + "show": true + }, + { + "$id": "default", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.2 + } + } + } + } + }, + "transparency": 0 + }, + { + "$id": "hover", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + }, + "transparency": 0 + }, + { + "$id": "selected", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.6 + } + } + } + } + }, + "transparency": 0 + }, + { + "$id": "disabled", + "fillColor": { + "solid": { + "color": "#F0F0F0" + } + }, + "transparency": 0 + } + ], + "icon": [ + { + "show": true + }, + { + "$id": "default", + "iconSize": 20, + "horizontalAlignment": "left", + "leftMargin": 12, + "topMargin": 6, + "bottomMargin": 8, + "lineWeight": 2, + "lineColor": { + "solid": { + "color": "background" + } + } + }, + { + "$id": "disabled", + "lineColor": { + "solid": { + "color": "foregroundNeutralTertiary" + } + } + } + ] + } + }, + "pageNavigator": { + "*": { + "background": [ + { + "show": false + } + ], + "visualHeader": [ + { + "show": false + } + ], + "title": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "border": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "text": [ + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "leftMargin": 12, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "selected", + "fontSize": 10.5, + "leftMargin": 12, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + } + ], + "shape": [ + { + "$id": "default", + "tileShape": "rectangle", + "roundEdge": 4 + } + ], + "outline": [ + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + } + }, + { + "$id": "hover", + "lineColor": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "press", + "lineColor": { + "solid": { + "color": "disabledText" + } + } + }, + { + "$id": "selected", + "transparency": 100 + } + ], + "fill": [ + { + "$id": "hover", + "fillColor": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "press", + "fillColor": { + "solid": { + "color": "#E0E0E0" + } + } + }, + { + "$id": "selected", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + } + } + ], + "layout": [ + { + "orientation": 2, + "cellPadding": 4 + } + ] + }, + "Tab": { + "background": [ + { + "show": false + } + ], + "visualHeader": [ + { + "show": false + } + ], + "title": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "border": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "text": [ + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "leftMargin": 6, + "rightMargin": 6, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "hover", + "fontColor": { + "solid": { + "color": "foreground" + } + } + }, + { + "$id": "press", + "fontColor": { + "solid": { + "color": "foreground" + } + } + }, + { + "$id": "selected", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "leftMargin": 6, + "rightMargin": 6, + "topMargin": 6, + "bottomMargin": 8 + } + ], + "shape": [ + { + "$id": "default", + "tileShape": "rectangle", + "roundEdge": 4 + } + ], + "outline": [ + { + "show": false + } + ], + "fill": [ + { + "show": true + }, + { + "$id": "default", + "transparency": 100 + }, + { + "$id": "hover", + "fillColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "transparency": 0 + }, + { + "$id": "press", + "fillColor": { + "solid": { + "color": "#E0E0E0" + } + }, + "transparency": 0 + }, + { + "$id": "selected", + "transparency": 100 + } + ], + "accentBar": [ + { + "show": false + }, + { + "$id": "hover", + "show": true, + "position": "Bottom", + "weight": 2, + "color": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "press", + "show": true, + "position": "Bottom", + "weight": 2, + "color": { + "solid": { + "color": "disabledText" + } + } + }, + { + "$id": "selected", + "show": true, + "position": "Bottom", + "weight": 2, + "color": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.2 + } + } + } + } + } + } + ], + "layout": [ + { + "orientation": 2, + "cellPadding": 4 + } + ] + } + }, + "bookmarkNavigator": { + "*": { + "background": [ + { + "show": false + } + ], + "visualHeader": [ + { + "show": false + } + ], + "title": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "border": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "text": [ + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "leftMargin": 12, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "selected", + "fontSize": 10.5, + "leftMargin": 12, + "rightMargin": 12, + "topMargin": 6, + "bottomMargin": 8 + } + ], + "shape": [ + { + "$id": "default", + "tileShape": "rectangle", + "roundEdge": 4 + } + ], + "outline": [ + { + "$id": "default", + "lineColor": { + "solid": { + "color": "shapeStroke" + } + }, + "weight": 1 + }, + { + "$id": "hover", + "lineColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "weight": 1 + }, + { + "$id": "press", + "lineColor": { + "solid": { + "color": "disabledText" + } + }, + "weight": 1 + }, + { + "$id": "selected", + "transparency": 100 + } + ], + "fill": [ + { + "$id": "hover", + "fillColor": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "press", + "fillColor": { + "solid": { + "color": "#E0E0E0" + } + } + }, + { + "$id": "selected", + "fillColor": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.4 + } + } + } + } + } + } + ], + "layout": [ + { + "orientation": 2, + "cellPadding": 4 + } + ] + }, + "Tab": { + "background": [ + { + "show": false + } + ], + "visualHeader": [ + { + "show": false + } + ], + "title": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "border": [ + { + "show": false + } + ], + "dropShadow": [ + { + "show": false + } + ], + "text": [ + { + "$id": "default", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "leftMargin": 6, + "rightMargin": 6, + "topMargin": 6, + "bottomMargin": 8 + }, + { + "$id": "hover", + "fontColor": { + "solid": { + "color": "foreground" + } + } + }, + { + "$id": "press", + "fontColor": { + "solid": { + "color": "foreground" + } + } + }, + { + "$id": "selected", + "fontSize": 10.5, + "fontColor": { + "solid": { + "color": "foreground" + } + }, + "leftMargin": 6, + "rightMargin": 6, + "topMargin": 6, + "bottomMargin": 8 + } + ], + "shape": [ + { + "$id": "default", + "tileShape": "rectangle", + "roundEdge": 4 + } + ], + "outline": [ + { + "show": false + } + ], + "fill": [ + { + "show": true + }, + { + "$id": "default", + "transparency": 100 + }, + { + "$id": "hover", + "fillColor": { + "solid": { + "color": "backgroundNeutral" + } + }, + "transparency": 0 + }, + { + "$id": "press", + "fillColor": { + "solid": { + "color": "#E0E0E0" + } + }, + "transparency": 0 + }, + { + "$id": "selected", + "transparency": 100 + } + ], + "accentBar": [ + { + "show": false + }, + { + "$id": "hover", + "show": true, + "position": "Bottom", + "weight": 2, + "color": { + "solid": { + "color": "backgroundNeutral" + } + } + }, + { + "$id": "press", + "show": true, + "position": "Bottom", + "weight": 2, + "color": { + "solid": { + "color": "disabledText" + } + } + }, + { + "$id": "selected", + "show": true, + "position": "Bottom", + "weight": 2, + "color": { + "solid": { + "color": { + "expr": { + "ThemeDataColor": { + "ColorId": 2, + "Percent": -0.2 + } + } + } + } + } + } + ], + "layout": [ + { + "orientation": 2, + "cellPadding": 4 + } + ] + } + }, + "textbox": { + "*": { + "background": [ + { + "show": false + } + ], + "general": [ + { + "keepLayerOrder": true + } + ], + "visualHeader": [ + { + "show": false + } + ], + "title": [ + { + "show": false + } + ], + "subTitle": [ + { + "show": false + } + ], + "padding": [ + { + "left": 0, + "top": 0, + "right": 0, + "bottom": 0 + } + ], + "border": [ + { + "show": false + } + ] + } + }, + "page": { + "*": { + "pageSize": [ + { + "pageSizeTypes": "Custom", + "pageSizeWidth": 1920, + "pageSizeHeight": 1080 + } + ], + "displayArea": [ + { + "verticalAlignment": "Middle" + } + ], + "outspace": [ + { + "color": { + "solid": { + "color": "#E6E6E6" + } + } + } + ], + "background": [ + { + "color": { + "solid": { + "color": "#FAFAFA" + } + }, + "transparency": 14 + } + ] + } + } + } +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/definition.pbir b/tests/fixtures/SalesSample/SalesSample.Report/definition.pbir new file mode 100644 index 0000000..67ffc4e --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/definition.pbir @@ -0,0 +1,9 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definitionProperties/2.0.0/schema.json", + "version": "4.0", + "datasetReference": { + "byPath": { + "path": "../SalesSample.SemanticModel" + } + } +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/page.json b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/page.json new file mode 100644 index 0000000..2dcd95a --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/page.json @@ -0,0 +1,8 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/page/2.1.0/schema.json", + "name": "e087e4b90fb242dfb36a", + "displayName": "Overview", + "displayOption": "FitToPage", + "height": 720, + "width": 1280 +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/visuals/1c9e5a7d3b2f4e6a8d01/visual.json b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/visuals/1c9e5a7d3b2f4e6a8d01/visual.json new file mode 100644 index 0000000..84c2cd7 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/visuals/1c9e5a7d3b2f4e6a8d01/visual.json @@ -0,0 +1,22 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/visualContainer/2.9.0/schema.json", + "name": "1c9e5a7d3b2f4e6a8d01", + "position": { "x": 40, "y": 40, "z": 1000, "width": 320, "height": 160, "tabOrder": 1000 }, + "visual": { + "visualType": "cardVisual", + "query": { + "queryState": { + "Data": { + "projections": [ + { + "field": { "Measure": { "Expression": { "SourceRef": { "Entity": "Sales" } }, "Property": "Total Sales" } }, + "queryRef": "Sales.Total Sales", + "nativeQueryRef": "Total Sales" + } + ] + } + } + }, + "drillFilterOtherVisuals": true + } +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/visuals/7b2d4f6a8c0e1a3b5d02/visual.json b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/visuals/7b2d4f6a8c0e1a3b5d02/visual.json new file mode 100644 index 0000000..9e582f3 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/e087e4b90fb242dfb36a/visuals/7b2d4f6a8c0e1a3b5d02/visual.json @@ -0,0 +1,42 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/visualContainer/2.9.0/schema.json", + "name": "7b2d4f6a8c0e1a3b5d02", + "position": { "x": 40, "y": 240, "z": 2000, "width": 760, "height": 420, "tabOrder": 2000 }, + "visual": { + "visualType": "clusteredColumnChart", + "query": { + "queryState": { + "Category": { + "projections": [ + { + "field": { "Column": { "Expression": { "SourceRef": { "Entity": "Product" } }, "Property": "Category" } }, + "queryRef": "Product.Category", + "nativeQueryRef": "Category", + "active": true + } + ] + }, + "Y": { + "projections": [ + { + "field": { "Measure": { "Expression": { "SourceRef": { "Entity": "Sales" } }, "Property": "Total Sales" } }, + "queryRef": "Sales.Total Sales", + "nativeQueryRef": "Total Sales" + } + ] + } + } + }, + "visualContainerObjects": { + "title": [ + { + "properties": { + "show": { "expr": { "Literal": { "Value": "true" } } }, + "text": { "expr": { "Literal": { "Value": "'Sales by category'" } } } + } + } + ] + }, + "drillFilterOtherVisuals": true + } +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/pages.json b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/pages.json new file mode 100644 index 0000000..97f5542 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/definition/pages/pages.json @@ -0,0 +1,7 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/pagesMetadata/1.1.0/schema.json", + "pageOrder": [ + "e087e4b90fb242dfb36a" + ], + "activePageName": "e087e4b90fb242dfb36a" +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/definition/report.json b/tests/fixtures/SalesSample/SalesSample.Report/definition/report.json new file mode 100644 index 0000000..5cae0df --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/definition/report.json @@ -0,0 +1,35 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/report/3.3.0/schema.json", + "themeCollection": { + "baseTheme": { + "name": "Fluent2-CY26SU10", + "reportVersionAtImport": { + "visual": "2.11.0", + "report": "3.4.0", + "page": "2.3.1" + }, + "type": "SharedResources" + } + }, + "resourcePackages": [ + { + "name": "SharedResources", + "type": "SharedResources", + "items": [ + { + "name": "Fluent2-CY26SU10", + "path": "BaseThemes/Fluent2-CY26SU10.json", + "type": "BaseTheme" + } + ] + } + ], + "settings": { + "useStylableVisualContainerHeader": true, + "exportDataMode": "AllowSummarized", + "defaultDrillFilterOtherVisuals": true, + "allowChangeFilterTypes": true, + "useEnhancedTooltips": true, + "useDefaultAggregateDisplayName": true + } +} diff --git a/tests/fixtures/SalesSample/SalesSample.Report/definition/version.json b/tests/fixtures/SalesSample/SalesSample.Report/definition/version.json new file mode 100644 index 0000000..6e4d155 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.Report/definition/version.json @@ -0,0 +1,4 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/report/definition/versionMetadata/1.0.0/schema.json", + "version": "2.0.0" +} diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/.platform b/tests/fixtures/SalesSample/SalesSample.SemanticModel/.platform new file mode 100644 index 0000000..0b9bc3c --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/.platform @@ -0,0 +1,11 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/gitIntegration/platformProperties/2.0.0/schema.json", + "metadata": { + "type": "SemanticModel", + "displayName": "SalesSample" + }, + "config": { + "version": "2.0", + "logicalId": "2b7e4c1a-6d3f-4e8b-9c0a-5f1d2e3b4a60" + } +} diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition.pbism b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition.pbism new file mode 100644 index 0000000..21b0111 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition.pbism @@ -0,0 +1,5 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/item/semanticModel/definitionProperties/1.0.0/schema.json", + "version": "4.2", + "settings": {} +} diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/database.tmdl b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/database.tmdl new file mode 100644 index 0000000..ffba38a --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/database.tmdl @@ -0,0 +1,3 @@ +database + compatibilityLevel: 1601 + diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/model.tmdl b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/model.tmdl new file mode 100644 index 0000000..fdd29d7 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/model.tmdl @@ -0,0 +1,15 @@ +model Model + culture: en-US + defaultPowerBIDataSourceVersion: powerBI_V3 + sourceQueryCulture: en-US + dataAccessOptions + legacyRedirects + returnErrorValuesAsNull + +annotation __PBI_TimeIntelligenceEnabled = 0 + +ref table Sales +ref table Date +ref table Product +ref table Targets + diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/relationships.tmdl b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/relationships.tmdl new file mode 100644 index 0000000..5b38f25 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/relationships.tmdl @@ -0,0 +1,9 @@ +relationship 5f0c2a4e-9b1d-4c3a-8e2f-1a6b7c8d9e01 + fromColumn: Sales.DateKey + toColumn: Date.DateKey + +relationship 8a3d1f6b-2c4e-4d5f-9a1b-3c5d7e9f1a02 + crossFilteringBehavior: bothDirections + fromColumn: Sales.ProductID + toColumn: Product.ProductID + diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Date.tmdl b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Date.tmdl new file mode 100644 index 0000000..4c897c1 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Date.tmdl @@ -0,0 +1,41 @@ +/// Calendar. Grain: one row per day. +table Date + + column DateKey + dataType: int64 + isHidden + formatString: 0 + summarizeBy: none + sourceColumn: DateKey + + column Date + dataType: dateTime + formatString: yyyy-mm-dd + summarizeBy: none + sourceColumn: Date + + column Month + dataType: string + summarizeBy: none + sourceColumn: Month + + partition Date = m + mode: import + source = + let + Source = #table( + type table [DateKey = Int64.Type, Date = Date.Type, Month = Text.Type], + { + {20260105, #date(2026, 1, 5), "2026-01"}, + {20260112, #date(2026, 1, 12), "2026-01"}, + {20260203, #date(2026, 2, 3), "2026-02"}, + {20260214, #date(2026, 2, 14), "2026-02"}, + {20260302, #date(2026, 3, 2), "2026-03"}, + {20260318, #date(2026, 3, 18), "2026-03"} + } + ) + in + Source + + annotation PBI_ResultType = Table + diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Product.tmdl b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Product.tmdl new file mode 100644 index 0000000..e5406c8 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Product.tmdl @@ -0,0 +1,36 @@ +table Product + + column ProductID + dataType: int64 + isHidden + formatString: 0 + summarizeBy: none + sourceColumn: ProductID + + column Product + dataType: string + summarizeBy: none + sourceColumn: Product + + column Category + dataType: string + summarizeBy: none + sourceColumn: Category + + partition Product = m + mode: import + source = + let + Source = #table( + type table [ProductID = Int64.Type, Product = Text.Type, Category = Text.Type], + { + {1, "Laptop", "Hardware"}, + {2, "Monitor", "Hardware"}, + {3, "Support plan", "Services"} + } + ) + in + Source + + annotation PBI_ResultType = Table + diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Sales.tmdl b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Sales.tmdl new file mode 100644 index 0000000..05e8195 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Sales.tmdl @@ -0,0 +1,75 @@ +/// Order lines. Grain: one row per order line. +table Sales + + /// Sum of the order line amount. + measure 'Total Sales' = SUM(Sales[Amount]) + formatString: #,##0.00 + displayFolder: Revenue + + measure 'total cost' = SUM(Sales[Cost]) + formatString: #,##0.00 + + /// Gross margin as a share of sales. + measure 'Margin %' = + IFERROR( + ( [Total Sales] - [total cost] ) / [Total Sales], + BLANK() + ) + formatString: 0.0% + displayFolder: Revenue + + /// Placeholder kept for a future forecast KPI. + measure Forecast = BLANK() + displayFolder: Planning + + column OrderId + dataType: int64 + formatString: 0 + summarizeBy: none + sourceColumn: OrderId + + column DateKey + dataType: int64 + isHidden + formatString: 0 + summarizeBy: none + sourceColumn: DateKey + + column ProductID + dataType: int64 + formatString: 0 + summarizeBy: none + sourceColumn: ProductID + + column Amount + dataType: decimal + formatString: #,##0.00 + summarizeBy: sum + sourceColumn: Amount + + column Cost + dataType: decimal + formatString: #,##0.00 + summarizeBy: sum + sourceColumn: Cost + + partition Sales = m + mode: import + source = + let + Source = #table( + type table [OrderId = Int64.Type, DateKey = Int64.Type, ProductID = Int64.Type, Amount = Currency.Type, Cost = Currency.Type], + { + {1, 20260105, 1, 120, 70}, + {2, 20260112, 2, 80, 50}, + {3, 20260203, 1, 150, 90}, + {4, 20260214, 3, 60, 20}, + {5, 20260302, 2, 95, 55}, + {6, 20260318, 3, 40, 15} + } + ) + in + Source + + annotation PBI_ResultType = Table + diff --git a/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Targets.tmdl b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Targets.tmdl new file mode 100644 index 0000000..6cc27de --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.SemanticModel/definition/tables/Targets.tmdl @@ -0,0 +1,31 @@ +/// Monthly sales targets typed in by Finance. Grain: one row per month. +table Targets + + column Month + dataType: string + summarizeBy: none + sourceColumn: Month + + column Target + dataType: decimal + formatString: #,##0.00 + summarizeBy: sum + sourceColumn: Target + + partition Targets = m + mode: import + source = + let + Source = #table( + type table [Month = Text.Type, Target = Currency.Type], + { + {"2026-01", 200}, + {"2026-02", 220}, + {"2026-03", 150} + } + ) + in + Source + + annotation PBI_ResultType = Table + diff --git a/tests/fixtures/SalesSample/SalesSample.pbip b/tests/fixtures/SalesSample/SalesSample.pbip new file mode 100644 index 0000000..cd71af8 --- /dev/null +++ b/tests/fixtures/SalesSample/SalesSample.pbip @@ -0,0 +1,14 @@ +{ + "$schema": "https://developer.microsoft.com/json-schemas/fabric/pbip/pbipProperties/1.0.0/schema.json", + "version": "1.0", + "artifacts": [ + { + "report": { + "path": "SalesSample.Report" + } + } + ], + "settings": { + "enableAutoRecovery": true + } +} diff --git a/tests/helpers.py b/tests/helpers.py new file mode 100644 index 0000000..2f1e7e2 --- /dev/null +++ b/tests/helpers.py @@ -0,0 +1,72 @@ +"""Shared paths and loaders for the test suite (stdlib unittest; run with `npm test`).""" + +import importlib.util +import os +import shutil +import stat +import sys +import tempfile +from pathlib import Path + +sys.dont_write_bytecode = True + +ROOT = Path(__file__).resolve().parent.parent +FIXTURE = ROOT / "tests" / "fixtures" / "SalesSample" +QUALITY_SCRIPTS = ROOT / ".agent" / "skills" / "pbi-quality-rules" / "scripts" +PBIR_SCRIPTS = ROOT / ".agent" / "skills" / "pbi-report-layer-pbir" / "scripts" + + +def load_module(path: Path, name: str): + """Import a script file as a module (scripts are not a package).""" + if str(path.parent) not in sys.path: + sys.path.insert(0, str(path.parent)) + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +def copy_fixture(tmp: Path) -> Path: + """Copy the SalesSample fixture into `tmp` and return the copy's project folder.""" + dest = tmp / "SalesSample" + shutil.copytree(FIXTURE, dest, ignore=shutil.ignore_patterns(".pbi")) + return dest + + +def write_stub(folder: Path, name: str, stdout: str, exit_code: int = 0) -> Path: + """Create an executable that records its argv to <name>.args and prints `stdout`.""" + script = folder / name + out_file = folder / f"{name}.out" + out_file.write_text(stdout, encoding="utf-8") + script.write_text( + f"#!{sys.executable}\n" + "import sys, pathlib\n" + f"pathlib.Path({str(folder / (name + '.args'))!r}).write_text('\\n'.join(sys.argv[1:]))\n" + f"sys.stdout.write(pathlib.Path({str(out_file)!r}).read_text(encoding='utf-8'))\n" + f"sys.exit({exit_code})\n", + encoding="utf-8", + ) + script.chmod(script.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) + return script + + +def stub_args(folder: Path, name: str) -> list: + return (folder / f"{name}.args").read_text(encoding="utf-8").split("\n") + + +class TempDirMixin: + def setUp(self): + super().setUp() + self._tmp = tempfile.TemporaryDirectory(prefix="adk-test-") + self.tmp = Path(self._tmp.name) + + def tearDown(self): + self._tmp.cleanup() + super().tearDown() + + +def clean_env(**overrides) -> dict: + env = {k: v for k, v in os.environ.items() if k not in ("TABULAR_EDITOR_PATH", "PBIR_AUTHOR_CLI")} + env.update(overrides) + return env diff --git a/tests/test_bpa.py b/tests/test_bpa.py new file mode 100644 index 0000000..f0458db --- /dev/null +++ b/tests/test_bpa.py @@ -0,0 +1,236 @@ +import contextlib +import io +import json +import unittest +from pathlib import Path +from unittest import mock + +from helpers import FIXTURE, QUALITY_SCRIPTS, TempDirMixin, copy_fixture, load_module, stub_args, write_stub + +bpa = load_module(QUALITY_SCRIPTS / "bpa_fallback.py", "bpa_fallback") +run_bpa = load_module(QUALITY_SCRIPTS / "run_bpa.py", "run_bpa") + +# The fixture carries exactly these violations on purpose (see tests/fixtures/SalesSample). +FIXTURE_VIOLATIONS = { + ("DAX_EMPTY_EXPRESSION", "'Sales'[Forecast]", "error"), + ("DAX_IFERROR_USAGE", "'Sales'[Margin %]", "warning"), + ("DOC_MEASURE_DESCRIPTION", "'Sales'[total cost]", "warning"), + ("NAMING_MEASURE_DISPLAYFOLDER", "'Sales'[total cost]", "warning"), + ("NAMING_MEASURE_PASCAL", "'Sales'[total cost]", "warning"), + ("DOC_TABLE_DESCRIPTION", "'Product'", "warning"), + ("MODEL_BIDIRECTIONAL", "'Sales'[ProductID] -> 'Product'[ProductID]", "warning"), + ("MODEL_ISLAND_TABLE", "'Targets'", "warning"), + ("DOC_KEY_COLUMN_HIDDEN", "'Sales'[ProductID]", "info"), +} + + +def run_main(module, args): + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + code = module.main([str(a) for a in args]) + return code, out.getvalue(), err.getvalue() + + +def triples(result): + return {(v["rule_id"], v["object"], v["severity"]) for v in result["violations"]} + + +class FallbackOnFixture(unittest.TestCase): + def test_reports_exactly_the_planted_violations(self): + code, out, _ = run_main(bpa, [FIXTURE / "SalesSample.pbip", "--format", "json"]) + result = json.loads(out) + self.assertEqual(triples(result), FIXTURE_VIOLATIONS) + self.assertEqual(result["summary"], {"error": 1, "warning": 7, "info": 1}) + self.assertEqual(result["skipped_rules"], []) + self.assertEqual(code, 1, "an error-severity violation must fail the run") + + def test_every_shipped_rule_has_a_fallback_implementation(self): + rule_ids = {r["ID"] for r in bpa.load_rules(bpa.DEFAULT_RULES)} + self.assertEqual(rule_ids - set(bpa.CHECKS), set()) + + def test_all_model_path_forms_resolve_to_the_same_model(self): + sm = FIXTURE / "SalesSample.SemanticModel" + expected = ("tmdl", sm / "definition") + for path in (FIXTURE / "SalesSample.pbip", FIXTURE, sm, sm / "definition", sm / "definition.pbism"): + with self.subTest(path=path): + kind, resolved = bpa.resolve_model(path) + self.assertEqual((kind, resolved.resolve()), (expected[0], expected[1].resolve())) + + def test_missing_path_is_a_usage_error(self): + code, _, err = run_main(bpa, ["/definitely/not/here"]) + self.assertEqual(code, 2) + self.assertIn("does not exist", err) + + +class TmdlParser(TempDirMixin, unittest.TestCase): + def write_model(self, files): + definition = self.tmp / "M.SemanticModel" / "definition" + (definition / "tables").mkdir(parents=True) + (definition / "model.tmdl").write_text("model Model\n\tculture: en-US\n", encoding="utf-8") + for name, text in files.items(): + (definition / name).write_text(text, encoding="utf-8") + return definition + + def test_multiline_fenced_and_quoted_constructs(self): + definition = self.write_model({ + "tables/Facts.tmdl": ( + "/// Facts table.\n" + "table 'Fact''s Table'\n" + "\n" + "\t/// Multi-line, empty after all.\n" + "\tmeasure Empty =\n" + "\t\t\tBLANK()\n" + "\t\tdisplayFolder: X\n" + "\n" + "\tmeasure 'Fenced' = ```\n" + "\t\t\tVAR x = 1\n" + "\t\t\tRETURN IFERROR(x, 0)\n" + "\t\t\t```\n" + "\t\tdescription: \"Has a description property\"\n" + "\t\tdisplayFolder: \"Quoted Folder\"\n" + "\n" + "\tcolumn CustomerID\n" + "\t\tdataType: int64\n" + "\t\tsourceColumn: CustomerID\n" + "\n" + "\tpartition P = m\n" + "\t\tmode: import\n" + "\t\tsource =\n" + "\t\t\t\tlet x = 1 in x\n" + "\n" + "\tisHidden\n" + ), + "tables/LocalDateTable_abc.tmdl": "table LocalDateTable_abc\n\tisHidden\n", + "relationships.tmdl": ( + "relationship r1\n" + "\tcrossFilteringBehavior: bothDirections\n" + "\tfromColumn: 'Fact''s Table'.CustomerID\n" + "\ttoColumn: 'Fact''s Table'.'CustomerID'\n" + ), + }) + model = bpa.parse_tmdl_folder(definition) + tables = {t.name: t for t in model.tables} + facts = tables["Fact's Table"] + self.assertEqual(facts.description, "Facts table.") + self.assertTrue(facts.is_hidden, "table-level isHidden after members must be read") + measures = {m.name: m for m in facts.measures} + self.assertEqual(measures["Empty"].expression, "BLANK()") + self.assertEqual(measures["Empty"].description, "Multi-line, empty after all.") + self.assertIn("IFERROR", measures["Fenced"].expression) + self.assertEqual(measures["Fenced"].description, "Has a description property") + self.assertEqual(measures["Fenced"].display_folder, "Quoted Folder") + self.assertEqual([c.name for c in facts.columns], ["CustomerID"]) + rel = model.relationships[0] + self.assertEqual((rel.from_table, rel.from_column, rel.to_table), ("Fact's Table", "CustomerID", "Fact's Table")) + + result = bpa.analyze(model, bpa.load_rules(bpa.DEFAULT_RULES)) + got = triples(result) + self.assertIn(("DAX_EMPTY_EXPRESSION", "'Fact''s Table'[Empty]", "error"), got) + self.assertIn(("DAX_IFERROR_USAGE", "'Fact''s Table'[Fenced]", "warning"), got) + self.assertIn(("PERF_AUTO_DATETIME", "'LocalDateTable_abc'", "warning"), got) + self.assertIn(("DOC_KEY_COLUMN_HIDDEN", "'Fact''s Table'[CustomerID]", "info"), got) + island = {o for r, o, _ in got if r == "MODEL_ISLAND_TABLE"} + self.assertEqual(island, set(), "auto-date tables and related tables are not islands") + + def test_model_bim_is_supported(self): + bim = self.tmp / "model.bim" + bim.write_text(json.dumps({"model": { + "tables": [{"name": "T", "description": "d", "measures": [ + {"name": "m", "expression": ["IFERROR(1,", "0)"], "displayFolder": "F"}]}], + "relationships": [], + }}), encoding="utf-8") + result = bpa.analyze(bpa.load_model(bim), bpa.load_rules(bpa.DEFAULT_RULES)) + self.assertEqual( + {(r, o) for r, o, _ in triples(result)}, + {("NAMING_MEASURE_PASCAL", "'T'[m]"), ("DAX_IFERROR_USAGE", "'T'[m]"), + ("DOC_MEASURE_DESCRIPTION", "'T'[m]"), ("MODEL_ISLAND_TABLE", "'T'")}, + ) + + def test_rule_without_fallback_is_skipped_not_passed(self): + rules = self.tmp / "rules.json" + rules.write_text(json.dumps([{"ID": "CUSTOM_X", "Name": "x", "Severity": 3, "Scope": "Measure", + "Expression": "true"}]), encoding="utf-8") + code, out, _ = run_main(bpa, [FIXTURE, "--rules", rules]) + self.assertEqual(code, 0) + self.assertIn("SKIPPED [CUSTOM_X]", out) + + +TE_OUTPUT = """Loading model... +Running Best Practice Analyzer... +================================= +##vso[task.logissue type=error;]Measure 'Sales'[Forecast] violates rule "Measure has an empty expression" +##vso[task.logissue type=warning;]Calculated Column 'Sales'[Margin] violates rule "Avoid IFERROR (masks underlying errors)" +##vso[task.logissue type=warning;]Relationship 'Sales'[ProductID] --> 'Product'[ProductID] violates rule "Bidirectional relationship" +Column 'Sales'[ProductID] violates rule "Key columns should be hidden" +##vso[task.logissue type=error;]Error on rule 'Broken rule': Unknown identifier 'Foo' +================================= +##vso[task.complete result=Failed;]Done. +""" + + +class TabularEditorRunner(TempDirMixin, unittest.TestCase): + def test_command_line(self): + cmd = run_bpa.build_command("TE.exe", Path("M/definition"), Path("rules.json")) + self.assertEqual(cmd, ["TE.exe", str(Path("M/definition")), "-A", "rules.json", "-V"]) + + def test_parses_severity_type_object_and_rule_errors(self): + rules = bpa.load_rules(bpa.DEFAULT_RULES) + result = run_bpa.parse_te_output(TE_OUTPUT, rules, "m") + self.assertEqual( + [(v["severity"], v["object_type"], v["object"], v["rule_id"]) for v in result["violations"]], + [ + ("error", "Measure", "'Sales'[Forecast]", "DAX_EMPTY_EXPRESSION"), + ("warning", "Calculated Column", "'Sales'[Margin]", "DAX_IFERROR_USAGE"), + ("warning", "Relationship", "'Sales'[ProductID] --> 'Product'[ProductID]", "MODEL_BIDIRECTIONAL"), + ("info", "Column", "'Sales'[ProductID]", "DOC_KEY_COLUMN_HIDDEN"), + ], + ) + self.assertEqual(result["rule_errors"], [{"rule_name": "Broken rule", "error": "Unknown identifier 'Foo'"}]) + self.assertEqual(result["summary"], {"error": 1, "warning": 2, "info": 1}) + + def test_clean_run_exits_zero(self): + te = write_stub(self.tmp, "TabularEditor", "Running Best Practice Analyzer...\nNo objects in violation of Best Practices.\n") + code, out, _ = run_main(run_bpa, [FIXTURE, "--te", te]) + self.assertEqual(code, 0) + self.assertIn("BPA engine: Tabular Editor 2", out) + + def test_runs_te_against_the_definition_folder(self): + te = write_stub(self.tmp, "TabularEditor", TE_OUTPUT, exit_code=1) + project = copy_fixture(self.tmp) + code, out, _ = run_main(run_bpa, [project / "SalesSample.pbip", "--te", te, "--format", "json"]) + args = stub_args(self.tmp, "TabularEditor") + self.assertEqual(Path(args[0]).resolve(), (project / "SalesSample.SemanticModel" / "definition").resolve()) + self.assertEqual(args[1:3], ["-A", str(bpa.DEFAULT_RULES.resolve())]) + self.assertEqual(args[3], "-V") + self.assertEqual(json.loads(out)["engine"], "tabular-editor-2") + self.assertEqual(code, 1) + + def test_te_crash_before_bpa_is_a_tool_failure(self): + te = write_stub(self.tmp, "TabularEditor", "Error loading model: boom\n", exit_code=1) + code, _, err = run_main(run_bpa, [FIXTURE, "--te", te]) + self.assertEqual(code, 2) + self.assertIn("before the BPA ran", err) + + def test_without_te_uses_the_fallback_or_fails_when_told_to(self): + with mock.patch.object(run_bpa, "find_tabular_editor", return_value=None): + code, out, err = run_main(run_bpa, [FIXTURE, "--format", "json"]) + self.assertEqual(json.loads(out)["engine"], "python-fallback") + self.assertIn("Python fallback", err) + self.assertEqual(code, 1) + code, _, err = run_main(run_bpa, [FIXTURE, "--no-fallback"]) + self.assertEqual(code, 2) + + def test_locator_precedence(self): + exe = write_stub(self.tmp, "te-env", "") + self.assertEqual(run_bpa.find_tabular_editor(None, {"TABULAR_EDITOR_PATH": str(exe)}), str(exe)) + self.assertIsNone(run_bpa.find_tabular_editor(str(self.tmp / "missing.exe"), {"TABULAR_EDITOR_PATH": str(exe)})) + install = self.tmp / "pf" / "Tabular Editor" + install.mkdir(parents=True) + (install / "TabularEditor.exe").write_text("") + with mock.patch.object(run_bpa.shutil, "which", return_value=None): + found = run_bpa.find_tabular_editor(None, {"ProgramFiles(x86)": str(self.tmp / "pf")}) + self.assertEqual(found, str(install / "TabularEditor.exe")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_cli_init.py b/tests/test_cli_init.py new file mode 100644 index 0000000..e909785 --- /dev/null +++ b/tests/test_cli_init.py @@ -0,0 +1,71 @@ +import shutil +import subprocess +import unittest + +from helpers import ROOT, TempDirMixin + +NODE = shutil.which("node") +HAS_DEPS = (ROOT / "node_modules" / "commander").is_dir() + + +@unittest.skipUnless(NODE and HAS_DEPS, "node and `npm install` are required for CLI tests") +class Init(TempDirMixin, unittest.TestCase): + def init(self, target, *extra): + return subprocess.run( + [NODE, str(ROOT / "bin" / "cli.js"), "init", "--target", target, "--path", str(self.tmp), *extra], + capture_output=True, text=True, timeout=60, + ) + + def assert_files(self, *rels): + for rel in rels: + self.assertTrue((self.tmp / rel).is_file(), f"missing {rel}") + + def test_each_target_installs_its_native_layout(self): + expected = { + "claude": [".claude/CLAUDE.md", ".claude/agents/pbi-visual-reviewer.md", + ".claude/commands/validate-pbi.md", ".claude/skills/pbi-quality-rules/scripts/run_bpa.py"], + "copilot": [".github/copilot-instructions.md", ".github/agents/pbi-visual-reviewer.agent.md", + ".github/prompts/validate-pbi.prompt.md", ".github/skills/pbi-quality-rules/bpa-rules.json"], + "opencode": ["AGENTS.md", ".opencode/agents/pbi-visual-reviewer.md", + ".opencode/skills/pbi-semantic-layer-tmdl/SKILL.md"], + "cursor": [".cursor/rules/agentic-data-kit.mdc", ".cursor/commands/validate-pbi.md", + ".cursor/skills/pbi-report-layer-pbir/SKILL.md"], + "antigravity": [".agent/rules/rules.md", ".agent/agents/pbi-visual-reviewer.md", "scripts/sync_agents.py"], + } + for target, files in expected.items(): + with self.subTest(target=target): + shutil.rmtree(self.tmp) + self.tmp.mkdir() + proc = self.init(target) + self.assertEqual(proc.returncode, 0, proc.stderr + proc.stdout) + self.assert_files(*files) + self.assertEqual(list(self.tmp.rglob("__pycache__")), []) + + def test_copilot_merges_into_an_existing_github_folder(self): + own = self.tmp / ".github" / "workflows" / "ci.yml" + own.parent.mkdir(parents=True) + own.write_text("on: push\n", encoding="utf-8") + proc = self.init("copilot") + self.assertEqual(proc.returncode, 0, proc.stderr + proc.stdout) + self.assertEqual(own.read_text(encoding="utf-8"), "on: push\n") + self.assertFalse((self.tmp / ".github" / "workflows" / "sync-check.yml").exists(), + "the kit's own CI workflow must never be installed") + self.assertNotEqual(self.init("copilot").returncode, 0, "second run without --force must refuse") + forced = self.init("copilot", "--force") + self.assertEqual(forced.returncode, 0, forced.stderr) + self.assertTrue(own.exists(), "--force must only replace the kit's own entries") + + def test_opencode_refuses_to_overwrite_an_existing_agents_md(self): + (self.tmp / "AGENTS.md").write_text("mine", encoding="utf-8") + proc = self.init("opencode") + self.assertNotEqual(proc.returncode, 0) + self.assertEqual((self.tmp / "AGENTS.md").read_text(encoding="utf-8"), "mine") + + def test_unknown_target(self): + proc = self.init("emacs") + self.assertNotEqual(proc.returncode, 0) + self.assertIn("cursor", proc.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_sync.py b/tests/test_sync.py new file mode 100644 index 0000000..6137e3f --- /dev/null +++ b/tests/test_sync.py @@ -0,0 +1,92 @@ +import shutil +import unittest + +import yaml + +from helpers import ROOT, TempDirMixin, load_module + +sync = load_module(ROOT / "scripts" / "sync_agents.py", "sync_agents") + + +def frontmatter(path): + meta, _ = sync.parse_frontmatter(path.read_text(encoding="utf-8")) + return meta + + +class GeneratedFolders(TempDirMixin, unittest.TestCase): + def test_committed_generated_folders_match_the_source(self): + self.assertEqual(sync.check(), [], "run: python3 scripts/sync_agents.py") + + def test_check_reports_every_kind_of_drift(self): + expected, actual = self.tmp / "expected", self.tmp / "actual" + sync.build(expected) + shutil.copytree(expected, actual) + (actual / ".claude" / "CLAUDE.md").write_text("hand edit", encoding="utf-8") + (actual / ".cursor" / "commands" / "old.md").write_text("stale", encoding="utf-8") + (actual / "AGENTS.md").unlink() + (actual / ".opencode" / "AGENTS.md").write_text("legacy", encoding="utf-8") + self.assertEqual( + sorted(sync.stale_paths(expected, actual)), + sorted([ + "outdated .claude/CLAUDE.md", + "extra .cursor/commands/old.md", + "missing AGENTS.md", + "legacy .opencode/AGENTS.md", + ]), + ) + + def test_sync_keeps_unmanaged_github_files(self): + workflow = self.tmp / ".github" / "workflows" / "ci.yml" + workflow.parent.mkdir(parents=True) + workflow.write_text("on: push\n", encoding="utf-8") + stale = self.tmp / ".github" / "agents" / "removed-agent.agent.md" + stale.parent.mkdir(parents=True) + stale.write_text("old", encoding="utf-8") + sync.sync(self.tmp) + self.assertTrue(workflow.exists()) + self.assertFalse(stale.exists(), "agents deleted from .agent/ must disappear from generated folders") + self.assertEqual(sync.stale_paths(self.tmp, self.tmp), []) + + +class ReviewerToolRestrictions(TempDirMixin, unittest.TestCase): + """The reviewer is read-only; every generated format must keep it read-only.""" + + @classmethod + def setUpClass(cls): + import tempfile + from pathlib import Path + cls._out = tempfile.TemporaryDirectory(prefix="adk-sync-") + cls.out = Path(cls._out.name) + sync.build(cls.out) + + @classmethod + def tearDownClass(cls): + cls._out.cleanup() + + def test_claude_keeps_the_source_allow_list(self): + self.assertEqual(frontmatter(self.out / ".claude/agents/pbi-visual-reviewer.md")["tools"], "Read, Grep, Glob") + + def test_copilot_gets_mapped_aliases(self): + meta = frontmatter(self.out / ".github/agents/pbi-visual-reviewer.agent.md") + self.assertEqual(meta["tools"], ["read", "search"]) + + def test_opencode_denies_edit_and_bash(self): + meta = frontmatter(self.out / ".opencode/agents/pbi-visual-reviewer.md") + self.assertEqual((meta["mode"], meta["permission"]), ("subagent", {"edit": "deny", "bash": "deny"})) + + def test_cursor_gets_skills_rule_and_commands(self): + self.assertTrue((self.out / ".cursor/skills/pbi-report-layer-pbir/scripts/validate_pbir.py").is_file()) + rule = yaml.safe_load((self.out / ".cursor/rules/agentic-data-kit.mdc").read_text(encoding="utf-8").split("---")[1]) + self.assertIs(rule["alwaysApply"], True) + self.assertTrue((self.out / ".cursor/commands/validate-pbi.md").is_file()) + + def test_unknown_tool_name_fails_instead_of_widening_access(self): + with self.assertRaises(SystemExit): + sync.copilot_tools("Read, Teleport", "x") + self.assertEqual(sync.copilot_tools("Read, Grep, Glob, Edit, Write, Bash", "x"), + ["read", "search", "edit", "execute"]) + self.assertEqual(sync.opencode_permission("Read, Write"), {"edit": "allow", "bash": "deny"}) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_validate_pbir.py b/tests/test_validate_pbir.py new file mode 100644 index 0000000..3f0a371 --- /dev/null +++ b/tests/test_validate_pbir.py @@ -0,0 +1,165 @@ +import contextlib +import io +import json +import os +import shutil +import unittest +from pathlib import Path +from unittest import mock + +from helpers import FIXTURE, PBIR_SCRIPTS, TempDirMixin, clean_env, copy_fixture, load_module, stub_args, write_stub + +vp = load_module(PBIR_SCRIPTS / "validate_pbir.py", "validate_pbir") + +PAGE = "e087e4b90fb242dfb36a" +CARD = "1c9e5a7d3b2f4e6a8d01" + + +def run_main(args, env=None): + out, err = io.StringIO(), io.StringIO() + with mock.patch.dict(os.environ, env if env is not None else clean_env(), clear=True), \ + contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + code = vp.main([str(a) for a in args]) + return code, out.getvalue(), err.getvalue() + + +def codes(result, severity=None): + return sorted(d["code"] for d in result["diagnostics"] if severity in (None, d["severity"])) + + +class Structural(TempDirMixin, unittest.TestCase): + def setUp(self): + super().setUp() + self.project = copy_fixture(self.tmp) + self.report = self.project / "SalesSample.Report" + self.pages = self.report / "definition" / "pages" + + def edit_json(self, path: Path, change): + data = json.loads(path.read_text(encoding="utf-8")) + change(data) + path.write_text(json.dumps(data), encoding="utf-8") + + def check(self): + return vp.structural_check(self.report) + + def test_fixture_is_clean(self): + result = self.check() + self.assertEqual((result["result"], result["errorCount"], result["warningCount"]), ("succeeded", 0, 0)) + + def test_pages_json_out_of_sync_with_folders(self): + self.edit_json(self.pages / "pages.json", lambda d: d.update(pageOrder=["ghost"], activePageName="other")) + result = self.check() + self.assertEqual(codes(result, "error"), ["ACTIVE_PAGE_UNKNOWN", "PAGE_MISSING"]) + self.assertEqual(codes(result, "warning"), ["PAGE_NOT_IN_ORDER"]) + + def test_visual_problems(self): + visual = self.pages / PAGE / "visuals" / CARD / "visual.json" + + def break_visual(d): + d["name"] = "renamed" + del d["position"]["height"] + del d["visual"]["visualType"] + self.edit_json(visual, break_visual) + result = self.check() + self.assertEqual(codes(result), ["NAME_MISMATCH", "POSITION_INVALID", "VISUAL_TYPE_MISSING"]) + self.assertTrue(all(d["file"] == f"definition/pages/{PAGE}/visuals/{CARD}/visual.json" for d in result["diagnostics"])) + + def test_visual_outside_canvas_is_a_warning(self): + visual = self.pages / PAGE / "visuals" / CARD / "visual.json" + self.edit_json(visual, lambda d: d["position"].update(x=1200)) + result = self.check() + self.assertEqual((codes(result, "warning"), result["errorCount"]), (["OUTSIDE_CANVAS"], 0)) + self.assertEqual(result["result"], "succeededWithWarnings") + + def test_invalid_json_is_reported_once(self): + (self.pages / PAGE / "visuals" / CARD / "visual.json").write_text("{ not json", encoding="utf-8") + self.assertEqual(codes(self.check()), ["JSON_INVALID"]) + + def test_broken_model_binding(self): + shutil.rmtree(self.project / "SalesSample.SemanticModel") + self.assertEqual(codes(self.check()), ["MODEL_BINDING_BROKEN"]) + + def test_registered_resources_must_exist(self): + (self.report / "StaticResources" / "SharedResources" / "BaseThemes" / "Fluent2-CY26SU10.json").unlink() + + def custom_theme(d): + d["themeCollection"]["customTheme"] = {"name": "Brand-1a2b3c.json", "type": "RegisteredResources"} + self.edit_json(self.report / "definition" / "report.json", custom_theme) + self.assertEqual(codes(self.check()), ["RESOURCE_MISSING", "THEME_NOT_REGISTERED"]) + + def test_pbir_legacy_layout(self): + shutil.rmtree(self.report / "definition") + (self.report / "report.json").write_text("{}", encoding="utf-8") + self.assertEqual(codes(self.check()), ["PBIR_LEGACY_FORMAT"]) + + def test_cli_exit_codes(self): + code, out, _ = run_main([self.project, "--structural-only"]) + self.assertEqual(code, 0) + self.assertIn("structural fallback", out) + (self.pages / PAGE / "visuals" / CARD / "visual.json").write_text("{", encoding="utf-8") + self.assertEqual(run_main([self.project, "--structural-only"])[0], 1) + self.assertEqual(run_main([self.tmp / "nope"])[0], 2) + + def test_report_resolution(self): + expected = self.report.resolve() + for path in (self.project / "SalesSample.pbip", self.project, self.report): + with self.subTest(path=path): + self.assertEqual(vp.resolve_report(path), expected) + + +# Real envelopes captured from powerbi-report-author 0.5.0. +CLI_FAILED = {"data": {"result": "failed", "errorCount": 1, "warningCount": 1, "reportPath": "R", "diagnostics": { + "PBIR_VISUAL_TYPE_UNKNOWN": {"severity": "warning", "items": [ + {"message": "Unknown visualType \"cardd\"", "file": "{report}/definition/pages/p/visuals/v1/visual.json", + "path": "visual.visualType"}]}, + "PBIR_SCHEMA_VALIDATION_ERROR": {"severity": "error", "items": [ + {"message": "Schema validation: /position must have required property 'height'", + "file": "{report}/definition/pages/p/visuals/v1/visual.json", "path": "/position"}]}, +}}} +CLI_ERROR = {"error": {"message": "Path does not exist: /x", "code": "REPORT_PATH_NOT_FOUND", "retryable": False}} + + +class ReportAuthorCli(TempDirMixin, unittest.TestCase): + def envelope(self, data, report: Path) -> str: + return json.dumps(data).replace("{report}", str(report)) + + def test_parses_diagnostics_relative_to_the_report(self): + report = FIXTURE / "SalesSample.Report" + result = vp.parse_cli_output(self.envelope(CLI_FAILED, report.resolve()), report.resolve()) + self.assertEqual((result["errorCount"], result["warningCount"], result["result"]), (1, 1, "failed")) + self.assertEqual({d["file"] for d in result["diagnostics"]}, {"definition/pages/p/visuals/v1/visual.json"}) + self.assertEqual(codes(result, "error"), ["PBIR_SCHEMA_VALIDATION_ERROR"]) + + def test_error_envelope_is_a_tool_failure(self): + with self.assertRaises(vp.UsageError): + vp.parse_cli_output(json.dumps(CLI_ERROR), FIXTURE) + + def test_uses_the_cli_when_configured(self): + report = (FIXTURE / "SalesSample.Report").resolve() + cli = write_stub(self.tmp, "powerbi-report-author", self.envelope(CLI_FAILED, report), exit_code=1) + code, out, _ = run_main([FIXTURE / "SalesSample.pbip", "--offline", "--format", "json"], + env=clean_env(PBIR_AUTHOR_CLI=str(cli))) + self.assertEqual(stub_args(self.tmp, "powerbi-report-author"), ["validate", str(report), "--no-schema"]) + self.assertEqual(json.loads(out)["engine"], "powerbi-report-author") + self.assertEqual(code, 1) + + def test_structural_only_ignores_the_cli(self): + cli = write_stub(self.tmp, "powerbi-report-author", json.dumps(CLI_ERROR)) + code, out, _ = run_main([FIXTURE, "--structural-only"], env=clean_env(PBIR_AUTHOR_CLI=str(cli))) + self.assertEqual(code, 0) + self.assertFalse((self.tmp / "powerbi-report-author.args").exists()) + + +REAL_CLI = vp.find_cli() + + +@unittest.skipUnless(REAL_CLI, "powerbi-report-author not installed (set PBIR_AUTHOR_CLI or add it to PATH)") +class RealCli(unittest.TestCase): + def test_fixture_passes_microsoft_validate(self): + report = vp.resolve_report(FIXTURE) + result = vp.run_cli(REAL_CLI, report, offline=True) + self.assertEqual((result["result"], result["errorCount"]), ("succeeded", 0), result["diagnostics"]) + + +if __name__ == "__main__": + unittest.main()