From ded723bd12edf51d8afea48ad5b7d5c535eeb599 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 30 Sep 2026 15:46:52 +0000 Subject: [PATCH 001/550] =?UTF-8?q?fix(studio):=20the=20account=20menu=20i?= =?UTF-8?q?s=20reachable=20on=20every=20screen=20=E2=80=94=20home=20and=20?= =?UTF-8?q?Build=20no=20longer=20hide=20it?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The account menu (change password, sign out, sign out my other sessions) was mounted beside the theme toggle in the context bar (.hdr), and the OBSERVA chrome hides that whole bar on the home and Build screens (app.css, ux.css). So until a pack was open, a signed-in user had no way to sign out or change a password — the README even said "top right once a pack is open". setupIdentityChip now appends the chip to the chrome's action cluster (.observa-hdr .observa-actions, beside Advanced): boot mounts the chrome first, and no mode rule hides it, so the same menu is there in home, Build and pack-open modes with the same behaviour. Un-hiding only the chip was not an option: the bar is hidden whole, so a child could only show by being pulled out of flow — a restyle. No CSS changes: the chip and its popover keep their rules, which are not scoped to the context bar. The identity rules are unchanged (no chip without a session or in the open postures), and so is the ORG chip: the context-bar copy stays with the pack pickers, and the OBSERVA bar already carries its own on every screen (updateObservaOrgChip, the same orgChipModel rule). server/test-authz.mjs pins the mount (the cluster, appended last, after the chrome is mounted) and scans every studio stylesheet: the context bar is hidden in home and build (the reason for the move) and no mode rule hides the chrome, its action cluster or the chip. Browser-verified with Playwright on an identity-local server: the button is visible and opens with change password / sign out my other sessions / sign out on the home screen, in Build and with a pack open (also at 1200 and 960 px), sign out works from the home screen, and with sign-in off no chip renders. README's Quickstart now says "top right on every screen". Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 4 ++-- docs/CHANGELOG.md | 1 + server/test-authz.mjs | 29 +++++++++++++++++++++++++++++ studio/app.mjs | 17 ++++++++++++----- 4 files changed, 44 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 0b41e0a5..3c465777 100644 --- a/README.md +++ b/README.md @@ -248,8 +248,8 @@ npm run dev Open `http://127.0.0.1:8000` and sign in with **admin / admin** — first boot seeds this default user and asks for a password change at sign-in (skippable for now; it asks again each sign-in until a real password -lands — or change it any time from the account menu, top right once a -pack is open, which also has **sign out my other sessions**). From +lands — or change it any time from the account menu, top right on every +screen, which also has **sign out my other sessions**). From there it's a signed-in app: your packs, deploy audit and run history belong to you. (`OBSERVOGRAM_AUTH=off` skips login entirely for a throwaway open sandbox.) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index e3c68eae..59a1d0e8 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,6 +7,7 @@ STORE_PLAN slice 3, second of two PRs (after 3a, below). Owners and org admins m - **Owner routes** (`/api/admin/*`, whatever org the request is in): `GET` / `POST /api/admin/users` (a local user; the first local user while no enabled local owner exists becomes an owner where local users can sign in), `POST …/users/:id/disable`, `…/enable`, `…/password` (a temporary password: every session ends and the next sign-in must replace it, no skip; local users only, never the caller's own row), `…/signout` (sign out everywhere; an IdP user can sign in again at the IdP — disable to keep them out), `PUT …/users/:id/owner` (grant or revoke), `GET` / `POST /api/admin/orgs` (the creator is the new org's first admin; `adopt` takes over a non-empty `orgs//`), `DELETE /api/admin/orgs/:id` (soft; the files stay), `GET` / `PUT /api/admin/join-role` (`admin` only with `"confirm": true`; it backfills nobody). Paths name users by `users.id`. - **Admin routes** (`/api/org*`, the request's org; no path names an org, so an org admin never reaches another one): `PATCH /api/org` (the name), `GET /api/org/members` (admins only: the list carries logins and emails), `POST /api/org/members` (an existing user by exact login or by the one enabled user whose sign-in verified that email — never a new row; a member already gets the role asked), `PATCH` and `DELETE /api/org/members/:userId`. The org's last enabled admin is demoted or removed only by an owner, on all three paths (the upsert included); a shell's `orgs -- add-member` / `remove-member` still can. To an org admin, no such user, a disabled one and several email matches are one 404; an owner gets the detail. - **Sign out my other sessions.** `POST /auth/signout-others`, the account menu's new item (local and OIDC sign-in): the caller's own session only — never the pwflow cookie of a pending forced change, never the bearer (401) — with `X-Observogram-CSRF: 1`; every other cookie of the user is refused from its next request, and this browser's is re-issued at the new epoch. The re-issued cookie keeps the session's own expiry rather than getting a fresh TTL (the design re-issued it as a sign-in does): signing out elsewhere never extends a session, so it cannot keep one alive past the moment the IdP stops letting its user in. +- **Studio fixes.** The account menu is on every screen: it moved from the context bar — hidden on the home and Build screens, so until a pack was open a signed-in user had no way to sign out or change a password — to the OBSERVA bar beside Advanced, which every screen shows. The same menu everywhere (who you are, change password…, sign out my other sessions, sign out), still none without sign-in; the ORG chip is unchanged (the OBSERVA bar already carries its own on every screen). `server/test-authz.mjs` pins the mount and that no mode rule hides its bar. - **Identity changes need the CSRF header in every posture.** Every identity-API mutation and `POST /auth/signout-others` answer 403 `csrf` without `X-Observogram-CSRF: 1` — from a session, and also from `local` on a server without sign-in; the bearer is exempt, and the identity API refuses it anyway (it is an operator: 403 `role`). - **The open postures.** Without sign-in the caller is `local`, an owner. On a loopback server the identity API answers only a request sent straight to a loopback address (`Host` `localhost`, `127.x.x.x` or `[::1]`; no `Forwarded`, `Via`, `X-Forwarded-*`, `X-Real-IP` or client-IP header such as `CF-Connecting-IP` or `True-Client-IP`; an `Origin`, if any, naming that host), else 403 `posture` naming the CLIs — a DNS-rebinding page or a proxy cannot plant an owner that outlives the posture. Beyond loopback (`OBSERVOGRAM_INSECURE_NO_AUTH=1`) it is closed (403 `posture`). A direct request to `POST /api/admin/orgs` answers 409 in both — a second org needs identity; under `OBSERVOGRAM_AUTH=off` the text says to restart without it. Every existing route is unchanged there. - **Refusals by kind.** A rule's refusal is 400 (bad input), 404 (no such user, org or member) or 409 (the state forbids it), naming a way out that works; 403 stays an authorization denial, with `denied`. diff --git a/server/test-authz.mjs b/server/test-authz.mjs index a0a81ac9..65da814e 100644 --- a/server/test-authz.mjs +++ b/server/test-authz.mjs @@ -575,6 +575,35 @@ test('the account menu\'s "sign out my other sessions": the route it posts to, a assert.match(fn, /chip\.querySelector\('\.hdr-user-menu-id'\)\.textContent = signOutOthersText\(status, body\);/); }); +// The account menu is on every screen. setupIdentityChip mounts it in the +// chrome's action cluster (.observa-hdr .observa-actions), which boot mounts +// first and no mode rule hides. The context bar (.hdr) it used to sit in IS +// hidden on the home and Build screens — where a signed-in user then had no +// way to sign out or change a password until a pack was open. +test('the account menu mounts in the one bar every screen shows — not the context bar the home and Build screens hide', () => { + const src = withoutComments(readFileSync(join(STUDIO, 'app.mjs'), 'utf8')); + const fn = functionSource(src, 'setupIdentityChip'); + assert.match(fn, /const actions = document\.querySelector\('\.observa-hdr \.observa-actions'\);/, 'the chrome\'s action cluster'); + assert.match(fn, /\n {2}actions\.appendChild\(chip\);\n\}$/, 'the chip is its last child'); + assert.doesNotMatch(fn, /insertBefore\(chip,/, 'not beside the theme toggle in the context bar'); + const boot = functionSource(src, 'boot'); + assert.ok(boot.indexOf('installObservaChrome();') < boot.indexOf('setupIdentityChip();'), 'boot mounts the chrome before the chip'); + + // Every selector a mode rule hides, from every studio stylesheet. + const hidden = []; + for (const f of readdirSync(STUDIO).filter((n) => n.endsWith('.css'))) { + const css = readFileSync(join(STUDIO, f), 'utf8').replace(/\/\*[\s\S]*?\*\//g, ' '); + for (const m of css.matchAll(/([^{}]+)\{([^{}]*)\}/g)) { + if (!/(display\s*:\s*none|visibility\s*:\s*hidden)/.test(m[2])) continue; + for (const sel of m[1].split(',')) if (sel.includes('[data-mode=')) hidden.push(sel.trim()); + } + } + for (const mode of ['home', 'build']) { + assert.ok(hidden.some((s) => s.endsWith(`[data-mode="${mode}"] .hdr`)), `the context bar is hidden in ${mode} mode (why the menu moved)`); + } + for (const s of hidden) assert.doesNotMatch(s, /observa-hdr|observa-actions|hdr-user/, `a mode rule hides the account menu's bar: ${s}`); +}); + // ---------- completeness: every route is classified and guarded (§14.1) ---------- // // The expected class of every route, written from STORE_PLAN §5 — NOT diff --git a/studio/app.mjs b/studio/app.mjs index 011b2b9f..17fe1894 100644 --- a/studio/app.mjs +++ b/studio/app.mjs @@ -5101,21 +5101,28 @@ function syncApiLink() { } // Identity chip — only renders when the server runs in an identity -// posture and a session exists. +// posture and a session exists. The account menu mounts in the chrome's +// action cluster (installObservaChrome runs first in boot): the one bar +// every screen shows. The context bar (.hdr) is hidden on the home and +// Build screens (app.css, ux.css), so a menu there left a signed-in user +// no way to sign out or change a password until a pack was open. function setupIdentityChip() { const me = state.identity; if (!me?.authenticated) return; - const anchor = $('#theme-toggle'); - if (!anchor || document.getElementById('hdr-user')) return; + const actions = document.querySelector('.observa-hdr .observa-actions'); + if (!actions || document.getElementById('hdr-user')) return; // Org indicator (orgChipModel) — a switcher when the user belongs to // several orgs, a static label for one org that is not the default one, // nothing for the default org alone. Switching reloads: every view is a // projection of the active org's workspace, so a clean re-boot is the // honest refresh. + // This copy sits with the pack pickers in the context bar; the OBSERVA + // bar's own ORG chip (updateObservaOrgChip) shows on every screen. const orgs = me.orgs || []; const orgChip = orgChipModel(orgs, getActiveOrg()); - if (orgChip.kind !== 'none' && !document.getElementById('hdr-org')) { + const anchor = $('#theme-toggle'); + if (anchor && orgChip.kind !== 'none' && !document.getElementById('hdr-org')) { const wrap = document.createElement('span'); wrap.id = 'hdr-org'; wrap.className = 'hdr-org'; @@ -5190,7 +5197,7 @@ function setupIdentityChip() { await fetch('/auth/logout', { method: 'POST', headers: { ...authHeaders() } }).catch(() => {}); window.location.assign('/auth/login'); }); - anchor.parentNode.insertBefore(chip, anchor); + actions.appendChild(chip); } // The inline script in already applied the persisted/system theme From 65e0a678536a39fe3228c5787f9e305953bfa31c Mon Sep 17 00:00:00 2001 From: Carlos Montero Date: Thu, 1 Oct 2026 00:33:22 +0200 Subject: [PATCH 002/550] fix(studio): the account menu gives way beside the tabs; the chrome's height is measured Review of the move found two layout regressions, both measured in a browser: - At a laptop width (1101-1280px) a login of a dozen characters took the tabs' room; their titles wrapped, the chrome grew by ~15px, and the context bar - sticky at the constant --observa-chrome-h - slid under it by that much on scroll, as did every strip pinned below it. The chrome's height is now measured into the variable (trackChromeHeight, as trackContextBarHeight already does for the context bar), once its tabs are in it and on every resize, and the chip's name gives way to ten characters at that width, as the Advanced label already did. - At phone width the action cluster could not shrink, so the chip pushed the chrome - and its popover, anchored to it - past the viewport's edge. The name is its own span, hidden but for assistive technology at 720px and under; the cluster may give way before the tabs do; the popover is capped at the viewport's width. The test's boot-order guard no longer passes vacuously when the chrome call is missing, and pins the measurement and the give-way rules. Co-Authored-By: Claude Fable 5.1 --- docs/CHANGELOG.md | 2 +- server/test-authz.mjs | 17 ++++++++++++++++- studio/app.mjs | 24 +++++++++++++++++++++++- studio/ux.css | 14 ++++++++++++++ 4 files changed, 54 insertions(+), 3 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 7edff646..418f8d12 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -19,7 +19,7 @@ STORE_PLAN slice 3, second of two PRs (after 3a, below). Owners and org admins m - **Owner routes** (`/api/admin/*`, whatever org the request is in): `GET` / `POST /api/admin/users` (a local user; the first local user while no enabled local owner exists becomes an owner where local users can sign in), `POST …/users/:id/disable`, `…/enable`, `…/password` (a temporary password: every session ends and the next sign-in must replace it, no skip; local users only, never the caller's own row), `…/signout` (sign out everywhere; an IdP user can sign in again at the IdP — disable to keep them out), `PUT …/users/:id/owner` (grant or revoke), `GET` / `POST /api/admin/orgs` (the creator is the new org's first admin; `adopt` takes over a non-empty `orgs//`), `DELETE /api/admin/orgs/:id` (soft; the files stay), `GET` / `PUT /api/admin/join-role` (`admin` only with `"confirm": true`; it backfills nobody). Paths name users by `users.id`. - **Admin routes** (`/api/org*`, the request's org; no path names an org, so an org admin never reaches another one): `PATCH /api/org` (the name), `GET /api/org/members` (admins only: the list carries logins and emails), `POST /api/org/members` (an existing user by exact login or by the one enabled user whose sign-in verified that email — never a new row; a member already gets the role asked), `PATCH` and `DELETE /api/org/members/:userId`. The org's last enabled admin is demoted or removed only by an owner, on all three paths (the upsert included); a shell's `orgs -- add-member` / `remove-member` still can. To an org admin, no such user, a disabled one and several email matches are one 404; an owner gets the detail. - **Sign out my other sessions.** `POST /auth/signout-others`, the account menu's new item (local and OIDC sign-in): the caller's own session only — never the pwflow cookie of a pending forced change, never the bearer (401) — with `X-Observogram-CSRF: 1`; every other cookie of the user is refused from its next request, and this browser's is re-issued at the new epoch. The re-issued cookie keeps the session's own expiry rather than getting a fresh TTL (the design re-issued it as a sign-in does): signing out elsewhere never extends a session, so it cannot keep one alive past the moment the IdP stops letting its user in. -- **Studio fixes.** The account menu is on every screen: it moved from the context bar — hidden on the home and Build screens, so until a pack was open a signed-in user had no way to sign out or change a password — to the OBSERVA bar beside Advanced, which every screen shows. The same menu everywhere (who you are, change password…, sign out my other sessions, sign out), still none without sign-in; the ORG chip is unchanged (the OBSERVA bar already carries its own on every screen). `server/test-authz.mjs` pins the mount and that no mode rule hides its bar. +- **Studio fixes.** The account menu is on every screen: it moved from the context bar — hidden on the home and Build screens, so until a pack was open a signed-in user had no way to sign out or change a password — to the OBSERVA bar beside Advanced, which every screen shows. The same menu everywhere (who you are, change password…, sign out my other sessions, sign out), still none without sign-in; the ORG chip is unchanged (the OBSERVA bar already carries its own on every screen). Beside the tabs the chip's name gives way — ten characters at a laptop width, the glyph alone at phone width (its title and menu say it in full) — so the tab titles keep their room and the popover stays on screen; and the chrome's height is now measured into `--observa-chrome-h` (as the context bar's already was), so the sticky context bar sits flush under it whatever wraps, where the constant had left it 15px under a two-line chrome. `server/test-authz.mjs` pins the mount, the measurement, the give-way rules and that no mode rule hides its bar. - **Identity changes need the CSRF header in every posture.** Every identity-API mutation and `POST /auth/signout-others` answer 403 `csrf` without `X-Observogram-CSRF: 1` — from a session, and also from `local` on a server without sign-in; the bearer is exempt, and the identity API refuses it anyway (it is an operator: 403 `role`). - **The open postures.** Without sign-in the caller is `local`, an owner. On a loopback server the identity API answers only a request sent straight to a loopback address (`Host` `localhost`, `127.x.x.x` or `[::1]`; no `Forwarded`, `Via`, `X-Forwarded-*`, `X-Real-IP` or client-IP header such as `CF-Connecting-IP` or `True-Client-IP`; an `Origin`, if any, naming that host), else 403 `posture` naming the CLIs — a DNS-rebinding page or a proxy cannot plant an owner that outlives the posture. Beyond loopback (`OBSERVOGRAM_INSECURE_NO_AUTH=1`) it is closed (403 `posture`). A direct request to `POST /api/admin/orgs` answers 409 in both — a second org needs identity; under `OBSERVOGRAM_AUTH=off` the text says to restart without it. Every existing route is unchanged there. - **Refusals by kind.** A rule's refusal is 400 (bad input), 404 (no such user, org or member) or 409 (the state forbids it), naming a way out that works; 403 stays an authorization denial, with `denied`. diff --git a/server/test-authz.mjs b/server/test-authz.mjs index 65da814e..8e71f4da 100644 --- a/server/test-authz.mjs +++ b/server/test-authz.mjs @@ -587,7 +587,22 @@ test('the account menu mounts in the one bar every screen shows — not the cont assert.match(fn, /\n {2}actions\.appendChild\(chip\);\n\}$/, 'the chip is its last child'); assert.doesNotMatch(fn, /insertBefore\(chip,/, 'not beside the theme toggle in the context bar'); const boot = functionSource(src, 'boot'); - assert.ok(boot.indexOf('installObservaChrome();') < boot.indexOf('setupIdentityChip();'), 'boot mounts the chrome before the chip'); + const chromeAt = boot.indexOf('installObservaChrome();'); + assert.ok(chromeAt >= 0 && chromeAt < boot.indexOf('setupIdentityChip();'), 'boot mounts the chrome before the chip'); + // The chrome's height is measured into --observa-chrome-h (the context bar + // and the sticky strips pin under it): beside the account menu the tab + // titles wrap at a laptop width, and the constant alone left the context + // bar under the chrome by the difference. The chip's name gives way there + // (ten characters) and at phone width (the glyph), so the tabs keep their + // room and the popover stays on screen. + assert.match(src, /syncObservaTabs\(\);\s*trackChromeHeight\(hdr\);/, 'the chrome is measured once its tabs are in it'); + assert.match(functionSource(src, 'trackChromeHeight'), /setProperty\('--observa-chrome-h', `\$\{Math\.ceil\(hdr\.getBoundingClientRect\(\)\.height\)\}px`\)[\s\S]*new ResizeObserver\(apply\)\.observe\(hdr\);[\s\S]*addEventListener\('resize', apply\);[\s\S]*apply\(\);\s*\}$/); + assert.match(fn, /\$\{escapeHtml\(me\.name \|\| me\.email \|\| me\.sub\)\}<\/span>/, 'the name is its own span, so a stylesheet can let it give way'); + const ux = readFileSync(join(STUDIO, 'ux.css'), 'utf8'); + const at1280 = ux.indexOf('@media (max-width: 1280px) {'); + const cap = ux.indexOf('.observa-actions .hdr-user-btn { max-width: 10ch; }'); + const at720 = ux.search(/@media \(max-width: 720px\) \{\r?\n\s*\.observa-actions \.hdr-user-name \{ position: absolute; width: 1px;/); + assert.ok(at1280 >= 0 && cap > at1280 && at720 > cap, 'the chip gives way at a laptop width (ten characters) and at phone width (its glyph)'); // Every selector a mode rule hides, from every studio stylesheet. const hidden = []; diff --git a/studio/app.mjs b/studio/app.mjs index b4486d56..53d519c5 100644 --- a/studio/app.mjs +++ b/studio/app.mjs @@ -1298,6 +1298,23 @@ function trackContextBarHeight() { apply(); } +// The context bar and those strips pin under the chrome by +// --observa-chrome-h. ux.css holds its single-row height; the chrome grows +// when its tab titles wrap (a laptop width, with the account menu beside +// them), and a constant would leave the context bar under the chrome by +// the difference — so measure it, as the context bar is. +function trackChromeHeight(hdr) { + const apply = () => { + document.body.style.setProperty('--observa-chrome-h', `${Math.ceil(hdr.getBoundingClientRect().height)}px`); + }; + if (typeof ResizeObserver === 'function') new ResizeObserver(apply).observe(hdr); + // A tab in the background gets no resize notifications until it is shown + // again — and not always then. + window.addEventListener('resize', apply); + document.addEventListener('visibilitychange', apply); + apply(); +} + function installObservaChrome() { if (document.querySelector('.observa-hdr')) return; document.body.classList.add('chrome-observa'); @@ -1402,6 +1419,8 @@ function installObservaChrome() { // The tab cards: the analysis journey's three (or the BUILD journey's // three in build mode), one renderer — syncObservaTabs wires the clicks. syncObservaTabs(); + // Measured with the tabs in it: they set its height. + trackChromeHeight(hdr); // Wire the Advanced menu — deep tools off the main workflow. const advToggle = hdr.querySelector('.observa-adv-toggle'); @@ -5153,6 +5172,9 @@ function setupIdentityChip() { // Account menu: who you are, change password (stand-alone mode — OIDC // passwords belong to the IdP), sign out my other sessions, sign out. + // The name is its own span: beside the tabs it gives way (ux.css — ten + // characters at a laptop width, the glyph alone at phone width), so the + // tab titles keep their room; the title and the menu say it in full. const chip = document.createElement('span'); chip.id = 'hdr-user'; chip.className = 'hdr-user'; @@ -5161,7 +5183,7 @@ function setupIdentityChip() { // buttons are natively focusable and honest about what this is. chip.innerHTML = ` + title="signed in as ${escapeHtml(me.email || me.sub)} (${escapeHtml(me.mode)})">⏣ ${escapeHtml(me.name || me.email || me.sub)} ▾
${escapeHtml(artefact.title || artefact.id)}
${artefact.desc ? `
${escapeHtml(artefact.desc)}
` : ''} + ${readers ? `
${escapeHtml(readers)}
` : ''}
${sourcePill} ${artefact.tool ? `${escapeHtml(artefact.tool)}` : ''} @@ -4368,6 +4407,14 @@ function compareSummaryHtml(digest, cmp, ctx) { const cosmeticCount = driftedEntries.filter(e => diagDriftCost(e).className === 'cosmetic').length; const qItems = []; if (cmp.missing) qItems.push(`
  • ${cmp.missing} ${drift ? `declared artefact${s(cmp.missing)} not seen live` : `artefact${s(cmp.missing)} from the ${escapeHtml(r.bNoun)} missing from the ${escapeHtml(r.aNoun)}`}
    • `); + // What the missing metrics cost, where the pack that holds them says what + // reads them: counted once each, the alert rules by name. + const gapPack = drift ? state.pack : state.packB; + const gapReaders = readersDigest((digest?.rows || []).flatMap(row => (drift ? row.onlyInA : row.onlyInB).map(e => entryMemberArtefacts(e, gapPack, row.L)))); + if (gapReaders.read) { + const n = gapReaders.alerts.length + gapReaders.panels.length + gapReaders.rules.length; + qItems.push(`
  • ${escapeHtml(readerCounts(gapReaders))} in the ${escapeHtml(drift ? r.aNoun : r.bNoun)} read${n === 1 ? 's' : ''} ${gapReaders.read === 1 ? 'a metric' : `${gapReaders.read} metrics`} ${drift ? 'not seen live' : `the ${escapeHtml(r.aNoun)} does not hold`}${gapReaders.alerts.length ? ` — alert rule${gapReaders.alerts.length === 1 ? '' : 's'}: ${escapeHtml(alertNames(gapReaders.alerts))}` : ''}
    • `); + } if (cmp.changed) qItems.push(`
  • ${cmp.changed} shared artefact${s(cmp.changed)} with changed fields${decisionCount ? ` — ${decisionCount} affect${decisionCount === 1 ? 's' : ''} decisions (objectives, thresholds, queries, routing)` : ''}${cosmeticCount ? `${decisionCount ? ',' : ' —'} ${cosmeticCount} cosmetic` : ''}
    • `); const qualityPanel = `
    @@ -4495,7 +4542,11 @@ function renderCompareReview(digest, cmp, ctx) { const MAX_FIELDS = 8; const missingSide = drift ? 'a' : 'b'; - const missing = (digest?.rows || []).flatMap(row => (drift ? row.onlyInA : row.onlyInB).map(e => ({ L: row.L, e }))); + // What an alert rule reads leads the queue, then what anything reads + // (studio/metric-readers.mjs); each row says its readers. + const missingPack = drift ? state.pack : state.packB; + const missing = (digest?.rows || []).flatMap(row => (drift ? row.onlyInA : row.onlyInB).map(e => ({ L: row.L, e, readers: metricReaders(entryMemberArtefacts(e, missingPack, row.L)) }))) + .sort((x, y) => readerRank(x.readers) - readerRank(y.readers)); const changed = (digest?.rows || []).flatMap(row => row.drifted.map(e => ({ L: row.L, e, cost: diagDriftCost(e) }))) .sort((x, y) => y.cost.weight - x.cost.weight); @@ -4506,12 +4557,13 @@ function renderCompareReview(digest, cmp, ctx) { ? 'The declared pack promises these; the live pack shows no behavioural counterpart.' : `The ${escapeHtml(r.bNoun)} declares these; the ${escapeHtml(r.aNoun)} has no behavioural counterpart.`}

      - ${missing.slice(0, MAX_MISSING).map(({ L, e }) => { + ${missing.slice(0, MAX_MISSING).map(({ L, e, readers }) => { const art = e.artefact || (missingSide === 'a' ? e.a : e.b); + const readBy = readerLine(readers); return `
    • ${escapeHtml(L)} - ${escapeHtml(diffEntryLabel(e))} + ${escapeHtml(diffEntryLabel(e))}${readBy ? `${escapeHtml(readBy)}` : ''} ${art?.id ? `${escapeHtml(art.id)}` : ''} ${openBtn(art, L, missingSide, 'Open')}
      • `; diff --git a/studio/metric-readers.mjs b/studio/metric-readers.mjs new file mode 100644 index 00000000..d27ad078 --- /dev/null +++ b/studio/metric-readers.mjs @@ -0,0 +1,185 @@ +// studio/metric-readers.mjs +// +// What reads a metric one pack holds and the other does not. +// +// A repository scan records, on every metric it declares, where the metric +// comes from (`spec.origin_kind`: emitted by source code, the output of a +// recording rule, or only named in a query) and what in the repository reads +// it (`spec.used_by`: `alert:`, `dashboard:/`, +// `recording_rule:`). Compare lists such a metric under "Only in A" +// and stops there. These helpers answer the next question — what does its +// absence from the other pack cost? — from what the pack itself says: the +// alert rules, dashboard panels and recording rules that read it. +// +// The wording stays with what is known. "Reads a metric Pack B does not +// hold" is a fact of the two packs; "this alert can never fire" is not (a +// rule built on absent() fires exactly then), so nothing here says it. +// +// Pure and zero-import (safe under node for the headless tests); the Compare +// view in compare-view.mjs reads it. + +// The scanner's reader prefixes (tools/lib/crawler.mjs, `usedBy`). +const READER_PREFIX = { alert: 'alerts', recording_rule: 'rules', dashboard: 'panels' }; + +// Where a metric comes from, strongest claim first: a metric the code emits +// AND a query names is, for this purpose, one the code emits. +const ORIGINS = ['source-code', 'recording-rule-output', 'promql-reference']; +const ORIGIN_KEY = { 'source-code': 'code', 'recording-rule-output': 'ruleOutput', 'promql-reference': 'query' }; + +const nameOf = (art) => String(art?.spec?.name || art?.title || ''); +const uniqueSorted = (xs) => [...new Set(xs)].sort((x, y) => x.localeCompare(y)); + +/** + * The readers of one metric. `artefacts` is every artefact that stands for it + * — one, or the members of a metric family (a histogram's _bucket / _count / + * _sum series are separate artefacts and a query names the series, not the + * family). + * + * → { origin: 'code' | 'ruleOutput' | 'query' | '', alerts: [name], + * rules: [record], panels: ['/'], dashboards: [id], + * count, known } + * + * `known` is false when no artefact carries the scanner's record at all (a + * live draft's metric, a hand-written pack): nothing can be said, which is + * not the same as "nothing reads it". A recording rule is not a reader of + * its own output. + */ +export function metricReaders(artefacts) { + const arts = (Array.isArray(artefacts) ? artefacts : [artefacts]).filter(Boolean); + const own = new Set(arts.map(nameOf).filter(Boolean)); + const out = { alerts: [], rules: [], panels: [] }; + let known = false; + let origin = ''; + for (const art of arts) { + const spec = art.spec || {}; + if (Array.isArray(spec.used_by) || spec.origin_kind) known = true; + const o = String(spec.origin_kind || ''); + if (ORIGINS.includes(o) && (!origin || ORIGINS.indexOf(o) < ORIGINS.indexOf(origin))) origin = o; + for (const ref of Array.isArray(spec.used_by) ? spec.used_by : []) { + const text = String(ref || ''); + const at = text.indexOf(':'); + const bucket = READER_PREFIX[text.slice(0, at)]; + const name = text.slice(at + 1); + if (at < 1 || !bucket || !name) continue; + if (bucket === 'rules' && own.has(name)) continue; + out[bucket].push(name); + } + } + const alerts = uniqueSorted(out.alerts); + const rules = uniqueSorted(out.rules); + const panels = uniqueSorted(out.panels); + const dashboards = uniqueSorted(panels.map((p) => p.split('/')[0])); + return { + origin: ORIGIN_KEY[origin] || '', + alerts, rules, panels, dashboards, + count: alerts.length + rules.length + panels.length, + known, + }; +} + +/** + * Every metric of a group at once. `metrics` is a list of artefact lists, one + * per metric (see metricReaders). Readers are counted once however many of + * the metrics they read. + * + * → { metrics, known, read, unread, alerts, rules, panels, dashboards, + * unreadByOrigin: { code, ruleOutput, query, other } } + */ +export function readersDigest(metrics) { + const all = (metrics || []).map(metricReaders); + const known = all.filter((r) => r.known); + const unread = known.filter((r) => r.count === 0); + const unreadByOrigin = { code: 0, ruleOutput: 0, query: 0, other: 0 }; + for (const r of unread) unreadByOrigin[r.origin || 'other']++; + return { + metrics: all.length, + known: known.length, + read: known.length - unread.length, + unread: unread.length, + alerts: uniqueSorted(known.flatMap((r) => r.alerts)), + rules: uniqueSorted(known.flatMap((r) => r.rules)), + panels: uniqueSorted(known.flatMap((r) => r.panels)), + dashboards: uniqueSorted(known.flatMap((r) => r.dashboards)), + unreadByOrigin, + }; +} + +const count = (n, one, many = `${one}s`) => `${n} ${n === 1 ? one : many}`; +const andList = (parts) => (parts.length <= 1 + ? parts.join('') + : `${parts.slice(0, -1).join(', ')} and ${parts[parts.length - 1]}`); + +// "5 alert rules, 34 dashboard panels and 7 recording rules", or ''. +export function readerCounts({ alerts = [], panels = [], rules = [] } = {}) { + return andList([ + alerts.length ? count(alerts.length, 'alert rule') : '', + panels.length ? count(panels.length, 'dashboard panel') : '', + rules.length ? count(rules.length, 'recording rule') : '', + ].filter(Boolean)); +} + +// The alert rules by name: they are the readers a person acts on first. +export const ALERT_NAMES_SHOWN = 6; +export function alertNames(alerts = []) { + if (!alerts.length) return ''; + const shown = alerts.slice(0, ALERT_NAMES_SHOWN); + const more = alerts.length - shown.length; + return shown.join(', ') + (more > 0 ? ` and ${more} more` : ''); +} + +/** + * One card's line: what reads this metric, or where it comes from when + * nothing does. '' when the pack says nothing about it. + */ +export function readerLine(readers) { + if (!readers?.known) return ''; + if (readers.count) { + const parts = [ + readers.alerts.length === 1 ? `alert rule ${readers.alerts[0]}` : (readers.alerts.length ? count(readers.alerts.length, 'alert rule') : ''), + readers.panels.length ? count(readers.panels.length, 'dashboard panel') : '', + readers.rules.length === 1 ? `recording rule ${readers.rules[0]}` : (readers.rules.length ? count(readers.rules.length, 'recording rule') : ''), + ].filter(Boolean); + return `Read by ${andList(parts)}`; + } + if (readers.origin === 'code') return 'Emitted by code; nothing in this pack reads it'; + if (readers.origin === 'ruleOutput') return 'A recording rule’s output; nothing in this pack reads it'; + return ''; +} + +/** + * The group's paragraph. `holder` and `other` are the pack letters ('A', + * 'B'); `otherIsLive` says the other pack was drafted from a live system, so + * "does not hold" can be said as what it is: the live system did not report + * it. A group may hold other artefacts too (a scrape job beside the metrics): + * the numbers are of the metrics the pack says something about. + * '' when the pack says nothing about any of the metrics. + */ +export function readersSentence(digest, { holder = 'A', other = 'B', otherIsLive = false } = {}) { + if (!digest?.known) return ''; + const parts = []; + const lead = otherIsLive + ? `Pack ${other} was drafted from the live system, which reported ${digest.known === 1 ? 'no such metric' : `none of these ${digest.known} metrics`}.` + : ''; + if (lead) parts.push(lead); + if (digest.read) { + const names = alertNames(digest.alerts); + parts.push(`In Pack ${holder}, ${readerCounts(digest)} read ${digest.read === digest.known ? (digest.known === 1 ? 'it' : 'them') : `${digest.read} of them`}${digest.dashboards.length ? `, on ${count(digest.dashboards.length, 'dashboard')}` : ''}.${names ? ` Alert rules: ${names}.` : ''}`); + } + if (digest.unread) { + const u = digest.unreadByOrigin; + const what = andList([ + u.code ? `${u.code} emitted by code` : '', + u.ruleOutput ? `${count(u.ruleOutput, 'recording-rule output')}` : '', + u.query + u.other ? `${u.query + u.other} named only in a query` : '', + ].filter(Boolean)); + parts.push(`${digest.read ? `The other ${digest.unread}` : `${digest.unread === digest.known && digest.known > 1 ? 'All ' : ''}${digest.unread}`} ${digest.unread === 1 ? 'is' : 'are'} read by nothing in Pack ${holder}${what ? ` (${what})` : ''}.`); + } + return parts.join(' '); +} + +// For ordering a group: what an alert reads first, then what anything reads, +// then the rest. Unknown sorts with the rest. +export function readerRank(readers) { + if (!readers?.known || !readers.count) return 2; + return readers.alerts.length ? 0 : 1; +} diff --git a/studio/reskin.css b/studio/reskin.css index c5979178..c25fe35b 100644 --- a/studio/reskin.css +++ b/studio/reskin.css @@ -519,6 +519,8 @@ body.chrome-observa .compare-col-group.is-both { color: var(--og-muted); } body.chrome-observa .compare-col-group.is-unchecked { color: var(--og-warn); } body.chrome-observa .compare-col-group-cards { font: 400 12px/1.3 var(--og-font); color: var(--og-muted); } body.chrome-observa .compare-col-group-why { font: 400 13px/1.45 var(--og-font); color: var(--og-muted); } +body.chrome-observa .compare-side-card .card-readers { font: 500 12px/1.4 var(--og-font); color: var(--og-text-2); } +body.chrome-observa .compare-missing-readers { font: 400 12.5px/1.4 var(--og-font); color: var(--og-muted); } body.chrome-observa .compare-side-card .diff-chip.chip-unchecked { background: var(--og-warn-bg); color: var(--og-warn); } body.chrome-observa .compare-view .compare-layer-col-a .compare-side-card.is-unchecked, body.chrome-observa .compare-view .compare-layer-col-b .compare-side-card.is-unchecked { border-left-color: var(--og-warn-line); } diff --git a/studio/ux-diagnose.css b/studio/ux-diagnose.css index 75b22eb6..9539834c 100644 --- a/studio/ux-diagnose.css +++ b/studio/ux-diagnose.css @@ -342,6 +342,12 @@ tr.diag-flash > td { animation: diag-flash-bg 1.8s ease-out; } .compare-view .compare-layer-col-b .compare-side-card.is-unchecked { border-left: 3px dashed var(--ux-warn); } .compare-side-card .diff-chip.chip-unchecked { background: var(--ux-warn-bg); color: var(--ux-warn); } .compare-summary.has-unchecked { grid-template-columns: repeat(6, 1fr); } +/* What reads a metric only one pack holds (studio/metric-readers.mjs): on + the card, and beside the row in the review queue. */ +.compare-side-card .card-readers { margin-top: 6px; font: 500 12px/1.4 var(--sans); color: var(--ink-2); overflow-wrap: anywhere; } +.compare-view[data-cview="list"] .compare-side-card .card-readers, +.compare-view[data-cview="tiles"] .compare-side-card .card-readers { display: none; } +.compare-missing-readers { display: block; margin-top: 2px; font: 400 12.5px/1.4 var(--sans); color: var(--ux-ink-3); } .compare-cell.c-unchecked { border-top: 3px solid var(--ux-warn); } @media (max-width: 900px) { .compare-summary.has-unchecked { grid-template-columns: 1fr 1fr; } } diff --git a/tools/test-metric-readers.mjs b/tools/test-metric-readers.mjs new file mode 100644 index 00000000..197284d6 --- /dev/null +++ b/tools/test-metric-readers.mjs @@ -0,0 +1,156 @@ +// tools/test-metric-readers.mjs +// +// What reads a metric one pack holds and the other does not +// (studio/metric-readers.mjs): the readers a repository scan records on each +// metric, counted once per group, said in words that claim no more than the +// two packs show — and the Compare view that prints them. + +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import { crawlFiles } from './lib/crawler.mjs'; +import { adapt } from './lib/adapter.mjs'; +import { + metricReaders, readersDigest, readerCounts, alertNames, readerLine, readersSentence, readerRank, ALERT_NAMES_SHOWN, +} from '../studio/metric-readers.mjs'; + +const metric = (name, origin_kind, used_by) => ({ id: `M-${name}`, title: name, spec: { name, origin_kind, used_by } }); + +test('readers come from used_by: alert rules, dashboard panels and recording rules, each once', () => { + const r = metricReaders(metric('rabbitmq_up', 'promql-reference', [ + 'alert:RabbitMQDown', 'alert:RabbitMQDown', 'dashboard:unified/queue_depth', 'dashboard:unified/throughput', + 'dashboard:ops/queue_depth', 'recording_rule:mq:up:avg5m', 'nonsense', 'webhook:x', 'alert:', + ])); + assert.deepEqual(r.alerts, ['RabbitMQDown']); + assert.deepEqual(r.panels, ['ops/queue_depth', 'unified/queue_depth', 'unified/throughput']); + assert.deepEqual(r.dashboards, ['ops', 'unified']); + assert.deepEqual(r.rules, ['mq:up:avg5m']); + assert.equal(r.count, 5); + assert.equal(r.origin, 'query'); + assert.equal(r.known, true); +}); + +test('a recording rule is not a reader of its own output; another rule reading it is', () => { + const r = metricReaders(metric('finops:cpu:usage_5m', 'recording-rule-output', [ + 'recording_rule:finops:cpu:usage_5m', 'recording_rule:finops:cost:index_1h', 'dashboard:finops/cpu', + ])); + assert.deepEqual(r.rules, ['finops:cost:index_1h']); + assert.equal(r.origin, 'ruleOutput'); + assert.equal(r.count, 2); +}); + +test('a metric family is read through its series: the members\' readers are one set', () => { + const r = metricReaders([ + metric('order_duration_seconds', 'source-code', []), + metric('order_duration_seconds_bucket', 'promql-reference', ['alert:SlowOrders', 'dashboard:shop/latency']), + metric('order_duration_seconds_count', 'promql-reference', ['dashboard:shop/latency', 'dashboard:shop/rate']), + ]); + assert.deepEqual(r.alerts, ['SlowOrders']); + assert.deepEqual(r.panels, ['shop/latency', 'shop/rate']); + assert.equal(r.origin, 'code', 'what the code emits is emitted by code, whatever else names it'); +}); + +test('a pack that records nothing about a metric is unknown, not unread', () => { + const live = metricReaders({ id: 'METRIC-1', title: 'up', spec: { name: 'up' } }); + assert.equal(live.known, false); + assert.equal(readerLine(live), ''); + assert.equal(readerRank(live), 2); + const d = readersDigest([[{ id: 'METRIC-1', spec: { name: 'up' } }]]); + assert.deepEqual([d.metrics, d.known, d.read, d.unread], [1, 0, 0, 0]); + assert.equal(readersSentence(d), ''); +}); + +test('the digest counts a reader once however many metrics it reads', () => { + const d = readersDigest([ + [metric('container_cpu', 'promql-reference', ['dashboard:k8s/cpu', 'recording_rule:finops:cpu'])], + [metric('container_mem', 'promql-reference', ['dashboard:k8s/cpu', 'dashboard:k8s/mem'])], + [metric('rabbitmq_up', 'promql-reference', ['alert:RabbitMQDown'])], + [metric('orders_failed_total', 'promql-reference', ['alert:OrderFailures'])], + [metric('kx_trades_today', 'source-code', [])], + [metric('finops:cpu', 'recording-rule-output', ['recording_rule:finops:cpu'])], + ]); + assert.deepEqual([d.metrics, d.known, d.read, d.unread], [6, 6, 4, 2]); + assert.deepEqual(d.alerts, ['OrderFailures', 'RabbitMQDown']); + assert.deepEqual(d.panels, ['k8s/cpu', 'k8s/mem']); + assert.deepEqual(d.dashboards, ['k8s']); + assert.deepEqual(d.rules, ['finops:cpu']); + assert.deepEqual(d.unreadByOrigin, { code: 1, ruleOutput: 1, query: 0, other: 0 }); + assert.equal(readerCounts(d), '2 alert rules, 2 dashboard panels and 1 recording rule'); +}); + +test('the words: counts, names, and no claim the packs do not support', () => { + assert.equal(readerCounts({ alerts: ['A'] }), '1 alert rule'); + assert.equal(readerCounts({ alerts: ['A'], rules: ['r'] }), '1 alert rule and 1 recording rule'); + assert.equal(readerCounts({}), ''); + const many = Array.from({ length: ALERT_NAMES_SHOWN + 3 }, (_, i) => `Alert${i}`); + assert.equal(alertNames(many), `${many.slice(0, ALERT_NAMES_SHOWN).join(', ')} and 3 more`); + assert.equal(alertNames([]), ''); + + assert.equal(readerLine(metricReaders(metric('rabbitmq_up', 'promql-reference', ['alert:RabbitMQDown']))), 'Read by alert rule RabbitMQDown'); + assert.equal(readerLine(metricReaders(metric('c', 'promql-reference', ['alert:A', 'alert:B', 'dashboard:d/p', 'recording_rule:r']))), + 'Read by 2 alert rules, 1 dashboard panel and recording rule r'); + assert.equal(readerLine(metricReaders(metric('kx_trades_today', 'source-code', []))), 'Emitted by code; nothing in this pack reads it'); + assert.equal(readerLine(metricReaders(metric('x:y', 'recording-rule-output', ['recording_rule:x:y']))), 'A recording rule’s output; nothing in this pack reads it'); + assert.equal(readerLine(metricReaders(metric('q', 'promql-reference', []))), ''); + + const d = readersDigest([ + [metric('rabbitmq_up', 'promql-reference', ['alert:RabbitMQDown'])], + [metric('container_cpu', 'promql-reference', ['dashboard:k8s/cpu'])], + [metric('kx_trades_today', 'source-code', [])], + ]); + const live = readersSentence(d, { holder: 'A', other: 'B', otherIsLive: true }); + assert.equal(live, 'Pack B was drafted from the live system, which reported none of these 3 metrics. ' + + 'In Pack A, 1 alert rule and 1 dashboard panel read 2 of them, on 1 dashboard. Alert rules: RabbitMQDown. ' + + 'The other 1 is read by nothing in Pack A (1 emitted by code).'); + const repo = readersSentence(d, { holder: 'B', other: 'A' }); + assert.ok(!/live system/.test(repo) && /^In Pack B, /.test(repo), 'between two repository packs nothing is said about a live system'); + for (const s of [live, repo]) assert.ok(!/never fire|cannot fire|broken|blind/i.test(s), 'no verdict the packs do not show'); + + const allRead = readersSentence(readersDigest([[metric('a', 'promql-reference', ['alert:A'])], [metric('b', 'promql-reference', ['alert:A'])]])); + assert.equal(allRead, 'In Pack A, 1 alert rule read them. Alert rules: A.'); + const mixed = readersSentence(readersDigest([[metric('a', 'promql-reference', ['alert:A'])], [{ id: 'SCRAPE-1', spec: { job: 'x' } }]]), { otherIsLive: true }); + assert.equal(mixed, 'Pack B was drafted from the live system, which reported no such metric. In Pack A, 1 alert rule read it. Alert rules: A.', + 'an artefact the pack says nothing about is not counted among the metrics'); + const noneRead = readersSentence(readersDigest([[metric('a', 'source-code', [])], [metric('b', 'source-code', [])]])); + assert.equal(noneRead, 'All 2 are read by nothing in Pack A (2 emitted by code).'); +}); + +test('rank: what an alert reads first, then what anything reads, then the rest', () => { + assert.equal(readerRank(metricReaders(metric('a', 'promql-reference', ['alert:A', 'dashboard:d/p']))), 0); + assert.equal(readerRank(metricReaders(metric('b', 'promql-reference', ['dashboard:d/p']))), 1); + assert.equal(readerRank(metricReaders(metric('c', 'source-code', []))), 2); +}); + +test('a scanned repository carries the record these helpers read', () => { + const { canonical } = crawlFiles({ + 'shop/k8s/prometheus/rules.yaml': [ + 'groups:', + '- name: shop', + ' rules:', + ' - record: shop:orders:rate5m', + ' expr: sum(rate(shop_orders_total[5m]))', + ' - alert: ShopQueueDown', + ' expr: shop_queue_up == 0', + ' for: 5m', + ' labels: {severity: critical}', + '', + ].join('\n'), + }, { repoName: 'shop', environment: 'prod', now: '2026-10-02T08:00:00.000Z' }); + const pack = adapt(canonical); + const metrics = (pack.layers.L2 || []).filter(a => a.spec?.origin_kind); + const byName = Object.fromEntries(metrics.map(a => [a.spec.name, a])); + assert.deepEqual(metricReaders(byName.shop_queue_up).alerts, ['ShopQueueDown'], 'the alert rule that reads a metric is on the metric'); + assert.deepEqual(metricReaders(byName.shop_orders_total).rules, ['shop:orders:rate5m'], 'so is the recording rule'); + assert.equal(metricReaders(byName['shop:orders:rate5m']).count, 0, 'and a rule does not read its own output'); +}); + +test('Compare prints the readers: the group paragraph, each card and the summary', () => { + const src = fs.readFileSync(new URL('../studio/compare-view.mjs', import.meta.url), 'utf8'); + assert.match(src, /from '\.\/metric-readers\.mjs'/, 'the view reads the helper'); + const col = src.slice(src.indexOf('function renderCompareLayerColumn'), src.indexOf('function compareKeyOf')); + assert.match(col, /readersSentence\(/, 'the Only-in group says what reads its metrics'); + assert.match(col, /readerRank\(/, 'and leads with what an alert reads'); + const card = src.slice(src.indexOf('function renderCompareCard'), src.indexOf('function renderCompareCard') + 4000); + assert.match(card, /readerLine\(/, 'a card says what reads it'); + assert.match(src, /compare-readers-item/, 'the summary names the readers among the quality gaps'); +}); From 231a7a0581a34bf005de50549b268dee8373306d Mon Sep 17 00:00:00 2001 From: Carlos Montero Date: Fri, 2 Oct 2026 09:29:52 +0200 Subject: [PATCH 007/550] docs(changelog): pick a folder walks the repository itself Co-Authored-By: Claude Fable 5.1 --- docs/CHANGELOG.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 99de209b..7e7aa514 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -2,6 +2,12 @@ ## Unreleased +### Pick a folder walks the repository itself: its dependencies are never listed +**Scan a repo → pick a folder** used a folder input, which hands the page every file under the folder before the scan rule can apply: 105,810 files in 11,874 folders for a repository whose scan reads 631 files in 157. Since 0.5.0 the rule kept the rest from being *read*; the browser still listed them all before the page saw the first one. +- **`walkScanFolder`** (`tools/lib/crawler.mjs`) walks a directory handle with the shared rule (`scanSkipsName`, `SCAN_EXT`) and does not enter a skipped folder, so `node_modules`, `.git`, build output and agent worktrees are never asked for their entries. Paths are rooted at the picked folder's name, as `webkitRelativePath` is, in name order; the picked folder is entered whatever it is called. +- **The studio** uses `showDirectoryPicker` where the browser has it (Chromium, on localhost or https) and keeps the folder input as the fallback: Firefox, Safari, plain http on a LAN address, a frame the API is refused in. The staged line counts the files as they are read. *Drop a folder* already walked this way. +- **Tests.** `tools/test-crawl.mjs`: the walk over a stand-in handle tree — what it reads, in what order, and that a skipped folder's entries are never asked for — and the studio's use of it with both fallbacks, pinned in source. + ### Fixed: one scan or one live draft is one pack in the catalogue **Scan a repo → use this pack** and **New from live → use this pack** each left two entries in the pack catalogue: *krystalinex-core (scanned)* and *krystalinex-core*, *production-live (live MCP draft)* and *production-live* — so the home screen read *krystalinex-core · 2 packs* and every live-discovered service *2 live drafts*. The route that answers a scan or a draft registers the pack (`registered.id` — `POST /api/crawl`, `/api/crawl-github`, `/api/draft-from-mcp`); the studio's adopt then posted the YAML to `POST /api/validate`, which registers what it validates, under a second id. Studio only (no server, engine or output change): - **Adopt selects the registered entry.** `registeredOrValidated` (`studio/api.mjs`) answers `registered.id` while the catalogue lists it, and goes through `/api/validate` only when the route registered nothing (the pack failed validation — its errors are the toast) or the entry has gone since (a reset). `adoptCrawlResult` and `adoptDraftFromMcpResult` (`studio/app.mjs`) use it, so the folder scan, the GitHub-URL scan, the *New from live* panel and the home screen's **Render the manifest** all load the entry under its label (*… (scanned)*, *owner/repo (repo scan)*, *… (live MCP draft)*). From 06b37cefbc3aaedf7a1a64708a19dd93ec785416 Mon Sep 17 00:00:00 2001 From: Carlos Montero Date: Fri, 2 Oct 2026 09:49:39 +0200 Subject: [PATCH 008/550] fix(studio): the account chip keeps its caret, closes when Advanced opens, and the chrome is measured once MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review of this PR against current develop: - develop already measures the chrome's height (syncContextBarHeight, with the re-skin). trackChromeHeight was a second observer writing the same property; it is gone, and the changelog no longer claims the fix. - the laptop-width cap was on the button (max-width: 10ch, border-box with the re-skin's padding): the seeded admin read "Ad…" and the caret was clipped. The cap is on the name span; the glyph and the caret stay. - the Advanced toggle stops its click from bubbling, so the account menu stayed open under the Advanced menu. Its outside-click closer listens in the capture phase. - .observa-actions is labelled for what it holds. server/test-authz.mjs pins one measurer, the name cap and the closer. Co-Authored-By: Claude Fable 5.1 --- docs/CHANGELOG.md | 2 +- server/test-authz.mjs | 22 +++++++++++++--------- studio/app.mjs | 25 ++++--------------------- studio/ux.css | 7 +++---- 4 files changed, 21 insertions(+), 35 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 2b3bb26d..4c056c52 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -3,7 +3,7 @@ ## Unreleased ### Fixed: the account menu is reachable on every screen -Studio only. The account menu is on every screen: it moved from the context bar — hidden on the home and Build screens, so until a pack was open a signed-in user had no way to sign out or change a password — to the OBSERVA bar beside Advanced, which every screen shows. The same menu everywhere (who you are, change password…, sign out my other sessions, sign out), still none without sign-in; the ORG chip is unchanged (the OBSERVA bar already carries its own on every screen). Beside the tabs the chip's name gives way — ten characters at a laptop width, the glyph alone at phone width (its title and menu say it in full) — so the tab titles keep their room and the popover stays on screen; and the chrome's height is now measured into `--observa-chrome-h` (as the context bar's already was), so the sticky context bar sits flush under it whatever wraps, where the constant had left it 15px under a two-line chrome. `server/test-authz.mjs` pins the mount, the measurement, the give-way rules and that no mode rule hides its bar. +Studio only. The account menu is on every screen: it moved from the context bar — hidden on the home and Build screens, so until a pack was open a signed-in user had no way to sign out or change a password — to the OBSERVA bar beside Advanced, which every screen shows. The same menu everywhere (who you are, change password…, sign out my other sessions, sign out), still none without sign-in; the ORG chip is unchanged (the OBSERVA bar already carries its own on every screen). Beside the tabs the chip's name gives way — ten characters at a laptop width, the glyph alone at phone width (its title and menu say it in full); the glyph and the caret that marks it a menu always show — so the tab titles keep their room and the popover stays on screen. Opening Advanced closes it (that toggle stops its click from bubbling, so the two menus could sit open one over the other). `server/test-authz.mjs` pins the mount, the give-way rules, the closer and that no mode rule hides its bar. ### Fixed: one scan or one live draft is one pack in the catalogue **Scan a repo → use this pack** and **New from live → use this pack** each left two entries in the pack catalogue: *krystalinex-core (scanned)* and *krystalinex-core*, *production-live (live MCP draft)* and *production-live* — so the home screen read *krystalinex-core · 2 packs* and every live-discovered service *2 live drafts*. The route that answers a scan or a draft registers the pack (`registered.id` — `POST /api/crawl`, `/api/crawl-github`, `/api/draft-from-mcp`); the studio's adopt then posted the YAML to `POST /api/validate`, which registers what it validates, under a second id. Studio only (no server, engine or output change): - **Adopt selects the registered entry.** `registeredOrValidated` (`studio/api.mjs`) answers `registered.id` while the catalogue lists it, and goes through `/api/validate` only when the route registered nothing (the pack failed validation — its errors are the toast) or the entry has gone since (a reset). `adoptCrawlResult` and `adoptDraftFromMcpResult` (`studio/app.mjs`) use it, so the folder scan, the GitHub-URL scan, the *New from live* panel and the home screen's **Render the manifest** all load the entry under its label (*… (scanned)*, *owner/repo (repo scan)*, *… (live MCP draft)*). diff --git a/server/test-authz.mjs b/server/test-authz.mjs index fe12ebc3..f3a16d0d 100644 --- a/server/test-authz.mjs +++ b/server/test-authz.mjs @@ -589,18 +589,22 @@ test('the account menu mounts in the one bar every screen shows — not the cont const boot = functionSource(src, 'boot'); const chromeAt = boot.indexOf('installObservaChrome();'); assert.ok(chromeAt >= 0 && chromeAt < boot.indexOf('setupIdentityChip();'), 'boot mounts the chrome before the chip'); - // The chrome's height is measured into --observa-chrome-h (the context bar - // and the sticky strips pin under it): beside the account menu the tab - // titles wrap at a laptop width, and the constant alone left the context - // bar under the chrome by the difference. The chip's name gives way there - // (ten characters) and at phone width (the glyph), so the tabs keep their - // room and the popover stays on screen. - assert.match(src, /syncObservaTabs\(\);\s*trackChromeHeight\(hdr\);/, 'the chrome is measured once its tabs are in it'); - assert.match(functionSource(src, 'trackChromeHeight'), /setProperty\('--observa-chrome-h', `\$\{Math\.ceil\(hdr\.getBoundingClientRect\(\)\.height\)\}px`\)[\s\S]*new ResizeObserver\(apply\)\.observe\(hdr\);[\s\S]*addEventListener\('resize', apply\);[\s\S]*apply\(\);\s*\}$/); + // Beside the tabs the chip's name gives way — ten characters at a laptop + // width, the glyph at phone width — so the tabs keep their room and the + // popover stays on screen. The cap is on the name, never the button: the + // glyph and the caret that marks it a menu stay. (The chrome's height is + // measured by syncContextBarHeight, so a chrome the chip makes taller still + // has the context bar flush under it.) + assert.match(functionSource(src, 'syncContextBarHeight'), /setProperty\('--observa-chrome-h',/, 'the chrome\'s height is measured, not assumed'); + assert.doesNotMatch(src, /trackChromeHeight/, 'by one measurer, not two'); + // Its outside-click closer listens on the way down: the Advanced toggle + // beside it stops its click from bubbling, and both menus stayed open. + assert.match(fn, /document\.addEventListener\('click', \(e\) => \{ if \(!chip\.contains\(e\.target\)\) setOpen\(false\); \}, true\);/, 'opening Advanced closes the account menu'); assert.match(fn, /\$\{escapeHtml\(me\.name \|\| me\.email \|\| me\.sub\)\}<\/span>/, 'the name is its own span, so a stylesheet can let it give way'); const ux = readFileSync(join(STUDIO, 'ux.css'), 'utf8'); const at1280 = ux.indexOf('@media (max-width: 1280px) {'); - const cap = ux.indexOf('.observa-actions .hdr-user-btn { max-width: 10ch; }'); + const cap = ux.indexOf('.observa-actions .hdr-user-name { display: inline-block; max-width: 10ch;'); + assert.ok(!ux.includes('.observa-actions .hdr-user-btn { max-width'), 'the button itself is not capped: that clipped the caret'); const at720 = ux.search(/@media \(max-width: 720px\) \{\r?\n\s*\.observa-actions \.hdr-user-name \{ position: absolute; width: 1px;/); assert.ok(at1280 >= 0 && cap > at1280 && at720 > cap, 'the chip gives way at a laptop width (ten characters) and at phone width (its glyph)'); diff --git a/studio/app.mjs b/studio/app.mjs index 3d4b6c25..b4ff1590 100644 --- a/studio/app.mjs +++ b/studio/app.mjs @@ -1304,23 +1304,6 @@ function trackContextBarHeight() { syncContextBarHeight(); } -// The context bar and those strips pin under the chrome by -// --observa-chrome-h. ux.css holds its single-row height; the chrome grows -// when its tab titles wrap (a laptop width, with the account menu beside -// them), and a constant would leave the context bar under the chrome by -// the difference — so measure it, as the context bar is. -function trackChromeHeight(hdr) { - const apply = () => { - document.body.style.setProperty('--observa-chrome-h', `${Math.ceil(hdr.getBoundingClientRect().height)}px`); - }; - if (typeof ResizeObserver === 'function') new ResizeObserver(apply).observe(hdr); - // A tab in the background gets no resize notifications until it is shown - // again — and not always then. - window.addEventListener('resize', apply); - document.addEventListener('visibilitychange', apply); - apply(); -} - function installObservaChrome() { if (document.querySelector('.observa-hdr')) return; document.body.classList.add('chrome-observa'); @@ -1376,7 +1359,7 @@ function installObservaChrome() { -
      +
      @@ -2898,7 +2898,7 @@ function renderHomeView() {
      @@ -2908,7 +2908,7 @@ function renderHomeView() { +
    diff --git a/studio/otlp-view.mjs b/studio/otlp-view.mjs index 625803d5..45a97576 100644 --- a/studio/otlp-view.mjs +++ b/studio/otlp-view.mjs @@ -209,7 +209,7 @@ function renderOtlpBody(host, canonical, layered) {
    ${hasOtlpReceiver ? 'pass' : 'fail'}
    - Spec v1.3 §3 — every pack MUST declare an otlp receiver. + Spec v1.4 §3 — every pack MUST declare an otlp receiver. The OTLP-out column is informational: many production stacks intentionally use native protocols downstream (Prometheus remote-write, Loki native, Tempo OTLP) for backend efficiency. diff --git a/studio/schema-view.mjs b/studio/schema-view.mjs index 1c1310c0..b3f10c7a 100644 --- a/studio/schema-view.mjs +++ b/studio/schema-view.mjs @@ -31,7 +31,7 @@ export function renderSchemaView(host) { sectionHead.className = 'section-head'; sectionHead.innerHTML = ` SCHEMA - Canonical manifest · ObservabilityPack v1.3${focusBadge} + Canonical manifest · ObservabilityPack v1.4${focusBadge} ${escapeHtml(pack?.meta?.binding || 'unknown binding')} `; wrap.appendChild(sectionHead); @@ -77,13 +77,13 @@ export function renderSchemaView(host) { ✓ Validates against the canonical - ObservabilityPack v1.3 JSON Schema. + ObservabilityPack v1.4 JSON Schema. Packs that fail validation never appear in the catalog.
    - Schema source: vendor/observability-pack-spec/v1.3/observability-pack.schema.json · - Spec document + Schema source: vendor/observability-pack-spec/v1.4/observability-pack.schema.json · + Spec document
    `; wrap.appendChild(validation); diff --git a/tools/fixtures/build/orders-api.tier-2.instantiate.json b/tools/fixtures/build/orders-api.tier-2.instantiate.json index c4d003b4..81e2ecf2 100644 --- a/tools/fixtures/build/orders-api.tier-2.instantiate.json +++ b/tools/fixtures/build/orders-api.tier-2.instantiate.json @@ -871,7 +871,7 @@ } } }, - "canonicalYaml": "# ObservabilityPack orders-api — built from the library (kafka@1.0.0,http-service@1.0.0) at tier-2\n# Todos: 21 (metadata.annotations library.todo.*). Spec v1.3.\napiVersion: observability.platform/v1\nkind: ObservabilityPack\nmetadata:\n name: orders-api\n version: 0.1.0\n binding: otel-grafanalabs\n owners:\n - team-orders\n imports:\n - ref: platform/std-budget-policy@2.1\n bindings:\n service: orders-api\n environments:\n - prod\n criticality: tier-2\n labels:\n source: library\n tier: tier-2\n library.entries: kafka,http-service\n annotations:\n library.source: kafka@1.0.0,http-service@1.0.0\n library.format: v1\n library.tier: tier-2\n library.environment: prod\n library.toggles: slos,policy,routes,dashboards,validation\n library.slis: kafka_broker_availability,kafka_consumer_group_lag_seconds,kafka_partition_replica_health,kafka_produce_latency_p99,kafka_fetch_latency_p99,http_service_availability,http_service_latency_p99\n library.params: \"{}\"\n library.evidence: kafka:recorded-live,http-service:semconv\n library.evidence.slis.kafka_broker_availability: \"recorded-live: kafka.md §1.4 — up{job=\\\"kafka-broker\\\"} = 1 on the lab (one target)\"\n library.evidence.slis.kafka_consumer_group_lag_seconds: \"recorded-live: kafka.md §1.3-1.4 — kafka_exporter 1.10.0 families and the expression executed (two consumer groups)\"\n library.evidence.slis.kafka_partition_replica_health: \"recorded-live: kafka.md §1.3-1.4 — both families present and the bool form executed; an ISR shortfall itself was not observable on RF 1 (§1.6)\"\n library.evidence.slis.kafka_produce_latency_p99: \"recorded-live: kafka.md §1.2, §1.4 — kafka_network_requestmetrics_totaltimems{request=\\\"Produce\\\",quantile=\\\"0.99\\\"} read from the JMX exposition and the expression executed\"\n library.evidence.slis.kafka_fetch_latency_p99: \"recorded-live: kafka.md §1.2, §1.4 — the LocalTimeMs quantile gauge and the expression executed; the TotalTimeMs trap measured\"\n library.evidence.slis.http_service_availability: \"semconv: OTel semconv v1.27.0 HTTP metrics + Prometheus compatibility naming; spelling not verified live in this repo (evidence.notes)\"\n library.evidence.slis.http_service_latency_p99: \"semconv: OTel semconv v1.27.0 HTTP metrics + Prometheus compatibility naming; spelling not verified live in this repo (evidence.notes)\"\n library.todoCount: \"21\"\n library.todo.alerting.routes[0]: \"channels.0.msteams: Chat channel for SEV1/SEV2: placeholder '#orders-api-oncall' (param oncall_channel) — The Teams channel the SEV1 and SEV2 routes post to. · channels.1.voice: Voice (pager) target for SEV1: placeholder 'pagerduty://orders-api' (param pager_service) — The voice channel of the SEV1 route (tier-2 and tier-1; L4.MUST.tier1_voice_route).\"\n library.todo.alerting.routes[1]: \"channels.0.msteams: Chat channel for SEV1/SEV2: placeholder '#orders-api-oncall' (param oncall_channel) — The Teams channel the SEV1 and SEV2 routes post to.\"\n library.todo.alerting.routes[2]: \"channels.0.msteams: Chat channel for SEV3: placeholder '#orders-api-team' (param team_channel) — The Teams channel the SEV3 route posts to.\"\n library.todo.baselines: \"mttd_target_p50: MTTD / MTTR targets are the tier-2 defaults, not measured: set them from the service's incident history\"\n library.todo.pipelines.exporters.logs: \"endpoints.0: Loki OTLP ingest endpoint: placeholder 'http://loki:3100/otlp' (param logs_otlp_endpoint) — Where the collector pushes logs over OTLP/HTTP (pipelines.exporters.logs).\"\n library.todo.pipelines.exporters.metrics: \"endpoint: Prometheus remote-write URL: placeholder 'http://prometheus:9090/api/v1/write' (param remote_write_url) — Where the collector remote-writes metrics (pipelines.exporters.metrics).\"\n library.todo.pipelines.exporters.traces: \"endpoint: Tempo OTLP ingest endpoint: placeholder 'tempo:4317' (param traces_otlp_endpoint) — Where the collector pushes traces over OTLP/gRPC (pipelines.exporters.traces).\"\n library.todo.pipelines.receivers[1]: \"scrape_configs.0.static_configs.0.targets.0: Broker JMX exporter targets: placeholder 'kafka-broker.kafka:9404' (param kafka.broker_targets) — host:port of every broker's JMX Prometheus exporter (comma-separated in the scrape config). · scrape_configs.1.static_configs.0.targets.0: kafka_exporter target: placeholder 'kafka-exporter.kafka:9308' (param kafka.exporter_target) — host:port of kafka_exporter.\"\n library.todo.remediation[0]: \"runbook: write the runbook file:///broker-down.md\"\n library.todo.remediation[1]: \"runbook: write the runbook file:///under-replicated-partitions.md\"\n library.todo.storage.logs: \"version: Loki version you run: placeholder '3.7' (param loki_version) — The version of the Loki that stores the logs (telemetry.backends logs-loki version.declared, storage.logs.version); min stays 3.0.\"\n library.todo.storage.metrics: \"version: Prometheus version you run: placeholder '3.14' (param prometheus_version) — The version of the Prometheus that stores the metrics (telemetry.backends metrics-prom version.declared, storage.metrics.version); min stays 2.53, the floor the expressions are known to work from, and tier-1 enforces the block. · remote_write.0.url: Prometheus remote-write URL: placeholder 'http://prometheus:9090/api/v1/write' (param remote_write_url) — Where the collector remote-writes metrics (pipelines.exporters.metrics).\"\n library.todo.storage.traces: \"version: Tempo version you run: placeholder '2.10' (param tempo_version) — The version of the Tempo that stores the traces (telemetry.backends traces-tempo version.declared, storage.traces.version); min stays 2.5.\"\n library.todo.telemetry.backends.logs-loki: \"version.declared: Loki version you run: placeholder '3.7' (param loki_version) — The version of the Loki that stores the logs (telemetry.backends logs-loki version.declared, storage.logs.version); min stays 3.0. · endpoints.0: Loki endpoint: placeholder 'http://loki:3100' (param logs_endpoint) — Where the logs backend answers (telemetry.backends logs-loki).\"\n library.todo.telemetry.backends.metrics-prom: \"version.declared: Prometheus version you run: placeholder '3.14' (param prometheus_version) — The version of the Prometheus that stores the metrics (telemetry.backends metrics-prom version.declared, storage.metrics.version); min stays 2.53, the floor the expressions are known to work from, and tier-1 enforces the block. · endpoints.0: Prometheus query endpoint: placeholder 'http://prometheus:9090' (param metrics_endpoint) — Where the metrics backend answers PromQL (telemetry.backends metrics-prom).\"\n library.todo.telemetry.backends.traces-tempo: \"version.declared: Tempo version you run: placeholder '2.10' (param tempo_version) — The version of the Tempo that stores the traces (telemetry.backends traces-tempo version.declared, storage.traces.version); min stays 2.5. · endpoints.0: Tempo endpoint: placeholder 'http://tempo:3200' (param traces_endpoint) — Where the traces backend answers (telemetry.backends traces-tempo).\"\n library.todo.validation.chaos_experiments.kafka-broker-network-partition: \"target: Broker workload (chaos target): placeholder 'kafka-broker' (param kafka.broker_workload) — The broker workload (StatefulSet / pods) the chaos experiments target.\"\n library.todo.validation.chaos_experiments.kafka-broker-pod-kill: \"target: Broker workload (chaos target): placeholder 'kafka-broker' (param kafka.broker_workload) — The broker workload (StatefulSet / pods) the chaos experiments target.\"\n library.todo.validation.synthetic_checks.http-service-health-probe: \"target: Health endpoint URL: placeholder 'http://orders-api:8080/health' (param http-service.health_url) — The URL the blackbox probe hits to prove the service answers.\"\n library.todo.validation.synthetic_checks.kafka-consumer-group-health: \"target: Bootstrap servers: placeholder 'kafka.kafka:9092' (param kafka.bootstrap) — The bootstrap address the synthetic produce/consume canary connects to.\"\n library.todo.validation.synthetic_checks.kafka-produce-consume-canary: \"target: Bootstrap servers: placeholder 'kafka.kafka:9092' (param kafka.bootstrap) — The bootstrap address the synthetic produce/consume canary connects to.\"\nspec:\n otel:\n semconv: 1.27.0\n resource_attributes:\n required:\n - service.name\n - service.namespace\n - service.version\n - deployment.environment\n custom:\n - kafka.cluster.id\n - kafka.broker.id\n sdk:\n languages:\n - java\n - go\n - node\n sampling:\n policy: parentbased_traceidratio\n ratio: 0.1\n propagators:\n - tracecontext\n - baggage\n log_correlation: true\n telemetry:\n backends:\n - id: metrics-prom\n signal: metrics\n product: prometheus\n version:\n declared: \"3.14\"\n min: \"2.53\"\n gating: warn\n endpoints:\n - http://prometheus:9090\n default: true\n - id: logs-loki\n signal: logs\n product: loki\n version:\n declared: \"3.7\"\n min: \"3.0\"\n gating: warn\n endpoints:\n - http://loki:3100\n default: true\n - id: traces-tempo\n signal: traces\n product: tempo\n version:\n declared: \"2.10\"\n min: \"2.5\"\n gating: warn\n endpoints:\n - http://tempo:3200\n default: true\n environments:\n prod:\n criticality: tier-2\n backends:\n metrics: metrics-prom\n logs: logs-loki\n traces: traces-tempo\n slis:\n - id: kafka_broker_availability\n type: ratio\n description: Fraction of brokers reporting ready vs. the cluster's declared broker count.\n semconv_metric: kafka.broker.up\n good: sum(up{job=\"kafka-broker\"} == bool 1)\n total: count(up{job=\"kafka-broker\"})\n - id: kafka_consumer_group_lag_seconds\n type: threshold\n description: Maximum consumer-group lag in seconds equivalent, per consumer group (message-count lag over the group's rolling consumption rate).\n semconv_metric: messaging.kafka.consumer.lag\n query: |\n max by (consumergroup) (\n kafka_consumergroup_lag /\n (rate(kafka_consumergroup_current_offset[5m]) > 0)\n )\n threshold: 60\n unit: seconds\n - id: kafka_partition_replica_health\n type: ratio\n description: Fraction of partitions whose in-sync replica count equals their replica count (== bool, one per healthy partition).\n semconv_metric: kafka.partition.in_sync_replicas\n good: sum(kafka_topic_partition_in_sync_replica == bool kafka_topic_partition_replicas)\n total: count(kafka_topic_partition_replicas)\n - id: kafka_produce_latency_p99\n type: threshold\n description: 99th-percentile total time of Produce requests on the broker, in seconds (queue, local, replication wait, response send) — what a producer waits for an ack.\n semconv_metric: kafka.broker.request.latency\n query: max(kafka_network_requestmetrics_totaltimems{request=\"Produce\",quantile=\"0.99\"}) / 1000\n threshold: 0.1\n unit: seconds\n - id: kafka_fetch_latency_p99\n type: threshold\n description: 99th-percentile local (broker processing) time of FetchConsumer requests, in seconds.\n semconv_metric: kafka.broker.request.latency\n query: max(kafka_network_requestmetrics_localtimems{request=\"FetchConsumer\",quantile=\"0.99\"}) / 1000\n threshold: 0.05\n unit: seconds\n - id: http_service_availability\n type: ratio\n description: Fraction of HTTP requests not answered with a 5xx status, from the request-duration histogram's _count series (semconv http.server.request.duration, label http_response_status_code).\n semconv_metric: http.server.request.duration\n good: sum(rate(http_server_request_duration_seconds_count{job=\"orders-api\",http_response_status_code!~\"5..\"}[5m]))\n total: sum(rate(http_server_request_duration_seconds_count{job=\"orders-api\"}[5m]))\n - id: http_service_latency_p99\n type: threshold\n description: 99th-percentile server-side request duration in seconds, from the semconv http.server.request.duration histogram.\n semconv_metric: http.server.request.duration\n query: |\n histogram_quantile(0.99,\n sum by (le)(rate(http_server_request_duration_seconds_bucket{job=\"orders-api\"}[5m])))\n threshold: 0.5\n unit: seconds\n slos:\n - id: kafka_broker_availability_99_9\n sli: kafka_broker_availability\n objective: 0.999\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_consumer_group_lag_seconds_99\n sli: kafka_consumer_group_lag_seconds\n objective: 0.99\n window: 7d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_partition_replica_health_99_95\n sli: kafka_partition_replica_health\n objective: 0.9995\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_produce_latency_p99_99\n sli: kafka_produce_latency_p99\n objective: 0.99\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_fetch_latency_p99_99\n sli: kafka_fetch_latency_p99\n objective: 0.99\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: http_service_availability_99_5\n sli: http_service_availability\n objective: 0.995\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: http_service_latency_p99_99\n sli: http_service_latency_p99\n objective: 0.99\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n pipelines:\n receivers:\n - name: otlp\n protocols:\n - grpc\n - http\n endpoint: 0.0.0.0:4317\n - name: prometheus\n scrape_configs:\n - job_name: kafka-broker\n scrape_interval: 30s\n static_configs:\n -\n targets:\n - kafka-broker.kafka:9404\n - job_name: kafka-exporter\n scrape_interval: 30s\n static_configs:\n -\n targets:\n - kafka-exporter.kafka:9308\n processors:\n - name: memory_limiter\n limit_percentage: 80\n - name: batch\n timeout: 10s\n - name: resource\n attributes:\n - key: deployment.environment\n value: prod\n action: upsert\n exporters:\n metrics:\n kind: prometheusremotewrite\n endpoint: http://prometheus:9090/api/v1/write\n external_labels:\n service: orders-api\n logs:\n kind: otlphttp\n endpoints:\n - http://loki:3100/otlp\n traces:\n kind: otlp\n endpoint: tempo:4317\n storage:\n metrics:\n backend: prometheus\n version: \"3.14\"\n retention: 90d\n remote_write:\n - url: http://prometheus:9090/api/v1/write\n logs:\n backend: loki\n version: \"3.7\"\n retention: 30d\n traces:\n backend: tempo\n version: \"2.10\"\n retention: 7d\n sampling: head-based\n queries:\n recording_rules:\n - name: orders_api:kafka_broker_availability:ratio_5m\n expr: ref:slis.kafka_broker_availability\n interval: 30s\n - name: orders_api:kafka_consumer_group_lag_seconds:value_5m\n expr: ref:slis.kafka_consumer_group_lag_seconds\n interval: 30s\n - name: orders_api:kafka_partition_replica_health:ratio_5m\n expr: ref:slis.kafka_partition_replica_health\n interval: 30s\n - name: orders_api:kafka_produce_latency_p99:value_5m\n expr: ref:slis.kafka_produce_latency_p99\n interval: 30s\n - name: orders_api:kafka_fetch_latency_p99:value_5m\n expr: ref:slis.kafka_fetch_latency_p99\n interval: 30s\n - name: orders_api:http_service_availability:ratio_5m\n expr: ref:slis.http_service_availability\n interval: 30s\n - name: orders_api:http_service_latency_p99:value_5m\n expr: ref:slis.http_service_latency_p99\n interval: 30s\n derived_views:\n - id: golden_signals_orders_api\n bind: ref:platform/golden-signals-view\n params:\n service: orders-api\n - id: kafka_per_topic_throughput\n bind: ref:platform/per-resource-rollup\n params:\n metric: kafka_server_brokertopicmetrics_messagesin_total{topic!=\"\"}\n by:\n - topic\n - id: kafka_per_consumergroup_lag\n bind: ref:platform/per-resource-rollup\n params:\n sli: kafka_consumer_group_lag_seconds\n by:\n - consumergroup\n - id: http_service_per_route_latency\n bind: ref:platform/per-resource-rollup\n params:\n sli: http_service_latency_p99\n by:\n - http_route\n - id: http_service_per_route_traffic\n bind: ref:platform/per-resource-rollup\n params:\n metric: http_server_request_duration_seconds_count\n by:\n - http_route\n - http_request_method\n dashboards:\n - id: orders-api-overview\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n source: file://dashboards/orders-api-overview.json\n panel_bindings:\n - panel: slis-kafka-broker-availability\n binds_to: slis.kafka_broker_availability\n - panel: slis-kafka-consumer-group-lag-seconds\n binds_to: slis.kafka_consumer_group_lag_seconds\n - panel: slis-kafka-partition-replica-health\n binds_to: slis.kafka_partition_replica_health\n - panel: slis-kafka-produce-latency-p99\n binds_to: slis.kafka_produce_latency_p99\n - panel: slis-kafka-fetch-latency-p99\n binds_to: slis.kafka_fetch_latency_p99\n - panel: slis-http-service-availability\n binds_to: slis.http_service_availability\n - panel: slis-http-service-latency-p99\n binds_to: slis.http_service_latency_p99\n - panel: slos-kafka-broker-availability-99-9\n binds_to: slos.kafka_broker_availability_99_9\n - panel: slos-kafka-consumer-group-lag-seconds-99\n binds_to: slos.kafka_consumer_group_lag_seconds_99\n - panel: slos-kafka-partition-replica-health-99-95\n binds_to: slos.kafka_partition_replica_health_99_95\n - panel: slos-kafka-produce-latency-p99-99\n binds_to: slos.kafka_produce_latency_p99_99\n - panel: slos-kafka-fetch-latency-p99-99\n binds_to: slos.kafka_fetch_latency_p99_99\n - panel: slos-http-service-availability-99-5\n binds_to: slos.http_service_availability_99_5\n - panel: slos-http-service-latency-p99-99\n binds_to: slos.http_service_latency_p99_99\n - id: orders-api-slo-burn\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n template: ref:platform/slo-burn-template\n params:\n slos:\n - kafka_broker_availability_99_9\n - kafka_consumer_group_lag_seconds_99\n - kafka_partition_replica_health_99_95\n - kafka_produce_latency_p99_99\n - kafka_fetch_latency_p99_99\n - http_service_availability_99_5\n - http_service_latency_p99_99\n - id: kafka-kafka-consumer-lag\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n source: file://dashboards/kafka-kafka-consumer-lag.json\n panel_bindings:\n - panel: slis-kafka-consumer-group-lag-seconds\n binds_to: slis.kafka_consumer_group_lag_seconds\n - panel: slos-kafka-consumer-group-lag-seconds-99\n binds_to: slos.kafka_consumer_group_lag_seconds_99\n - panel: view-kafka-per-consumergroup-lag\n binds_to: ref:queries.kafka_per_consumergroup_lag\n - id: kafka-kafka-throughput\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n source: file://dashboards/kafka-kafka-throughput.json\n panel_bindings:\n - panel: view-kafka-per-topic-throughput\n binds_to: ref:queries.kafka_per_topic_throughput\n policy:\n burn_rate_alerts:\n - slo: kafka_broker_availability_99_9\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV1\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV2\n - slo: kafka_consumer_group_lag_seconds_99\n windows:\n - short: 10m\n long: 1h\n factor: 10\n severity: SEV2\n - short: 1h\n long: 6h\n factor: 4\n severity: SEV3\n - slo: kafka_partition_replica_health_99_95\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV1\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV2\n - slo: kafka_produce_latency_p99_99\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV2\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV3\n - slo: kafka_fetch_latency_p99_99\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV2\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV3\n - slo: http_service_availability_99_5\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV1\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV2\n - slo: http_service_latency_p99_99\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV2\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV3\n alerting:\n routes:\n - severity: SEV1\n channels:\n - msteams: \"#orders-api-oncall\"\n - voice: pagerduty://orders-api\n - severity: SEV2\n channels:\n - msteams: \"#orders-api-oncall\"\n - severity: SEV3\n channels:\n - msteams: \"#orders-api-team\"\n dedup: ref:platform/std-dedup\n suppress:\n - maintenance_windows\n - deploy_freezes\n remediation:\n - trigger: alert:kafka_broker_availability_99_9_burn_14x_5m_1h\n runbook: file://runbooks/broker-down.md\n automation: argo-workflow://restart-broker-with-quorum-check\n guardrails:\n max_invocations_per_hour: 2\n requires_human_above: SEV1\n rollback_on_failure: true\n cooldown_after_success: 30m\n circuit_breaker:\n failures: 2\n window: 1h\n - trigger: alert:kafka_partition_replica_health_99_95_burn_14x_5m_1h\n runbook: file://runbooks/under-replicated-partitions.md\n automation: argo-workflow://reassign-partitions-with-throttle\n guardrails:\n max_invocations_per_hour: 1\n requires_human_above: SEV2\n rollback_on_failure: false\n cooldown_after_success: 1h\n baselines:\n mttd_target_p50: 5m\n mttd_target_p95: 15m\n mttr_target_p50: 1h\n mttr_target_p95: 4h\n measurement_source: incident-mgmt + alertmanager firing timestamps\n review_cadence: monthly\n regression_gate: warn_only\n validation:\n synthetic_checks:\n - id: kafka-produce-consume-canary\n kind: k6\n target: kafka.kafka:9092\n interval: 1m\n assertions:\n - produce_success: true\n - round_trip_latency_lt: 500ms\n on_fail_severity: SEV2\n - id: kafka-consumer-group-health\n kind: k6\n target: kafka.kafka:9092\n interval: 1m\n assertions:\n - consumer_groups_assigned: true\n - current_offset_advancing: true\n on_fail_severity: SEV2\n - id: http-service-health-probe\n kind: blackbox-exporter\n target: http://orders-api:8080/health\n interval: 1m\n assertions:\n - status_code: 200\n - latency_lt: 1s\n on_fail_severity: SEV2\n chaos_experiments:\n - id: kafka-broker-pod-kill\n engine: chaos-mesh\n target: kafka-broker\n steady_state_hypothesis: ref:slos.kafka_broker_availability_99_9\n fault:\n kind: pod-failure\n fraction: 0.33\n duration: 90s\n expected_alerts:\n - kafka_broker_availability_99_9_burn_14x_5m_1h\n expected_mttd: 90s\n schedule: monthly\n environment: staging\n - id: kafka-broker-network-partition\n engine: chaos-mesh\n target: kafka-broker\n steady_state_hypothesis: ref:slos.kafka_partition_replica_health_99_95\n fault:\n kind: network-partition\n duration: 3m\n expected_alerts:\n - kafka_partition_replica_health_99_95_burn_14x_5m_1h\n expected_mttd: 2m\n schedule: monthly\n environment: staging\n", + "canonicalYaml": "# ObservabilityPack orders-api — built from the library (kafka@1.0.0,http-service@1.0.0) at tier-2\n# Todos: 21 (metadata.annotations library.todo.*). Spec v1.4.\napiVersion: observability.platform/v1\nkind: ObservabilityPack\nmetadata:\n name: orders-api\n version: 0.1.0\n binding: otel-grafanalabs\n owners:\n - team-orders\n imports:\n - ref: platform/std-budget-policy@2.1\n bindings:\n service: orders-api\n environments:\n - prod\n criticality: tier-2\n labels:\n source: library\n tier: tier-2\n library.entries: kafka,http-service\n annotations:\n library.source: kafka@1.0.0,http-service@1.0.0\n library.format: v1\n library.tier: tier-2\n library.environment: prod\n library.toggles: slos,policy,routes,dashboards,validation\n library.slis: kafka_broker_availability,kafka_consumer_group_lag_seconds,kafka_partition_replica_health,kafka_produce_latency_p99,kafka_fetch_latency_p99,http_service_availability,http_service_latency_p99\n library.params: \"{}\"\n library.evidence: kafka:recorded-live,http-service:semconv\n library.evidence.slis.kafka_broker_availability: \"recorded-live: kafka.md §1.4 — up{job=\\\"kafka-broker\\\"} = 1 on the lab (one target)\"\n library.evidence.slis.kafka_consumer_group_lag_seconds: \"recorded-live: kafka.md §1.3-1.4 — kafka_exporter 1.10.0 families and the expression executed (two consumer groups)\"\n library.evidence.slis.kafka_partition_replica_health: \"recorded-live: kafka.md §1.3-1.4 — both families present and the bool form executed; an ISR shortfall itself was not observable on RF 1 (§1.6)\"\n library.evidence.slis.kafka_produce_latency_p99: \"recorded-live: kafka.md §1.2, §1.4 — kafka_network_requestmetrics_totaltimems{request=\\\"Produce\\\",quantile=\\\"0.99\\\"} read from the JMX exposition and the expression executed\"\n library.evidence.slis.kafka_fetch_latency_p99: \"recorded-live: kafka.md §1.2, §1.4 — the LocalTimeMs quantile gauge and the expression executed; the TotalTimeMs trap measured\"\n library.evidence.slis.http_service_availability: \"semconv: OTel semconv v1.27.0 HTTP metrics + Prometheus compatibility naming; spelling not verified live in this repo (evidence.notes)\"\n library.evidence.slis.http_service_latency_p99: \"semconv: OTel semconv v1.27.0 HTTP metrics + Prometheus compatibility naming; spelling not verified live in this repo (evidence.notes)\"\n library.todoCount: \"21\"\n library.todo.alerting.routes[0]: \"channels.0.msteams: Chat channel for SEV1/SEV2: placeholder '#orders-api-oncall' (param oncall_channel) — The Teams channel the SEV1 and SEV2 routes post to. · channels.1.voice: Voice (pager) target for SEV1: placeholder 'pagerduty://orders-api' (param pager_service) — The voice channel of the SEV1 route (tier-2 and tier-1; L4.MUST.tier1_voice_route).\"\n library.todo.alerting.routes[1]: \"channels.0.msteams: Chat channel for SEV1/SEV2: placeholder '#orders-api-oncall' (param oncall_channel) — The Teams channel the SEV1 and SEV2 routes post to.\"\n library.todo.alerting.routes[2]: \"channels.0.msteams: Chat channel for SEV3: placeholder '#orders-api-team' (param team_channel) — The Teams channel the SEV3 route posts to.\"\n library.todo.baselines: \"mttd_target_p50: MTTD / MTTR targets are the tier-2 defaults, not measured: set them from the service's incident history\"\n library.todo.pipelines.exporters.logs: \"endpoints.0: Loki OTLP ingest endpoint: placeholder 'http://loki:3100/otlp' (param logs_otlp_endpoint) — Where the collector pushes logs over OTLP/HTTP (pipelines.exporters.logs).\"\n library.todo.pipelines.exporters.metrics: \"endpoint: Prometheus remote-write URL: placeholder 'http://prometheus:9090/api/v1/write' (param remote_write_url) — Where the collector remote-writes metrics (pipelines.exporters.metrics).\"\n library.todo.pipelines.exporters.traces: \"endpoint: Tempo OTLP ingest endpoint: placeholder 'tempo:4317' (param traces_otlp_endpoint) — Where the collector pushes traces over OTLP/gRPC (pipelines.exporters.traces).\"\n library.todo.pipelines.receivers[1]: \"scrape_configs.0.static_configs.0.targets.0: Broker JMX exporter targets: placeholder 'kafka-broker.kafka:9404' (param kafka.broker_targets) — host:port of every broker's JMX Prometheus exporter (comma-separated in the scrape config). · scrape_configs.1.static_configs.0.targets.0: kafka_exporter target: placeholder 'kafka-exporter.kafka:9308' (param kafka.exporter_target) — host:port of kafka_exporter.\"\n library.todo.remediation[0]: \"runbook: write the runbook file:///broker-down.md\"\n library.todo.remediation[1]: \"runbook: write the runbook file:///under-replicated-partitions.md\"\n library.todo.storage.logs: \"version: Loki version you run: placeholder '3.7' (param loki_version) — The version of the Loki that stores the logs (telemetry.backends logs-loki version.declared, storage.logs.version); min stays 3.0.\"\n library.todo.storage.metrics: \"version: Prometheus version you run: placeholder '3.14' (param prometheus_version) — The version of the Prometheus that stores the metrics (telemetry.backends metrics-prom version.declared, storage.metrics.version); min stays 2.53, the floor the expressions are known to work from, and tier-1 enforces the block. · remote_write.0.url: Prometheus remote-write URL: placeholder 'http://prometheus:9090/api/v1/write' (param remote_write_url) — Where the collector remote-writes metrics (pipelines.exporters.metrics).\"\n library.todo.storage.traces: \"version: Tempo version you run: placeholder '2.10' (param tempo_version) — The version of the Tempo that stores the traces (telemetry.backends traces-tempo version.declared, storage.traces.version); min stays 2.5.\"\n library.todo.telemetry.backends.logs-loki: \"version.declared: Loki version you run: placeholder '3.7' (param loki_version) — The version of the Loki that stores the logs (telemetry.backends logs-loki version.declared, storage.logs.version); min stays 3.0. · endpoints.0: Loki endpoint: placeholder 'http://loki:3100' (param logs_endpoint) — Where the logs backend answers (telemetry.backends logs-loki).\"\n library.todo.telemetry.backends.metrics-prom: \"version.declared: Prometheus version you run: placeholder '3.14' (param prometheus_version) — The version of the Prometheus that stores the metrics (telemetry.backends metrics-prom version.declared, storage.metrics.version); min stays 2.53, the floor the expressions are known to work from, and tier-1 enforces the block. · endpoints.0: Prometheus query endpoint: placeholder 'http://prometheus:9090' (param metrics_endpoint) — Where the metrics backend answers PromQL (telemetry.backends metrics-prom).\"\n library.todo.telemetry.backends.traces-tempo: \"version.declared: Tempo version you run: placeholder '2.10' (param tempo_version) — The version of the Tempo that stores the traces (telemetry.backends traces-tempo version.declared, storage.traces.version); min stays 2.5. · endpoints.0: Tempo endpoint: placeholder 'http://tempo:3200' (param traces_endpoint) — Where the traces backend answers (telemetry.backends traces-tempo).\"\n library.todo.validation.chaos_experiments.kafka-broker-network-partition: \"target: Broker workload (chaos target): placeholder 'kafka-broker' (param kafka.broker_workload) — The broker workload (StatefulSet / pods) the chaos experiments target.\"\n library.todo.validation.chaos_experiments.kafka-broker-pod-kill: \"target: Broker workload (chaos target): placeholder 'kafka-broker' (param kafka.broker_workload) — The broker workload (StatefulSet / pods) the chaos experiments target.\"\n library.todo.validation.synthetic_checks.http-service-health-probe: \"target: Health endpoint URL: placeholder 'http://orders-api:8080/health' (param http-service.health_url) — The URL the blackbox probe hits to prove the service answers.\"\n library.todo.validation.synthetic_checks.kafka-consumer-group-health: \"target: Bootstrap servers: placeholder 'kafka.kafka:9092' (param kafka.bootstrap) — The bootstrap address the synthetic produce/consume canary connects to.\"\n library.todo.validation.synthetic_checks.kafka-produce-consume-canary: \"target: Bootstrap servers: placeholder 'kafka.kafka:9092' (param kafka.bootstrap) — The bootstrap address the synthetic produce/consume canary connects to.\"\nspec:\n otel:\n semconv: 1.27.0\n resource_attributes:\n required:\n - service.name\n - service.namespace\n - service.version\n - deployment.environment\n custom:\n - kafka.cluster.id\n - kafka.broker.id\n sdk:\n languages:\n - java\n - go\n - node\n sampling:\n policy: parentbased_traceidratio\n ratio: 0.1\n propagators:\n - tracecontext\n - baggage\n log_correlation: true\n telemetry:\n backends:\n - id: metrics-prom\n signal: metrics\n product: prometheus\n version:\n declared: \"3.14\"\n min: \"2.53\"\n gating: warn\n endpoints:\n - http://prometheus:9090\n default: true\n - id: logs-loki\n signal: logs\n product: loki\n version:\n declared: \"3.7\"\n min: \"3.0\"\n gating: warn\n endpoints:\n - http://loki:3100\n default: true\n - id: traces-tempo\n signal: traces\n product: tempo\n version:\n declared: \"2.10\"\n min: \"2.5\"\n gating: warn\n endpoints:\n - http://tempo:3200\n default: true\n environments:\n prod:\n criticality: tier-2\n backends:\n metrics: metrics-prom\n logs: logs-loki\n traces: traces-tempo\n slis:\n - id: kafka_broker_availability\n type: ratio\n description: Fraction of brokers reporting ready vs. the cluster's declared broker count.\n semconv_metric: kafka.broker.up\n good: sum(up{job=\"kafka-broker\"} == bool 1)\n total: count(up{job=\"kafka-broker\"})\n - id: kafka_consumer_group_lag_seconds\n type: threshold\n description: Maximum consumer-group lag in seconds equivalent, per consumer group (message-count lag over the group's rolling consumption rate).\n semconv_metric: messaging.kafka.consumer.lag\n query: |\n max by (consumergroup) (\n kafka_consumergroup_lag /\n (rate(kafka_consumergroup_current_offset[5m]) > 0)\n )\n threshold: 60\n unit: seconds\n - id: kafka_partition_replica_health\n type: ratio\n description: Fraction of partitions whose in-sync replica count equals their replica count (== bool, one per healthy partition).\n semconv_metric: kafka.partition.in_sync_replicas\n good: sum(kafka_topic_partition_in_sync_replica == bool kafka_topic_partition_replicas)\n total: count(kafka_topic_partition_replicas)\n - id: kafka_produce_latency_p99\n type: threshold\n description: 99th-percentile total time of Produce requests on the broker, in seconds (queue, local, replication wait, response send) — what a producer waits for an ack.\n semconv_metric: kafka.broker.request.latency\n query: max(kafka_network_requestmetrics_totaltimems{request=\"Produce\",quantile=\"0.99\"}) / 1000\n threshold: 0.1\n unit: seconds\n - id: kafka_fetch_latency_p99\n type: threshold\n description: 99th-percentile local (broker processing) time of FetchConsumer requests, in seconds.\n semconv_metric: kafka.broker.request.latency\n query: max(kafka_network_requestmetrics_localtimems{request=\"FetchConsumer\",quantile=\"0.99\"}) / 1000\n threshold: 0.05\n unit: seconds\n - id: http_service_availability\n type: ratio\n description: Fraction of HTTP requests not answered with a 5xx status, from the request-duration histogram's _count series (semconv http.server.request.duration, label http_response_status_code).\n semconv_metric: http.server.request.duration\n good: sum(rate(http_server_request_duration_seconds_count{job=\"orders-api\",http_response_status_code!~\"5..\"}[5m]))\n total: sum(rate(http_server_request_duration_seconds_count{job=\"orders-api\"}[5m]))\n - id: http_service_latency_p99\n type: threshold\n description: 99th-percentile server-side request duration in seconds, from the semconv http.server.request.duration histogram.\n semconv_metric: http.server.request.duration\n query: |\n histogram_quantile(0.99,\n sum by (le)(rate(http_server_request_duration_seconds_bucket{job=\"orders-api\"}[5m])))\n threshold: 0.5\n unit: seconds\n slos:\n - id: kafka_broker_availability_99_9\n sli: kafka_broker_availability\n objective: 0.999\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_consumer_group_lag_seconds_99\n sli: kafka_consumer_group_lag_seconds\n objective: 0.99\n window: 7d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_partition_replica_health_99_95\n sli: kafka_partition_replica_health\n objective: 0.9995\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_produce_latency_p99_99\n sli: kafka_produce_latency_p99\n objective: 0.99\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: kafka_fetch_latency_p99_99\n sli: kafka_fetch_latency_p99\n objective: 0.99\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: http_service_availability_99_5\n sli: http_service_availability\n objective: 0.995\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n - id: http_service_latency_p99_99\n sli: http_service_latency_p99\n objective: 0.99\n window: 30d\n error_budget_policy: ref:platform/std-budget-policy\n pipelines:\n receivers:\n - name: otlp\n protocols:\n - grpc\n - http\n endpoint: 0.0.0.0:4317\n - name: prometheus\n scrape_configs:\n - job_name: kafka-broker\n scrape_interval: 30s\n static_configs:\n -\n targets:\n - kafka-broker.kafka:9404\n - job_name: kafka-exporter\n scrape_interval: 30s\n static_configs:\n -\n targets:\n - kafka-exporter.kafka:9308\n processors:\n - name: memory_limiter\n limit_percentage: 80\n - name: batch\n timeout: 10s\n - name: resource\n attributes:\n - key: deployment.environment\n value: prod\n action: upsert\n exporters:\n metrics:\n kind: prometheusremotewrite\n endpoint: http://prometheus:9090/api/v1/write\n external_labels:\n service: orders-api\n logs:\n kind: otlphttp\n endpoints:\n - http://loki:3100/otlp\n traces:\n kind: otlp\n endpoint: tempo:4317\n storage:\n metrics:\n backend: prometheus\n version: \"3.14\"\n retention: 90d\n remote_write:\n - url: http://prometheus:9090/api/v1/write\n logs:\n backend: loki\n version: \"3.7\"\n retention: 30d\n traces:\n backend: tempo\n version: \"2.10\"\n retention: 7d\n sampling: head-based\n queries:\n recording_rules:\n - name: orders_api:kafka_broker_availability:ratio_5m\n expr: ref:slis.kafka_broker_availability\n interval: 30s\n - name: orders_api:kafka_consumer_group_lag_seconds:value_5m\n expr: ref:slis.kafka_consumer_group_lag_seconds\n interval: 30s\n - name: orders_api:kafka_partition_replica_health:ratio_5m\n expr: ref:slis.kafka_partition_replica_health\n interval: 30s\n - name: orders_api:kafka_produce_latency_p99:value_5m\n expr: ref:slis.kafka_produce_latency_p99\n interval: 30s\n - name: orders_api:kafka_fetch_latency_p99:value_5m\n expr: ref:slis.kafka_fetch_latency_p99\n interval: 30s\n - name: orders_api:http_service_availability:ratio_5m\n expr: ref:slis.http_service_availability\n interval: 30s\n - name: orders_api:http_service_latency_p99:value_5m\n expr: ref:slis.http_service_latency_p99\n interval: 30s\n derived_views:\n - id: golden_signals_orders_api\n bind: ref:platform/golden-signals-view\n params:\n service: orders-api\n - id: kafka_per_topic_throughput\n bind: ref:platform/per-resource-rollup\n params:\n metric: kafka_server_brokertopicmetrics_messagesin_total{topic!=\"\"}\n by:\n - topic\n - id: kafka_per_consumergroup_lag\n bind: ref:platform/per-resource-rollup\n params:\n sli: kafka_consumer_group_lag_seconds\n by:\n - consumergroup\n - id: http_service_per_route_latency\n bind: ref:platform/per-resource-rollup\n params:\n sli: http_service_latency_p99\n by:\n - http_route\n - id: http_service_per_route_traffic\n bind: ref:platform/per-resource-rollup\n params:\n metric: http_server_request_duration_seconds_count\n by:\n - http_route\n - http_request_method\n dashboards:\n - id: orders-api-overview\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n source: file://dashboards/orders-api-overview.json\n panel_bindings:\n - panel: slis-kafka-broker-availability\n binds_to: slis.kafka_broker_availability\n - panel: slis-kafka-consumer-group-lag-seconds\n binds_to: slis.kafka_consumer_group_lag_seconds\n - panel: slis-kafka-partition-replica-health\n binds_to: slis.kafka_partition_replica_health\n - panel: slis-kafka-produce-latency-p99\n binds_to: slis.kafka_produce_latency_p99\n - panel: slis-kafka-fetch-latency-p99\n binds_to: slis.kafka_fetch_latency_p99\n - panel: slis-http-service-availability\n binds_to: slis.http_service_availability\n - panel: slis-http-service-latency-p99\n binds_to: slis.http_service_latency_p99\n - panel: slos-kafka-broker-availability-99-9\n binds_to: slos.kafka_broker_availability_99_9\n - panel: slos-kafka-consumer-group-lag-seconds-99\n binds_to: slos.kafka_consumer_group_lag_seconds_99\n - panel: slos-kafka-partition-replica-health-99-95\n binds_to: slos.kafka_partition_replica_health_99_95\n - panel: slos-kafka-produce-latency-p99-99\n binds_to: slos.kafka_produce_latency_p99_99\n - panel: slos-kafka-fetch-latency-p99-99\n binds_to: slos.kafka_fetch_latency_p99_99\n - panel: slos-http-service-availability-99-5\n binds_to: slos.http_service_availability_99_5\n - panel: slos-http-service-latency-p99-99\n binds_to: slos.http_service_latency_p99_99\n - id: orders-api-slo-burn\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n template: ref:platform/slo-burn-template\n params:\n slos:\n - kafka_broker_availability_99_9\n - kafka_consumer_group_lag_seconds_99\n - kafka_partition_replica_health_99_95\n - kafka_produce_latency_p99_99\n - kafka_fetch_latency_p99_99\n - http_service_availability_99_5\n - http_service_latency_p99_99\n - id: kafka-kafka-consumer-lag\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n source: file://dashboards/kafka-kafka-consumer-lag.json\n panel_bindings:\n - panel: slis-kafka-consumer-group-lag-seconds\n binds_to: slis.kafka_consumer_group_lag_seconds\n - panel: slos-kafka-consumer-group-lag-seconds-99\n binds_to: slos.kafka_consumer_group_lag_seconds_99\n - panel: view-kafka-per-consumergroup-lag\n binds_to: ref:queries.kafka_per_consumergroup_lag\n - id: kafka-kafka-throughput\n provider:\n kind: grafana\n version: \"12.4\"\n schemaVersion: 41\n folder: orders-api\n source: file://dashboards/kafka-kafka-throughput.json\n panel_bindings:\n - panel: view-kafka-per-topic-throughput\n binds_to: ref:queries.kafka_per_topic_throughput\n policy:\n burn_rate_alerts:\n - slo: kafka_broker_availability_99_9\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV1\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV2\n - slo: kafka_consumer_group_lag_seconds_99\n windows:\n - short: 10m\n long: 1h\n factor: 10\n severity: SEV2\n - short: 1h\n long: 6h\n factor: 4\n severity: SEV3\n - slo: kafka_partition_replica_health_99_95\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV1\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV2\n - slo: kafka_produce_latency_p99_99\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV2\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV3\n - slo: kafka_fetch_latency_p99_99\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV2\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV3\n - slo: http_service_availability_99_5\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV1\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV2\n - slo: http_service_latency_p99_99\n windows:\n - short: 5m\n long: 1h\n factor: 14\n severity: SEV2\n - short: 30m\n long: 6h\n factor: 6\n severity: SEV3\n alerting:\n routes:\n - severity: SEV1\n channels:\n - msteams: \"#orders-api-oncall\"\n - voice: pagerduty://orders-api\n - severity: SEV2\n channels:\n - msteams: \"#orders-api-oncall\"\n - severity: SEV3\n channels:\n - msteams: \"#orders-api-team\"\n dedup: ref:platform/std-dedup\n suppress:\n - maintenance_windows\n - deploy_freezes\n remediation:\n - trigger: alert:kafka_broker_availability_99_9_burn_14x_5m_1h\n runbook: file://runbooks/broker-down.md\n automation: argo-workflow://restart-broker-with-quorum-check\n guardrails:\n max_invocations_per_hour: 2\n requires_human_above: SEV1\n rollback_on_failure: true\n cooldown_after_success: 30m\n circuit_breaker:\n failures: 2\n window: 1h\n - trigger: alert:kafka_partition_replica_health_99_95_burn_14x_5m_1h\n runbook: file://runbooks/under-replicated-partitions.md\n automation: argo-workflow://reassign-partitions-with-throttle\n guardrails:\n max_invocations_per_hour: 1\n requires_human_above: SEV2\n rollback_on_failure: false\n cooldown_after_success: 1h\n baselines:\n mttd_target_p50: 5m\n mttd_target_p95: 15m\n mttr_target_p50: 1h\n mttr_target_p95: 4h\n measurement_source: incident-mgmt + alertmanager firing timestamps\n review_cadence: monthly\n regression_gate: warn_only\n validation:\n synthetic_checks:\n - id: kafka-produce-consume-canary\n kind: k6\n target: kafka.kafka:9092\n interval: 1m\n assertions:\n - produce_success: true\n - round_trip_latency_lt: 500ms\n on_fail_severity: SEV2\n - id: kafka-consumer-group-health\n kind: k6\n target: kafka.kafka:9092\n interval: 1m\n assertions:\n - consumer_groups_assigned: true\n - current_offset_advancing: true\n on_fail_severity: SEV2\n - id: http-service-health-probe\n kind: blackbox-exporter\n target: http://orders-api:8080/health\n interval: 1m\n assertions:\n - status_code: 200\n - latency_lt: 1s\n on_fail_severity: SEV2\n chaos_experiments:\n - id: kafka-broker-pod-kill\n engine: chaos-mesh\n target: kafka-broker\n steady_state_hypothesis: ref:slos.kafka_broker_availability_99_9\n fault:\n kind: pod-failure\n fraction: 0.33\n duration: 90s\n expected_alerts:\n - kafka_broker_availability_99_9_burn_14x_5m_1h\n expected_mttd: 90s\n schedule: monthly\n environment: staging\n - id: kafka-broker-network-partition\n engine: chaos-mesh\n target: kafka-broker\n steady_state_hypothesis: ref:slos.kafka_partition_replica_health_99_95\n fault:\n kind: network-partition\n duration: 3m\n expected_alerts:\n - kafka_partition_replica_health_99_95_burn_14x_5m_1h\n expected_mttd: 2m\n schedule: monthly\n environment: staging\n", "todos": [ { "path": "alerting.routes[0]", diff --git a/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.25.0.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.25.0.golden.yaml index a9edf3d6..53f5f146 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.25.0.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.25.0.golden.yaml @@ -1,4 +1,4 @@ -# Alertmanager config compiled from ObservabilityPack v1.3 +# Alertmanager config compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.27.0.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.27.0.golden.yaml index 11736d86..9fefa958 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.27.0.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.27.0.golden.yaml @@ -1,4 +1,4 @@ -# Alertmanager config compiled from ObservabilityPack v1.3 +# Alertmanager config compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.28.0.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.28.0.golden.yaml index 3ca538c2..0174bead 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.28.0.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__alertmanager__alertmanager-0.28.0.golden.yaml @@ -1,4 +1,4 @@ -# Alertmanager config compiled from ObservabilityPack v1.3 +# Alertmanager config compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-grafana-managed-all.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-grafana-managed-all.golden.yaml index fc7099d2..1ba2b656 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-grafana-managed-all.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-grafana-managed-all.golden.yaml @@ -1,4 +1,4 @@ -# Grafana-managed rules compiled from ObservabilityPack v1.3 +# Grafana-managed rules compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Format: Grafana 9+ provisioning YAML (apiVersion: 1). # Apply via: copy under provisioning/alerting/ OR POST to /api/v1/provisioning/alert-rules diff --git a/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-prometheus-slo-hostile_availability_99.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-prometheus-slo-hostile_availability_99.golden.yaml index 6868a3ab..253d3fc8 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-prometheus-slo-hostile_availability_99.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__artifact__rules-prometheus-slo-hostile_availability_99.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules — SLO hostile_availability_99 compiled from ObservabilityPack v1.3 +# Prometheus rules — SLO hostile_availability_99 compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.80.0.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.80.0.golden.yaml index 1a83c4f6..668bb6d9 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.80.0.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.80.0.golden.yaml @@ -1,4 +1,4 @@ -# OTel Collector compiled from ObservabilityPack v1.3 +# OTel Collector compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.90.0.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.90.0.golden.yaml index 1a83c4f6..668bb6d9 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.90.0.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.90.0.golden.yaml @@ -1,4 +1,4 @@ -# OTel Collector compiled from ObservabilityPack v1.3 +# OTel Collector compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.96.0.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.96.0.golden.yaml index 85714b96..6da57d8a 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.96.0.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__otel-collector__otel-collector-0.96.0.golden.yaml @@ -1,4 +1,4 @@ -# OTel Collector compiled from ObservabilityPack v1.3 +# OTel Collector compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__mimir-2.15.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__mimir-2.15.golden.yaml index d0a18567..ae340a02 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__mimir-2.15.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__mimir-2.15.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.30.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.30.golden.yaml index 400878c4..1de4c0a2 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.30.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.30.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.50.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.50.golden.yaml index c3d568e1..1879c380 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.50.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-2.50.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-3.0.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-3.0.golden.yaml index c3d568e1..1879c380 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-3.0.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-3.0.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-9.9.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-9.9.golden.yaml index c3d568e1..1879c380 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-9.9.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__prometheus-9.9.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__victoriametrics-1.99.golden.yaml b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__victoriametrics-1.99.golden.yaml index a5b8746a..7d1c4691 100644 --- a/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__victoriametrics-1.99.golden.yaml +++ b/tools/fixtures/golden/compile/edge-hostile-names__prometheus-rules__victoriametrics-1.99.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: edge-hostile-names · version 0.1.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.25.0.golden.yaml b/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.25.0.golden.yaml index 54ffea91..d5558701 100644 --- a/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.25.0.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.25.0.golden.yaml @@ -1,4 +1,4 @@ -# Alertmanager config compiled from ObservabilityPack v1.3 +# Alertmanager config compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.27.0.golden.yaml b/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.27.0.golden.yaml index fe814c5e..4027e252 100644 --- a/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.27.0.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.27.0.golden.yaml @@ -1,4 +1,4 @@ -# Alertmanager config compiled from ObservabilityPack v1.3 +# Alertmanager config compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.28.0.golden.yaml b/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.28.0.golden.yaml index c0644ac2..f208a377 100644 --- a/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.28.0.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__alertmanager__alertmanager-0.28.0.golden.yaml @@ -1,4 +1,4 @@ -# Alertmanager config compiled from ObservabilityPack v1.3 +# Alertmanager config compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-all.golden.yaml b/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-all.golden.yaml index 44ec1083..d6802cc0 100644 --- a/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-all.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-all.golden.yaml @@ -1,4 +1,4 @@ -# Grafana-managed rules compiled from ObservabilityPack v1.3 +# Grafana-managed rules compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Format: Grafana 9+ provisioning YAML (apiVersion: 1). # Apply via: copy under provisioning/alerting/ OR POST to /api/v1/provisioning/alert-rules diff --git a/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-assurance.golden.yaml b/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-assurance.golden.yaml index 869c7e11..ee26a9bb 100644 --- a/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-assurance.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-assurance.golden.yaml @@ -1,4 +1,4 @@ -# Grafana-managed rules — assurance compiled from ObservabilityPack v1.3 +# Grafana-managed rules — assurance compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Format: Grafana 9+ provisioning YAML (apiVersion: 1). # Apply via: copy under provisioning/alerting/ OR POST to /api/v1/provisioning/alert-rules diff --git a/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-slo-checkout_latency_99_5_p99_300ms.golden.yaml b/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-slo-checkout_latency_99_5_p99_300ms.golden.yaml index 7fa8f60b..cb4cc0e3 100644 --- a/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-slo-checkout_latency_99_5_p99_300ms.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__artifact__rules-grafana-managed-slo-checkout_latency_99_5_p99_300ms.golden.yaml @@ -1,4 +1,4 @@ -# Grafana-managed rules — SLO checkout_latency_99_5_p99_300ms compiled from ObservabilityPack v1.3 +# Grafana-managed rules — SLO checkout_latency_99_5_p99_300ms compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Format: Grafana 9+ provisioning YAML (apiVersion: 1). # Apply via: copy under provisioning/alerting/ OR POST to /api/v1/provisioning/alert-rules diff --git a/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-assurance.golden.yaml b/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-assurance.golden.yaml index d95f07ef..f56350a0 100644 --- a/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-assurance.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-assurance.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules — assurance compiled from ObservabilityPack v1.3 +# Prometheus rules — assurance compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-api_availability_99_9.golden.yaml b/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-api_availability_99_9.golden.yaml index ea48943f..de57995b 100644 --- a/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-api_availability_99_9.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-api_availability_99_9.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules — SLO api_availability_99_9 compiled from ObservabilityPack v1.3 +# Prometheus rules — SLO api_availability_99_9 compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-checkout_latency_99_5_p99_300ms.golden.yaml b/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-checkout_latency_99_5_p99_300ms.golden.yaml index c4a8f59e..0714440e 100644 --- a/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-checkout_latency_99_5_p99_300ms.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__artifact__rules-prometheus-slo-checkout_latency_99_5_p99_300ms.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules — SLO checkout_latency_99_5_p99_300ms compiled from ObservabilityPack v1.3 +# Prometheus rules — SLO checkout_latency_99_5_p99_300ms compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.80.0.golden.yaml b/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.80.0.golden.yaml index de9b2ffb..c9d55180 100644 --- a/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.80.0.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.80.0.golden.yaml @@ -1,4 +1,4 @@ -# OTel Collector compiled from ObservabilityPack v1.3 +# OTel Collector compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.90.0.golden.yaml b/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.90.0.golden.yaml index de9b2ffb..c9d55180 100644 --- a/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.90.0.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.90.0.golden.yaml @@ -1,4 +1,4 @@ -# OTel Collector compiled from ObservabilityPack v1.3 +# OTel Collector compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.96.0.golden.yaml b/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.96.0.golden.yaml index 5abbd16e..72b17498 100644 --- a/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.96.0.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__otel-collector__otel-collector-0.96.0.golden.yaml @@ -1,4 +1,4 @@ -# OTel Collector compiled from ObservabilityPack v1.3 +# OTel Collector compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__prometheus-rules__mimir-2.15.golden.yaml b/tools/fixtures/golden/compile/payment-service__prometheus-rules__mimir-2.15.golden.yaml index 16eb0cbc..3964fa08 100644 --- a/tools/fixtures/golden/compile/payment-service__prometheus-rules__mimir-2.15.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__prometheus-rules__mimir-2.15.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.30.golden.yaml b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.30.golden.yaml index ebfde47e..80b8ddc0 100644 --- a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.30.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.30.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.50.golden.yaml b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.50.golden.yaml index 1bd0cfea..288181dc 100644 --- a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.50.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-2.50.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-3.0.golden.yaml b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-3.0.golden.yaml index 1bd0cfea..288181dc 100644 --- a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-3.0.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-3.0.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-9.9.golden.yaml b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-9.9.golden.yaml index 1bd0cfea..288181dc 100644 --- a/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-9.9.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__prometheus-rules__prometheus-9.9.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/payment-service__prometheus-rules__victoriametrics-1.99.golden.yaml b/tools/fixtures/golden/compile/payment-service__prometheus-rules__victoriametrics-1.99.golden.yaml index 6b63ab33..2439c6d8 100644 --- a/tools/fixtures/golden/compile/payment-service__prometheus-rules__victoriametrics-1.99.golden.yaml +++ b/tools/fixtures/golden/compile/payment-service__prometheus-rules__victoriametrics-1.99.golden.yaml @@ -1,4 +1,4 @@ -# Prometheus rules compiled from ObservabilityPack v1.3 +# Prometheus rules compiled from ObservabilityPack v1.4 # Pack: payment-service · version 1.5.0 # Source of truth — DO NOT hand-edit. Re-emit from the pack. # Generated by tools/lib/compile.mjs diff --git a/tools/fixtures/golden/compile/shared-sli__artifact__rules-grafana-managed-all.golden.yaml b/tools/fixtures/golden/compile/shared-sli__artifact__rules-grafana-managed-all.golden.yaml index 5bc186f3..a85d8454 100644 --- a/tools/fixtures/golden/compile/shared-sli__artifact__rules-grafana-managed-all.golden.yaml +++ b/tools/fixtures/golden/compile/shared-sli__artifact__rules-grafana-managed-all.golden.yaml @@ -1,4 +1,4 @@ -# Grafana-managed rules compiled from ObservabilityPack v1.3 +# Grafana-managed rules compiled from ObservabilityPack v1.4 # Pack: shared-sli · version 0.1.0 # Format: Grafana 9+ provisioning YAML (apiVersion: 1). # Apply via: copy under provisioning/alerting/ OR POST to /api/v1/provisioning/alert-rules diff --git a/tools/lib/validator.mjs b/tools/lib/validator.mjs index 214b40b3..ee9aa873 100644 --- a/tools/lib/validator.mjs +++ b/tools/lib/validator.mjs @@ -14,6 +14,13 @@ // A misplaced field therefore reads `$.spec.slis[N].good_when: matches // forbidden 'not' schema`; a bad value `not in enum ["below","above"]`. // +// Spec 1.4 (RFC-0003) adds `alerting.rules[]`, the operational (non-SLO) +// alert rules: `name` and `expr` required, `engine` a closed enum +// ($defs/AlertEngine, absent means prometheus), `severity` the pack's +// SEV1..SEV4. `routes` stays the only required key of `alerting`, so every +// 1.3 pack validates unchanged; a rule missing `expr` reads +// `$.spec.alerting.rules[N]: missing required key 'expr'`. +// // Supported keywords: // type, enum, const, pattern, minLength, maxLength, // minimum, maximum, exclusiveMinimum, exclusiveMaximum, @@ -25,7 +32,7 @@ export const REQUIRED_API_VERSION = 'observability.platform/v1'; export const REQUIRED_KIND = 'ObservabilityPack'; -export const SPEC_VERSION = '1.3'; +export const SPEC_VERSION = '1.4'; /** The vendored spec directory and its schema, relative to the repo root (Node callers resolve them; plain strings, browser-safe). */ export const SPEC_DIR = `vendor/observability-pack-spec/v${SPEC_VERSION}`; export const SPEC_SCHEMA_PATH = `${SPEC_DIR}/observability-pack.schema.json`; diff --git a/tools/sync-spec.mjs b/tools/sync-spec.mjs index c62d823f..9ee6121b 100644 --- a/tools/sync-spec.mjs +++ b/tools/sync-spec.mjs @@ -5,7 +5,7 @@ * Refreshes the vendored copy of the ObservabilityPack spec under * vendor/observability-pack-spec/. Pulls the four canonical files (SOURCE_FILES) from * MoebiusX/otel-observability-pack via `gh api`, writes them under v/ — the - * version is read from the header table of the fetched spec (`| Spec version | 1.3 |`), never + * version is read from the header table of the fetched spec (`| Spec version | 1.4 |`), never * typed here — recomputes sha256 checksums, and rewrites VERSIONS.json. * * Usage: @@ -13,7 +13,7 @@ * node tools/sync-spec.mjs --ref # sync to a specific branch, tag or sha * node tools/sync-spec.mjs --check # verify on-disk checksums match VERSIONS.json; exit 1 on drift * - * A spec bump lands in a new directory (v1.2/ → v1.3/) and VERSIONS.json follows it; the previous + * A spec bump lands in a new directory (v1.3/ → v1.4/) and VERSIONS.json follows it; the previous * directory is not deleted here — remove it in the commit that moves tools/lib/validator.mjs * SPEC_VERSION (git history keeps it). * diff --git a/tools/test-validator.mjs b/tools/test-validator.mjs index 0e418d05..1184a188 100644 --- a/tools/test-validator.mjs +++ b/tools/test-validator.mjs @@ -3,12 +3,15 @@ * tools/test-validator.mjs — the vendored spec through tools/lib/validator.mjs. * * Spec 1.3 (RFC-0002) adds `good_when: below | above` to threshold and distribution SLIs and forbids - * it on ratio and custom ones with a `not` sub-schema, the form this walker enforces. This suite - * pins: the version constants agree with the vendored manifest and the one directory on disk; the - * 1.3 schema accepts good_when where the spec allows it and rejects it where it does not, naming - * the error; a bad value is named; and every pack the repo ships or tests with — all 1.2-shaped, - * none carries good_when — validates unchanged, so a 1.2 pack reaches the same verdict. Then the - * drift guard: no code, studio text, workflow or current-version doc names another spec version. + * it on ratio and custom ones with a `not` sub-schema, the form this walker enforces. Spec 1.4 + * (RFC-0003) adds `alerting.rules[]`, the operational (non-SLO) alert rules: name and expr required, + * engine a closed enum, routes still the only required key. This suite pins: the version constants + * agree with the vendored manifest and the one directory on disk; the schema accepts good_when where + * the spec allows it and rejects it where it does not, naming the error; it accepts alerting.rules + * and names a rule missing expr, an unknown engine, a non-SEV severity; and every pack the repo ships + * or tests with — all 1.3-shaped, none carries alerting.rules — validates unchanged, so a 1.3 pack + * reaches the same verdict. Then the drift guard: no code, studio text, workflow or current-version + * doc names another spec version. */ import { test } from 'node:test'; @@ -26,17 +29,17 @@ const clone = (v) => JSON.parse(JSON.stringify(v)); const example = () => parseYaml(readFileSync(EXAMPLE, 'utf8')); const errorsOf = (pack) => validateCanonical(pack, SCHEMA); -test('the version constants, the vendored manifest and the one directory on disk agree (spec 1.3)', () => { - assert.equal(SPEC_VERSION, '1.3'); - assert.equal(SPEC_DIR, 'vendor/observability-pack-spec/v1.3'); - assert.equal(SPEC_SCHEMA_PATH, 'vendor/observability-pack-spec/v1.3/observability-pack.schema.json'); +test('the version constants, the vendored manifest and the one directory on disk agree (spec 1.4)', () => { + assert.equal(SPEC_VERSION, '1.4'); + assert.equal(SPEC_DIR, 'vendor/observability-pack-spec/v1.4'); + assert.equal(SPEC_SCHEMA_PATH, 'vendor/observability-pack-spec/v1.4/observability-pack.schema.json'); const manifest = JSON.parse(readFileSync(resolve(ROOT, 'vendor/observability-pack-spec/VERSIONS.json'), 'utf8')); assert.equal(manifest.schema, SPEC_VERSION, 'VERSIONS.json names the spec version the validator serves'); assert.ok(Object.keys(manifest.files).every(f => f.startsWith(`v${SPEC_VERSION}/`)), 'every vendored file sits under the current version directory'); const dirs = readdirSync(resolve(ROOT, 'vendor/observability-pack-spec')).filter(n => /^v\d+\.\d+$/.test(n) && statSync(join(ROOT, 'vendor/observability-pack-spec', n)).isDirectory()); - assert.deepEqual(dirs, [`v${SPEC_VERSION}`], 'one version directory: 1.3 validates every 1.2 pack, so 1.2 is history (git keeps it)'); + assert.deepEqual(dirs, [`v${SPEC_VERSION}`], 'one version directory: 1.4 validates every 1.3 pack (as 1.3 did every 1.2 pack), so 1.3 is history (git keeps it)'); for (const f of ['observability-pack.schema.json', 'spec.md', 'examples/payment-service.pack.yaml', 'docs/maturity-model.md']) assert.ok(existsSync(resolve(ROOT, SPEC_DIR, f)), f); - // The schema is the 1.3 one: good_when on the SLI, forbidden by `not` on ratio and custom. + // The schema carries 1.3's good_when on the SLI, forbidden by `not` on ratio and custom. const sli = SCHEMA.$defs.SLI; assert.deepEqual(sli.properties.good_when.enum, ['below', 'above']); assert.equal(sli.properties.good_when.default, 'below'); @@ -45,7 +48,17 @@ test('the version constants, the vendored manifest and the one directory on disk assert.deepEqual(branch('custom').properties, { good_when: { not: {} } }); assert.equal(branch('threshold').properties, undefined); assert.equal(branch('distribution').properties, undefined); - assert.match(readFileSync(resolve(ROOT, SPEC_DIR, 'spec.md'), 'utf8'), /^\| Spec version \| 1\.3 \|/m); + // And 1.4's alerting.rules: AlertRule (name and expr required, closed) under Alerting, whose only required key is still routes. + const alerting = SCHEMA.$defs.Alerting; + assert.deepEqual(alerting.required, ['routes']); + assert.deepEqual(alerting.properties.rules.items, { $ref: '#/$defs/AlertRule' }); + const rule = SCHEMA.$defs.AlertRule; + assert.deepEqual(rule.required, ['name', 'expr']); + assert.equal(rule.additionalProperties, false); + assert.deepEqual(Object.keys(rule.properties).sort(), ['annotations', 'engine', 'expr', 'for', 'labels', 'name', 'severity', 'source']); + assert.deepEqual(rule.properties.engine, { $ref: '#/$defs/AlertEngine' }); + assert.deepEqual(SCHEMA.$defs.AlertEngine.enum, ['prometheus', 'mimir', 'thanos', 'victoriametrics', 'loki', 'grafana', 'alertmanager']); + assert.match(readFileSync(resolve(ROOT, SPEC_DIR, 'spec.md'), 'utf8'), /^\| Spec version \| 1\.4 \|/m); // package.json validates the vendored example through the same directory. const pkg = JSON.parse(readFileSync(resolve(ROOT, 'package.json'), 'utf8')); assert.equal(pkg.scripts.validate, `node tools/validate-pack.mjs ${SPEC_DIR}/examples/payment-service.pack.yaml`); @@ -91,21 +104,53 @@ test('good_when is refused on ratio and custom SLIs through the `not` sub-schema assert.deepEqual(errorsOf(pack), [`$.spec.slis[${lat}].good_when: expected string, got boolean`]); }); -// Every schema-valid pack the repo ships or tests with, 1.2-shaped (none carries good_when): the 1.3 schema accepts -// each unchanged. (tools/fixtures/compile/ holds the compiler's SHAPE fixtures — partial packs and hostile names, +test('alerting.rules (spec 1.4): the example declares three operational rules; name and expr are required, engine and severity are closed, routes alone still suffice', () => { + const pack = example(); + const rules = pack.spec.alerting.rules; + assert.deepEqual(rules.map(r => r.name), ['PaymentServicePodRestarting', 'PaymentDbConnectionPoolSaturated', 'PaymentCertificateExpiringSoon']); + assert.deepEqual(rules.map(r => r.engine), ['prometheus', undefined, 'grafana'], 'the second rule leaves engine absent: absent means prometheus'); + assert.ok(rules.every(r => r.expr && r.for && r.severity && r.labels?.severity && r.source), 'each states expr, for, severity, labels.severity and source'); + // Absent and empty mean the same: no operational rules. A 1.3-shaped alerting block validates unchanged. + delete pack.spec.alerting.rules; + assert.deepEqual(errorsOf(pack), []); + pack.spec.alerting.rules = []; + assert.deepEqual(errorsOf(pack), []); + // The shape: name and expr required; an unknown engine, a non-SEV severity, an unknown property are each refused and named. + pack.spec.alerting.rules = [{ name: 'HighCPU', expr: 'avg(rate(node_cpu_seconds_total{mode!="idle"}[5m])) > 0.9' }]; + assert.deepEqual(errorsOf(pack), [], 'name and expr alone suffice'); + pack.spec.alerting.rules = [{ name: 'HighCPU' }]; + assert.deepEqual(errorsOf(pack), ["$.spec.alerting.rules[0]: missing required key 'expr'"]); + pack.spec.alerting.rules = [{ expr: 'up == 0' }]; + assert.deepEqual(errorsOf(pack), ["$.spec.alerting.rules[0]: missing required key 'name'"]); + pack.spec.alerting.rules = [{ name: 'HighCPU', expr: 'up == 0', engine: 'nagios' }]; + assert.deepEqual(errorsOf(pack), ['$.spec.alerting.rules[0].engine: not in enum ["prometheus","mimir","thanos","victoriametrics","loki","grafana","alertmanager"], got "nagios"']); + pack.spec.alerting.rules = [{ name: 'HighCPU', expr: 'up == 0', severity: 'critical' }]; + assert.deepEqual(errorsOf(pack), ['$.spec.alerting.rules[0].severity: not in enum ["SEV1","SEV2","SEV3","SEV4"], got "critical"'], "the engine's label belongs in labels.severity"); + pack.spec.alerting.rules = [{ name: 'HighCPU', expr: 'up == 0', keep_firing_for: '5m' }]; + assert.equal(errorsOf(pack).length, 1); + assert.match(errorsOf(pack)[0], /^\$\.spec\.alerting\.rules\[0\]/); + // A rule as the crawler emits one: the engine's labels verbatim beside the pack's severity, provenance in source. + pack.spec.alerting.rules = [{ name: 'Payment API — 5xx spike', expr: 'sum(rate(http_requests_total{code=~"5.."}[5m])) > 10', for: '0s', severity: 'SEV2', engine: 'grafana', + labels: { severity: 'critical', team: 'payments' }, annotations: { summary: '5xx above 10/s' }, source: 'grafana/provisioning/alerting/payments.yaml#payments/api/Payment API — 5xx spike' }]; + assert.deepEqual(errorsOf(pack), [], 'a Grafana title with spaces and punctuation is a valid name'); +}); + +// Every schema-valid pack the repo ships or tests with, 1.3-shaped (none carries alerting.rules; none carries good_when +// either, 1.2-shaped as they were): the 1.4 schema accepts each unchanged. (tools/fixtures/compile/ holds the compiler's SHAPE fixtures — partial packs and hostile names, // never schema-valid under any version — so they are not here.) const yamlPacks = (dir) => (existsSync(resolve(ROOT, dir)) ? readdirSync(resolve(ROOT, dir)).filter(f => f.endsWith('.pack.yaml')).map(f => `${dir}/${f}`) : []); const FIXTURES = [ ...yamlPacks('examples'), ...yamlPacks('reference-packs'), ...yamlPacks('tools/fixtures/library'), 'tools/fixtures/site/fixture.pack.yaml', 'tools/fixtures/golden-crawl.pack.json', ]; -test('every 1.2-shaped pack of the repo validates unchanged against the 1.3 schema', () => { +test('every 1.3-shaped pack of the repo validates unchanged against the 1.4 schema', () => { assert.ok(FIXTURES.length >= 12, `fixtures found: ${FIXTURES.length}`); for (const rel of FIXTURES) { const text = readFileSync(resolve(ROOT, rel), 'utf8'); const pack = rel.endsWith('.json') ? JSON.parse(text) : parseYaml(text); assert.deepEqual(errorsOf(pack), [], rel); - assert.ok((pack.spec.slis || []).every(s => !('good_when' in s)), `${rel} is 1.2-shaped: no SLI carries good_when`); + assert.ok(!('rules' in (pack.spec.alerting || {})), `${rel} is 1.3-shaped: alerting carries no rules`); + assert.ok((pack.spec.slis || []).every(s => !('good_when' in s)), `${rel} is 1.2-shaped too: no SLI carries good_when`); // The meaning is unchanged too: with an explicit `below` on every threshold / distribution SLI the pack still validates. const stated = clone(pack); for (const s of stated.spec.slis || []) if (s.type === 'threshold' || s.type === 'distribution') s.good_when = 'below'; @@ -124,9 +169,10 @@ const SCANNED = [ ...walk(resolve(ROOT, 'studio')).filter(p => /\.(mjs|html)$/.test(p)), ...['package.json', '.github/workflows/ci.yml', 'README.md', 'docs/MODEL.md', 'docs/CONFORMANCE.md', 'docs/gen-site.md', 'docs/MCP_INTEGRATION.md', 'docs/ADAPTER.md', 'docs/BUILD_JOURNEY.md', 'library/README.md', 'vendor/observability-pack-spec/README.md'].map(f => resolve(ROOT, f)), ]; -// Lines that name an older version on purpose: the spec's own lineage, RFC-0001 (a 1.2 sibling layer), the layered -// JSON that predates the canonical manifest, the VersionSpec history. -const HISTORY = /RFC-0001|pre-v1\.2|predates (the )?canonical|Spec v1\.2 §VersionSpec|SPEC_v1\.2_GAP|spec 1\.2 → 1\.3|spec 1\.2 -> 1\.3|1\.2 pack|1\.2 packs|1\.2 SLI|1\.2 reader|1\.2 meaning|1\.2 shaped|1\.2-shaped|1\.2 fixture|1\.2 board|1\.2 commit|stopped at 1\.2|every 1\.2|a 1\.2 |v1\.2\/ → v1\.3\/|remove v1\.2\/|v1\.2\/ stays|\(1\.2\)|1\.2 could express/; +// Lines that name an older version on purpose: the spec's own lineage, RFC-0001 (a 1.2 sibling layer), RFC-0002 (1.3's +// good_when — every mention of the direction of a bound dates it), the layered JSON that predates the canonical manifest, +// the VersionSpec history. +const HISTORY = /RFC-0001|RFC-0002|good_when|[Tt]he direction of|since spec 1\.3|has a direction|carries a direction|its direction|the direction of|side of (?:a|the|its) [^.]*bound|the bound|a floor|a ceiling|1\.3 → 1\.4|1\.3 -> 1\.4|v1\.3\/ → v1\.4\/|remove v1\.3\/|1\.3 pack|1\.3 packs|1\.3-shaped|1\.3 example|every 1\.3|a 1\.3 |develop\) 1\.3|develop` 1\.3|pre-v1\.2|predates (the )?canonical|Spec v1\.2 §VersionSpec|SPEC_v1\.2_GAP|spec 1\.2 → 1\.3|spec 1\.2 -> 1\.3|1\.2 pack|1\.2 packs|1\.2 SLI|1\.2 reader|1\.2 meaning|1\.2 shaped|1\.2-shaped|1\.2 fixture|1\.2 board|1\.2 commit|stopped at 1\.2|every 1\.2|a 1\.2 |v1\.2\/ → v1\.3\/|remove v1\.2\/|v1\.2\/ stays|\(1\.2\)|1\.2 could express/; test('no scanned file names a spec version other than the current one', () => { // `(?!\.\d|\d)`: not a longer version (1.2.3, 1.23) — a sentence-final "v1.2." still counts. const other = new RegExp(`\\b(?:spec|Spec|canonical|schema|pack|manifest|ObservabilityPack|valid|version)\\s+v?1\\.\\d(?!\\.\\d|\\d)|vendor/observability-pack-spec/v1\\.\\d(?!\\.\\d|\\d)|\\bv1\\.\\d(?!\\.\\d|\\d)\\b`, 'g'); @@ -139,12 +185,12 @@ test('no scanned file names a spec version other than the current one', () => { for (const m of line.matchAll(other)) if (!m[0].includes(SPEC_VERSION)) offenders.push(`${rel}:${i + 1}: ${m[0]}`); }); } - assert.deepEqual(offenders, [], 'every current-version mention follows SPEC_VERSION (1.3)'); + assert.deepEqual(offenders, [], 'every current-version mention follows SPEC_VERSION (1.4)'); }); -// The studio's links to upstream: the footer's "spec v1.3", the Schema view's "ObservabilityPack v1.3 JSON Schema" -// and "Spec document", the Conformance view's "maturity rubric". Upstream's default branch is develop (1.3) and main -// stopped at the 1.2 commit, so a link labelled with the version must open the commit VERSIONS.json vendored — never +// The studio's links to upstream: the footer's "spec v1.4", the Schema view's "ObservabilityPack v1.4 JSON Schema" +// and "Spec document", the Conformance view's "maturity rubric". Upstream's default branch is develop (1.3), main +// stopped at the 1.2 commit and 1.4 is on its RFC branch, so a link labelled with the version must open the commit VERSIONS.json vendored — never // a branch, which serves whatever it serves that day. After a re-vendor (node tools/sync-spec.mjs) the hrefs follow // upstream.commit; this test names the ones that do not. test('every studio link into the upstream spec repo opens the vendored commit, not a branch', () => { @@ -158,6 +204,6 @@ test('every studio link into the upstream spec repo opens the vendored commit, n for (const m of line.matchAll(link)) { seen.push(rel); if (m[1] !== upstream.commit) offenders.push(`${rel}:${i + 1}: blob/${m[1]}`); } }); } - assert.deepEqual(offenders, [], `a studio link into ${upstream.repo} opens blob/${upstream.commit} (VERSIONS.json upstream.commit; upstream main serves 1.2)`); + assert.deepEqual(offenders, [], `a studio link into ${upstream.repo} opens blob/${upstream.commit} (VERSIONS.json upstream.commit; upstream main serves 1.2, develop 1.3)`); assert.deepEqual([...new Set(seen)].sort(), ['studio/conformance-view.mjs', 'studio/index.html', 'studio/schema-view.mjs'], 'the footer, the Schema view and the Conformance view link upstream'); }); diff --git a/vendor/observability-pack-spec/README.md b/vendor/observability-pack-spec/README.md index 60ca0d6f..c5873ded 100644 --- a/vendor/observability-pack-spec/README.md +++ b/vendor/observability-pack-spec/README.md @@ -8,17 +8,19 @@ is reproducible offline and drift against upstream is detectable. | Vendor path | Upstream path | |---|---| -| `v1.3/observability-pack.schema.json` | `schema/observability-pack.schema.json` | -| `v1.3/spec.md` | `spec/ObservabilityPack-Spec.md` | -| `v1.3/examples/payment-service.pack.yaml` | `examples/payment-service.pack.yaml` | -| `v1.3/docs/maturity-model.md` | `docs/maturity-model.md` | +| `v1.4/observability-pack.schema.json` | `schema/observability-pack.schema.json` | +| `v1.4/spec.md` | `spec/ObservabilityPack-Spec.md` | +| `v1.4/examples/payment-service.pack.yaml` | `examples/payment-service.pack.yaml` | +| `v1.4/docs/maturity-model.md` | `docs/maturity-model.md` | `VERSIONS.json` records the spec version, the upstream ref and commit, the fetch timestamp, and the per-file SHA-256 and byte count of every vendored file (byte for byte the upstream blob — `git show -: | sha256sum` reproduces each checksum). Spec 1.3 (RFC-0002, accepted 2026-09-23) -adds `good_when: below | above` on `threshold` and `distribution` SLIs; it validates every 1.2 pack -unchanged, so one version directory is vendored at a time — `tools/lib/validator.mjs` `SPEC_VERSION` -names it, and every path in the repo is derived from that constant. +: | sha256sum` reproduces each checksum). Spec 1.4 (RFC-0003, proposed 2026-10-03) +adds `alerting.rules[]`, a home for the operational (non-SLO) alert rules a service runs — `name` and +`expr` required, `engine` a closed enum, `routes` still the only required key — so it validates every +1.3 pack unchanged (as 1.3, RFC-0002's `good_when`, validated every 1.2 pack); one version directory is +vendored at a time — `tools/lib/validator.mjs` `SPEC_VERSION` names it, and every path in the repo is +derived from that constant. ## Refreshing @@ -27,18 +29,18 @@ names it, and every path in the repo is derived from that constant. node tools/sync-spec.mjs # Sync to a specific branch, tag or commit -node tools/sync-spec.mjs --ref 98be4ae8 +node tools/sync-spec.mjs --ref 6674072e # Verify on-disk files match VERSIONS.json (CI-friendly) node tools/sync-spec.mjs --check ``` -The version directory is read from the fetched spec's own header (`| Spec version | 1.3 |`), never typed: +The version directory is read from the fetched spec's own header (`| Spec version | 1.4 |`), never typed: a spec bump lands in a new `v/` and `VERSIONS.json` follows it. The previous directory is removed in the commit that moves `SPEC_VERSION` (git history keeps it). The sync script uses the `gh` CLI for authentication and rate-limit headroom; no npm deps. The studio links into the upstream repository at the vendored commit (`upstream.commit`), never at a branch: -upstream `main` still serves 1.2 while `develop` carries 1.3, so a link labelled with the version must open the -file the label names. After a refresh, move the hrefs in `studio/index.html` (footer), `studio/schema-view.mjs` +upstream `main` still serves 1.2 and `develop` 1.3 while 1.4 is on its RFC branch, so a link labelled with the +version must open the file the label names. After a refresh, move the hrefs in `studio/index.html` (footer), `studio/schema-view.mjs` and `studio/conformance-view.mjs` to the new commit; `tools/test-validator.mjs` names the ones left behind. diff --git a/vendor/observability-pack-spec/VERSIONS.json b/vendor/observability-pack-spec/VERSIONS.json index 2522a22a..aaf45d5a 100644 --- a/vendor/observability-pack-spec/VERSIONS.json +++ b/vendor/observability-pack-spec/VERSIONS.json @@ -1,28 +1,28 @@ { - "schema": "1.3", + "schema": "1.4", "upstream": { "repo": "MoebiusX/otel-observability-pack", - "ref": "develop", - "commit": "98be4ae8c05899c066b9882e0498feb850afa387" + "ref": "6674072e4d3593f3e8345c4032a6eec23b7b19b7", + "commit": "6674072e4d3593f3e8345c4032a6eec23b7b19b7" }, - "fetchedAt": "2026-09-24T08:43:33Z", + "fetchedAt": "2026-10-03T12:53:22Z", "files": { - "v1.3/spec.md": { + "v1.4/spec.md": { "sourcePath": "spec/ObservabilityPack-Spec.md", - "sha256": "71ba2d9a7331199deb52d918e9eebad0d6d9ff94708d041defa6d64042792a68", - "bytes": 32095 + "sha256": "6fd908d981a541aca7e3c336b3921245f4fda434e25aa80ad7ee6ccd628795cc", + "bytes": 35689 }, - "v1.3/observability-pack.schema.json": { + "v1.4/observability-pack.schema.json": { "sourcePath": "schema/observability-pack.schema.json", - "sha256": "4c3377421701009b52098d00d1fdaa98fb27c7810db1503ed03e4037ecb6f29c", - "bytes": 28519 + "sha256": "2d7537e83f0b5f97e9936435f9cda7f715c46b0aca3115ea6668b5730c636b4d", + "bytes": 31685 }, - "v1.3/examples/payment-service.pack.yaml": { + "v1.4/examples/payment-service.pack.yaml": { "sourcePath": "examples/payment-service.pack.yaml", - "sha256": "6d8dc694ab5ebfea8506818c36b137dd959f75a786968667b86d27056323e710", - "bytes": 21965 + "sha256": "938f1d68391511651a4fd922826923754a0506130e2f9b0aed0ce4370dfbe44b", + "bytes": 24052 }, - "v1.3/docs/maturity-model.md": { + "v1.4/docs/maturity-model.md": { "sourcePath": "docs/maturity-model.md", "sha256": "def23e1f9c1ace2cb9ef702c58e234c047197aef9aeba1742e31c6bb2da37de4", "bytes": 13763 diff --git a/vendor/observability-pack-spec/v1.3/docs/maturity-model.md b/vendor/observability-pack-spec/v1.4/docs/maturity-model.md similarity index 100% rename from vendor/observability-pack-spec/v1.3/docs/maturity-model.md rename to vendor/observability-pack-spec/v1.4/docs/maturity-model.md diff --git a/vendor/observability-pack-spec/v1.3/examples/payment-service.pack.yaml b/vendor/observability-pack-spec/v1.4/examples/payment-service.pack.yaml similarity index 91% rename from vendor/observability-pack-spec/v1.3/examples/payment-service.pack.yaml rename to vendor/observability-pack-spec/v1.4/examples/payment-service.pack.yaml index 6182648e..005fd606 100644 --- a/vendor/observability-pack-spec/v1.3/examples/payment-service.pack.yaml +++ b/vendor/observability-pack-spec/v1.4/examples/payment-service.pack.yaml @@ -466,6 +466,43 @@ spec: channels: - { msteams: "#payments-team" } + # Operational (non-SLO) alert rules, spec 1.4 — the rules that say something + # is wrong now rather than that an error budget is being spent. `name` is + # the rule's exact name (the key a reconciler joins on against the live + # engine); `severity` is the pack's routing severity, the engine's own + # label stays in `labels`. Burn-rate alerts stay under policy above. + rules: + - name: PaymentServicePodRestarting + expr: increase(kube_pod_container_status_restarts_total{namespace="payments",container="payment-service"}[15m]) > 3 + for: 10m + severity: SEV3 + engine: prometheus + labels: { severity: warning, team: payments } + annotations: + summary: "payment-service restarted more than 3 times in 15m" + runbook_url: "https://runbooks.example.internal/payments/pod-restarts" + source: observability/alerts.yml#payment-service.operational/PaymentServicePodRestarting + + - name: PaymentDbConnectionPoolSaturated + expr: sum(pg_stat_activity_count{datname="payments"}) / sum(pg_settings_max_connections{datname="payments"}) > 0.9 + for: 5m + severity: SEV2 + labels: { severity: critical, team: payments } + annotations: + summary: "payments database connection pool above 90% for 5m" + runbook_url: "https://runbooks.example.internal/payments/db-pool" + source: observability/alerts.yml#payment-service.operational/PaymentDbConnectionPoolSaturated + + - name: PaymentCertificateExpiringSoon # Grafana-managed rule over the blackbox probe's series + expr: min(probe_ssl_earliest_cert_expiry{job="blackbox",instance=~"https://payments.example.com.*"}) - time() < 14 * 24 * 3600 + for: 1h + severity: SEV3 + engine: grafana + labels: { severity: warning, team: platform-sre } + annotations: + summary: "payments.example.com TLS certificate expires in under 14 days" + source: grafana/provisioning/alerting/payments.yaml#payments/certificates/PaymentCertificateExpiringSoon + dedup: ref:platform/std-dedup suppress: [ maintenance_windows, deploy_freezes, dependency_outage ] diff --git a/vendor/observability-pack-spec/v1.3/observability-pack.schema.json b/vendor/observability-pack-spec/v1.4/observability-pack.schema.json similarity index 90% rename from vendor/observability-pack-spec/v1.3/observability-pack.schema.json rename to vendor/observability-pack-spec/v1.4/observability-pack.schema.json index 9e02efd6..079db6f1 100644 --- a/vendor/observability-pack-spec/v1.3/observability-pack.schema.json +++ b/vendor/observability-pack-spec/v1.4/observability-pack.schema.json @@ -376,6 +376,29 @@ "email": { "type": "string" }, "webhook": { "type": "string", "format": "uri" } } }, + "AlertEngine": { "type": "string", "enum": ["prometheus","mimir","thanos","victoriametrics","loki","grafana","alertmanager"], + "description": "The engine that evaluates an operational alert rule's `expr` (spec 1.4). A closed enum, unlike the open Product registry, because a reader dispatches on it: `prometheus`, `mimir`, `thanos` and `victoriametrics` evaluate PromQL in a ruler; `loki` evaluates LogQL; `grafana` is a Grafana-managed (unified alerting) rule whose `expr` is the query of its data node or, when the rule is built from expression nodes only, the condition reference; `alertmanager` is a rule expressed as an Alertmanager matcher set over other alerts. Every value is the name the Product registry gives that product, so a rule joins to `telemetry.backends[].product`. Absent means `prometheus`, the default binding's ruler." }, + + "AlertRule": { + "type": "object", + "required": ["name","expr"], + "additionalProperties": false, + "description": "One operational (non-SLO) alert rule (spec 1.4): a rule a repository or a running engine holds that is not a burn-rate alert on an SLO. `name` is the rule's exact name (`alert:` in a Prometheus or Loki rule file, `title` of a Grafana-managed rule) and is the key a reconciler joins on against the live engine; it is not a Slug on purpose. Burn-rate alerts stay in `policy.burn_rate_alerts`; a rule MUST NOT appear in both.", + "properties": { + "name": { "type": "string", "minLength": 1, "maxLength": 190 }, + "expr": { "type": "string", "minLength": 1, + "description": "The rule's condition in the engine's own language: PromQL, LogQL, or for a Grafana-managed rule the query of its data node or the condition reference (`C`) when the rule has no query node. Not a `Ref`: it is evaluated by the engine, not resolved by the pack." }, + "severity": { "$ref": "#/$defs/Severity", + "description": "The pack's routing severity, so `alerting.routes` apply to the rule. The engine's own label (`critical`, `warning`, …) stays verbatim in `labels.severity`." }, + "for": { "$ref": "#/$defs/Duration" }, + "labels": { "type": "object", "additionalProperties": { "type": "string" } }, + "annotations": { "type": "object", "additionalProperties": { "type": "string" } }, + "engine": { "$ref": "#/$defs/AlertEngine" }, + "source": { "type": "string", "minLength": 1, + "description": "Provenance: where the rule was read from, as `#/` for a rule file, or the folder and group for a Grafana-managed rule. Informational; never used for matching." } + } + }, + "Alerting": { "type": "object", "required": ["routes"], @@ -385,6 +408,8 @@ "properties": { "severity": { "$ref": "#/$defs/Severity" }, "channels": { "type": "array", "minItems": 1, "items": { "$ref": "#/$defs/Channel" } }, "match": { "type": "object", "additionalProperties": { "type": "string" } } } } }, + "rules": { "type": "array", "items": { "$ref": "#/$defs/AlertRule" }, + "description": "Operational (non-SLO) alert rules (spec 1.4): every alert rule of the service that is not a burn-rate alert on an SLO. Optional; `routes` stays the only required key, so every 1.3 pack validates unchanged. An empty array is allowed and means the same as absent: no operational rules." }, "dedup": { "$ref": "#/$defs/Ref" }, "suppress": { "type": "array", "items": { "type": "string", "enum": ["maintenance_windows","deploy_freezes","dependency_outage"] } } } diff --git a/vendor/observability-pack-spec/v1.3/spec.md b/vendor/observability-pack-spec/v1.4/spec.md similarity index 88% rename from vendor/observability-pack-spec/v1.3/spec.md rename to vendor/observability-pack-spec/v1.4/spec.md index 7fff7542..2e0a64ed 100644 --- a/vendor/observability-pack-spec/v1.3/spec.md +++ b/vendor/observability-pack-spec/v1.4/spec.md @@ -2,11 +2,11 @@ | | | |---|---| -| Spec version | 1.3 | +| Spec version | 1.4 | | Status | Draft for review | | Author | Carlos Montero | | First publication | 2026-05-08 | -| Last updated | 2026-09-23 | +| Last updated | 2026-10-03 | | Default binding | `otel-elastic-prometheus-grafana` | | Audience | Service owners, SREs, platform engineers, security & compliance, leadership | @@ -36,9 +36,9 @@ This document defines the contract for an ObservabilityPack: its conceptual mode ### 1.3 Versioning -This document — the standard itself — carries an explicit **spec version** (see the header table; currently **1.3**), a two-part `major.minor` number: the minor part moves for backward-compatible additions and clarifications, the major part for breaking changes to the contract. The lineage: **1.0** was the generic observability standard; **1.1** added the OpenTelemetry instrumentation contract as a separate concern; **1.2** consolidates the two into this single unified document; **1.3** gives `threshold` and `distribution` SLIs a direction (`good_when`, §5.1), an additive minor. Because the consolidation and the addition both preserve the manifest contract (`apiVersion` stays `observability.platform/v1`, existing packs still validate and keep their meaning), each is a backward-compatible minor bump rather than a breaking `2.0`. Beyond the spec version, five independent versioning axes appear inside the contract and should not be confused with one another: +This document — the standard itself — carries an explicit **spec version** (see the header table; currently **1.4**), a two-part `major.minor` number: the minor part moves for backward-compatible additions and clarifications, the major part for breaking changes to the contract. The lineage: **1.0** was the generic observability standard; **1.1** added the OpenTelemetry instrumentation contract as a separate concern; **1.2** consolidates the two into this single unified document; **1.3** gives `threshold` and `distribution` SLIs a direction (`good_when`, §5.1), an additive minor; **1.4** gives operational (non-SLO) alert rules a home (`alerting.rules`, §5.8), an additive minor. Because the consolidation and the additions all preserve the manifest contract (`apiVersion` stays `observability.platform/v1`, existing packs still validate and keep their meaning), each is a backward-compatible minor bump rather than a breaking `2.0`. Beyond the spec version, five independent versioning axes appear inside the contract and should not be confused with one another: -- **Spec version** (`1.3`) — the version of this prose standard, in the header table. Owned by the platform engineering team. `major.minor`; bumped whenever this document changes. +- **Spec version** (`1.4`) — the version of this prose standard, in the header table. Owned by the platform engineering team. `major.minor`; bumped whenever this document changes. - **`apiVersion: observability.platform/v1`** — the stable API surface for pack manifests, in the Kubernetes-style sense. A breaking change to the manifest shape would bump this to `v2`. Tracks the spec's major version but is not identical to it: an editorial spec patch does not move `apiVersion`. - **`metadata.version`** on each pack — SemVer per pack instance, owned by the service team. Bumped on every pack change. Unrelated to the spec. - **Binding name** (currently `otel-elastic-prometheus-grafana`) — the realisation contract for a specific stack. Future bindings live as separate documents under `bindings/` and do not bump the `apiVersion`. @@ -286,10 +286,33 @@ Routes triggered policy events to humans or automation. The default binding supp Three suppression contexts: `maintenance_windows`, `deploy_freezes`, `dependency_outage`. Suppression MUST NOT silence the underlying alert in the audit log. +**Operational alert rules (`alerting.rules`, since 1.4).** Not every alert a service runs is a burn-rate alert on an SLO. A pod restarting, a connection pool near its limit, a certificate about to expire, an error-log spike: these are operational alerts — they say something is wrong now, not that an error budget is being spent — and they outnumber burn-rate alerts in every real repository and every running Grafana or Prometheus. Spec 1.3 had no home for them: `policy.burn_rate_alerts` requires an SLO, so a reader that wanted to be faithful to the pack had to drop every other rule, and a reconciler comparing the pack with the live engine found nothing to match the engine's rules against although the repository held the very files they were provisioned from. `alerting.rules[]` declares them, one entry per rule, as the engine holds them: + +```yaml +alerting: + routes: [ ... ] + rules: + - name: PaymentServicePodRestarting # the rule's exact name: the key a reconciler joins on + expr: increase(kube_pod_container_status_restarts_total{namespace="payments",container="payment-service"}[15m]) > 3 + for: 10m + severity: SEV3 # the pack's routing severity; the engine's own label stays in labels + engine: prometheus # default; also mimir | thanos | victoriametrics | loki | grafana | alertmanager + labels: { severity: warning, team: payments } + annotations: { summary: "payment-service restarted more than 3 times in 15m" } + source: observability/alerts.yml#payment-service.operational/PaymentServicePodRestarting +``` + +`name` and `expr` are required; everything else is optional. `name` is the rule's exact name — `alert:` in a Prometheus or Loki rule file, `title` of a Grafana-managed rule — and is deliberately not a Slug: it is the join key against the live engine, so it must be the engine's spelling. `expr` is the condition in the engine's own language (PromQL, LogQL; for a Grafana-managed rule the query of its data node, or the condition reference when the rule is built from expression nodes only); it is evaluated by the engine, never resolved by the pack. `engine` names the evaluator with the Product registry's name for it, so a rule joins to `telemetry.backends[].product`; absent means `prometheus`. `severity` is the pack's SEV1–SEV4 so that `routes` apply to the rule; the engine's own `labels.severity` (`critical`, `warning`, …) is kept verbatim in `labels`. `source` records where the rule was read from and is never used for matching. + +The division of labour is by kind, not by engine: a rule that guards an SLO's recorded series is a burn-rate alert and belongs in `policy.burn_rate_alerts`; every other rule belongs here. A rule MUST NOT appear in both. `routes` stays the only required key of `alerting`, so every 1.3 pack validates unchanged, and an absent or empty `rules` means the service declares no operational rules. + **Conformance:** - MUST: tier-1 SEV1 routes include at least one voice channel. - MUST: every alert has at least one routing rule. +- MUST: every `alerting.rules[]` entry has a `name` and an `expr`; an `engine` outside the enumerated set is invalid; absent means `prometheus`. +- MUST: a rule appears in `alerting.rules` or is a burn-rate alert in `policy.burn_rate_alerts`, never both. - SHOULD: chat-only routing reserved for SEV3 and below. +- SHOULD: an operational rule declares `severity` so that a route applies to it. ### 5.9 Self-healing remediation (L4) @@ -513,6 +536,7 @@ Exceptions are time-bounded (default 90 days), reviewed by the platform engineer | `policy.burn_rate_alerts` | Prometheus alerting rules | Alertmanager | | `dashboards` | Grafana dashboard JSON (schemaVersion >= 39) | Grafana 11 | | `alerting.routes` | Alertmanager route tree + receivers | Alertmanager / PagerDuty | +| `alerting.rules` | Prometheus / Loki alerting rule group YAML, or Grafana-managed (provisioned) alert rules, per `engine` | Prometheus / Mimir ruler, Loki ruler, Grafana unified alerting | | `remediation` | Argo Events Sensor + WorkflowTemplate | Argo Workflows | | `baselines` | Derived MTTD/MTTR metrics | Platform observability service | | `validation.chaos` | Chaos Mesh Workflow/Schedule CRDs | Chaos Mesh | From 52210bc3e3be07a3964a296f5096c01de3f094a1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 13:08:09 +0000 Subject: [PATCH 026/550] feat(crawler): operational alerts are kept in alerting.rules, name for name, instead of dropped MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every alert rule the crawler discovers that is not an SLO burn-rate alert was dropped at the policy step with "Excluded N operational alert(s) from burn-rate policy … They remain available as alerting signals" — a promise nothing kept: spec 1.3 had no field for them, so they were emitted nowhere. A live Grafana snapshot reconciled against the crawled pack therefore matched zero alerts although the repository held the very provisioning files those rules came from, with identical titles. Spec 1.4 (RFC-0003, vendored in the previous commit) gives them a home, and the crawler now fills it: spec.alerting.rules[] carries each operational rule with its EXACT name (the join key a live listing is reconciled on — never slugged), expr, for, severity, labels, annotations, engine and source. The classification is unchanged: a rule whose expression references a recorded series of an SLO the repo defines is a burn-rate alert and stays in policy.burn_rate_alerts exactly as before (same linking, same _drop, same merge per SLO); every other rule is operational and goes to alerting.rules; a rule is in one place, never both. A repository with no recording rules still reads every alert as an SLO contract (the tier-3 fallback) and then declares no operational rules — that fallback is also unchanged. tools/lib/sli-inference.mjs — the one reading for the repository and for live, as for SLIs and burn-rate entries: - operationalAlertRule(rule, { engine, source }): name from alert: / title: / name; expr from alertRuleExpr; severity the pack's SEV1..SEV4 from labels.severity through normalizeSeverity (alert-routes.mjs, the mapping the Alertmanager routes already use), only when the rule states a severity label; for through canonicalRuleDuration (`600s` in a file and `duration: 600` from a ruler are one `10m`; none or zero omitted); labels and annotations verbatim as string maps, the engine's own severity word among them; engine and source as the caller knows them. Null when the rule has no name or no expression. - operationalAlertRules(rules, hasSlo, { engineOf, sourceOf }): the complement of recordedSloForExpr — kept entries and the names skipped. - alertRuleExpr(rule): expr, or query (a ruler), or for a Grafana-managed rule the first query node's model.expr (model a mapping or a JSON string; expression nodes — reduce, math, threshold, classic_conditions, resample, sql, or datasource __expr__ — are skipped), else the condition reference, as spec 1.4 §5.8 allows. burnCandidateFromAlertRule reads its expr through it, so a Grafana-provisioned burn-rate alert that references a recorded SLO series links like a Prometheus one. - isGrafanaManagedRule(rule): `title` + `data[]`. tools/lib/crawler.mjs: - every alert rule that reads is kept raw with its file and group (walkPrometheusRule, the per-rule shape #138 introduced; a rule #138 skips because its model JSON does not parse is not declared either). After the burn-rate step the operational ones are read with operationalAlertRules; evidence RULE- records each source (`#/`), summary.discovered.alertRules counts them, and spec.alerting.rules is emitted only when there are some (absent means none, spec 1.4), so a repo without operational alerts produces the same pack as before. - the "Excluded … remain available" warning is replaced by a true summary line: "N operational alert rule(s) kept in alerting.rules — not SLO burn-rate alerts (no recorded-ratio reference): . M burn-rate alert(s) in policy.burn_rate_alerts." A rule with no expression is named in a second line and not declared. - engine (spec 1.4 $defs/AlertEngine, the Product registry's names): grafana for a Grafana-managed rule by its shape; loki for a rule that ships under a loki path or whose expression is LogQL (a stream selector followed by a line filter, a parser stage or a range); prometheus for the rule-file format otherwise — the default, since a rule file cannot tell a Prometheus ruler from a Mimir, Thanos or VictoriaMetrics one. - detectArtefactKind: a Grafana provisioning file is a rule file by its path (`provisioning/alerting/.yaml`) or by its shape (a group with a `title` + `data[]` rule), not only by a *rules / *alerts name. tools/crawl-repo.mjs prints the count beside the burn-rate alerts. Goldens: tools/fixtures/golden-crawl.pack.json is byte-identical — its one alert references the recorded ratio and is a burn-rate alert, so the fixture has no operational rule and alerting.rules is absent (verified by npm run test:golden). npm run test:golden:compile is byte-identical: the compile targets consume policy.burn_rate_alerts and alerting.routes only. The crawl, scan-live, fetch-live, validator, contract-guard and compile suites pass; eslint 0 errors, no new warning. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- tools/crawl-repo.mjs | 1 + tools/lib/crawler.mjs | 100 ++++++++++++++++++++++++------ tools/lib/sli-inference.mjs | 117 +++++++++++++++++++++++++++++++++++- 3 files changed, 199 insertions(+), 19 deletions(-) diff --git a/tools/crawl-repo.mjs b/tools/crawl-repo.mjs index 987e10be..6fc10357 100644 --- a/tools/crawl-repo.mjs +++ b/tools/crawl-repo.mjs @@ -155,6 +155,7 @@ async function main() { `# backends : ${summary.discovered.backends}`, `# recording rules : ${summary.discovered.recordingRules}`, `# burn-rate alerts : ${summary.discovered.burnRateAlerts}`, + `# alert rules : ${summary.discovered.alertRules} (operational, alerting.rules)`, `# metric definitions: ${summary.discovered.metricDefinitions}`, `# scrape jobs : ${summary.discovered.scrapeJobs}`, `# dashboards : ${summary.discovered.dashboards}`, diff --git a/tools/lib/crawler.mjs b/tools/lib/crawler.mjs index 91dabb09..baabf7a3 100644 --- a/tools/lib/crawler.mjs +++ b/tools/lib/crawler.mjs @@ -25,6 +25,7 @@ import { parse as parseYaml, parseAll as parseYamlAll, emit as emitYaml } from ' import { inferSlisFromRecordingRules, ruleNameToSliId, burnCandidateFromAlertRule, recordedSloForExpr, mergeBurnAlertsBySlo, defaultBurnWindows, + operationalAlertRules, isGrafanaManagedRule, alertRuleExpr, } from './sli-inference.mjs'; import { materializeL2XFromBackends } from './l2x.mjs'; import { routesFromAlertmanagerConfig } from './alert-routes.mjs'; @@ -219,6 +220,9 @@ export function detectArtefactKind(relPath, content) { if (/alertmanager(\.config)?\.ya?ml$/.test(lower)) return 'alertmanager'; if (/(^|\/)(otel|otelcol|collector)[-_a-z]*\.ya?ml$/.test(lower)) return 'otel-collector'; if (/(rules?|alerts?|recording|burn[-_ ]?rate)\.ya?ml$/.test(lower)) return 'prometheus-rules'; + // Grafana unified-alerting provisioning (`provisioning/alerting/.yaml`): + // rule groups of `title` + `data[]` rules, walked by the rules walker. + if (/(^|\/)provisioning\/alerting\/[^/]+\.ya?ml$/.test(lower)) return 'prometheus-rules'; // Content sniff fallback. let obj; @@ -226,7 +230,7 @@ export function detectArtefactKind(relPath, content) { if (!obj || typeof obj !== 'object') return 'unknown'; if (obj.groups && Array.isArray(obj.groups) - && obj.groups.some(g => g.rules?.some(r => 'record' in r || 'alert' in r))) { + && obj.groups.some(g => g.rules?.some(r => r && typeof r === 'object' && ('record' in r || 'alert' in r || isGrafanaManagedRule(r))))) { return 'prometheus-rules'; } if (Array.isArray(obj.scrape_configs) @@ -449,7 +453,7 @@ export function crawlFiles(filesInput, opts = {}) { diffScopeMode, }, discovered: { - backends: 0, recordingRules: 0, burnRateAlerts: 0, + backends: 0, recordingRules: 0, burnRateAlerts: 0, alertRules: 0, metricDefinitions: 0, scrapeJobs: 0, pipelines: 0, dashboards: 0, alertingRoutes: 0, extendedSurfaces: 0, @@ -467,6 +471,7 @@ export function crawlFiles(filesInput, opts = {}) { const backends = []; const recordingRules = []; const burnRateAlerts = []; + const alertRules = []; // every alert rule found, raw, with its file and group (classified below) const metricDefinitions = []; const scrapeJobs = []; const dashboards = []; @@ -500,14 +505,14 @@ export function crawlFiles(filesInput, opts = {}) { try { switch (f.kind) { case 'docker-compose': walkDockerCompose(f, backends, pipelines, evidence, summary); break; - case 'prometheus-rules': walkPrometheusRules(f, recordingRules, burnRateAlerts, metricDefinitions, evidence, summary); break; + case 'prometheus-rules': walkPrometheusRules(f, recordingRules, burnRateAlerts, alertRules, metricDefinitions, evidence, summary); break; case 'prometheus-scrape-config': walkPrometheusScrapeConfig(f, scrapeJobs, evidence, summary); break; case 'actuator-metrics-config': walkActuatorMetricsConfig(f, scrapeJobs, evidence, summary, repoName); break; case 'metric-source-code': walkMetricSourceCode(f, metricDefinitions, evidence, summary, repoName); break; case 'alertmanager': walkAlertmanager(f, alertingRoutes, evidence, summary); break; case 'otel-collector': walkOtelCollector(f, pipelines, evidence, summary); break; case 'grafana-dashboard':walkGrafanaDashboard(f, dashboards, metricDefinitions, evidence, summary); break; - case 'helm-template': walkHelmTemplate(f, { backends, recordingRules, burnRateAlerts, metricDefinitions, scrapeJobs, alertingRoutes, dashboards, pipelines, repoName }, evidence, summary); break; + case 'helm-template': walkHelmTemplate(f, { backends, recordingRules, burnRateAlerts, alertRules, metricDefinitions, scrapeJobs, alertingRoutes, dashboards, pipelines, repoName }, evidence, summary); break; case 'helm-values': walkHelmValues(f, backends, pipelines, evidence, summary, disabledComponents); break; case 'k8s-workload': walkK8sWorkload(f, backends, pipelines, evidence, summary); break; // Chart.yaml itself carries no observability contracts — the payloads @@ -579,18 +584,44 @@ export function crawlFiles(filesInput, opts = {}) { } } - // Drop operational (non-SLO) alerts, then fold the remaining burn-rate - // alerts that now share a recording-rule SLO into one entry per SLO, - // unioning their windows so the emitted policy stays valid. - const droppedOps = burnRateAlerts.filter(a => a._drop).length; - if (droppedOps) { - summary.warnings.push(`Excluded ${droppedOps} operational alert(s) from burn-rate policy — not SLO burn-rate alerts (no recorded-ratio reference). They remain available as alerting signals.`); - } + // Take the operational (non-SLO) alerts out of the burn-rate policy, then + // fold the remaining burn-rate alerts that now share a recording-rule SLO + // into one entry per SLO, unioning their windows so the emitted policy + // stays valid. for (let i = burnRateAlerts.length - 1; i >= 0; i--) { if (burnRateAlerts[i]._drop) burnRateAlerts.splice(i, 1); } mergeBurnAlertsBySlo(burnRateAlerts); + // The operational alerts are not lost: spec 1.4 keeps them in + // spec.alerting.rules, one entry per rule with its EXACT name (the key a + // live Grafana or ruler listing is reconciled on), its expression, wait, + // labels, annotations, the engine that evaluates it and where it was read + // from (sli-inference.mjs operationalAlertRule — the reading the live + // fetcher applies to a ruler's rules). The classification is the one + // above, exactly: a rule whose expression references a recorded SLO + // series is a burn-rate alert and stays in the policy; every other rule + // is operational, when the repo records SLIs at all — a repo with no + // recording rules still reads every alert as an SLO contract (the tier-3 + // fallback), and then declares no operational rules. + const ruleMeta = new Map(alertRules.map(r => [r.rule, r])); + const operational = haveRecordedSlis + ? operationalAlertRules(alertRules.map(r => r.rule), (id) => sloMap.has(id), { + engineOf: (rule) => alertRuleEngine(rule, ruleMeta.get(rule)?.relPath), + sourceOf: (rule) => ruleMeta.get(rule)?.source, + }) + : { kept: [], skipped: [] }; + summary.discovered.alertRules = operational.kept.length; + if (operational.kept.length) { + const names = operational.kept.map(r => r.name); + const shown = names.slice(0, 8).join(', ') + (names.length > 8 ? `, … (${names.length - 8} more)` : ''); + summary.warnings.push(`${operational.kept.length} operational alert rule(s) kept in alerting.rules — not SLO burn-rate alerts (no recorded-ratio reference): ${shown}. ${burnRateAlerts.length} burn-rate alert(s) in policy.burn_rate_alerts.`); + } + if (operational.skipped.length) { + summary.warnings.push(`${operational.skipped.length} alert rule(s) state no expression and are not declared: ${operational.skipped.slice(0, 8).join(', ')}.`); + } + operational.kept.forEach((r, i) => { evidence[`RULE-${i + 1}`] = r.source; }); + // If still no SLI/SLO discovered, fill the minimum tier-3 stub so the // result validates. @@ -785,7 +816,11 @@ export function crawlFiles(filesInput, opts = {}) { queries: { recording_rules: recordingRules }, dashboards, policy: { burn_rate_alerts: burnRateAlerts.map(({ slo, windows }) => ({ slo, windows })) }, - alerting: { routes: alertingRoutes }, + alerting: { + routes: alertingRoutes, + // Absent means "no operational rules" (spec 1.4): the key appears only when there are some. + ...(operational.kept.length ? { rules: operational.kept } : {}), + }, baselines, validation: { synthetic_checks: [{ @@ -1617,10 +1652,10 @@ function walkK8sWorkload(f, backends, pipelines, evidence, summary) { // like a rule shipped as a standalone file. If the template embeds nothing, we // strip the scaffolding and treat the document itself as a single manifest. function walkHelmTemplate(f, buckets, evidence, summary) { - const { backends, recordingRules, burnRateAlerts, metricDefinitions, scrapeJobs, alertingRoutes, dashboards, pipelines, repoName } = buckets; + const { backends, recordingRules, burnRateAlerts, alertRules, metricDefinitions, scrapeJobs, alertingRoutes, dashboards, pipelines, repoName } = buckets; const route = (kind, sub) => { switch (kind) { - case 'prometheus-rules': walkPrometheusRules(sub, recordingRules, burnRateAlerts, metricDefinitions, evidence, summary); return true; + case 'prometheus-rules': walkPrometheusRules(sub, recordingRules, burnRateAlerts, alertRules, metricDefinitions, evidence, summary); return true; case 'prometheus-scrape-config': walkPrometheusScrapeConfig(sub, scrapeJobs, evidence, summary); return true; case 'actuator-metrics-config': walkActuatorMetricsConfig(sub, scrapeJobs, evidence, summary, repoName); return true; case 'alertmanager': walkAlertmanager(sub, alertingRoutes, evidence, summary); return true; @@ -1721,6 +1756,29 @@ function inferEndpoint(svc, product) { return null; } +// A LogQL condition: a stream selector (`{app="x"}`) that is not a metric's +// label matcher — it opens the expression or follows an operator / paren — +// and is followed by a line filter (`|=`, `|~`, `!=`, `!~`), a parser stage +// (`| json`, `| logfmt`, …) or a range (`[5m]`), as `rate({app="x"}[5m])` is. +const LOGQL_STAGE_RE = /\{[^}]*\}\s*(?:\|[=~]|![=~]|\|\s*(?:json|logfmt|pattern|regexp|unpack|unwrap|line_format|label_format|drop|keep|decolorize)\b)/; +const LOGQL_RANGE_RE = /(?:^|[(\s,])\{[^}]*\}\s*\[/; +function looksLikeLogQL(expr) { + const e = String(expr || ''); + return LOGQL_STAGE_RE.test(e) || LOGQL_RANGE_RE.test(e); +} + +// The engine that evaluates an alert rule, in the Product registry's words +// (spec 1.4 $defs/AlertEngine): a Grafana-managed rule by its shape +// (`title` + `data[]`), a Loki rule by the path it ships under or by its +// LogQL, everything else the Prometheus rule-file format — `prometheus`, +// the default, since a rule file cannot tell a Prometheus ruler from a +// Mimir, Thanos or VictoriaMetrics one (the backends do). +function alertRuleEngine(rule, relPath) { + if (isGrafanaManagedRule(rule)) return 'grafana'; + if (/(^|[/_.-])loki([/_.-]|$)/i.test(String(relPath || '')) || looksLikeLogQL(alertRuleExpr(rule))) return 'loki'; + return 'prometheus'; +} + // The PromQL a Grafana unified-alerting rule evaluates. A provisioned rule // carries no `expr`; its queries sit in `data[].model`, one entry per // datasource query or expression node. The model is a mapping in Grafana's @@ -1741,7 +1799,7 @@ function grafanaRuleExpr(rule) { return null; } -function walkPrometheusRules(f, recordingRules, burnRateAlerts, metricDefinitions, evidence, summary) { +function walkPrometheusRules(f, recordingRules, burnRateAlerts, alertRules, metricDefinitions, evidence, summary) { // A rule that cannot be read is skipped on its own; the file and its other // rules survive, and one warning per file says how many were lost and why. let ruleCount = 0; @@ -1752,7 +1810,7 @@ function walkPrometheusRules(f, recordingRules, burnRateAlerts, metricDefinition for (const rule of group.rules || []) { ruleCount++; try { - walkPrometheusRule(f, group, rule, recordingRules, burnRateAlerts, metricDefinitions, evidence, summary); + walkPrometheusRule(f, group, rule, recordingRules, burnRateAlerts, alertRules, metricDefinitions, evidence, summary); } catch (e) { skipped.push(e?.message || String(e)); } @@ -1767,7 +1825,7 @@ function walkPrometheusRules(f, recordingRules, burnRateAlerts, metricDefinition } } -function walkPrometheusRule(f, group, rule, recordingRules, burnRateAlerts, metricDefinitions, evidence, summary) { +function walkPrometheusRule(f, group, rule, recordingRules, burnRateAlerts, alertRules, metricDefinitions, evidence, summary) { if (rule.record) { const id = `QRY-${recordingRules.length + 1}-${slug(rule.record).slice(0, 16)}`; const expr = typeof rule.expr === 'string' ? rule.expr : String(rule.expr || ''); @@ -1799,8 +1857,14 @@ function walkPrometheusRule(f, group, rule, recordingRules, burnRateAlerts, metr // the shared derivation (sli-inference.mjs) the live fetcher applies // to the ruler's rules: one rule set, one policy. const grafanaExpr = typeof rule.expr === 'string' ? null : grafanaRuleExpr(rule); - const candidate = burnCandidateFromAlertRule(grafanaExpr === null ? rule : { ...rule, expr: grafanaExpr }); + const read = grafanaExpr === null ? rule : { ...rule, expr: grafanaExpr }; + const candidate = burnCandidateFromAlertRule(read); const { alertName, expr } = candidate; + // Every alert rule that reads is kept raw as well, with where it was + // read from: the ones that turn out to guard no SLO are declared in + // alerting.rules (crawlFiles), name for name. A rule skipped above + // (its model could not be read) is not declared either. + alertRules.push({ rule: read, relPath: f.relPath, source: `${f.relPath}#${group.name || '_'}/${alertName}` }); if (!burnRateAlerts.some(a => a.slo === candidate.slo)) { burnRateAlerts.push(candidate); const id = `pol-${burnRateAlerts.length}`; diff --git a/tools/lib/sli-inference.mjs b/tools/lib/sli-inference.mjs index f7d26dfc..d7671556 100644 --- a/tools/lib/sli-inference.mjs +++ b/tools/lib/sli-inference.mjs @@ -17,10 +17,15 @@ // contributes (the SLO it guards, its window) is derived here too, so the // rules a repository declares and the rules a ruler reports turn into the // SAME spec.policy.burn_rate_alerts — see burnAlertsFromAlertRules below. +// And the rules that guard no SLO — the operational alerts spec 1.4 keeps +// in spec.alerting.rules — are read into that shape here as well +// (operationalAlertRule), so a rule file and a ruler's listing of the same +// rule become the same entry, name for name. // // Pure ESM, no Node APIs. import { symbolSlug } from './slug.mjs'; +import { normalizeSeverity } from './alert-routes.mjs'; // `::` — Prometheus recording-rule naming convention. const RULE_NAME_RE = /^([a-z][a-z0-9_]*):([a-z][a-z0-9_]*):([a-z0-9_]+)$/; @@ -217,17 +222,127 @@ export function recordedSloForExpr(expr, hasSlo) { return null; } +// Grafana expression-node types: a `data[]` node of one of these is a +// reduce / math / threshold step over another node, not the query itself. +const GRAFANA_EXPRESSION_TYPES = new Set(['reduce', 'math', 'threshold', 'classic_conditions', 'resample', 'sql']); + +/** A Grafana-managed (unified alerting) rule as provisioning YAML or the ruler API spells it: `title` + `data[]`. */ +export function isGrafanaManagedRule(rule) { + return !!rule && typeof rule === 'object' && typeof rule.title === 'string' && Array.isArray(rule.data); +} + +// A Grafana data node's `model`: an object in provisioning YAML, sometimes a +// JSON string on the wire. Unparseable is `null`, never a throw — one bad +// node must not lose the rule. +function grafanaModel(node) { + const m = node?.model; + if (m && typeof m === 'object') return m; + if (typeof m === 'string') { try { return JSON.parse(m); } catch { return null; } } + return null; +} + +/** + * The condition an alerting rule evaluates, in the engine's own words. + * A Prometheus or Loki rule says it in `expr`. A Grafana-managed rule + * says it in its query node (`data[].model.expr`, the first node that is + * not an expression step); a rule built from expression nodes only has no + * query text, and its `condition` reference (`C`) stands for it, as spec + * 1.4 §5.8 allows. Empty when the rule states none. + */ +export function alertRuleExpr(rule) { + if (typeof rule?.expr === 'string' && rule.expr.trim()) return rule.expr; + if (rule?.expr != null && typeof rule.expr !== 'object') return String(rule.expr); + if (typeof rule?.query === 'string' && rule.query.trim()) return rule.query; + if (Array.isArray(rule?.data)) { + for (const node of rule.data) { + const model = grafanaModel(node); + if (!model || typeof model.expr !== 'string' || !model.expr.trim()) continue; + if (GRAFANA_EXPRESSION_TYPES.has(model.type) || node?.datasourceUid === '__expr__' || model.datasource?.uid === '__expr__') continue; + return model.expr; + } + if (typeof rule.condition === 'string' && rule.condition.trim()) return rule.condition.trim(); + const first = rule.data.find(n => typeof n?.refId === 'string' && n.refId.trim())?.refId; + if (first) return first.trim(); + } + return ''; +} + /** The entry an alerting rule starts as, keyed by the id its NAME derives. */ export function burnCandidateFromAlertRule(rule) { const alertName = rule?.alert || rule?.title || rule?.name || ''; return { slo: symbolSlug(alertName).replace(/-burn-?rate.*$/, '_99'), windows: alertRuleWindows(rule), - expr: typeof rule?.expr === 'string' ? rule.expr : String(rule?.expr || ''), + expr: alertRuleExpr(rule), alertName, }; } +// A label / annotation map as the schema wants it: string values only. +function stringMap(value) { + const out = {}; + if (!value || typeof value !== 'object' || Array.isArray(value)) return out; + for (const [k, v] of Object.entries(value)) { + if (v === null || v === undefined) continue; + out[k] = typeof v === 'string' ? v : typeof v === 'object' ? JSON.stringify(v) : String(v); + } + return out; +} + +/** + * An operational (non-SLO) alert rule as spec 1.4 `alerting.rules[]` + * declares it, read from a rule as a rule file (`alert:` / `title:`) or a + * ruler (`name`, `query`, `duration`) spells it — the one reading for the + * repository and for live, so the two sides pair by name: + * name the rule's exact name (the join key; never slugged) + * expr alertRuleExpr: PromQL / LogQL, or a Grafana query node + * severity the pack's SEV1..SEV4 from labels.severity (normalizeSeverity), + * only when the rule states a severity label + * for canonicalRuleDuration: `600s` in a file and `duration: 600` + * from a ruler are one `10m`; none / zero is omitted + * labels, annotations verbatim (string values), the engine's severity + * word among them + * engine, source what the caller knows: the evaluator and the + * provenance (`#/`) + * Null when the rule has no name or no expression — nothing to declare. + */ +export function operationalAlertRule(rule, { engine, source } = {}) { + const name = String(rule?.alert ?? rule?.title ?? rule?.name ?? '').trim(); + const expr = alertRuleExpr(rule).trim(); + if (!name || !expr) return null; + const out = { name, expr }; + const labels = stringMap(rule?.labels); + const annotations = stringMap(rule?.annotations); + if (labels.severity) out.severity = normalizeSeverity(labels.severity); + const wait = canonicalRuleDuration(rule?.for ?? rule?.duration); + if (wait) out.for = wait; + if (Object.keys(labels).length) out.labels = labels; + if (Object.keys(annotations).length) out.annotations = annotations; + if (engine) out.engine = engine; + if (source) out.source = source; + return out; +} + +/** + * Operational rules among `rules`: the ones whose expression references + * no recorded series of an SLO `hasSlo` knows — the complement of the + * burn-rate classification (recordedSloForExpr), so a rule is one or the + * other, never both. Each is read with operationalAlertRule; `engineOf` + * and `sourceOf` supply what only the caller knows. Rules with no name or + * expression are skipped and returned in `skipped`. + */ +export function operationalAlertRules(rules, hasSlo, { engineOf = () => undefined, sourceOf = () => undefined } = {}) { + const kept = []; + const skipped = []; + for (const rule of rules || []) { + if (recordedSloForExpr(alertRuleExpr(rule), hasSlo)) continue; + const entry = operationalAlertRule(rule, { engine: engineOf(rule), source: sourceOf(rule) }); + if (entry) kept.push(entry); + else skipped.push(String(rule?.alert ?? rule?.title ?? rule?.name ?? '').trim() || '(unnamed)'); + } + return { kept, skipped }; +} + /** * Fold entries that share an SLO into one, unioning their windows * (de-duplicated by short/long/factor). MUTATES and returns `alerts`. From 7446237bfe0fc8c3c1d49d41e91a0251402ce9ac Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 13:13:18 +0000 Subject: [PATCH 027/550] feat(adapter): alerting.rules project as L4 alerting artefacts keyed by name; the live fetcher declares them too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tools/lib/adapter.mjs: spec.alerting.rules[i] projects into the layered object's L4 alerting column beside the routes — RULE-NN, title the rule's exact name, desc the engine, wait, the pack's severity and the engine's own label when they differ, tool by engine (PrometheusRule for prometheus / mimir / thanos / victoriametrics, the operator's rendering; Grafana alerting for a Grafana-managed rule; Loki ruler; Alertmanager), tags alert-rule + engine + severity, source = sourceOf(alerting.rules[i]) so mcp.verified.* and *.scaffold.* stamps apply as for every artefact. tools/lib/artefact-model.mjs: RULE- classifies as alert_rule; its identity is the rule's name (lower-cased, as recording rules key on their record name) — never the positional index — so the rule a repository declares and the same rule a live Grafana or ruler reports are ONE artefact in a comparison (diff.mjs pairs on identityKeyOf). Behaviour is expr (normalised like every expression), for, labels and severity; `engine` is left out on purpose — a rule file cannot tell a Prometheus ruler from a VictoriaMetrics one and a live listing may know more, which is reader knowledge, not deployed behaviour; `source` and `annotations` were volatile already. tools/fetch-live-pack.mjs: the alerting rules the ruler reports that bind to no SLO (linkedBurn.unlinked — counted until now in mcp.discovered.alert_rules_operational and carried nowhere) are declared in spec.alerting.rules through the same reading the crawler applies to a rule file (sli-inference.mjs operationalAlertRule), engine from what the answering tool attests (the contract registry's `attests`: grafana, mimir, vmalert → victoriametrics; a plain Prometheus-API answer states none), each stamped mcp.verified.alerting.rules[i] unless the ruler reports it unhealthy. A rule listed without its expression cannot be declared (expr is required) and is left to alert_rule_names. Studio: the RULE- prefix in the card-kind table, the drawer (a panel with name, engine, severity, for, the expression, labels, annotations and source), the Discover board (an optional "Operational alert rules" group under L4) and the Build model's type table and symbol map (RULE-NN → alerting.rules[i]). Conformance (tools/lib/conformance.mjs) is unaffected: it reads alerting.routes and policy.burn_rate_alerts only. The traceability graph and blast radius give an unlisted kind the default weight and never verify it live — the rules appear in the compare and the layers, not in the requirement chains; a later change can add them. The compile goldens are byte-identical (compile targets consume policy and routes). Tests. tools/test-adapt.mjs: L4.alerting is 6 for the 1.4 example (three routes, three rules); RULE-01..03 titled by name, tool by engine, Declared, tagged, desc as stated; and the pairing — a "live" copy of the example whose rules carry no engine, no source, no annotations and a trailing newline in every expression pairs rule for rule (3 inBoth), the unchanged ones aligned, the one whose expression changed drifted on expr, a rule only the live side holds "live, not declared". tools/test-scan-live.mjs: the repository's DiskAlmostFull and the ruler's DiskAlmostFull pair as one alert_rule and read aligned with no delta although the file says prometheus and vmalert attests victoriametrics; and when the latency rules are not deployed, the latency alerting rule the ruler still evaluates reads as a live operational rule — the one "live, not declared" entry, and a true one. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- studio/build-model.mjs | 3 ++- studio/card-html.mjs | 1 + studio/discover-board.mjs | 1 + studio/drawer.mjs | 19 ++++++++++++++++ tools/fetch-live-pack.mjs | 29 +++++++++++++++++++++++- tools/lib/adapter.mjs | 37 +++++++++++++++++++++++++++++- tools/lib/artefact-model.mjs | 16 +++++++++++++ tools/test-adapt.mjs | 44 +++++++++++++++++++++++++++++++++++- tools/test-scan-live.mjs | 18 ++++++++++++++- 9 files changed, 163 insertions(+), 5 deletions(-) diff --git a/studio/build-model.mjs b/studio/build-model.mjs index 4ce7a537..07acf621 100644 --- a/studio/build-model.mjs +++ b/studio/build-model.mjs @@ -1356,6 +1356,7 @@ export const ARTEFACT_TYPES = [ { match: /^POL-/, type: 'Burn-rate alerts', purpose: 'when an objective is at risk' }, { match: /^FCST-/, type: 'Forecasts', purpose: 'an objective at risk before it burns' }, { match: /^ALR-/, type: 'Alert routes', purpose: 'who is told, and how' }, + { match: /^RULE-/, type: 'Alert rules', purpose: 'operational alerts: something is wrong now' }, { match: /^HEAL-/, type: 'Remediations', purpose: 'what runs when it breaks' }, { match: /^SYN-/, type: 'Synthetic probes', purpose: 'checks that the service answers' }, { match: /^CHAOS-/, type: 'Chaos experiments', purpose: 'proof that the alerts fire' }, @@ -1557,7 +1558,7 @@ export function artefactSymbol(a) { if (fixed[id]) return fixed[id]; const indexed = [ [/^PIP-RCV-(\d+)$/, 'pipelines.receivers'], [/^PIP-PRC-(\d+)$/, 'pipelines.processors'], [/^QRY-(\d+)$/, 'queries.recording_rules'], - [/^POL-(\d+)$/, 'policy.burn_rate_alerts'], [/^FCST-(\d+)$/, 'policy.forecasts'], [/^ALR-(\d+)$/, 'alerting.routes'], + [/^POL-(\d+)$/, 'policy.burn_rate_alerts'], [/^FCST-(\d+)$/, 'policy.forecasts'], [/^ALR-(\d+)$/, 'alerting.routes'], [/^RULE-(\d+)$/, 'alerting.rules'], [/^HEAL-(\d+)$/, 'remediation'], [/^MESH-(\d+)$/, 'mesh'], [/^COL-(\d+)$/, 'collection'], ]; for (const [re, head] of indexed) { const m = re.exec(id); if (m) return `${head}[${Number(m[1]) - 1}]`; } diff --git a/studio/card-html.mjs b/studio/card-html.mjs index 9fd43b96..0142b53d 100644 --- a/studio/card-html.mjs +++ b/studio/card-html.mjs @@ -114,6 +114,7 @@ export const ARTEFACT_KINDS = [ ['POL-', 'Burn-rate alert', 'Warns when an objective burns its error budget too fast.'], ['FCST-', 'Forecast', 'Predicts when a budget or capacity runs out.'], ['ALR-', 'Alert route', 'Sends alerts of one severity to the people who act on them.'], + ['RULE-', 'Alert rule', 'An operational alert the engine evaluates: something is wrong now, not a budget burning.'], ['HEAL-', 'Self-healing action', 'Runs a remediation when an alert fires.'], ['BASE-', 'Baselines', 'Records normal behaviour to compare against.'], ['CHAOS-', 'Chaos experiment', 'Breaks something on purpose to prove the alerts fire.'], diff --git a/studio/discover-board.mjs b/studio/discover-board.mjs index 7c173bd2..9f80c055 100644 --- a/studio/discover-board.mjs +++ b/studio/discover-board.mjs @@ -47,6 +47,7 @@ export const BOARD_LAYERS = { L4: { flow: true, groups: [ { id: 'pol', title: 'Policy & detection', prefixes: ['POL-', 'FCST-'] }, { id: 'alr', title: 'Alert routing', prefixes: ['ALR-'], draw: 'route' }, + { id: 'rule', title: 'Operational alert rules', prefixes: ['RULE-'], optional: true }, { id: 'heal', title: 'Remediation', prefixes: ['HEAL-'] }, ] }, L5: { groups: [ diff --git a/studio/drawer.mjs b/studio/drawer.mjs index fb10fea4..865de664 100644 --- a/studio/drawer.mjs +++ b/studio/drawer.mjs @@ -181,6 +181,7 @@ function renderTypedPanels(artefact, def) { if (id.startsWith('POL-')) return panelBurnRate(s); if (id.startsWith('FCST-')) return panelForecast(s); if (id.startsWith('ALR-')) return panelAlertRoute(s); + if (id.startsWith('RULE-')) return panelAlertRule(s); if (id.startsWith('HEAL-')) return panelRemediation(s); if (id.startsWith('BASE-')) return panelBaselines(s); if (id.startsWith('CHAOS-')) return panelChaos(s); @@ -582,6 +583,24 @@ function panelForecast(s) { return p; } +// An operational alert rule (spec 1.4 alerting.rules): the rule as its +// engine holds it — name, expression, wait, the pack's severity beside the +// engine's own label, and where it was read from. +function panelAlertRule(s) { + const p = panel('Alert rule', 'p-alert'); + p.appendChild(dl([ + ['name', s.name], + ['engine', s.engine || 'prometheus (default)'], + ['severity', s.severity || '—'], + ['for', s.for || '—'], + ])); + if (s.expr) { const sec = subpanel('expr'); const pre = document.createElement('pre'); pre.textContent = s.expr; sec.appendChild(pre); p.appendChild(sec); } + if (s.labels && Object.keys(s.labels).length) { const sec = subpanel('labels'); sec.appendChild(dl(Object.entries(s.labels))); p.appendChild(sec); } + if (s.annotations && Object.keys(s.annotations).length) { const sec = subpanel('annotations'); sec.appendChild(dl(Object.entries(s.annotations))); p.appendChild(sec); } + if (s.source) p.appendChild(dl([['source', s.source]])); + return p; +} + function panelAlertRoute(s) { const p = panel('Alert route', 'p-alert'); p.appendChild(dl([['severity', s.severity]])); diff --git a/tools/fetch-live-pack.mjs b/tools/fetch-live-pack.mjs index 56a2db9d..ec57cf12 100644 --- a/tools/fetch-live-pack.mjs +++ b/tools/fetch-live-pack.mjs @@ -49,7 +49,7 @@ import { dirname, resolve } from 'node:path'; import { fileURLToPath } from 'node:url'; import { emit as emitYaml, parse as parseYaml } from './lib/mini-yaml.mjs'; import { validateCanonical, SPEC_VERSION } from './lib/validator.mjs'; -import { inferSlisFromRecordingRules, ruleNameToSliId, burnAlertsFromAlertRules } from './lib/sli-inference.mjs'; +import { inferSlisFromRecordingRules, ruleNameToSliId, burnAlertsFromAlertRules, operationalAlertRule } from './lib/sli-inference.mjs'; import { materializeL2XFromBackends } from './lib/l2x.mjs'; import { routesFromAlertmanagerConfig } from './lib/alert-routes.mjs'; import { backendForScrapeJob, knownBackendProduct } from './lib/backend-products.mjs'; @@ -1921,6 +1921,30 @@ export function buildCanonicalPack({ annotations['mcp.discovered.alert_rules_linked'] = String(linkedCount); annotations['mcp.discovered.alert_rules_operational'] = String(linkedBurn.unlinked.length); } + // The operational rules are not dropped either: spec 1.4 keeps them in + // spec.alerting.rules, read with the same reading the crawler applies to + // a rule file (operationalAlertRule), so the rule a repository declares + // and the rule the ruler reports pair by name. The engine is what the + // answering tool attests (the registry's `attests`: grafana, mimir, + // vmalert → victoriametrics); a plain Prometheus-API answer states none, + // which means prometheus. A rule the ruler reports unhealthy is declared + // but not stamped Verified, as a burn entry fed by such a rule is not. + const liveAlertRules = []; + { + const unlinked = new Set(linkedBurn.unlinked); + const unhealthy = new Set(unhealthyRuleNames(discoveredAlerts)); + const attested = productAttestedByTool(String(probeResults?.alert_rules?.tool || '').split('+')[0]); + const engine = attested === 'vmalert' ? 'victoriametrics' + : ['grafana', 'mimir', 'loki', 'thanos', 'alertmanager'].includes(attested) ? attested : undefined; + for (const a of discoveredAlerts) { + if (!a?.name || !unlinked.has(a.name) || a?.labels?.kind === 'forecast') continue; + if (liveAlertRules.some(r => r.name === a.name)) continue; + const entry = operationalAlertRule(a, { engine }); + if (!entry) continue; // a rule the ruler listed without its expression cannot be declared (expr is required) + liveAlertRules.push(entry); + if (!unhealthy.has(a.name)) markVerified(`alerting.rules[${liveAlertRules.length - 1}]`); + } + } if (burnMapping.severityInferred.length) { annotations['mcp.discovered.alert_rules_severity_inferred'] = burnMapping.severityInferred.slice(0, 64).join(','); } @@ -2029,6 +2053,9 @@ export function buildCanonicalPack({ severity: 'SEV1', channels: [{ msteams: '#platform-oncall' }], }], + // Operational alert rules (spec 1.4), read above beside the burn-rate + // mapping; absent means the ruler reported none. + ...(liveAlertRules.length ? { rules: liveAlertRules } : {}), }; if (liveRoutes.length) { liveRoutes.forEach((_, i) => markVerified(`alerting.routes[${i}]`)); diff --git a/tools/lib/adapter.mjs b/tools/lib/adapter.mjs index ca75450f..367902b1 100644 --- a/tools/lib/adapter.mjs +++ b/tools/lib/adapter.mjs @@ -122,7 +122,10 @@ export function adapt(canonical, opts = {}) { ...adaptBurnRateAlerts(ctx), ...adaptForecasts(ctx), ], - alerting: adaptAlertingRoutes(ctx), + alerting: [ + ...adaptAlertingRoutes(ctx), + ...adaptAlertRules(ctx), + ], healing: adaptRemediation(ctx), }, L5: [ @@ -705,6 +708,38 @@ function adaptAlertingRoutes(ctx) { }); } +// Operational (non-SLO) alert rules, spec 1.4 alerting.rules[] (RFC-0003): +// one artefact per rule, titled by the rule's exact name — the key the +// artefact model pairs on (alert_rule identity = name), so a rule the +// crawler read from a repository and the same rule a live Grafana or ruler +// reports are one artefact in a comparison. `tool` names the evaluator the +// way the operator renders rules: a PrometheusRule for the PromQL rulers, +// Grafana alerting for a Grafana-managed rule, the Loki ruler for LogQL. +const ALERT_RULE_TOOL = { + prometheus: 'PrometheusRule', mimir: 'PrometheusRule', thanos: 'PrometheusRule', victoriametrics: 'PrometheusRule', + loki: 'Loki ruler', grafana: 'Grafana alerting', alertmanager: 'Alertmanager', +}; +function adaptAlertRules(ctx) { + return (ctx.spec.alerting?.rules || []).map((r, i) => { + const engine = r.engine || 'prometheus'; // absent means prometheus (spec 1.4) + return { + id: `RULE-${pad(i + 1)}`, + title: r.name, + desc: [ + `${engine} alert rule`, + r.for ? `for ${r.for}` : null, + r.severity ? r.severity : null, + r.labels?.severity && r.labels.severity !== r.severity ? `labels.severity ${r.labels.severity}` : null, + ].filter(Boolean).join(' · '), + tool: ALERT_RULE_TOOL[engine] || 'PrometheusRule', + tags: ['alert-rule', engine, r.severity].filter(Boolean), + source: ctx.sourceOf(`alerting.rules[${i}]`), + spec: r, + mcp: ctx.mcpEvidence(`alerting.rules[${i}]`), + }; + }); +} + function adaptRemediation(ctx) { return (ctx.spec.remediation || []).map((r, i) => ({ id: `HEAL-${pad(i + 1)}`, diff --git a/tools/lib/artefact-model.mjs b/tools/lib/artefact-model.mjs index 6a839b62..d7bdba59 100644 --- a/tools/lib/artefact-model.mjs +++ b/tools/lib/artefact-model.mjs @@ -180,6 +180,7 @@ export function classify(artefact) { if (id.startsWith('POL-')) return 'burn_rate'; if (id.startsWith('FCST-')) return 'forecast'; if (id.startsWith('ALR-')) return 'alert_route'; + if (id.startsWith('RULE-')) return 'alert_rule'; if (id.startsWith('HEAL-')) return 'remediation'; if (id === 'BASE-01') return 'baselines'; if (id.startsWith('CHAOS-')) return 'chaos'; @@ -263,6 +264,11 @@ const IDENTITY = { // decision-bearing drift on the paired route instead; same-severity // duplicates survive via the diff's occurrence ordinals and `collisions`. alert_route: (s) => ({ severity: low(s.severity) }), + // An operational alert rule (spec 1.4 alerting.rules) IS its name: the + // exact name the engine evaluates it under — `alert:` in a rule file, + // `title` of a Grafana-managed rule, `name` in a ruler's listing — is + // what a repository and a live listing share. Never the RULE-NN index. + alert_rule: (s) => ({ name: low(s.name) }), remediation: (s) => ({ trigger: low(s.trigger) }), // spec.baselines is a singular object (fixed BASE-01 id, at most one per // pack) — empty identity is the documented singleton invariant, like otel. @@ -326,6 +332,16 @@ function behaviorFor(kind, spec) { if (kind === 'scrape_job') { return canonicalize({ job: spec.job }); } + if (kind === 'alert_rule') { + // What the rule does: its expression (normalised like every expr), its + // wait, its labels and the pack's severity. Not `engine`: a rule file + // cannot tell a Prometheus ruler from a Mimir or VictoriaMetrics one + // and a live listing may know more than the repository — reader + // knowledge, not deployed behaviour. `source` and `annotations` are + // volatile already. + const { engine: _engine, ...rest } = stripRefFields(spec) || {}; + return canonicalize(rest); + } if (kind === 'sli') { // An SLI's `good` and `total` are expressions like its `query`: the // same cosmetic normalisation applies (a ruler returns its rule bodies diff --git a/tools/test-adapt.mjs b/tools/test-adapt.mjs index ec9d8654..566aa0d9 100644 --- a/tools/test-adapt.mjs +++ b/tools/test-adapt.mjs @@ -15,6 +15,7 @@ import { resolve, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; import { parse as parseYaml } from './lib/mini-yaml.mjs'; import { adapt, listEnvironments } from './lib/adapter.mjs'; +import { diffPacks } from './lib/diff.mjs'; import { SPEC_DIR } from './lib/validator.mjs'; const __dirname = dirname(fileURLToPath(import.meta.url)); @@ -53,7 +54,7 @@ assert(prod.layers.L2.length === 24, 'L2 = 1 otel + 9 backends + 8 pip assert(prod.layers.L2X.length === 7, 'L2X = profiling + network + policy_engine + 2 mesh + 2 collection', prod.layers.L2X.length, 7); assert(prod.layers.L3.length === 16, 'L3 = 5 recording + 2 derived + 4 dashboards + 5 panels (expand)', prod.layers.L3.length, 16); assert(prod.layers.L4.policy.length === 7, 'L4.policy = 5 burn-rate + 2 forecasts', prod.layers.L4.policy.length, 7); -assert(prod.layers.L4.alerting.length === 3, 'L4.alerting = SEV1/SEV2/SEV3 routes', prod.layers.L4.alerting.length, 3); +assert(prod.layers.L4.alerting.length === 6, 'L4.alerting = SEV1/SEV2/SEV3 routes + 3 operational alert rules (the 1.4 example)', prod.layers.L4.alerting.length, 6); assert(prod.layers.L4.healing.length === 3, 'L4.healing = 3 remediations', prod.layers.L4.healing.length, 3); assert(prod.layers.L5.length === 6, 'L5 = 1 baseline + 3 chaos + 2 synthetic', prod.layers.L5.length, 6); assert(prod.layers.GOV.length === 3, 'GOV = 3 imports', prod.layers.GOV.length, 3); @@ -322,4 +323,45 @@ assert(adapt(legacyScopeFixture).meta.diffScopeMode === 'service', // ---------- summary ---------- +// ---------- operational alert rules (spec 1.4 alerting.rules) ---------- +// One L4 alerting artefact per rule, titled by the rule's exact name, beside +// the routes; the artefact model keys an alert_rule on that name, so the +// rule a repository declares and the same rule a live listing reports pair. +{ + const rules = prod.layers.L4.alerting.filter(a => a.id.startsWith('RULE-')); + assert(rules.map(a => a.id).join() === 'RULE-01,RULE-02,RULE-03', 'RULE-NN ids follow alerting.rules order', rules.map(a => a.id)); + assert(rules.map(a => a.title).join() === 'PaymentServicePodRestarting,PaymentDbConnectionPoolSaturated,PaymentCertificateExpiringSoon', + 'each alert rule artefact is titled by the rule\'s exact name', rules.map(a => a.title)); + assert(rules.map(a => a.tool).join() === 'PrometheusRule,PrometheusRule,Grafana alerting', + 'tool follows the engine: PrometheusRule for the PromQL rulers (absent engine means prometheus), Grafana alerting for a Grafana-managed rule', rules.map(a => a.tool)); + assert(rules.every(a => a.source === 'Declared') && rules[0].tags.includes('alert-rule') && rules[0].tags.includes('SEV3'), + 'alert rules project as Declared and carry the alert-rule tag and their severity', rules.map(a => [a.source, a.tags])); + assert(rules[0].desc === 'prometheus alert rule · for 10m · SEV3 · labels.severity warning', + 'desc states engine, wait, the pack\'s severity and the engine\'s own label when they differ', rules[0].desc); + assert(prod.layers.L4.alerting.filter(a => a.id.startsWith('ALR-')).length === 3, 'the three routes are still there beside the rules'); + + // Pairing: a "live" pack that holds the same rules under other spellings + // of everything but the name — a ruler's trailing newline in the + // expression, no engine, no source, no annotations — pairs rule for rule + // and reads aligned; a rule whose expression changed reads drifted; a + // rule only one side holds reads only there. + const live = clone(canonical); + live.spec.alerting.rules = canonical.spec.alerting.rules.map(({ name, expr, severity, labels }, i) => ({ + name, expr: `${expr}\n`, severity, labels, ...(canonical.spec.alerting.rules[i].for ? { for: canonical.spec.alerting.rules[i].for } : {}), + })); + live.spec.alerting.rules[1].expr = live.spec.alerting.rules[1].expr.replace('> 0.9', '> 0.95'); + live.spec.alerting.rules.push({ name: 'PaymentNodeDown', expr: 'up{job="payment-service"} == 0', severity: 'SEV1' }); + const d = diffPacks(prod, adapt(live)); + const ruleEntries = (bucket) => d.layers.L4[bucket].filter(e => e.key.startsWith('alert_rule::')); + assert(ruleEntries('inBoth').length === 3 && ruleEntries('inBoth').every(e => e.a.title === e.b.title), + 'the three rules pair by name — engine, source, annotations and a trailing newline in the expression are not identity', ruleEntries('inBoth').map(e => [e.key, e.match])); + assert(ruleEntries('inBoth').filter(e => e.match === 'aligned').map(e => e.a.title).sort().join() === 'PaymentCertificateExpiringSoon,PaymentServicePodRestarting', + 'the unchanged rules read aligned although the live side states no engine and no source', ruleEntries('inBoth').map(e => [e.a.title, e.match, e.deltas])); + const drifted = ruleEntries('inBoth').find(e => e.match === 'drifted'); + assert(drifted?.a.title === 'PaymentDbConnectionPoolSaturated' && drifted.deltas.map(x => x.field).join() === 'expr', + 'a rule whose expression changed reads drifted, on expr', drifted && [drifted.a.title, drifted.deltas]); + assert(ruleEntries('onlyInB').map(e => e.artefact.title).join() === 'PaymentNodeDown' && ruleEntries('onlyInA').length === 0, + 'a rule only the live side holds reads "live, not declared"', [ruleEntries('onlyInA').length, ruleEntries('onlyInB').map(e => e.artefact.title)]); +} + report('adapter'); diff --git a/tools/test-scan-live.mjs b/tools/test-scan-live.mjs index aee8ab71..e1e66c6c 100644 --- a/tools/test-scan-live.mjs +++ b/tools/test-scan-live.mjs @@ -382,6 +382,15 @@ assert(both('L4').burn_rate === 2, 'L4: the alerts that guard a recorded SLO pair as burn-rate entries on both sides', both('L4')); assert(both('L4').alert_route === 4, 'L4: all four routes pair — including the one whose address the repository takes from a deploy-time variable', both('L4')); +assert(both('L4').alert_rule === 1 + && repoPack.spec.alerting.rules.map(r => r.name).join() === 'DiskAlmostFull' + && livePack.spec.alerting.rules.map(r => r.name).join() === 'DiskAlmostFull', +'L4: the operational alert (spec 1.4 alerting.rules) pairs by its exact name — the rule file\'s `alert:` against the ruler\'s listing', [both('L4'), repoPack.spec.alerting.rules, livePack.spec.alerting.rules]); +{ + const rule = diff.layers.L4.inBoth.find(e => e.key.startsWith('alert_rule::')); + assert(rule?.match === 'aligned' && rule.deltas.length === 0 && rule.a.spec.engine === 'prometheus' && rule.b.spec.engine === 'victoriametrics', + 'the operational rule is aligned with no delta: the file says prometheus and the ruler attests victoriametrics, and the engine is reader knowledge, not behaviour; its `duration: 0` is no wait, like the file\'s absent `for`', rule && [rule.match, rule.deltas, rule.a.spec, rule.b.spec]); +} assert(both('L2').backend === 7, 'L2: the seven products the repository deploys are the seven the live side found evidence of', both('L2')); assert(both('L2').scrape_job === 3, 'L2: the three metrics scrape jobs pair', both('L2')); @@ -465,7 +474,14 @@ assert(both('L2').metric >= 11, 'L2: declared metrics pair with the series the s && missing.includes('sli::{"id":"shop_latency"}') && missing.includes('slo::{"id":"shop_latency_99"}') && missing.includes('burn_rate::{"slo":"shop_latency_99"}'), 'rules that are not deployed: the rules, the SLI / SLO they carry and the alert that guards it read "not live"', missing); - assert(missing.every(k => /latency/.test(k)) && d.summary.onlyInB === 0, 'and nothing else does', [missing, keysOf(d, 'onlyInB')]); + assert(missing.every(k => /latency/.test(k)), 'and nothing else does', missing); + // The latency alerting rule itself still runs live — but with its recorded + // series gone it guards no SLO there, so the live side reads it as an + // operational alert (alerting.rules) while the repository reads it as the + // burn-rate alert on shop_latency_99: the one "live, not declared" entry, + // and a true one — the rule is deployed, what it stands for is not. + assert(JSON.stringify(keysOf(d, 'onlyInB')) === JSON.stringify(['alert_rule::{"name":"shoplatencyhigh"}']), + 'the undeployed SLO\'s alerting rule, still evaluated live, reads as a live operational rule — and nothing else reads "live, not declared"', keysOf(d, 'onlyInB')); } { // The live Alertmanager has no route for DiskAlmostFull. From dbca847d77bd39077183a8bd0c5e04032d044725 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 13:16:28 +0000 Subject: [PATCH 028/550] =?UTF-8?q?test(crawler):=20acceptance=20fixture?= =?UTF-8?q?=20=E2=80=94=20six=20operational=20and=20three=20burn-rate=20ru?= =?UTF-8?q?les,=20Prometheus,=20Grafana=20provisioning=20and=20Loki?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tools/fixtures/crawl/operational-alerts/ is a service repository as the crawler meets one: a Prometheus rule file with four recording rules (two SLIs), one burn-rate alert and two operational alerts; Grafana unified-alerting provisioning YAML with two burn-rate rules (a model mapping and a model JSON string) and three operational rules — one titled "Payments: certificate expiring", a Grafana title with a colon and spaces, one with reduce and threshold expression nodes after its query node, one with `for: 0s` and `severity: page`; and a Loki rule file with one LogQL alert. One annotation value ends in ":" and one in " #". README says which rule is which. tools/test-crawl-alerting-rules.mjs (on `npm test`, `npm run test:crawl:alerting`) crawls it and pins: the pack validates against the vendored 1.4 schema; all N = 6 operational rules are in alerting.rules under their exact names (the Grafana title never slugged), with engine grafana / loki / prometheus by file, the query node's expr (not the reduce or threshold nodes; a JSON-string model read the same way), LogQL verbatim, the pack's severity from labels.severity (critical and page → SEV1, warning → SEV2) beside the engine's own word in labels, `for` as the canonical duration (300s → 5m, 0s omitted), labels and annotations verbatim, source `#/`, schema fields only, evidence RULE-1..6; the three burn-rate rules fold into M = 2 policy entries, one per recorded SLO, the fast (2m) and slow (30m) availability rules on the one entry; no rule is in both places and no operational rule became an SLO; summary.discovered.alertRules is 6 and the summary line states the counts and the names (the old "excluded … remain available" warning is gone); the pack's spec round-trips through mini-yaml (emit → parse → deep-equal), the ":"-suffixed scalar included — the quoting fix #138 merged, so this passes outright; the adapter shows six alerting artefacts titled by rule name, each keyed alert_rule::{name}, tool by engine, Declared, beside M = 2 burn-rate artefacts. One pre-existing quirk is pinned rather than hidden: the emitter writes a multi-line scalar as a literal block (`|`), which re-parses with a trailing newline the value did not have, so the crawler's multi-line crawler.discovered.* annotations differ by exactly that after a round trip. The test asserts the difference is that and nothing else; a mini-yaml fix (`|-`) will show up as that assertion turning strict. Nothing in alerting.rules is affected. The crawl golden is untouched (its fixture has no operational rule). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- package.json | 3 +- .../crawl/operational-alerts/README.md | 17 ++ .../provisioning/alerting/payments.yaml | 111 +++++++++++++ .../crawl/operational-alerts/loki/rules.yaml | 10 ++ .../operational-alerts/prometheus/rules.yml | 36 +++++ tools/test-crawl-alerting-rules.mjs | 151 ++++++++++++++++++ 6 files changed, 327 insertions(+), 1 deletion(-) create mode 100644 tools/fixtures/crawl/operational-alerts/README.md create mode 100644 tools/fixtures/crawl/operational-alerts/grafana/provisioning/alerting/payments.yaml create mode 100644 tools/fixtures/crawl/operational-alerts/loki/rules.yaml create mode 100644 tools/fixtures/crawl/operational-alerts/prometheus/rules.yml create mode 100644 tools/test-crawl-alerting-rules.mjs diff --git a/package.json b/package.json index 133ba726..9311bae5 100644 --- a/package.json +++ b/package.json @@ -62,6 +62,7 @@ "crawl": "node tools/crawl-repo.mjs", "test:adapt": "node tools/test-adapt.mjs", "test:crawl": "node tools/test-crawl.mjs", + "test:crawl:alerting": "node tools/test-crawl-alerting-rules.mjs", "test:diff": "node tools/test-diff.mjs", "test:promql": "node tools/test-promql.mjs", "test:diagnostic": "node tools/test-diagnostic-grade.mjs", @@ -104,7 +105,7 @@ "test:store:prestore:strict": "node tools/test-store-prestore-live.mjs --strict", "test:auth": "node server/test-auth-local.mjs && node server/test-auth-oidc.mjs && node --test server/test-authz.mjs && node --test server/test-identity-api.mjs", "users": "node tools/user-admin.mjs", - "test": "node --test server/test-auth-local.mjs server/test-auth-oidc.mjs server/test-authz.mjs server/test-deploy-helpers.mjs server/test-identity-api.mjs server/test-mcp-url.mjs server/test-smoke.mjs server/test-store-boot.mjs server/test-store-guards.mjs server/test-store-import.mjs server/test-store-ops.mjs server/test-store.mjs server/test-tenancy.mjs server/test-workspace.mjs tools/test-adapt.mjs tools/test-backend-validate.mjs tools/test-blast-radius.mjs tools/test-build-editor.mjs tools/test-build-info.mjs tools/test-build-model.mjs tools/test-chain-history.mjs tools/test-compile.mjs tools/test-contract-guard.mjs tools/test-contract-shapes.mjs tools/test-crawl.mjs tools/test-deploy-manifests.mjs tools/test-design-system.mjs tools/test-diagnostic-grade.mjs tools/test-diff.mjs tools/test-discover-rows.mjs tools/test-fetch-live.mjs tools/test-gen-pack.mjs tools/test-gen-site.mjs tools/test-golden-compile.mjs tools/test-golden-crawl.mjs tools/test-inventory-coverage.mjs tools/test-journey-notify.mjs tools/test-journey.mjs tools/test-legacy-pack.mjs tools/test-library.mjs tools/test-metric-readers.mjs tools/test-mini-yaml.mjs tools/test-neuron-model.mjs tools/test-packs.mjs tools/test-profiles.mjs tools/test-promql-canon.mjs tools/test-promql.mjs tools/test-record-fixtures.mjs tools/test-retrofeed.mjs tools/test-scan-live.mjs tools/test-schedule-snippets.mjs tools/test-schedule.mjs tools/test-stack-evidence.mjs tools/test-stack-self-metrics.mjs tools/test-studio-graph.mjs tools/test-studio-layout.mjs tools/test-svg-charts.mjs tools/test-traceability-graph.mjs tools/test-ux-kit.mjs tools/test-validator.mjs tools/test-verify-deploy.mjs tools/test-zip.mjs", + "test": "node --test server/test-auth-local.mjs server/test-auth-oidc.mjs server/test-authz.mjs server/test-deploy-helpers.mjs server/test-identity-api.mjs server/test-mcp-url.mjs server/test-smoke.mjs server/test-store-boot.mjs server/test-store-guards.mjs server/test-store-import.mjs server/test-store-ops.mjs server/test-store.mjs server/test-tenancy.mjs server/test-workspace.mjs tools/test-adapt.mjs tools/test-backend-validate.mjs tools/test-blast-radius.mjs tools/test-build-editor.mjs tools/test-build-info.mjs tools/test-build-model.mjs tools/test-chain-history.mjs tools/test-compile.mjs tools/test-contract-guard.mjs tools/test-contract-shapes.mjs tools/test-crawl.mjs tools/test-crawl-alerting-rules.mjs tools/test-deploy-manifests.mjs tools/test-design-system.mjs tools/test-diagnostic-grade.mjs tools/test-diff.mjs tools/test-discover-rows.mjs tools/test-fetch-live.mjs tools/test-gen-pack.mjs tools/test-gen-site.mjs tools/test-golden-compile.mjs tools/test-golden-crawl.mjs tools/test-inventory-coverage.mjs tools/test-journey-notify.mjs tools/test-journey.mjs tools/test-legacy-pack.mjs tools/test-library.mjs tools/test-metric-readers.mjs tools/test-mini-yaml.mjs tools/test-neuron-model.mjs tools/test-packs.mjs tools/test-profiles.mjs tools/test-promql-canon.mjs tools/test-promql.mjs tools/test-record-fixtures.mjs tools/test-retrofeed.mjs tools/test-scan-live.mjs tools/test-schedule-snippets.mjs tools/test-schedule.mjs tools/test-stack-evidence.mjs tools/test-stack-self-metrics.mjs tools/test-studio-graph.mjs tools/test-studio-layout.mjs tools/test-svg-charts.mjs tools/test-traceability-graph.mjs tools/test-ux-kit.mjs tools/test-validator.mjs tools/test-verify-deploy.mjs tools/test-zip.mjs", "lint": "eslint .", "coverage": "c8 npm test", "sync-spec": "node tools/sync-spec.mjs", diff --git a/tools/fixtures/crawl/operational-alerts/README.md b/tools/fixtures/crawl/operational-alerts/README.md new file mode 100644 index 00000000..00a1d24b --- /dev/null +++ b/tools/fixtures/crawl/operational-alerts/README.md @@ -0,0 +1,17 @@ +# Operational + burn-rate alert rules fixture (spec 1.4 `alerting.rules`) + +A service repository as the crawler meets one: recording rules that define +two SLIs, and alert rules of both kinds side by side in every engine's own +format. + +| File | Burn-rate rules (reference a recorded SLO series → `policy.burn_rate_alerts`) | Operational rules (→ `alerting.rules`) | +|---|---|---| +| `prometheus/rules.yml` | `PaymentsAvailabilityBurnFast` | `PaymentsPodRestarting`, `PaymentsDbPoolSaturated` | +| `grafana/provisioning/alerting/payments.yaml` | `PaymentsAvailabilityBurnSlow`, `PaymentsLatencyBurn` | `Payments: certificate expiring`, `Payments queue backlog`, `Payments 5xx spike` | +| `loki/rules.yaml` | — | `PaymentsErrorLogSpike` (LogQL) | + +Six operational rules (N = 6) with their exact names — one with a colon and +spaces, as Grafana titles have — and three burn-rate rules that fold into +two policy entries (M = 2, one per SLO). One annotation value ends in `:` +and one in ` #`, the scalars the YAML emitter must quote for the pack to +round-trip. Read by tools/test-crawl-alerting-rules.mjs; not a golden input. diff --git a/tools/fixtures/crawl/operational-alerts/grafana/provisioning/alerting/payments.yaml b/tools/fixtures/crawl/operational-alerts/grafana/provisioning/alerting/payments.yaml new file mode 100644 index 00000000..a32669be --- /dev/null +++ b/tools/fixtures/crawl/operational-alerts/grafana/provisioning/alerting/payments.yaml @@ -0,0 +1,111 @@ +apiVersion: 1 +groups: + - orgId: 1 + name: payments-slo + folder: Payments + interval: 1m + rules: + - uid: payments-availability-burn-slow + title: PaymentsAvailabilityBurnSlow + condition: C + for: 30m + data: + - refId: A + datasourceUid: prometheus + model: + editorMode: code + expr: payments:availability:ratio_5m < 0.995 + instant: true + refId: A + - refId: C + datasourceUid: __expr__ + model: + expression: A + refId: C + type: threshold + labels: + severity: SEV2 + - uid: payments-latency-burn + title: PaymentsLatencyBurn + condition: B + for: 10m + data: + - refId: A + datasourceUid: prometheus + model: "{\"editorMode\":\"code\",\"expr\":\"payments:latency:p99_5m > 0.3\",\"instant\":true,\"refId\":\"A\"}" + - refId: B + datasourceUid: __expr__ + model: "{\"expression\":\"A\",\"refId\":\"B\",\"type\":\"threshold\"}" + labels: + severity: SEV2 + - orgId: 1 + name: payments-operational + folder: Payments + interval: 1m + rules: + - uid: payments-cert-expiring + title: "Payments: certificate expiring" + condition: B + for: 1h + data: + - refId: A + datasourceUid: prometheus + model: + expr: min(probe_ssl_earliest_cert_expiry{job="blackbox",instance=~"https://payments.example.com.*"}) - time() < 14 * 24 * 3600 + refId: A + - refId: B + datasourceUid: __expr__ + model: + expression: A + refId: B + type: threshold + labels: + severity: warning + team: platform-sre + annotations: + summary: "payments.example.com TLS certificate expires in under 14 days" + dashboard: "https://grafana.example.internal/d/payments #certificates" + - uid: payments-queue-backlog + title: Payments queue backlog + condition: C + for: 15m + data: + - refId: A + datasourceUid: prometheus + model: + expr: sum(rabbitmq_queue_messages_ready{queue=~"payments\\..*"}) + refId: A + - refId: B + datasourceUid: __expr__ + model: + expression: A + reducer: last + refId: B + type: reduce + - refId: C + datasourceUid: __expr__ + model: + conditions: + - evaluator: + params: + - 10000 + type: gt + expression: B + refId: C + type: threshold + labels: + severity: critical + team: payments + - uid: payments-5xx-spike + title: Payments 5xx spike + condition: B + for: 0s + data: + - refId: A + datasourceUid: prometheus + model: "{\"expr\":\"sum(rate(http_requests_total{service=\\\"payments\\\",code=~\\\"5..\\\"}[5m])) > 10\",\"refId\":\"A\"}" + - refId: B + datasourceUid: __expr__ + model: "{\"expression\":\"A\",\"refId\":\"B\",\"type\":\"threshold\"}" + labels: + severity: page diff --git a/tools/fixtures/crawl/operational-alerts/loki/rules.yaml b/tools/fixtures/crawl/operational-alerts/loki/rules.yaml new file mode 100644 index 00000000..6029b244 --- /dev/null +++ b/tools/fixtures/crawl/operational-alerts/loki/rules.yaml @@ -0,0 +1,10 @@ +groups: + - name: payments.logs + rules: + - alert: PaymentsErrorLogSpike + expr: sum(rate({namespace="payments", app="payments"} |= "level=error" [5m])) > 5 + for: 10m + labels: + severity: warning + annotations: + summary: "payments logs more than 5 errors/s for 10m" diff --git a/tools/fixtures/crawl/operational-alerts/prometheus/rules.yml b/tools/fixtures/crawl/operational-alerts/prometheus/rules.yml new file mode 100644 index 00000000..125c9b0b --- /dev/null +++ b/tools/fixtures/crawl/operational-alerts/prometheus/rules.yml @@ -0,0 +1,36 @@ +groups: + - name: payments.slo + interval: 30s + rules: + - record: payments:availability:good_5m + expr: sum(rate(http_requests_total{service="payments",code!~"5.."}[5m])) + - record: payments:availability:total_5m + expr: sum(rate(http_requests_total{service="payments"}[5m])) + - record: payments:availability:ratio_5m + expr: payments:availability:good_5m / payments:availability:total_5m + - record: payments:latency:p99_5m + expr: histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket{service="payments"}[5m])) by (le)) + - name: payments.alerts + rules: + - alert: PaymentsAvailabilityBurnFast + expr: payments:availability:ratio_5m < 0.99 + for: 2m + labels: + severity: critical + team: payments + - alert: PaymentsPodRestarting + expr: increase(kube_pod_container_status_restarts_total{namespace="payments"}[15m]) > 3 + for: 10m + labels: + severity: warning + team: payments + annotations: + summary: "payments pod restarted more than 3 times in 15m" + runbook_url: "https://runbooks.example.internal/payments/pod-restarts" + - alert: PaymentsDbPoolSaturated + expr: sum(pg_stat_activity_count{datname="payments"}) / sum(pg_settings_max_connections{datname="payments"}) > 0.9 + for: 300s + labels: + severity: critical + annotations: + summary: "payments connection pool above 90% for 5m — see the runbook:" diff --git a/tools/test-crawl-alerting-rules.mjs b/tools/test-crawl-alerting-rules.mjs new file mode 100644 index 00000000..0e71bde7 --- /dev/null +++ b/tools/test-crawl-alerting-rules.mjs @@ -0,0 +1,151 @@ +#!/usr/bin/env node +/** + * tools/test-crawl-alerting-rules.mjs + * + * Acceptance test for spec 1.4 alerting.rules through the crawler and the + * adapter: a repository holding operational alerts and burn-rate alerts + * side by side — a Prometheus rule file, Grafana unified-alerting + * provisioning YAML and a Loki rule file (tools/fixtures/crawl/ + * operational-alerts/) — crawls into a pack that + * 1. validates against the vendored 1.4 schema; + * 2. carries every operational rule (N = 6) in spec.alerting.rules with + * its EXACT name, expression, wait, the pack's severity beside the + * engine's label, its engine and its source, and every burn-rate rule + * in spec.policy.burn_rate_alerts (3 rules → M = 2 entries, one per + * SLO) — a rule in one place, never both; + * 3. round-trips through mini-yaml (crawlToYaml → parse → deep-equal), + * colons at the end of a scalar and ` #` included; + * 4. projects, through the adapter, N alerting artefacts titled by rule + * name with distinct name-keyed identities. + * Exit 0 = pass. + */ + +import { readFileSync, readdirSync, statSync } from 'node:fs'; +import { join, relative } from 'node:path'; +import { crawlFiles, crawlToYaml, detectArtefactKind } from './lib/crawler.mjs'; +import { validateCanonical, SPEC_SCHEMA_PATH } from './lib/validator.mjs'; +import { parse as parseYaml } from './lib/mini-yaml.mjs'; +import { adapt } from './lib/adapter.mjs'; +import { identityKeyOf } from './lib/artefact-model.mjs'; +import { createHarness } from './lib/harness.mjs'; + +const { assert, report } = createHarness({ truncate: 600 }); +const SCHEMA = JSON.parse(readFileSync(new URL(`../${SPEC_SCHEMA_PATH}`, import.meta.url), 'utf8')); + +// The fixture directory as a file map, every file under it (README aside). +const ROOT = new URL('./fixtures/crawl/operational-alerts/', import.meta.url).pathname; +const FILES = {}; +(function walk(dir) { + for (const name of readdirSync(dir)) { + const p = join(dir, name); + if (statSync(p).isDirectory()) walk(p); + else if (name !== 'README.md') FILES[relative(ROOT, p).replace(/\\/g, '/')] = readFileSync(p, 'utf8'); + } +})(ROOT); + +const OPERATIONAL = [ + 'Payments: certificate expiring', 'Payments queue backlog', 'Payments 5xx spike', // Grafana provisioning + 'PaymentsErrorLogSpike', // Loki + 'PaymentsPodRestarting', 'PaymentsDbPoolSaturated', // Prometheus +]; +const BURN_RULES = ['PaymentsAvailabilityBurnFast', 'PaymentsAvailabilityBurnSlow', 'PaymentsLatencyBurn']; +const N = OPERATIONAL.length; // 6 +const M = 2; // policy entries: payments_availability_99, payments_latency_99 + +assert(Object.keys(FILES).length === 3, 'the fixture holds three rule files', Object.keys(FILES)); +assert(detectArtefactKind('grafana/provisioning/alerting/payments.yaml', FILES['grafana/provisioning/alerting/payments.yaml']) === 'prometheus-rules', + 'Grafana provisioning YAML is detected as a rule file'); + +const opts = { repoName: 'payments', now: '2026-10-03T00:00:00.000Z' }; +const { canonical, summary, evidence } = crawlFiles(FILES, opts); + +// 1. validates against 1.4 +assert(validateCanonical(canonical, SCHEMA).length === 0, 'the crawled pack validates against the vendored 1.4 schema', validateCanonical(canonical, SCHEMA).slice(0, 3)); + +// 2. every rule in its place +const rules = canonical.spec.alerting.rules || []; +const names = rules.map(r => r.name); +assert(names.length === N && [...names].sort().join('|') === [...OPERATIONAL].sort().join('|'), + `all ${N} operational rules are in alerting.rules under their exact names`, names, OPERATIONAL); +assert(names.includes('Payments: certificate expiring'), 'a Grafana title with a colon and spaces is kept as the name, never slugged'); +const slos = canonical.spec.policy.burn_rate_alerts.map(a => a.slo).sort(); +assert(slos.length === M && slos.join() === 'payments_availability_99,payments_latency_99', + `the ${BURN_RULES.length} burn-rate rules fold into M = ${M} policy entries, one per recorded SLO`, slos); +assert(!names.some(n => BURN_RULES.includes(n)), 'no burn-rate rule is also an operational rule'); +assert(!canonical.spec.policy.burn_rate_alerts.some(a => OPERATIONAL.some(n => a.slo.includes(n.toLowerCase().replace(/[^a-z0-9]+/g, '_')))), + 'no operational rule was manufactured into an SLO contract'); +const avail = canonical.spec.policy.burn_rate_alerts.find(a => a.slo === 'payments_availability_99'); +assert(avail.windows.map(w => w.short).sort().join() === '2m,30m', + 'the fast (Prometheus, 2m) and slow (Grafana, 30m) availability rules union their windows on the one entry', avail.windows); + +const by = Object.fromEntries(rules.map(r => [r.name, r])); +assert(['Payments: certificate expiring', 'Payments queue backlog', 'Payments 5xx spike'].every(n => by[n].engine === 'grafana') + && by.PaymentsErrorLogSpike.engine === 'loki' && by.PaymentsPodRestarting.engine === 'prometheus' && by.PaymentsDbPoolSaturated.engine === 'prometheus', + 'engine: grafana for provisioned rules, loki for the Loki rule file, prometheus for the Prometheus rule file', rules.map(r => [r.name, r.engine])); +assert(by['Payments queue backlog'].expr === 'sum(rabbitmq_queue_messages_ready{queue=~"payments\\\\..*"})', + 'a Grafana rule\'s expr is its query node (model mapping), not its reduce / threshold nodes', by['Payments queue backlog'].expr); +assert(by['Payments 5xx spike'].expr === 'sum(rate(http_requests_total{service="payments",code=~"5.."}[5m])) > 10', + 'a Grafana rule whose model is a JSON string reads the same way', by['Payments 5xx spike'].expr); +assert(by.PaymentsErrorLogSpike.expr === 'sum(rate({namespace="payments", app="payments"} |= "level=error" [5m])) > 5', 'LogQL is kept verbatim'); +assert(by.PaymentsPodRestarting.severity === 'SEV2' && by.PaymentsPodRestarting.labels.severity === 'warning' + && by.PaymentsDbPoolSaturated.severity === 'SEV1' && by.PaymentsDbPoolSaturated.labels.severity === 'critical' + && by['Payments 5xx spike'].severity === 'SEV1' && by['Payments 5xx spike'].labels.severity === 'page', + 'severity is the pack\'s SEV from labels.severity (critical / page → SEV1, warning → SEV2); the engine\'s word stays in labels', + rules.map(r => [r.name, r.severity, r.labels?.severity])); +assert(by.PaymentsDbPoolSaturated.for === '5m' && by.PaymentsPodRestarting.for === '10m' && by['Payments: certificate expiring'].for === '1h' && by['Payments 5xx spike'].for === undefined, + 'for is the canonical duration: 300s → 5m; 0s is no wait and omitted', rules.map(r => [r.name, r.for])); +assert(by.PaymentsPodRestarting.labels.team === 'payments' && by.PaymentsPodRestarting.annotations.runbook_url === 'https://runbooks.example.internal/payments/pod-restarts' + && by['Payments: certificate expiring'].annotations.dashboard === 'https://grafana.example.internal/d/payments #certificates', + 'labels and annotations are kept verbatim'); +assert(by.PaymentsPodRestarting.source === 'prometheus/rules.yml#payments.alerts/PaymentsPodRestarting' + && by['Payments queue backlog'].source === 'grafana/provisioning/alerting/payments.yaml#payments-operational/Payments queue backlog' + && by.PaymentsErrorLogSpike.source === 'loki/rules.yaml#payments.logs/PaymentsErrorLogSpike', + 'source is #/, as the crawler records every rule\'s provenance', rules.map(r => r.source)); +assert(rules.every(r => Object.keys(r).every(k => ['name', 'expr', 'severity', 'for', 'labels', 'annotations', 'engine', 'source'].includes(k))), 'a rule carries schema fields only'); +assert(Object.entries(evidence).filter(([k]) => /^RULE-\d+$/.test(k)).length === N && evidence['RULE-1'] === rules[0].source, + `evidence RULE-1..${N} records each rule's source`, Object.entries(evidence).filter(([k]) => /^RULE-/.test(k))); + +// the summary says so, truthfully +assert(summary.discovered.alertRules === N, `summary.discovered.alertRules counts the ${N} operational rules`, summary.discovered.alertRules); +assert(summary.discovered.burnRateAlerts === N + BURN_RULES.length, + 'summary.discovered.burnRateAlerts still counts every alert rule discovered before classification (unchanged semantics); alertRules counts the operational ones', summary.discovered.burnRateAlerts); +const line = summary.warnings.find(w => /operational alert rule\(s\) kept in alerting\.rules/.test(w)); +assert(line === `${N} operational alert rule(s) kept in alerting.rules — not SLO burn-rate alerts (no recorded-ratio reference): ${names.join(', ')}. ${M} burn-rate alert(s) in policy.burn_rate_alerts.`, + 'the summary line states the counts and the names', line); +assert(!summary.warnings.some(w => /Excluded .* operational alert/.test(w)), 'the "excluded … remain available" warning is gone'); + +// 3. YAML round-trip +const { yaml } = crawlToYaml(FILES, opts); +const back = parseYaml(yaml); +assert(JSON.stringify(back.spec) === JSON.stringify(canonical.spec), 'the pack\'s spec round-trips through mini-yaml: emit → parse → deep-equal (alerting.rules included)', + JSON.stringify(back.spec.alerting) === JSON.stringify(canonical.spec.alerting) ? 'difference outside alerting' : [back.spec.alerting.rules, canonical.spec.alerting.rules]); +// KNOWN, PRE-EXISTING and outside this change: the emitter writes a +// multi-line scalar as a literal block (`|`), which re-parses with a trailing +// newline the value did not have; the crawler's crawler.discovered.* +// annotations are multi-line JSON, so metadata differs by exactly that. +// Pinned here so a fix in mini-yaml (`|-`) shows up as this assertion turning +// strict: every other metadata key is byte-equal. +{ + const a = canonical.metadata.annotations, b = back.metadata.annotations; + const differing = Object.keys(a).filter(k => a[k] !== b[k]); + assert(JSON.stringify({ ...canonical.metadata, annotations: null }) === JSON.stringify({ ...back.metadata, annotations: null }) + && differing.every(k => a[k].includes('\n') && b[k] === `${a[k]}\n`), + 'metadata round-trips except the known trailing newline on multi-line annotation values (mini-yaml literal block `|`; not alerting.rules)', differing.map(k => [k, JSON.stringify(a[k].slice(-20)), JSON.stringify(b[k].slice(-20))])); +} +assert(back.spec.alerting.rules.find(r => r.name === 'PaymentsDbPoolSaturated').annotations.summary.endsWith('runbook:'), + 'a scalar ending in ":" survives the round-trip (mini-yaml quotes it)'); +assert(validateCanonical(back, SCHEMA).length === 0, 'the re-parsed pack validates too'); + +// 4. the adapter +const layered = adapt(canonical); +const artefacts = layered.layers.L4.alerting.filter(a => a.id.startsWith('RULE-')); +assert(artefacts.length === N && artefacts.map(a => a.title).join('|') === names.join('|'), + `the adapter shows ${N} alerting artefacts titled by rule name`, artefacts.map(a => a.title)); +assert(new Set(artefacts.map(identityKeyOf)).size === N && artefacts.every(a => identityKeyOf(a) === `alert_rule::{"name":"${a.title.toLowerCase()}"}`), + 'each is keyed by its name — the key a live listing of the same rules pairs on', artefacts.map(identityKeyOf)); +assert(artefacts.every(a => a.source === 'Declared') && artefacts.filter(a => a.tool === 'Grafana alerting').length === 3 + && artefacts.filter(a => a.tool === 'PrometheusRule').length === 2 && artefacts.filter(a => a.tool === 'Loki ruler').length === 1, + 'Declared, tool by engine', artefacts.map(a => [a.title, a.tool, a.source])); +assert(layered.layers.L4.policy.filter(a => a.id.startsWith('POL-')).length === M, `and M = ${M} burn-rate artefacts`, layered.layers.L4.policy.map(a => a.title)); + +report('crawl alerting.rules', `N = ${N} operational rules in alerting.rules, M = ${M} burn-rate entries in policy; valid, round-trips, projects.`); From fb3d3276df8bff23c99c0ff05e40b4d99fe4ee7f Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 13:17:59 +0000 Subject: [PATCH 029/550] =?UTF-8?q?docs:=20spec=201.4=20alerting.rules=20?= =?UTF-8?q?=E2=80=94=20crawled,=20shown,=20and=20paired=20with=20the=20liv?= =?UTF-8?q?e=20rules?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README: the crawler's sources name Grafana provisioning YAML and Loki rule files, and the crawl section says what happens to every alert rule: a burn-rate alert stays in the policy, every other rule is declared in spec.alerting.rules (spec 1.4) under its exact name — the name a live Grafana or ruler listing is reconciled on, so the two pair in Compare. docs/ADAPTER.md: the mapping table gains spec.alerting.rules[] → L4 . alerting RULE-{NN}, titled by name, tool by engine, keyed on the name by the artefact model. docs/MCP_INTEGRATION.md: the live-pack table and the annotation table say where the non-SLO rules go and how they are stamped; the burn-rate section gains the paragraph on operational rules (the one reading for both sides, what is compared and what is not, and the gap it closes); the "One reading for the repository and for live" table gains the alerting.rules row. docs/CHANGELOG.md (Unreleased): the problem, RFC-0003 and the upstream commit, and the adoption in four parts — vendor, crawler, adapter and Compare, tests. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- README.md | 13 +++++++++++-- docs/ADAPTER.md | 1 + docs/CHANGELOG.md | 7 +++++++ docs/MCP_INTEGRATION.md | 24 +++++++++++++++++++++--- 4 files changed, 40 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index d121d903..b30e3033 100644 --- a/README.md +++ b/README.md @@ -640,7 +640,8 @@ npm run validate-pack -- repo.pack.yaml The crawler reads source files such as: -- Prometheus rule files +- Prometheus rule files (recording rules, burn-rate and operational alerts) +- Grafana unified-alerting provisioning YAML and Loki rule files - Grafana dashboard JSON - Alertmanager config - OTel Collector config @@ -648,7 +649,15 @@ The crawler reads source files such as: - Docker Compose files It emits a canonical v1.4 pack plus crawler annotations describing what was -scanned and what was inferred. +scanned and what was inferred. Every alert rule is kept: a rule whose +expression references a recorded SLO series is a burn-rate alert +(`spec.policy.burn_rate_alerts`); every other rule — a pod restarting, a pool +saturated, a certificate expiring — is an operational alert and is declared in +`spec.alerting.rules` (spec 1.4) under its exact name, with its expression, +wait, severity, labels, engine (`prometheus`, `grafana`, `loki`) and the file +and group it was read from. The same name is what a live Grafana or ruler +listing is reconciled on, so the repository's rule and the running rule pair +in Compare. ### Fetch Live From MCP diff --git a/docs/ADAPTER.md b/docs/ADAPTER.md index 58b0b768..6c596d0e 100644 --- a/docs/ADAPTER.md +++ b/docs/ADAPTER.md @@ -95,6 +95,7 @@ The adapter walks each top-level spec section into a deterministic family of lay | `spec.policy.burn_rate_alerts[]` | L4 . policy | `POL-{NN}` | `refs` includes the bound SLO | | `spec.policy.forecasts[]` | L4 . policy | `FCST-{NN}` | | | `spec.alerting.routes[]` | L4 . alerting | `ALR-{NN}` | One per severity route | +| `spec.alerting.rules[]` | L4 . alerting | `RULE-{NN}` | One per operational alert rule (spec 1.4), titled by the rule's exact name; `tool` by engine (`PrometheusRule`, `Grafana alerting`, `Loki ruler`); the artefact model keys it on the name, so a repository's rule and a live listing's rule pair | | `spec.remediation[]` | L4 . healing | `HEAL-{NN}` | `refs` includes the trigger alert | | `spec.baselines` | L5 | `BASE-01` | | | `spec.validation.chaos_experiments[]` | L5 | `CHAOS-{NN}` | `refs` = steady-state SLO + each `expected_alerts` entry | diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 0365ab33..5dc1e4a2 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -2,6 +2,13 @@ ## Unreleased +### Spec 1.4: operational alert rules are kept in `alerting.rules`, and the repository's rules pair with the live ones +The crawler discovered every Prometheus and Grafana alert rule in a repository and kept only the burn-rate alerts (those referencing a recorded SLO series) in `spec.policy.burn_rate_alerts`; every other rule was dropped with *Excluded N operational alert(s) from burn-rate policy — … They remain available as alerting signals* — a promise nothing kept, since spec 1.3 had no field for them. Reconciled against a live Grafana snapshot, the crawled pack matched zero alerts although the repository held the very provisioning files those rules came from, with identical titles. RFC-0003 (upstream MoebiusX/otel-observability-pack, branch `codex/alerting-rules`, commit `6674072e`, proposed 2026-10-03) adds `spec.alerting.rules[]` — `name` and `expr` required; `severity` (the pack's SEV1–SEV4), `for`, `labels`, `annotations`, `engine` (`prometheus` default, `mimir`, `thanos`, `victoriametrics`, `loki`, `grafana`, `alertmanager`) and `source` optional; `routes` still the only required key, so every 1.3 pack validates unchanged. Observogram adopts it end to end: +- **Vendor.** `vendor/observability-pack-spec/v1.4/` holds the four upstream files byte for byte at that commit, `VERSIONS.json` records spec 1.4; `v1.3/` is removed (1.4 validates every 1.3 pack unchanged; git history keeps it). `SPEC_VERSION = '1.4'` and every derived path, banner, studio text and link follows (33 compile goldens and the build fixture moved by their version line alone). `tools/test-validator.mjs` pins the 1.4 schema's shape, its acceptance of `alerting.rules` and its refusal of a rule without `expr`, an unknown `engine` or a non-SEV `severity`, and every 1.3-shaped pack of the repo validating unchanged. +- **Crawler.** Operational alerts are declared in `spec.alerting.rules` with their exact names (never slugged — the key a live listing is reconciled on), expression, canonical `for`, the pack's severity from `labels.severity` beside the engine's own label, labels and annotations verbatim, `engine` (Grafana-managed by shape, Loki by path or LogQL, Prometheus otherwise) and `source` (`#/`). The burn-rate classification is unchanged. Grafana provisioning YAML is detected by path (`provisioning/alerting/`) or shape, and a Grafana rule's expression is read from its query node. The warning became a true summary line (*N operational alert rule(s) kept in alerting.rules … M burn-rate alert(s) in policy.burn_rate_alerts*); `summary.discovered.alertRules` counts them. The crawl golden is byte-identical (its fixture has no operational rule). +- **Adapter and Compare.** Each rule projects as an L4 alerting artefact (`RULE-NN`) titled by the rule's name, tool by engine (`PrometheusRule`, `Grafana alerting`, `Loki ruler`), beside the routes; the artefact model keys an `alert_rule` on its name and compares expression, wait, labels and severity — not `engine` or `source` — so a repository's rule and the running rule are one artefact. The live fetcher declares the ruler's non-SLO rules the same way (`operationalAlertRule`, one reading for both sides), engine from what the answering tool attests, each `mcp.verified.alerting.rules[i]` unless unhealthy. Conformance is unaffected. The studio shows them in the drawer, the Discover board and the Build model. +- **Tests.** `tools/test-crawl-alerting-rules.mjs` with the fixture `tools/fixtures/crawl/operational-alerts/`: six operational rules across a Prometheus rule file, Grafana provisioning YAML and a Loki rule file, three burn-rate rules folding into two policy entries — validates against 1.4, exact names, engines, severities, `for`, provenance, YAML round-trip, six adapter artefacts keyed by name. `test-scan-live`: the repository's `DiskAlmostFull` and the ruler's pair aligned although one side says prometheus and the other attests victoriametrics; a rule whose recorded series is not deployed reads as a live operational rule — a true "live, not declared". `test-adapt`: pairing, drift on `expr`, a live-only rule. + ### Proposed: the UPDATE journey — changing a pack that already exists Docs only; nothing is built. Build is for a team with no pack, and once a pack is registered the studio has no way to adapt it: add an SLO, an alert, a route, change an objective. `docs/UPDATE_JOURNEY.md` is the user journey for it, *Build or update a pack*: one journey with two starts, where `update` opens an existing pack on Build's own screen, lists the changes against the saved version, shows the saved pack beside the pack with the changes, saves the next version of the same pack and hands the difference to deploy. It records what the code allows today (Build is a generator and cannot load a pack; one SLO per SLI, one burn-rate alert per SLO, three fixed routes; a pack's id is its content hash and its version a constant), what each stage reuses and what is new, three slices — adding an SLO, an alert or a route arrives with the second, as additions recorded in the pack on top of Build's output — and eight decisions for the owner. `BUILD_JOURNEY.md` and `USER_JOURNEY.md` link to it. diff --git a/docs/MCP_INTEGRATION.md b/docs/MCP_INTEGRATION.md index 676bb2dd..1f19d6ca 100644 --- a/docs/MCP_INTEGRATION.md +++ b/docs/MCP_INTEGRATION.md @@ -48,7 +48,7 @@ diagnostic-grade drift: | Metrics | metric inventory and names observed from the live platform | | Scrape jobs | Prometheus/VictoriaMetrics scrape evidence | | Recording rules | full rule names and expressions where the MCP exposes them | -| Alert rules | Grafana/Prometheus alerting rules; burn-rate alerts are mapped from them per SLO, never synthesised | +| Alert rules | Grafana/Prometheus alerting rules; burn-rate alerts are mapped from them per SLO, never synthesised; every other rule is declared in `spec.alerting.rules` (spec 1.4) under its exact name | | Alerting routes | the routes of the configuration the running Alertmanager reports — severities and channels, secret addresses as `redacted:secret` | | Dashboards | Grafana dashboard metadata plus dashboard bodies, panels, variables, and targets | | Baselines | none yet — MTTD/MTTR are platform defaults stamped `Scaffold`; anomaly baselines are only counted (`mcp.baselinesComputed`) | @@ -153,7 +153,7 @@ JSON arrays (`annotationJson`) at 200 entries; error strings at 200 chars. | `mcp.capabilities.unobserved` | comma list of products | what the inventory lists and nothing showed to be running: supported, not deployed — never a backend | | `mcp.backends.evidence` | `=` joined by `\|` | per backend: the version probe (`metrics_query/vm_app_version`), the scrape job (`scrape job promtail`) or the product tool (`vmalert_rules`) behind it | | `mcp.derived.` | string | an entry READ from live rules without being attested as what the pack calls it — `policy.burn_rate_alerts[]` read from plain alerting rules names the rules; projects as `Declared`, neither `Verified` nor `Scaffold` | -| `mcp.discovered.alert_rules_linked` / `alert_rules_operational` | counts | alerting rules that guard a recorded SLO (read as burn-rate entries) / that guard none | +| `mcp.discovered.alert_rules_linked` / `alert_rules_operational` | counts | alerting rules that guard a recorded SLO (read as burn-rate entries) / that guard none — the latter are declared in `spec.alerting.rules` (spec 1.4), each stamped `mcp.verified.alerting.rules[]` unless the ruler reports it unhealthy | | `observogram.unobserved.` | reason string | an artefact family this fetch had no way to look at (see *What the fetch could not look at*) | | `mcp.stack.status` | `sampled` \| `not-attempted` | the stack self-metrics panel (step 2, "Stack self-metrics (sampling)"): whether it was sampled at all — signals, never verdicts; written whenever the fetcher ran the step-2 sampler (a caller that predates step 2 writes nothing) | | `mcp.stack.reason` | string | only when `not-attempted`: why (`metrics_query not exposed by this MCP (restricted tier)`) | @@ -282,7 +282,8 @@ first) and each emitted entry is stamped `mcp.verified.policy.burn_rate_alerts[< series' SLO and contributes its window (`for`, severity) to the SLO's entry; entries with fewer than two windows take the default pair; a rule that references no recorded series is an operational alert and no SLO - contract. Hand-written rules carry neither the compiler's labels nor its + contract — since spec 1.4 it is declared in `spec.alerting.rules` (below), + not dropped. Hand-written rules carry neither the compiler's labels nor its names — without this a platform with seventy alerting rules read as having no burn-rate policy, and every alert its repository declares read "not live". Such an entry is real (no scaffold marker) and **not** `Verified`: @@ -293,6 +294,22 @@ first) and each emitted entry is stamped `mcp.verified.policy.burn_rate_alerts[< - A burn group for an SLO nobody inferred, or one with a single window (the schema requires two), is not representable and is listed in `mcp.discovered.alert_rules_unmapped` instead of being padded. +- **Operational rules are declared, not lost (spec 1.4 `alerting.rules`).** + Every alerting rule the ruler reports that binds to no SLO is read with + the same reading the crawler applies to a rule file + (`sli-inference.mjs` `operationalAlertRule`): its exact `name` (the join + key), `expr` (`query` on the wire), `for` as the canonical duration + (`duration: 600` is `10m`), the pack's `severity` from `labels.severity` + beside the engine's own word in `labels`, and `engine` from what the + answering tool attests (the contract registry's `attests`: grafana, + mimir, vmalert → victoriametrics; a plain Prometheus-API answer states + none, which means prometheus). The repository's crawler declares the + same rules from the rule files, so the two sides pair rule for rule by + name in Compare — the engine is not compared (a rule file cannot tell a + Prometheus ruler from a VictoriaMetrics one), the expression, wait, + labels and severity are. Until 1.4 these rules were counted + (`alert_rules_operational`) and carried nowhere, and a live Grafana + snapshot matched zero alerts against the repository that provisioned them. - A rule with no recognisable severity gets one from its burn factor (`>= 10x` SEV1, `>= 5x` SEV2, else SEV3) and is listed in `mcp.discovered.alert_rules_severity_inferred`. @@ -379,6 +396,7 @@ both read, they read through one module: |---|---|---|---| | SLIs / SLOs | `sli-inference.mjs` `inferSlisFromRecordingRules` | rule files | the ruler's rule list — choices are made by rule NAME, never by arrival order | | Burn-rate entries | `sli-inference.mjs` `burnAlertsFromAlertRules` | alerting rule files (`for: 2m`) | the ruler's alerting rules (`duration: 120`) | +| Operational alert rules (spec 1.4 `alerting.rules`) | `sli-inference.mjs` `operationalAlertRule` | the rules of a rule file or Grafana provisioning YAML that guard no SLO (`alert:` / `title:`, `expr` or `data[].model.expr`) | the ruler's alerting rules that bind to no SLO (`name`, `query`, `duration`) — paired by exact name | | Routes | `alert-routes.mjs` `routesFromAlertmanagerConfig` | the Alertmanager config file (`${VAR}` → `unresolved:`) | the running configuration (`` → `redacted:secret`) | | Backends | `backend-products.mjs` | container images | scrape jobs, versions, product tools | | Extended surfaces | `l2x.mjs` | the backends above | the backends above | From cac025ed8679d020a7f937d8c824024719c75e3b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 3 Oct 2026 13:30:04 +0000 Subject: [PATCH 030/550] chore(vendor): spec 1.4 pinned at upstream develop (e64e5813, PR #9 merged), not the RFC branch RFC-0003 merged upstream (MoebiusX/otel-observability-pack#9), so the vendored copy is re-synced with `node tools/sync-spec.mjs` against the default branch: VERSIONS.json records ref develop and commit e64e58132a46364afd5432a86e67d470858936cf. All four blobs are byte-identical to the ones vendored from the RFC branch head (same sha256 and byte counts; `unchanged` on every file), so v1.4/ does not move; only the upstream block and fetchedAt do. The three studio links into the upstream repo (footer, Schema view, Conformance view) follow upstream.commit, as tools/test-validator.mjs requires; its comment and the vendor README's refresh example and branch note say develop now carries 1.4; the changelog names the merge. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01UhPGnqVJ24goSVtbKpdkLJ --- docs/CHANGELOG.md | 2 +- studio/conformance-view.mjs | 2 +- studio/index.html | 2 +- studio/schema-view.mjs | 4 ++-- tools/test-validator.mjs | 4 ++-- vendor/observability-pack-spec/README.md | 4 ++-- vendor/observability-pack-spec/VERSIONS.json | 6 +++--- 7 files changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 5dc1e4a2..5996929d 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -3,7 +3,7 @@ ## Unreleased ### Spec 1.4: operational alert rules are kept in `alerting.rules`, and the repository's rules pair with the live ones -The crawler discovered every Prometheus and Grafana alert rule in a repository and kept only the burn-rate alerts (those referencing a recorded SLO series) in `spec.policy.burn_rate_alerts`; every other rule was dropped with *Excluded N operational alert(s) from burn-rate policy — … They remain available as alerting signals* — a promise nothing kept, since spec 1.3 had no field for them. Reconciled against a live Grafana snapshot, the crawled pack matched zero alerts although the repository held the very provisioning files those rules came from, with identical titles. RFC-0003 (upstream MoebiusX/otel-observability-pack, branch `codex/alerting-rules`, commit `6674072e`, proposed 2026-10-03) adds `spec.alerting.rules[]` — `name` and `expr` required; `severity` (the pack's SEV1–SEV4), `for`, `labels`, `annotations`, `engine` (`prometheus` default, `mimir`, `thanos`, `victoriametrics`, `loki`, `grafana`, `alertmanager`) and `source` optional; `routes` still the only required key, so every 1.3 pack validates unchanged. Observogram adopts it end to end: +The crawler discovered every Prometheus and Grafana alert rule in a repository and kept only the burn-rate alerts (those referencing a recorded SLO series) in `spec.policy.burn_rate_alerts`; every other rule was dropped with *Excluded N operational alert(s) from burn-rate policy — … They remain available as alerting signals* — a promise nothing kept, since spec 1.3 had no field for them. Reconciled against a live Grafana snapshot, the crawled pack matched zero alerts although the repository held the very provisioning files those rules came from, with identical titles. RFC-0003 (upstream MoebiusX/otel-observability-pack, merged into develop as `e64e5813` via PR #9, proposed 2026-10-03) adds `spec.alerting.rules[]` — `name` and `expr` required; `severity` (the pack's SEV1–SEV4), `for`, `labels`, `annotations`, `engine` (`prometheus` default, `mimir`, `thanos`, `victoriametrics`, `loki`, `grafana`, `alertmanager`) and `source` optional; `routes` still the only required key, so every 1.3 pack validates unchanged. Observogram adopts it end to end: - **Vendor.** `vendor/observability-pack-spec/v1.4/` holds the four upstream files byte for byte at that commit, `VERSIONS.json` records spec 1.4; `v1.3/` is removed (1.4 validates every 1.3 pack unchanged; git history keeps it). `SPEC_VERSION = '1.4'` and every derived path, banner, studio text and link follows (33 compile goldens and the build fixture moved by their version line alone). `tools/test-validator.mjs` pins the 1.4 schema's shape, its acceptance of `alerting.rules` and its refusal of a rule without `expr`, an unknown `engine` or a non-SEV `severity`, and every 1.3-shaped pack of the repo validating unchanged. - **Crawler.** Operational alerts are declared in `spec.alerting.rules` with their exact names (never slugged — the key a live listing is reconciled on), expression, canonical `for`, the pack's severity from `labels.severity` beside the engine's own label, labels and annotations verbatim, `engine` (Grafana-managed by shape, Loki by path or LogQL, Prometheus otherwise) and `source` (`#/`). The burn-rate classification is unchanged. Grafana provisioning YAML is detected by path (`provisioning/alerting/`) or shape, and a Grafana rule's expression is read from its query node. The warning became a true summary line (*N operational alert rule(s) kept in alerting.rules … M burn-rate alert(s) in policy.burn_rate_alerts*); `summary.discovered.alertRules` counts them. The crawl golden is byte-identical (its fixture has no operational rule). - **Adapter and Compare.** Each rule projects as an L4 alerting artefact (`RULE-NN`) titled by the rule's name, tool by engine (`PrometheusRule`, `Grafana alerting`, `Loki ruler`), beside the routes; the artefact model keys an `alert_rule` on its name and compares expression, wait, labels and severity — not `engine` or `source` — so a repository's rule and the running rule are one artefact. The live fetcher declares the ruler's non-SLO rules the same way (`operationalAlertRule`, one reading for both sides), engine from what the answering tool attests, each `mcp.verified.alerting.rules[i]` unless unhealthy. Conformance is unaffected. The studio shows them in the drawer, the Discover board and the Build model. diff --git a/studio/conformance-view.mjs b/studio/conformance-view.mjs index d1061483..211011d9 100644 --- a/studio/conformance-view.mjs +++ b/studio/conformance-view.mjs @@ -28,7 +28,7 @@ import { wireSectionNav, wireUxActions, } from './ux-kit.mjs'; -const RUBRIC_URL = 'https://github.com/MoebiusX/otel-observability-pack/blob/6674072e4d3593f3e8345c4032a6eec23b7b19b7/docs/maturity-model.md'; +const RUBRIC_URL = 'https://github.com/MoebiusX/otel-observability-pack/blob/e64e58132a46364afd5432a86e67d470858936cf/docs/maturity-model.md'; const DIMENSIONS = ['L1', 'L2', 'L2X', 'L3', 'L4', 'L5']; diff --git a/studio/index.html b/studio/index.html index e91e4ba8..72fb80d9 100644 --- a/studio/index.html +++ b/studio/index.html @@ -488,7 +488,7 @@

    Deploy repository changes to live